It turns out that the cause of complication in backend request line
construction is a absolute-form in HTTP/1 request. In HTTP/2, we have
separated pseudo-header fields and no problem at all. In this commit,
we parse request URI in HTTP/1 frontend and extract values from it to
make backend logic simpler. This patch removes host header field
emission in HTTP/2 backend if :authority is emitted. It also rewrites
host header field with authority part in absolute-form URI as per RFC
7230.
We may later add append(const T &), where we require T for .size() and
.data(). We can coexist former append_cstr and new one. If we pass
string literal, append(const (&)[N]) is selected.
Previously when requests are issued to HTTP/2 downstream connection,
but it turns out that connection is down, handlers of those requests
are deleted. In some situations, we only know connection is down when
we write something to network, so we'd like to handle this kind of
situation in more robust manner. In this change, certain seconds
passed after last network activity, we first issue PING frame to
downstream connection before issuing new HTTP request. If writing
PING frame is failed, it means connection was lost. In this case,
instead of deleting handler, pending requests are migrated to new
HTTP2/ downstream connection, so that it can continue without
affecting upstream connection.
This commit adds functionality to customize access logging format in
nghttpx. The format variables are inspired by nginx. The default
format is combined format.
Previously read and write timeouts work independently. When we are
writing response to the client, read timeout still ticks (e.g., HTTP/2
or tunneled HTTPS connection). So read timeout may occur during long
download. This commit fixes this issue. This commit only fixes the
upstream part. We need similar fix for the downstream.
Previously we empties request headers after they are sent to
downstream in order to free memory. But it turns out that we use
request headers when rewriting location header response field. Also
user reported that request headers are useful to add new features.
This commits defers the deletion of request headers to the point when
response headers are deleted (which is after response headers are sent
to upstream client).
Previously we only update consumed flow control window when number of
bytes read in nghttp2 and spdylay callback is 0. Now we notify
nghttp2 library the consumed bytes even if number of bytes read > 0.
This change also uses newly added spdylay_session_consume() API, so we
require spdylay >= 1.3.0.
--no-location-rewrite option disallows location header rewrite on
--http2-bridge, --client and default mode. This option is useful when
connecting nghttpx proxy with --http2-bridge to backend nghttpx with
http2-proxy mode.
Reworked no automatic WINDOW_UPDATE feature. We added new API
nghttp2_session_consume() which tells the library how many bytes are
consumed by the application. Instead of submitting WINDOW_UPDATE by
the application, the library is now responsible to submit
WINDOW_UPDATE based on consumed bytes. This is more reliable method,
since it enables us to properly send WINDOW_UPDATE for stream and
connection individually. The previous implementation of nghttpx had
broken connection window management.
This change rewrites logging system of nghttpx. Previously access log
and error log are written to stderr or syslog and there was no option
to change stderr to something else. With this change, file path of
access log and error log can be configured separately and logging to
regular file is now added. To support rotating log, if SIGUSR1 signal
is received by nghttpx, it closes the current log files and reopen it
with the same name. The format of access log is changed and has same
look of apache's. But not all columns are not supported yet.
Previously, we use evbuffer_pullup(buf, -1) to linearize the memory
region and it may cause buffer copy. To avoid this, we use the return
value of evbuffer_get_contiguous_space() as 2nd parameter. According
to the libevent manual, by doing so evbuffer_pullup() will not copy or
modify any data in evbuffer.
It appears that some Android client does not like Connection: close in
200 CONNECT response. Browsers fine with this header field. It is
better to remove it. Squid does not emit it too.
If SPDY or HTTP/2 ustream is used and HTTP/2 downstream is used, only
call {spdylay,nghttp2}_resume_data when complete DATA frame was read
in backend to avoid to transmit too small DATA frame to the upstream.
4ed4efc does not disable TLS renegotiation at all, if client keeps
rengotiations without sending application data. In this change,
we intercept the raw incoming data from the client and if it is a
renegotiation, drop the connection immediately.
We thought that this kind of rewrite can be achieved by the configuration
of the backend severs, but in some configuration, however, it may get
complicated. So we decided to implement at least location rewrite in
nghttpx.
This commit also contains a fix to the bug which prevents the http2
backend request from concatenating header fields with the same value.
Now we set Downstream::set_response_connection_close(true) for
tunneled connections. Also call
Upstream::on_downstream_body_complete() callback when setting
MSG_COMPLETE in SpdySession when RST_STREAM is caught. Clean up EOF
handling in https_downstream_readcb.
Currently, resume_read() fails if on_read() returns -1 in case that
evbuffer_add failed, which means, most likely, memory allocation
failure. ClientHandler is marked "should be closed", but if
evbuffer_add is failed, write callback will not be invoked and its
marking is not evaluated. It will eventually be deleted when the
client is disconnected or backend failure though.
With --spdy-bridge option, it listens SPDY/HTTPS connections from
front end and forwards them to the backend in SPDY. The usage will be
written later. This change fixes the crash when more than 2
outstanding SpdyDownstreamConnection objects are added to SpdySession
and establishing connection to SPDY backend is failed.
INFO log and its surrounding code are now guarded by
LOG_ENABLED(SEVERITY) macro so that they don't run if log level
threshold is higher. This increases performance because log formatting
is somewhat expensive.
Added macros which log messages from the following components are
prefixed with their component name + object pointer address:
ListenHandler: LISTEN
ThreadEventReceiver: THREAD_RECV
Upstream: UPSTREAM
Downstream: DOWNSTREAM
DownstreamConnection: DCONN
SpdySession: DSPDY
In client mode, now SPDY connection to the backend server is
established per thread. The frontend connections which belong to the
same thread share the SPDY connection.
With --client-mode option, shrpx now accepts unencrypted HTTP
connections and communicates with backend server in SPDY. In short,
this is the "reversed" operation mode against normal mode. This may
be useful for testing purpose because it can sit between HTTP client
and shrpx "normal" mode.
When deciding whether to close the client connection, check
request_connection_close_ of Downstream in addition of
response_connection_close_. Also we only add "Connection: Keep-Alive"
header to the HTTP/1.0 or HTTP/0.9 clients.
To distinguish the to-be-installed programs and non-installable
example source code, the former programs, spdycat, spdydyd and shrpx,
were moved to src directory. spdynative was removed from Makefile
because it does not appeal to any users much.
Tatsuhiro Tsujikawa
Stefan Eissing
Lucas Pardue