To achieve host-path backend routing, we changed behaviour of
--backend-http2-connections-per-worker. It now sets the number of
HTTP/2 physical connections per pattern group if pattern is used in -b
option.
Fixes GH-292
There are many requests which changes its meaning when we rewrite
path. This is due to bad percent-encoding in URI; reserved characters
are just used without percent encoding. It seems this is common in ad
services, but I suspect more to come. For reverse proxying situation,
sane service most likely encodes URI properly, so probably this is not
an issue.
-b option syntax is now <HOST>,<PORT>[;<PATTERN>[:...]]. The optional
<PATTERN>s specify the request host and path it is used for. The
<PATTERN> can contain path, host + path or host. The matching rule is
closely designed to ServeMux in Go programming language.
Previously we send RST_STREAM when we send DATA with END_STREAM flag
set. With this commit, we also do this when we send HEADERS with
END_STREAM flag set.
Previously nghttp2_session_send() and nghttp2_session_mem_send() did
not send 24 bytes client magic byte string (MAGIC). We made
nghttp2_session_recv() and nghttp2_session_mem_recv() process MAGIC by
default, so it is natural to make library send MAGIC as well. This
commit makes nghttp2_session_send() and nghttp2_session_mem_send()
send MAGIC. This commit also replace "connection preface" with
"client magic", since we call MAGIC as "connection preface" but it is
just a part of connection preface. NGHTTP2_CLIENT_CONNECTION_PREFACE
macro was replaced with NGHTTP2_CLIENT_MAGIC. The already deprecated
NGHTTP2_CLIENT_CONNECTION_HEADER macro was removed permanently.
nghttp2_option_set_no_recv_client_preface() was renamed as
nghttp2_option_set_no_recv_client_magic(). NGHTTP2_ERR_BAD_PREFACE
was renamed as NGHTTP2_ERR_BAD_CLIENT_MAGIC.
HTTP/2 and HPACK are going to be published as RFC, but ALTSVC is still
in draft state. To make our API stable, it would be better to remove
ALTSVC API for 1.0.0 release.
Currently, we use same number of HTTP/2 sessions per worker with given
backend addresses. New option to specify the number of HTTP/2 session
per worker will follow.
nghttpx server push is initiated by looking for Link header field from
backend server response. Currently we only enable server push for
HTTP/1 backend and without HTTP/2 proxy mode. The URIs which have
rel=preload are eligible to resource to be pushed.
nghttp2_submit_shutdown_notice() is used to notify the client that
graceful shutdown is started. We expect that after this call, the
server application should send another GOAWAY using
nghttp2_submit_goaway() with appropriate last_stream_id. In this
commit, we also added nghttp2_session_get_last_proc_stream_id(), which
can be used as last_stream_id parameter.
This commit implements graceful shutdown in nghttpx. The integration
test for graceful shutdown is also added.
Previously when requests are issued to HTTP/2 downstream connection,
but it turns out that connection is down, handlers of those requests
are deleted. In some situations, we only know connection is down when
we write something to network, so we'd like to handle this kind of
situation in more robust manner. In this change, certain seconds
passed after last network activity, we first issue PING frame to
downstream connection before issuing new HTTP request. If writing
PING frame is failed, it means connection was lost. In this case,
instead of deleting handler, pending requests are migrated to new
HTTP2/ downstream connection, so that it can continue without
affecting upstream connection.
This commit limits the number of concurrent HTTP/1 downstream
connections to same host. By defualt, it is limited to 8 connections.
--backend-connections-per-frontend option was replaced with
--backend-http1-connections-per-host, which changes the maximum number
of connections per host. This limitation only kicks in when h2 proxy
is used (-s option).
This commit adds functionality to customize access logging format in
nghttpx. The format variables are inspired by nginx. The default
format is combined format.
This is not obvious but it makes intermediaries flush and forward DATA
frame boundary without excessive buffering. Since we have different
TCP connections frontend and backend, this may not work. This is
still experimental.
Use the same behaviour the current Google server does: start with 1300
TLS record size and after transmitting 1MiB, change record size to
16384. After 1 second idle time, reset to 1300. Only applies to
HTTP/2 and SPDY upstream connections.
Previously read and write timeouts work independently. When we are
writing response to the client, read timeout still ticks (e.g., HTTP/2
or tunneled HTTPS connection). So read timeout may occur during long
download. This commit fixes this issue. This commit only fixes the
upstream part. We need similar fix for the downstream.
Previously we empties request headers after they are sent to
downstream in order to free memory. But it turns out that we use
request headers when rewriting location header response field. Also
user reported that request headers are useful to add new features.
This commits defers the deletion of request headers to the point when
response headers are deleted (which is after response headers are sent
to upstream client).
It is not used by library for a while. It could be used to pass
unsupported extension frames to application, but its interface
requires library to buffer entire frame, which we'd like to avoid.
For unsupported extension frames, we will add new callbacks which does
not require buffering if they are required.
h2-14 now allows extensions to define new error codes. To allow
application callback to access such error codes, we uses uint32_t as
error_code type for structs and function parameters. Previously we
treated unknown error code as INTERNAL_ERROR, but this change removes
this and unknown error code is passed to application callback as is.
To make it possible to add new callbacks without bumping so name, we
decided to hide details of nghttp2_session_callbacks. We provide
setter like functions to set individual callback function.
Previously we only update consumed flow control window when number of
bytes read in nghttp2 and spdylay callback is 0. Now we notify
nghttp2 library the consumed bytes even if number of bytes read > 0.
This change also uses newly added spdylay_session_consume() API, so we
require spdylay >= 1.3.0.
This option limits the number of backend connections per frontend.
This is meaningful for the combination of HTTP/2 and SPDY frontend and
HTTP/1 backend.
--no-location-rewrite option disallows location header rewrite on
--http2-bridge, --client and default mode. This option is useful when
connecting nghttpx proxy with --http2-bridge to backend nghttpx with
http2-proxy mode.
Reworked no automatic WINDOW_UPDATE feature. We added new API
nghttp2_session_consume() which tells the library how many bytes are
consumed by the application. Instead of submitting WINDOW_UPDATE by
the application, the library is now responsible to submit
WINDOW_UPDATE based on consumed bytes. This is more reliable method,
since it enables us to properly send WINDOW_UPDATE for stream and
connection individually. The previous implementation of nghttpx had
broken connection window management.
Now concatenating header values with 0x00 as delimiter is not
necessary because HPACK reference set is removed and the order of
header field fed into HPACK encoder is preserved when they are
decoded.
This change rewrites logging system of nghttpx. Previously access log
and error log are written to stderr or syslog and there was no option
to change stderr to something else. With this change, file path of
access log and error log can be configured separately and logging to
regular file is now added. To support rotating log, if SIGUSR1 signal
is received by nghttpx, it closes the current log files and reopen it
with the same name. The format of access log is changed and has same
look of apache's. But not all columns are not supported yet.
For now, if request has request body, we'll issue RST_STREAM to inform
the peer to stop sending body. RST_STREAM may be sent before error
page header or data, so peer may receive RST_STREAM only.
Previously, we use evbuffer_pullup(buf, -1) to linearize the memory
region and it may cause buffer copy. To avoid this, we use the return
value of evbuffer_get_contiguous_space() as 2nd parameter. According
to the libevent manual, by doing so evbuffer_pullup() will not copy or
modify any data in evbuffer.
To make adding new option easier, we decided to make the details of
option struct private and hide it from public API. We provide
functions to set individual option value.
If SPDY or HTTP/2 ustream is used and HTTP/2 downstream is used, only
call {spdylay,nghttp2}_resume_data when complete DATA frame was read
in backend to avoid to transmit too small DATA frame to the upstream.
The profiler and benchmarking showed that calling evbuffer_add()
repeatedly is very costly. To avoid this, we buffer up small writes
into one large chunk and call evbuffer_add() less times.
This function behaves like nghttp2_session_send(), but it does not
use nghttp2_send_callback to send data. Instead, it returns the
serialized data to trasmit and its length to the caller.
Previously, there is inconsistency when on_frame_recv_callback
is called between HEADERS/PUSH_PROMISE and the other frames.
For former case, it is called before header block, in latter
case, it is called after whole frame is received. To make it
consistent, we call on_frame_recv_callback for HEADERS/PUSH_PROMISE
after its frame is fully received. Since on_frame_recv_callback
can signal the end of header block, we replaced on_end_headers_callback
with on_begin_headers_callback, which is called when the reception
of the header block is started.
nghttp2_data is added to nghttp2_frame union. When DATA is
received, nghttp2_on_frame_recv_callback is called. When DATA is
sent, nghttp2_on_frame_send_callback is called.
4ed4efc does not disable TLS renegotiation at all, if client keeps
rengotiations without sending application data. In this change,
we intercept the raw incoming data from the client and if it is a
renegotiation, drop the connection immediately.
nghttp2 library itself now accept octet header/value pairs,
completely not restricted by HTTP/1 header name/value rule.
The applications may impose restriction about them using
validators.
Now, in nghttp2_on_frame_recv_callback, nva and nvlen in
HEADERS and PUSH_PROMISE frames are always NULL and 0 respectively.
The header name/value pairs are emitted successive
nghttp2_on_header_callback functions. The end of header fields are
signaled with nghttp2_on_end_headers_callback function.
Since NGHTTP2_ERR_PAUSE for nghttp2_on_frame_recv_callback is
introduced to handle header block, it is now deprecated.
Instead, nghttp2_on_header_callback can be paused using
NGHTTP2_ERR_PAUSE.
We thought that this kind of rewrite can be achieved by the configuration
of the backend severs, but in some configuration, however, it may get
complicated. So we decided to implement at least location rewrite in
nghttpx.
This commit also contains a fix to the bug which prevents the http2
backend request from concatenating header fields with the same value.
Tatsuhiro Tsujikawa
Lucas Pardue