Previously we use 16K - 9 bytes (frame header) as frame payload size
so that whole frame fits in 1 TLS record size (16K). But it turns out
that in proxy use case, we will receive 16K payload from backend and
we have to split it into 2 odd looking frames (16K - 9 and 9), and
latter is highly inefficient. To avoid this situation, we decided to
use min frame payload size to 16K. Since we operates on TLS as stream
of data, we are not so much restricted in its record size.
Previously we treat stream in NGHTTP2_STREAM_RESERVED state specially,
that is we don't increment or decrement streams counts if stream is in
that state. Because of this, we don't change the stream state to
NGHTTP2_STREAM_CLOSING if stream is in NGHTTP2_STREAM_RESERVED. But
it turns out that it causes a problem. If client canceled pushed
stream before push response HEADERS, stream is still in
NGHTTP2_STREAM_RESERVED state. If push response HEADERS arrived in
this state, library happily accepts it and passed to application.
With this commit, this bug was corrected. We now change stream state
to NGHTTP2_STREAM_CLOSING even if it was in NGHTTP2_STREAM_RESERVED
state. We now use NGHTTP2_STREAM_FLAG_PUSH to determine whether we
have to increase/decrase stream count.
nghttp2_submit_shutdown_notice() is used to notify the client that
graceful shutdown is started. We expect that after this call, the
server application should send another GOAWAY using
nghttp2_submit_goaway() with appropriate last_stream_id. In this
commit, we also added nghttp2_session_get_last_proc_stream_id(), which
can be used as last_stream_id parameter.
This commit implements graceful shutdown in nghttpx. The integration
test for graceful shutdown is also added.
Initially, we use nghttp2_stream.data_item to refer only item with
DATA frame. But recently we use it to refer HEADERS frame as well.
So it is better to call just item rather than data_item. This applies
to all related functions.
Previously we handle idle streams as closed streams. We only keeps
sum of closed streams and active streams under max concurrent streams
limit, idle streams gets deleted earlier than client expects.
In this change, idle streams are kept in separate list and not handled
as closed streams. To mitigate possible attack vector to make
unlimited idle streams, we cap the number of idle streams in a half of
max concurrent streams. This is arbitrary choice. It may be adjusted
in the future when we have interop experience.
nghttp2_mem structure is introduced to hold custom memory allocator
functions and user supplied pointer. nghttp2_mem object can be passed
to nghttp2_session_client_new3(), nghttp2_session_server_new3(),
nghttp2_hd_deflate_new2() and nghttp2_hd_inflate_new2() to replace
standard malloc(), free(), calloc() and realloc(). nghttp2_mem
structure has user supplied pointer mem_user_data which can be used as
per session/object memory pool.
This change will utilize last_stream_id in GOAWAY extensively. When
GOAWAY is received with a last_stream_id, library closes all outgoing
streams whose stream_id > received last_stream_id.
nghttp2_on_stream_callback is called for each stream to be closed.
When GOAWAY is sent with a last_stream_id, library closes all incoming
streams whose stream_id > sent last_stream_id.
nghttp2_on_stream_callback is called for each stream to be closed.
If stream ID is not idle, it might be valid HEADERS. If stream ID is
idle, it is invalid regardless stream ID is even or odd, since client
is not expected to recieve request from server. nghttp2 library
historically allows this, but now we forbids this.
We make following HEADERS under priority control:
* push response HEADERS
* HEADERS submitted by nghttp2_submit_response
Currently, HEADERS submitted by nghttp2_submit_headers is not attached
to stream. This is because it may be used as non-final response
header and application may submit final response using
nghttp2_submit_response without checking non-final response header
transmission.
Allowing PRIORITY frame at anytime so that PRIORITY frame to idle
stream can create anchor node in dependency tree. In this change, we
open stream with new NGHTTP2_STREAM_IDLE state, which is linked in
session->closed_stream_head and is treated as if it is closed stream.
One difference is that if the stream is opened, we remove it from
linked list and change the state to the appropriate one. To O(1)
removal from linked list, we change session->closed_stream_head to
doubly linked list.
This change fixes the bug that stream is out of dependency tree if the
number of nodes in a dependency tree which we add new node to is
already maximum (NGHTTP2_MAX_DEP_TREE_LENGTH) and the number of
maximum concurrent streams is more than more than
NGHTTP2_MAX_DEP_TREE_LENGTH.
* Add NGHTTP2_HTTP_1_1_REQUIRED error code
* Allow transmission of WINDOW_UPDATE on reserved (remote)
* Allow reception of WINDOW_UPDATE on reserved (local)
* Treat frame larger than MAX_FRAME_SIZE as FRAME_SIZE_ERROR
ALPN identifier is still h2-14 to continue interop, since draft-14 and
-15 are binary compatible. The new error code was added in draft-15,
but HTTP/2 allows extensions can freely add new error code, so it is
not a problem.
This also means that at least one stream whose dpri is
NGHTTP2_STREAM_DPRI_TOP exists, its siblings descendants have no
chance to send streams, even if their parent stream has
NGHTTP2_STREAM_DPRI_NODATA.
Previously when connection level remote flow control window gets 0, we
mark the stream having DATA frame with
NGHTTP2_STREAM_FLAG_DEFERRED_FLOW_CONTROL. When connection level
WINDOW_UPDATE is received, we checks all existing streams, including
closed ones, and call nghttp2_stream_resume_deferred_data(). The
profiler shows this is expensive.
Now we prepare dedicated priority queue for DATA frames. And we don't
mark stream with NGHTTP2_STREAM_FLAG_DEFERRED_FLOW_CONTROL when DATA
cannot be sent solely due to connection level flow control. Instead,
we just queue DATA item to queue. We won't pop DATA item from queue
when connection level remote window size is 0. This way, we avoid the
expensive operation for all streams when WINDOW_UPDATE is arrived.
Previously we missed the case where stream->data_item is not deleted
and it caused leak. Now stream->data_item is properly deleted when
session is deleted. We decided not to delete data_item in
nghttp2_stream_free() since we need nghttp2_session to decide whether
data_item should be deleted or not there.
By default, nghttp2 library only handles HTTP/2 frames and does not
recognize first 24 bytes of client connection preface. This design
choice is done due to the fact that server may want to detect the
application protocol based on first few bytes on clear text
communication. But for simple servers which only speak HTTP/2, it is
easier for developers if nghttp2 library takes care of client
connection preface.
If this option is used with nonzero val, nghttp2 library checks first
24 bytes client connection preface. If it is not a valid one,
nghttp2_session_recv() and nghttp2_session_mem_recv() will return
error NGHTTP2_ERR_BAD_PREFACE, which is fatal error.
This commit moves frame_type parameter of
nghttp2_data_soruce_read_length_callback in front of stream_id
parameter. The motivation is that other callback is generally put
frame related parameters first. To make it consistent, we move
frame_type, which is frame ralted parameter, to the left.
Now it returns only stream's available remote window size, without
considering connection level window size. For connection-level window
size, nghttp2_session_get_remote_window_size() is added by this
commit. To get old behavior of
nghttp2_session_get_stream_remote_window_size() is use
min(nghttp2_session_get_stream_remote_window_size(),
nghttp2_session_get_remote_window_size()). The reason of this change
is that it is desirable to know just stream level window size without
taking into connection level window size. This is useful for
debugging purpose.
h2-14 now allows extensions to define new error codes. To allow
application callback to access such error codes, we uses uint32_t as
error_code type for structs and function parameters. Previously we
treated unknown error code as INTERNAL_ERROR, but this change removes
this and unknown error code is passed to application callback as is.
Motivation:
The send window size is currently fixed by a macro at compile time.
In order for users of the library to impact the send window size they
would have to change a macro at compile time. The window size may be dynamic
depending on the environment and deployment scheme. The library users
currently have no way to change this parameter.
Modifications:
Add a new optional callback method which is called before data is sent to
obtain the desired send window size. The callback return value will be
subject to a range check for the current session, stream, and settings
limits defined by flow control.
Result:
Library users have control over their send sizes.
Previously returning NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE from
on_header_callback moves input offset badly and it causes header
decompression error on the subsequent frames. This commit fix this
bug.
This commit makes handling of outgoing HEADERS and PUSH_PROMISE in the
same priority of other frames on the stream, so these frames are
processed in the order they are submitted. This allows application to
submit frames to a stream returned by nghttp2_submit_{request,
headers, push_promise} immediately. The only exception is
WINDOW_UPDATA frame, which requires nghttp2_stream object, which is
not created yet.
Reworked no automatic WINDOW_UPDATE feature. We added new API
nghttp2_session_consume() which tells the library how many bytes are
consumed by the application. Instead of submitting WINDOW_UPDATE by
the application, the library is now responsible to submit
WINDOW_UPDATE based on consumed bytes. This is more reliable method,
since it enables us to properly send WINDOW_UPDATE for stream and
connection individually. The previous implementation of nghttpx had
broken connection window management.
Previously we just assumed that if same settings ID is found in
SETTINGS, it is enough to process last seen entry. But it turns out
it is not enough for SETTINGS_HEADER_TABLE_SIZE. If we have 0 and
4096 for SETTINGS_HEADER_TABLE_SIZE in one SETTINGS, we must first
shrink dynamic table to 0 and then enlarge it to 4096. This means
that we have to remember the minimum value and last value.
Previously in inflater we reserve new ringbuffer when table size is
changed. This may be potentially a problem if new table size is very
large number. When inflater is not used directly by application, this
is not a problem because application can choose the buffer size. On
the other hand, if application uses inflater directly and it does not
have control of new buffer size (e.g., protocol dissector), then we
just fail to allocate large buffer in
nghttp2_hd_inflate_change_table_size() without actually use such huge
buffer. This change defers the actual allocation of buffer when it is
actually needed so that we will fail when it is absolutely needed.
Add last_stream_id parameter to nghttp2_submit_goaway(). To terminate
connection immediately with application chosen last stream ID,
nghttp2_session_terminate_session2() was added.
ALTSVC and BLOCKED frames are now extension frames. To add new
extension frame without modifying nghttp2_frame union, which causes so
name bump, we separated extension frames from core frames.
nghttp2_frame includes generic nghttp2_extension. The payload member
of nghttp2_extension will point to the structure of extension frame
payload. The frame types of extension frames are defined in
nghttp2_ext_frame_type.
Previously we use 2 separate buffer for each name and value. The
problem is we would waste buffer space for name because it is usually
small. Also tuning buffer size for each buffer separately is not
elegant and current HTTP server practice is that one buffer for 1
name/value pair. This commit unifies 2 buffers into 1.
We simulate resource sharing by decreasing weight. The thing is if
weight is wrapped, that item continues to send DATA until its weight
gets lowered under the other items. This commits fix this issue.
Previously stream ID was assigned just before HEADERS or PUSH_PROMISE
was serialized and nghttp2_submit_{request, headers, push_promise} did
not return stream ID. The application has to check assigned stream ID
using before_frame_send_callback. Now it is apparent that priority is
meant to DATA transfer only. Also application can reorder the
requests if it wants. Therefore we can assign stream ID in
nghttp2_submit_* functions and return stream ID from them. With this
change, now application does not have to check stream ID using
before_frame_send_callback and its code will be simplified.
We inherited gzip compression API from spdylay codebase. In spdylay,
the cost of having such API is almost free because spdylay requires
zlib for header compression. nghttp2 no longer uses gzip to header
compression. zlib dependency exists just for gzip compression API,
which is not an essential. So we decided to move gzip code to under
src and remove zlib dependency from libnghttp2 itself. As nghttp2
package, we depend on zlib to compile tools under src.
The library interface supports compressed DATA. The library does not
deflate nor inflate data payload. When sending data, an application
has to compress data and set NGHTTP2_DATA_FLAG_COMPRESSED to
data_flags parameter in nghttp2_data_source_read_callback. On
receiving, flags parameter in nghttp2_on_data_chunk_recv_callback
includes NGHTTP2_FLAG_COMPRESSED. An application should check the
flags and inflate data as necessary. Since compression context is per
frame, when DATA is seen in nghttp2_on_frame_recv_callback, an
application should reset compression context.
If stream with dpri value of no_data, we check any its descendant has
stream with dpri value of top. If so, we have to distribute of its
portion of weight to its descendants.
nghttp2_submit_{headers,request}: Return NGHTTP2_ERR_INVAILD_ARGUMENT
if pri_spec->type is invalid.
nghttp2_submit_push_promise: Return NGHTTP2_ERR_PROTO if issued by
client.
nghttp2_submit_altsvc: Return NGHTTP2_ERR_PROTO instead of
NGHTTP2_ERR_INVALID_STATE if issued by client.
Exceeding ACKed max concurrent streams results in connection error.
This change fixes the bug that num_{incoming,outgoing}_streams
is decremented wrongly if a stream is in reserved state and
RST_STREAM is send and its state is changed to NGHTTP2_STREAM_CLOSING.
This change also fixes the bug that transmission of push response
HEADERS does not increase num_outgoing_streams.
We need paddings regardless of payload and frame boundary to mitigate
certain attacks.
Since we handles CONTINUATION internally, we don't show FLAG_PAD_HIGH
and PAD_LOW flags of HEADERS in nghttp/nghttpd. We just show the
total paddings in HEADERS + CONTINUATION.
* Use 1 Huffman code table for both request and response
* Remove complicated deflater side table size management
* Add encoding context update
* Fix memory leak in inflater
Now previous padding options are removed and instead we added
select_padding_callback to select padding length for each frame
by application. If this callback is not implemented by application,
no padding is added.
This change also fixes the broken session_detect_idle_stream()
if stream_id is our side.
Previously, there is inconsistency when on_frame_recv_callback
is called between HEADERS/PUSH_PROMISE and the other frames.
For former case, it is called before header block, in latter
case, it is called after whole frame is received. To make it
consistent, we call on_frame_recv_callback for HEADERS/PUSH_PROMISE
after its frame is fully received. Since on_frame_recv_callback
can signal the end of header block, we replaced on_end_headers_callback
with on_begin_headers_callback, which is called when the reception
of the header block is started.
nghttp2_data is added to nghttp2_frame union. When DATA is
received, nghttp2_on_frame_recv_callback is called. When DATA is
sent, nghttp2_on_frame_send_callback is called.
Now incoming data is processed in very small buffer (up to 8 bytes)
using state machine. GOAWAY debug data can get to 16K - 1, and we
don't have callback for it. Since we don't want to buffer that
amount of data just for debugging, we currently discard it.
This change also makes parse_error callback not function.
It probably be removed from API.
This stream inflater can inflate incoming header block in streaming
fashion. Currently, we buffer up single name/value pair, but we chose
far more smaller buffer size than HTTP/2 frame size.
Remove the check to see that stream exists at the time when submitting
DATA, PRIORITY and WINDOW_UPDATE. We will do this check when we actually
serialize and send them off to the network (or application provided
buffer).
nghttp2 library itself now accept octet header/value pairs,
completely not restricted by HTTP/1 header name/value rule.
The applications may impose restriction about them using
validators.
Now, in nghttp2_on_frame_recv_callback, nva and nvlen in
HEADERS and PUSH_PROMISE frames are always NULL and 0 respectively.
The header name/value pairs are emitted successive
nghttp2_on_header_callback functions. The end of header fields are
signaled with nghttp2_on_end_headers_callback function.
Since NGHTTP2_ERR_PAUSE for nghttp2_on_frame_recv_callback is
introduced to handle header block, it is now deprecated.
Instead, nghttp2_on_header_callback can be paused using
NGHTTP2_ERR_PAUSE.
We tried several times about this subject, but for the current
HTTP/2.0 priority scheme, we think it is best to serve the highest
priroty streams first (interleaving streams if there are several
higest ones). There are an issue when aggregating several frontend
connections to one connection in backend, but it is HTTP/2.0
spec issue, rather than implementation.
The specification now says that index to the header table entry is
1-based. Since 0-based index is very handy to access arrays, we
internally uses 0-based index. We just convert it to 1-based when
we emit to the block and convert 1-based to 0-based on decoding.
Remove sorting headers from library code. The application must sort
them if necessary. nghttpx and nghttpd do the sorting of the headers
in stable way if names are equal.
nghttp2_session_client_new2 and nghttp2_session_server_new2 take
additional parameters which specifies session options.
nghttp2_set_option is somewhat crumsy because of type checking.
Now we use nghttp2_opt_set, which specifies individual options with
types. We changed the value of nghttp2_opt, so this change will
require re-compile.
It is not clear that SETTINGS_ENABLE_PUSH = 0 disallows HEADERS
to the reserved streams. For now, we just check the reception
and transmission of PUSH_PROMISE against SETTINGS_ENABLE_PUSH.
The NGHTTP2_ERR_PAUSE library error code is introduced to pause
the execution of nghttp2_session_mem_recv() when that error code
is returned from nghttp2_on_frame_recv_callback or
nghttp2_on_data_chunk_recv_callback. If this happens, the parameters
available for both callbacks are retained until the application
calls nghttp2_session_continue(). The application must retain
input bytes which was used to produce the frame.
After successful call of nghttp2_session_continue, the application
can continue to call nghttp2_session_mem_recv() to process
additional data.
This function is similar to nghttp2_submit_request and the
difference is it takes an array of nghttp2_nv as name/value pairs.
It is useful if name/value pairs is not NULL-terminated in the
application code.
Tatsuhiro Tsujikawa
Alexis La Goutte
Scott Mitchell
Daniel Stenberg