This commit makes handling of outgoing HEADERS and PUSH_PROMISE in the
same priority of other frames on the stream, so these frames are
processed in the order they are submitted. This allows application to
submit frames to a stream returned by nghttp2_submit_{request,
headers, push_promise} immediately. The only exception is
WINDOW_UPDATA frame, which requires nghttp2_stream object, which is
not created yet.
The application should be responsible for the size of incoming header
block size. Framing layer just passes everything (we have size limit
for one header/field though) to application.
Reworked no automatic WINDOW_UPDATE feature. We added new API
nghttp2_session_consume() which tells the library how many bytes are
consumed by the application. Instead of submitting WINDOW_UPDATE by
the application, the library is now responsible to submit
WINDOW_UPDATE based on consumed bytes. This is more reliable method,
since it enables us to properly send WINDOW_UPDATE for stream and
connection individually. The previous implementation of nghttpx had
broken connection window management.
Previously we just assumed that if same settings ID is found in
SETTINGS, it is enough to process last seen entry. But it turns out
it is not enough for SETTINGS_HEADER_TABLE_SIZE. If we have 0 and
4096 for SETTINGS_HEADER_TABLE_SIZE in one SETTINGS, we must first
shrink dynamic table to 0 and then enlarge it to 4096. This means
that we have to remember the minimum value and last value.
Since we do not call on_data_chunk_recv_callback for ignored DATA
chunk, if nghttp2_option_set_no_auto_connection_window_update is used,
application may not have a chance to send connection WINDOW_UPDATE.
To fix this, we accumulate those received bytes, and if it exceeds
certain number, we automatically send connection-level WINDOW_UPDATE.
This commits changes the upper bound of one header field size (the sum
of the length of name and value) to 64KiB by default. We may add an
option to change this upper bound in the future.
Add last_stream_id parameter to nghttp2_submit_goaway(). To terminate
connection immediately with application chosen last stream ID,
nghttp2_session_terminate_session2() was added.
ALTSVC and BLOCKED frames are now extension frames. To add new
extension frame without modifying nghttp2_frame union, which causes so
name bump, we separated extension frames from core frames.
nghttp2_frame includes generic nghttp2_extension. The payload member
of nghttp2_extension will point to the structure of extension frame
payload. The frame types of extension frames are defined in
nghttp2_ext_frame_type.
pending_local_max_concurrent_stream is, once local settings applied,
becomes NGHTTP2_INITIAL_MAX_CONCURRENT_STREAMS, which is very large
number. When adjusting number of streams, we have to take min of
local effective SETTINGS_MAX_CONCURRENT_STREAMS and pending one.
We simulate resource sharing by decreasing weight. The thing is if
weight is wrapped, that item continues to send DATA until its weight
gets lowered under the other items. This commits fix this issue.
Previously stream ID was assigned just before HEADERS or PUSH_PROMISE
was serialized and nghttp2_submit_{request, headers, push_promise} did
not return stream ID. The application has to check assigned stream ID
using before_frame_send_callback. Now it is apparent that priority is
meant to DATA transfer only. Also application can reorder the
requests if it wants. Therefore we can assign stream ID in
nghttp2_submit_* functions and return stream ID from them. With this
change, now application does not have to check stream ID using
before_frame_send_callback and its code will be simplified.
Currently, nghttpd server only compresses files whose extensions are
one of .html, .js, .css and .txt. nghttp advertises its support of
per-frame compression in SETTINGS frame. To implement this feature,
we added 2 public API: nghttp2_session_get_remote_settings() and
nghttp2_gzip_inflate_finished().
Callback function invoked to adjust priority value for request
HEADERS.
Since the application doesn’t know stream ID when it submits
requests, it may not be able to add correct priority value to HEADERS
frame and forced to use follwing PRIORITY frame. The purpose of this
callback is give the chance to the application to adjust priority
value with the latest information it has just before transmission so
that correct priority is included in HEADERS frame and it doesn’t
have to send additional PRIORITY frame.
The library interface supports compressed DATA. The library does not
deflate nor inflate data payload. When sending data, an application
has to compress data and set NGHTTP2_DATA_FLAG_COMPRESSED to
data_flags parameter in nghttp2_data_source_read_callback. On
receiving, flags parameter in nghttp2_on_data_chunk_recv_callback
includes NGHTTP2_FLAG_COMPRESSED. An application should check the
flags and inflate data as necessary. Since compression context is per
frame, when DATA is seen in nghttp2_on_frame_recv_callback, an
application should reset compression context.
To make adding new option easier, we decided to make the details of
option struct private and hide it from public API. We provide
functions to set individual option value.
Exceeding ACKed max concurrent streams results in connection error.
This change fixes the bug that num_{incoming,outgoing}_streams
is decremented wrongly if a stream is in reserved state and
RST_STREAM is send and its state is changed to NGHTTP2_STREAM_CLOSING.
This change also fixes the bug that transmission of push response
HEADERS does not increase num_outgoing_streams.
This function behaves like nghttp2_session_send(), but it does not
use nghttp2_send_callback to send data. Instead, it returns the
serialized data to trasmit and its length to the caller.
We need paddings regardless of payload and frame boundary to mitigate
certain attacks.
Since we handles CONTINUATION internally, we don't show FLAG_PAD_HIGH
and PAD_LOW flags of HEADERS in nghttp/nghttpd. We just show the
total paddings in HEADERS + CONTINUATION.
* Use 1 Huffman code table for both request and response
* Remove complicated deflater side table size management
* Add encoding context update
* Fix memory leak in inflater
Now previous padding options are removed and instead we added
select_padding_callback to select padding length for each frame
by application. If this callback is not implemented by application,
no padding is added.
This change also fixes the broken session_detect_idle_stream()
if stream_id is our side.
Previously, there is inconsistency when on_frame_recv_callback
is called between HEADERS/PUSH_PROMISE and the other frames.
For former case, it is called before header block, in latter
case, it is called after whole frame is received. To make it
consistent, we call on_frame_recv_callback for HEADERS/PUSH_PROMISE
after its frame is fully received. Since on_frame_recv_callback
can signal the end of header block, we replaced on_end_headers_callback
with on_begin_headers_callback, which is called when the reception
of the header block is started.
nghttp2_data is added to nghttp2_frame union. When DATA is
received, nghttp2_on_frame_recv_callback is called. When DATA is
sent, nghttp2_on_frame_send_callback is called.
Now incoming data is processed in very small buffer (up to 8 bytes)
using state machine. GOAWAY debug data can get to 16K - 1, and we
don't have callback for it. Since we don't want to buffer that
amount of data just for debugging, we currently discard it.
This change also makes parse_error callback not function.
It probably be removed from API.
This stream inflater can inflate incoming header block in streaming
fashion. Currently, we buffer up single name/value pair, but we chose
far more smaller buffer size than HTTP/2 frame size.
Now, in nghttp2_on_frame_recv_callback, nva and nvlen in
HEADERS and PUSH_PROMISE frames are always NULL and 0 respectively.
The header name/value pairs are emitted successive
nghttp2_on_header_callback functions. The end of header fields are
signaled with nghttp2_on_end_headers_callback function.
Since NGHTTP2_ERR_PAUSE for nghttp2_on_frame_recv_callback is
introduced to handle header block, it is now deprecated.
Instead, nghttp2_on_header_callback can be paused using
NGHTTP2_ERR_PAUSE.
We tried several times about this subject, but for the current
HTTP/2.0 priority scheme, we think it is best to serve the highest
priroty streams first (interleaving streams if there are several
higest ones). There are an issue when aggregating several frontend
connections to one connection in backend, but it is HTTP/2.0
spec issue, rather than implementation.
Previously we use largest stream ID received so far as last-stream-ID,
and it is irrevant that it is passed to the callback (thus upper layer).
Now the stream ID which is passed to callback is eligible to
last-stream-ID.
nghttp2_session_client_new2 and nghttp2_session_server_new2 take
additional parameters which specifies session options.
nghttp2_set_option is somewhat crumsy because of type checking.
Now we use nghttp2_opt_set, which specifies individual options with
types. We changed the value of nghttp2_opt, so this change will
require re-compile.
It is not clear that SETTINGS_ENABLE_PUSH = 0 disallows HEADERS
to the reserved streams. For now, we just check the reception
and transmission of PUSH_PROMISE against SETTINGS_ENABLE_PUSH.
Now we have SETTINGS synchronization, flow control error can be
detected strictly. If DATA frame is received with length > 0 and
current received window size is equal to or larger than local
window size (latter happens when we shirnk window size), it is
subject to FLOW_CONTROL_ERROR,
Use NGHTTP2_ERR_INVALID_STREAM_STATE when trying to send DATA or
WINDOW_UPDATE to reserved stream.
Also a bit refactored nghttp2_session_predicate_window_update_send
and nghttp2_session_predicate_data_send
The NGHTTP2_ERR_PAUSE library error code is introduced to pause
the execution of nghttp2_session_mem_recv() when that error code
is returned from nghttp2_on_frame_recv_callback or
nghttp2_on_data_chunk_recv_callback. If this happens, the parameters
available for both callbacks are retained until the application
calls nghttp2_session_continue(). The application must retain
input bytes which was used to produce the frame.
After successful call of nghttp2_session_continue, the application
can continue to call nghttp2_session_mem_recv() to process
additional data.
We call on_stream_close_callback even if stream->state is
NGHTTP2_STREAM_INITIAL. This will happen while sending request
HEADERS, a local endpoint receives RST_STREAM for that
stream. It may be PROTOCOL_ERROR, but without notifying stream
closure will hang the stream in a local endpoint.
Following cases are now treated as connection error of type
PROTOCOL_ERROR.
* Receiving HEADERS from client in reserved (local)
* Receiving PUSH_PROMISE against peer-initiated stream
* Receiving WINDOW_UPDATE against in reserved (local, remote)
Now NGHTTP2_OPT_NO_AUTO_WINDOW_UPDATE is split into 2 options:
NGHTTP2_OPT_NO_AUTO_STREAM_WINDOW_UPDATE and
NGHTTP2_OPT_NO_AUTO_CONNECTION_WINDOW_UPDATE.
This is preparation for the upcoming removal of END_FLOW_CONTROL
flag. For nghttpx, instead of using END_FLOW_CONTROL to disable
connection-level flow control, increase window size by large
enough value, which is friendly way to current chromium
implementation.
Fix the bug that connection-level local window is not updated
for the data is the last part of the stream. For the stream
level window may ignore this, connection-level window must
be updated. Also this change fixes the bug that connection-level
window is not updated for the ignored DATA frames.
Tatsuhiro Tsujikawa
Alexis La Goutte