Motivation:
The send window size is currently fixed by a macro at compile time.
In order for users of the library to impact the send window size they
would have to change a macro at compile time. The window size may be dynamic
depending on the environment and deployment scheme. The library users
currently have no way to change this parameter.
Modifications:
Add a new optional callback method which is called before data is sent to
obtain the desired send window size. The callback return value will be
subject to a range check for the current session, stream, and settings
limits defined by flow control.
Result:
Library users have control over their send sizes.
Previously returning NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE from
on_header_callback moves input offset badly and it causes header
decompression error on the subsequent frames. This commit fix this
bug.
This is partial revert of bbe4f5a3d1.
Only documentation is reverted. Since we have 2 queues to handle
maximum concurrent streams, we are not ready to allow immediate frame
submission for pending new frames.
This commit makes handling of outgoing HEADERS and PUSH_PROMISE in the
same priority of other frames on the stream, so these frames are
processed in the order they are submitted. This allows application to
submit frames to a stream returned by nghttp2_submit_{request,
headers, push_promise} immediately. The only exception is
WINDOW_UPDATA frame, which requires nghttp2_stream object, which is
not created yet.
The application should be responsible for the size of incoming header
block size. Framing layer just passes everything (we have size limit
for one header/field though) to application.
Reworked no automatic WINDOW_UPDATE feature. We added new API
nghttp2_session_consume() which tells the library how many bytes are
consumed by the application. Instead of submitting WINDOW_UPDATE by
the application, the library is now responsible to submit
WINDOW_UPDATE based on consumed bytes. This is more reliable method,
since it enables us to properly send WINDOW_UPDATE for stream and
connection individually. The previous implementation of nghttpx had
broken connection window management.
Previously we just assumed that if same settings ID is found in
SETTINGS, it is enough to process last seen entry. But it turns out
it is not enough for SETTINGS_HEADER_TABLE_SIZE. If we have 0 and
4096 for SETTINGS_HEADER_TABLE_SIZE in one SETTINGS, we must first
shrink dynamic table to 0 and then enlarge it to 4096. This means
that we have to remember the minimum value and last value.
Since we do not call on_data_chunk_recv_callback for ignored DATA
chunk, if nghttp2_option_set_no_auto_connection_window_update is used,
application may not have a chance to send connection WINDOW_UPDATE.
To fix this, we accumulate those received bytes, and if it exceeds
certain number, we automatically send connection-level WINDOW_UPDATE.
This commits changes the upper bound of one header field size (the sum
of the length of name and value) to 64KiB by default. We may add an
option to change this upper bound in the future.
Previously in inflater we reserve new ringbuffer when table size is
changed. This may be potentially a problem if new table size is very
large number. When inflater is not used directly by application, this
is not a problem because application can choose the buffer size. On
the other hand, if application uses inflater directly and it does not
have control of new buffer size (e.g., protocol dissector), then we
just fail to allocate large buffer in
nghttp2_hd_inflate_change_table_size() without actually use such huge
buffer. This change defers the actual allocation of buffer when it is
actually needed so that we will fail when it is absolutely needed.
Add last_stream_id parameter to nghttp2_submit_goaway(). To terminate
connection immediately with application chosen last stream ID,
nghttp2_session_terminate_session2() was added.
ALTSVC and BLOCKED frames are now extension frames. To add new
extension frame without modifying nghttp2_frame union, which causes so
name bump, we separated extension frames from core frames.
nghttp2_frame includes generic nghttp2_extension. The payload member
of nghttp2_extension will point to the structure of extension frame
payload. The frame types of extension frames are defined in
nghttp2_ext_frame_type.
Previously we have uint16_t as state member variable in
nghttp2_huff_decode structure to express -1 as failure. This is
because we have 256 valid states. However, we can express failed
state using flags member variable and make state uint8_t. This commit
does this and as a result the size of decoding table is reduced.
Previously we use 2 separate buffer for each name and value. The
problem is we would waste buffer space for name because it is usually
small. Also tuning buffer size for each buffer separately is not
elegant and current HTTP server practice is that one buffer for 1
name/value pair. This commit unifies 2 buffers into 1.
pending_local_max_concurrent_stream is, once local settings applied,
becomes NGHTTP2_INITIAL_MAX_CONCURRENT_STREAMS, which is very large
number. When adjusting number of streams, we have to take min of
local effective SETTINGS_MAX_CONCURRENT_STREAMS and pending one.
It is generally useful to know what is the cause of the error. Since
we expose HPACK API, it is friendly to tell application the
insufficient buffer size is a culprit.
Use this macro in Python sources. Python module constant
HD_DEFLATE_HD_TABLE_BUFSIZE_MAX was renamed as
DEFLATE_MAX_HEADER_TABLE_SIZE since the previous name was awkward.
We simulate resource sharing by decreasing weight. The thing is if
weight is wrapped, that item continues to send DATA until its weight
gets lowered under the other items. This commits fix this issue.
Previously stream ID was assigned just before HEADERS or PUSH_PROMISE
was serialized and nghttp2_submit_{request, headers, push_promise} did
not return stream ID. The application has to check assigned stream ID
using before_frame_send_callback. Now it is apparent that priority is
meant to DATA transfer only. Also application can reorder the
requests if it wants. Therefore we can assign stream ID in
nghttp2_submit_* functions and return stream ID from them. With this
change, now application does not have to check stream ID using
before_frame_send_callback and its code will be simplified.
We inherited gzip compression API from spdylay codebase. In spdylay,
the cost of having such API is almost free because spdylay requires
zlib for header compression. nghttp2 no longer uses gzip to header
compression. zlib dependency exists just for gzip compression API,
which is not an essential. So we decided to move gzip code to under
src and remove zlib dependency from libnghttp2 itself. As nghttp2
package, we depend on zlib to compile tools under src.
Currently, nghttpd server only compresses files whose extensions are
one of .html, .js, .css and .txt. nghttp advertises its support of
per-frame compression in SETTINGS frame. To implement this feature,
we added 2 public API: nghttp2_session_get_remote_settings() and
nghttp2_gzip_inflate_finished().
Callback function invoked to adjust priority value for request
HEADERS.
Since the application doesn’t know stream ID when it submits
requests, it may not be able to add correct priority value to HEADERS
frame and forced to use follwing PRIORITY frame. The purpose of this
callback is give the chance to the application to adjust priority
value with the latest information it has just before transmission so
that correct priority is included in HEADERS frame and it doesn’t
have to send additional PRIORITY frame.
The library interface supports compressed DATA. The library does not
deflate nor inflate data payload. When sending data, an application
has to compress data and set NGHTTP2_DATA_FLAG_COMPRESSED to
data_flags parameter in nghttp2_data_source_read_callback. On
receiving, flags parameter in nghttp2_on_data_chunk_recv_callback
includes NGHTTP2_FLAG_COMPRESSED. An application should check the
flags and inflate data as necessary. Since compression context is per
frame, when DATA is seen in nghttp2_on_frame_recv_callback, an
application should reset compression context.
If stream with dpri value of no_data, we check any its descendant has
stream with dpri value of top. If so, we have to distribute of its
portion of weight to its descendants.
nghttp2_submit_{headers,request}: Return NGHTTP2_ERR_INVAILD_ARGUMENT
if pri_spec->type is invalid.
nghttp2_submit_push_promise: Return NGHTTP2_ERR_PROTO if issued by
client.
nghttp2_submit_altsvc: Return NGHTTP2_ERR_PROTO instead of
NGHTTP2_ERR_INVALID_STATE if issued by client.
To make adding new option easier, we decided to make the details of
option struct private and hide it from public API. We provide
functions to set individual option value.
Tatsuhiro Tsujikawa
Scott Mitchell
Alexis La Goutte