To avoid buffer copy in nghttp2_data_source_read_callback, this commit
introduces NGHTTP2_DATA_FLAG_NO_COPY and nghttp2_send_data_callback.
By using NGHTTP2_DATA_FLAG_NO_COPY in
nghttp2_data_source_read_callback, application can avoid to copy
application data to given buffer. Instead, application has to
implement nghttp2_send_data_callback to send complete DATA frame by
itself. We see noticeable performance increase in nghttpd and
tiny-nghttpd using this new feature. On the other hand, nghttpx does
not show such difference, probably because buffer copy is not
bottleneck. Using nghttp2_send_data_callback adds complexity, so it
is recommended to measure the performance to see whether this extra
complexity worth it.
Previously API reference is gigantic one rst file and it is a bit hard
to use, especially when browsing similar functions. This commit
splits API reference into smaller fine grained files. The macros,
enums, types are now in its own file. Each API function has its own
file now. API reference doc is now index to above documentation
files. The apiref-header.rst is renamed as programmers-guide.rst and
becomes standalone document.
nghttp2_submit_request and nghttp2_submit_response will set
NGHTTP2_FLAG_END_STREAM after all given data is sent (data could be
0). This means we have no way to send trailers. In this commit, we
added NGHTTP2_DATA_FLAG_NO_END_STREAM flag. The application can set
this flag in *data_flags inside nghttp2_data_source_read_callback. If
NGHTTP2_DATA_FLAG_EOF is set, library automatically set
NGHTTP2_FLAG_END_STREAM. But if both NGHTTP2_DATA_FLAG_EOF and
NGHTTP2_DATA_FLAG_NO_END_STREAM are set, NGHTTP2_FLAG_END_STREAM will
not set by library. Then application can use new
nghttp2_submit_trailer() to send trailers. nghttp2_submit_trailer()
will set NGHTTP2_FLAG_END_STREAM and it is actually thing wrapper of
nghttp2_submit_headers().
This API function with nonzero |val| parameter disables HTTP Messaging
validation in nghttp2 library, so that application can use nghttp2
library for non-HTTP use.
nghttp2_submit_shutdown_notice() is used to notify the client that
graceful shutdown is started. We expect that after this call, the
server application should send another GOAWAY using
nghttp2_submit_goaway() with appropriate last_stream_id. In this
commit, we also added nghttp2_session_get_last_proc_stream_id(), which
can be used as last_stream_id parameter.
This commit implements graceful shutdown in nghttpx. The integration
test for graceful shutdown is also added.
This change makes sure that GOAWAY which terminates session is
immediately sent without blocked by other frames.
NGHTTP2_ERR_SESSION_CLOSING library error code was added to indicate
this situation to callback.
nghttp2_mem structure is introduced to hold custom memory allocator
functions and user supplied pointer. nghttp2_mem object can be passed
to nghttp2_session_client_new3(), nghttp2_session_server_new3(),
nghttp2_hd_deflate_new2() and nghttp2_hd_inflate_new2() to replace
standard malloc(), free(), calloc() and realloc(). nghttp2_mem
structure has user supplied pointer mem_user_data which can be used as
per session/object memory pool.
This change will utilize last_stream_id in GOAWAY extensively. When
GOAWAY is received with a last_stream_id, library closes all outgoing
streams whose stream_id > received last_stream_id.
nghttp2_on_stream_callback is called for each stream to be closed.
When GOAWAY is sent with a last_stream_id, library closes all incoming
streams whose stream_id > sent last_stream_id.
nghttp2_on_stream_callback is called for each stream to be closed.
* Add NGHTTP2_HTTP_1_1_REQUIRED error code
* Allow transmission of WINDOW_UPDATE on reserved (remote)
* Allow reception of WINDOW_UPDATE on reserved (local)
* Treat frame larger than MAX_FRAME_SIZE as FRAME_SIZE_ERROR
ALPN identifier is still h2-14 to continue interop, since draft-14 and
-15 are binary compatible. The new error code was added in draft-15,
but HTTP/2 allows extensions can freely add new error code, so it is
not a problem.
Previously when nghttp2_stream_resume_deferred_data() is called,
deferred flags in stream->flags are all cleared. This is not ideal
because if application returned NGHTTP2_ERR_DEFERRED, and also that
stream is deferred by flow control, then all flags are cleared and
read callback will be invoked again. This commit fixes this issue.
This changes error condition of nghttp2_session_resume_data().
Previously we return error if stream was deferred by flow control.
Now we don't return error in this case. We just clear
NGHTTP2_FLAG_DEFERRED_USER and if still
NGHTTP2_FLAG_DEFERRED_FLOW_CONTROL is set, just return 0.
By default, nghttp2 library only handles HTTP/2 frames and does not
recognize first 24 bytes of client connection preface. This design
choice is done due to the fact that server may want to detect the
application protocol based on first few bytes on clear text
communication. But for simple servers which only speak HTTP/2, it is
easier for developers if nghttp2 library takes care of client
connection preface.
If this option is used with nonzero val, nghttp2 library checks first
24 bytes client connection preface. If it is not a valid one,
nghttp2_session_recv() and nghttp2_session_mem_recv() will return
error NGHTTP2_ERR_BAD_PREFACE, which is fatal error.
This commit moves frame_type parameter of
nghttp2_data_soruce_read_length_callback in front of stream_id
parameter. The motivation is that other callback is generally put
frame related parameters first. To make it consistent, we move
frame_type, which is frame ralted parameter, to the left.
Now it returns only stream's available remote window size, without
considering connection level window size. For connection-level window
size, nghttp2_session_get_remote_window_size() is added by this
commit. To get old behavior of
nghttp2_session_get_stream_remote_window_size() is use
min(nghttp2_session_get_stream_remote_window_size(),
nghttp2_session_get_remote_window_size()). The reason of this change
is that it is desirable to know just stream level window size without
taking into connection level window size. This is useful for
debugging purpose.
It is not used by library for a while. It could be used to pass
unsupported extension frames to application, but its interface
requires library to buffer entire frame, which we'd like to avoid.
For unsupported extension frames, we will add new callbacks which does
not require buffering if they are required.
h2-14 now allows extensions to define new error codes. To allow
application callback to access such error codes, we uses uint32_t as
error_code type for structs and function parameters. Previously we
treated unknown error code as INTERNAL_ERROR, but this change removes
this and unknown error code is passed to application callback as is.
To make it possible to add new callbacks without bumping so name, we
decided to hide details of nghttp2_session_callbacks. We provide
setter like functions to set individual callback function.
Motivation:
The send window size is currently fixed by a macro at compile time.
In order for users of the library to impact the send window size they
would have to change a macro at compile time. The window size may be dynamic
depending on the environment and deployment scheme. The library users
currently have no way to change this parameter.
Modifications:
Add a new optional callback method which is called before data is sent to
obtain the desired send window size. The callback return value will be
subject to a range check for the current session, stream, and settings
limits defined by flow control.
Result:
Library users have control over their send sizes.
This is partial revert of bbe4f5a3d1.
Only documentation is reverted. Since we have 2 queues to handle
maximum concurrent streams, we are not ready to allow immediate frame
submission for pending new frames.
This commit makes handling of outgoing HEADERS and PUSH_PROMISE in the
same priority of other frames on the stream, so these frames are
processed in the order they are submitted. This allows application to
submit frames to a stream returned by nghttp2_submit_{request,
headers, push_promise} immediately. The only exception is
WINDOW_UPDATA frame, which requires nghttp2_stream object, which is
not created yet.
Reworked no automatic WINDOW_UPDATE feature. We added new API
nghttp2_session_consume() which tells the library how many bytes are
consumed by the application. Instead of submitting WINDOW_UPDATE by
the application, the library is now responsible to submit
WINDOW_UPDATE based on consumed bytes. This is more reliable method,
since it enables us to properly send WINDOW_UPDATE for stream and
connection individually. The previous implementation of nghttpx had
broken connection window management.
Previously we just assumed that if same settings ID is found in
SETTINGS, it is enough to process last seen entry. But it turns out
it is not enough for SETTINGS_HEADER_TABLE_SIZE. If we have 0 and
4096 for SETTINGS_HEADER_TABLE_SIZE in one SETTINGS, we must first
shrink dynamic table to 0 and then enlarge it to 4096. This means
that we have to remember the minimum value and last value.
Add last_stream_id parameter to nghttp2_submit_goaway(). To terminate
connection immediately with application chosen last stream ID,
nghttp2_session_terminate_session2() was added.
ALTSVC and BLOCKED frames are now extension frames. To add new
extension frame without modifying nghttp2_frame union, which causes so
name bump, we separated extension frames from core frames.
nghttp2_frame includes generic nghttp2_extension. The payload member
of nghttp2_extension will point to the structure of extension frame
payload. The frame types of extension frames are defined in
nghttp2_ext_frame_type.
It is generally useful to know what is the cause of the error. Since
we expose HPACK API, it is friendly to tell application the
insufficient buffer size is a culprit.
Use this macro in Python sources. Python module constant
HD_DEFLATE_HD_TABLE_BUFSIZE_MAX was renamed as
DEFLATE_MAX_HEADER_TABLE_SIZE since the previous name was awkward.
Previously stream ID was assigned just before HEADERS or PUSH_PROMISE
was serialized and nghttp2_submit_{request, headers, push_promise} did
not return stream ID. The application has to check assigned stream ID
using before_frame_send_callback. Now it is apparent that priority is
meant to DATA transfer only. Also application can reorder the
requests if it wants. Therefore we can assign stream ID in
nghttp2_submit_* functions and return stream ID from them. With this
change, now application does not have to check stream ID using
before_frame_send_callback and its code will be simplified.
We inherited gzip compression API from spdylay codebase. In spdylay,
the cost of having such API is almost free because spdylay requires
zlib for header compression. nghttp2 no longer uses gzip to header
compression. zlib dependency exists just for gzip compression API,
which is not an essential. So we decided to move gzip code to under
src and remove zlib dependency from libnghttp2 itself. As nghttp2
package, we depend on zlib to compile tools under src.
Currently, nghttpd server only compresses files whose extensions are
one of .html, .js, .css and .txt. nghttp advertises its support of
per-frame compression in SETTINGS frame. To implement this feature,
we added 2 public API: nghttp2_session_get_remote_settings() and
nghttp2_gzip_inflate_finished().
Callback function invoked to adjust priority value for request
HEADERS.
Since the application doesn’t know stream ID when it submits
requests, it may not be able to add correct priority value to HEADERS
frame and forced to use follwing PRIORITY frame. The purpose of this
callback is give the chance to the application to adjust priority
value with the latest information it has just before transmission so
that correct priority is included in HEADERS frame and it doesn’t
have to send additional PRIORITY frame.
The library interface supports compressed DATA. The library does not
deflate nor inflate data payload. When sending data, an application
has to compress data and set NGHTTP2_DATA_FLAG_COMPRESSED to
data_flags parameter in nghttp2_data_source_read_callback. On
receiving, flags parameter in nghttp2_on_data_chunk_recv_callback
includes NGHTTP2_FLAG_COMPRESSED. An application should check the
flags and inflate data as necessary. Since compression context is per
frame, when DATA is seen in nghttp2_on_frame_recv_callback, an
application should reset compression context.
Tatsuhiro Tsujikawa
Daniel Stenberg
Scott Mitchell