To avoid buffer copy in nghttp2_data_source_read_callback, this commit
introduces NGHTTP2_DATA_FLAG_NO_COPY and nghttp2_send_data_callback.
By using NGHTTP2_DATA_FLAG_NO_COPY in
nghttp2_data_source_read_callback, application can avoid to copy
application data to given buffer. Instead, application has to
implement nghttp2_send_data_callback to send complete DATA frame by
itself. We see noticeable performance increase in nghttpd and
tiny-nghttpd using this new feature. On the other hand, nghttpx does
not show such difference, probably because buffer copy is not
bottleneck. Using nghttp2_send_data_callback adds complexity, so it
is recommended to measure the performance to see whether this extra
complexity worth it.
nghttp2_submit_request and nghttp2_submit_response will set
NGHTTP2_FLAG_END_STREAM after all given data is sent (data could be
0). This means we have no way to send trailers. In this commit, we
added NGHTTP2_DATA_FLAG_NO_END_STREAM flag. The application can set
this flag in *data_flags inside nghttp2_data_source_read_callback. If
NGHTTP2_DATA_FLAG_EOF is set, library automatically set
NGHTTP2_FLAG_END_STREAM. But if both NGHTTP2_DATA_FLAG_EOF and
NGHTTP2_DATA_FLAG_NO_END_STREAM are set, NGHTTP2_FLAG_END_STREAM will
not set by library. Then application can use new
nghttp2_submit_trailer() to send trailers. nghttp2_submit_trailer()
will set NGHTTP2_FLAG_END_STREAM and it is actually thing wrapper of
nghttp2_submit_headers().
After we sent SETTINGS including ENABLE_PUSH = 0, peer may already
issue PUSH_PROMISE before receiving our SETTINGS and react it to
SETTINGS ACK. Previously we accept this PUSH_PROMISE. In this
commit, we check the pending ENABLE_PUSH value and if it means
disabling push, we refuse PUSH_PROMISE with RST_STREAM of error
REFUSED_STREAM.
This commit only affects the library behaviour unless
nghttp2_option_set_no_http_messaging() is used.
We like strict validation against header field name and value against
RFC 7230, but we have already so much web sites and libraries in
public internet which do not obey these rules. Simply just
terminating stream because of this may break web sites and it is too
disruptive. So we decided that we should be conservative here so
those header fields containing illegal characters are just ignored.
But we are conservative only for regular headers. We are strict for
pseudo headers since it is new to HTTP/2 and new implementations
should know the rules better.
This API function with nonzero |val| parameter disables HTTP Messaging
validation in nghttp2 library, so that application can use nghttp2
library for non-HTTP use.
Previously we did not check HTTP semantics and it is left out for
application. Although checking is relatively easy, but they are
scattered and error prone. We have implemented these checks in our
applications and also feel they are tedious. To make application
development a bit easier, this commit adds basic HTTP semantics
validation to library code. We do following checks:
server:
* HEADERS is either request header or trailer header. Other type of
header is disallowed.
client:
* HEADERS is either zero or more non-final response header or final
response header or trailer header. Other type of header is
disallowed.
For both:
* Check mandatory pseudo header fields.
* Make sure that content-length matches the amount of DATA we
received.
If validation fails, RST_STREAM of type PROTOCOL_ERROR is issued.
Previously we did not handle the situation where RST_STREAM is
submitted against a stream while requet HEADERS which opens that
stream is still in queue. Due to max concurrent streams limit,
RST_STREAM is sent first, and then request HEADERS, which effectively
voids RST_STREAM.
In this commit, we checks RST_STREAM against currently pending request
HEADERS in queue and if stream ID matches, we mark that HEADERS as
canceled and RST_STREAM is not sent in this case. The library will
call on_frame_not_sent_callback for the canceled HEADERS with error
code from RST_STREAM.
Previously we treat stream in NGHTTP2_STREAM_RESERVED state specially,
that is we don't increment or decrement streams counts if stream is in
that state. Because of this, we don't change the stream state to
NGHTTP2_STREAM_CLOSING if stream is in NGHTTP2_STREAM_RESERVED. But
it turns out that it causes a problem. If client canceled pushed
stream before push response HEADERS, stream is still in
NGHTTP2_STREAM_RESERVED state. If push response HEADERS arrived in
this state, library happily accepts it and passed to application.
With this commit, this bug was corrected. We now change stream state
to NGHTTP2_STREAM_CLOSING even if it was in NGHTTP2_STREAM_RESERVED
state. We now use NGHTTP2_STREAM_FLAG_PUSH to determine whether we
have to increase/decrase stream count.
nghttp2_submit_shutdown_notice() is used to notify the client that
graceful shutdown is started. We expect that after this call, the
server application should send another GOAWAY using
nghttp2_submit_goaway() with appropriate last_stream_id. In this
commit, we also added nghttp2_session_get_last_proc_stream_id(), which
can be used as last_stream_id parameter.
This commit implements graceful shutdown in nghttpx. The integration
test for graceful shutdown is also added.
This change makes sure that GOAWAY which terminates session is
immediately sent without blocked by other frames.
NGHTTP2_ERR_SESSION_CLOSING library error code was added to indicate
this situation to callback.
Initially, we use nghttp2_stream.data_item to refer only item with
DATA frame. But recently we use it to refer HEADERS frame as well.
So it is better to call just item rather than data_item. This applies
to all related functions.
Previously session_after_frame_sent is called after we detected all
data is sent. In nghttp2_session_mem_send, we only detect it in the
next call of the function. It means that if a frame data bearing
END_STREAM is on flight to the peer as a result of
nghttp2_session_mem_send, peer may get that data and knows the stream
closure and issues new stream. We may receive this new stream before
the next nghttp2_session_mem_send call, which means that we may
incorrectly assumes that peer violates maximum concurrent stream
limit. To fix this issue, we separate session_after_frame_sent into 2
functions: session_after_frame_sent1 and session_after_frame_sent2.
session_after_frame_sent1 handles on_frame_send_callback and stream
closure and we call this early in nghttp2_session_mem_send. This
makes number of streams are synchronized correctly with peer.
Previously we handle idle streams as closed streams. We only keeps
sum of closed streams and active streams under max concurrent streams
limit, idle streams gets deleted earlier than client expects.
In this change, idle streams are kept in separate list and not handled
as closed streams. To mitigate possible attack vector to make
unlimited idle streams, we cap the number of idle streams in a half of
max concurrent streams. This is arbitrary choice. It may be adjusted
in the future when we have interop experience.
nghttp2_mem structure is introduced to hold custom memory allocator
functions and user supplied pointer. nghttp2_mem object can be passed
to nghttp2_session_client_new3(), nghttp2_session_server_new3(),
nghttp2_hd_deflate_new2() and nghttp2_hd_inflate_new2() to replace
standard malloc(), free(), calloc() and realloc(). nghttp2_mem
structure has user supplied pointer mem_user_data which can be used as
per session/object memory pool.
This change will utilize last_stream_id in GOAWAY extensively. When
GOAWAY is received with a last_stream_id, library closes all outgoing
streams whose stream_id > received last_stream_id.
nghttp2_on_stream_callback is called for each stream to be closed.
When GOAWAY is sent with a last_stream_id, library closes all incoming
streams whose stream_id > sent last_stream_id.
nghttp2_on_stream_callback is called for each stream to be closed.
If stream ID is not idle, it might be valid HEADERS. If stream ID is
idle, it is invalid regardless stream ID is even or odd, since client
is not expected to recieve request from server. nghttp2 library
historically allows this, but now we forbids this.
We make following HEADERS under priority control:
* push response HEADERS
* HEADERS submitted by nghttp2_submit_response
Currently, HEADERS submitted by nghttp2_submit_headers is not attached
to stream. This is because it may be used as non-final response
header and application may submit final response using
nghttp2_submit_response without checking non-final response header
transmission.
Allowing PRIORITY frame at anytime so that PRIORITY frame to idle
stream can create anchor node in dependency tree. In this change, we
open stream with new NGHTTP2_STREAM_IDLE state, which is linked in
session->closed_stream_head and is treated as if it is closed stream.
One difference is that if the stream is opened, we remove it from
linked list and change the state to the appropriate one. To O(1)
removal from linked list, we change session->closed_stream_head to
doubly linked list.
This change fixes the bug that stream is out of dependency tree if the
number of nodes in a dependency tree which we add new node to is
already maximum (NGHTTP2_MAX_DEP_TREE_LENGTH) and the number of
maximum concurrent streams is more than more than
NGHTTP2_MAX_DEP_TREE_LENGTH.
* Add NGHTTP2_HTTP_1_1_REQUIRED error code
* Allow transmission of WINDOW_UPDATE on reserved (remote)
* Allow reception of WINDOW_UPDATE on reserved (local)
* Treat frame larger than MAX_FRAME_SIZE as FRAME_SIZE_ERROR
ALPN identifier is still h2-14 to continue interop, since draft-14 and
-15 are binary compatible. The new error code was added in draft-15,
but HTTP/2 allows extensions can freely add new error code, so it is
not a problem.
Previously when nghttp2_stream_resume_deferred_data() is called,
deferred flags in stream->flags are all cleared. This is not ideal
because if application returned NGHTTP2_ERR_DEFERRED, and also that
stream is deferred by flow control, then all flags are cleared and
read callback will be invoked again. This commit fixes this issue.
This changes error condition of nghttp2_session_resume_data().
Previously we return error if stream was deferred by flow control.
Now we don't return error in this case. We just clear
NGHTTP2_FLAG_DEFERRED_USER and if still
NGHTTP2_FLAG_DEFERRED_FLOW_CONTROL is set, just return 0.
Previously when connection level remote flow control window gets 0, we
mark the stream having DATA frame with
NGHTTP2_STREAM_FLAG_DEFERRED_FLOW_CONTROL. When connection level
WINDOW_UPDATE is received, we checks all existing streams, including
closed ones, and call nghttp2_stream_resume_deferred_data(). The
profiler shows this is expensive.
Now we prepare dedicated priority queue for DATA frames. And we don't
mark stream with NGHTTP2_STREAM_FLAG_DEFERRED_FLOW_CONTROL when DATA
cannot be sent solely due to connection level flow control. Instead,
we just queue DATA item to queue. We won't pop DATA item from queue
when connection level remote window size is 0. This way, we avoid the
expensive operation for all streams when WINDOW_UPDATE is arrived.
Previously we missed the case where stream->data_item is not deleted
and it caused leak. Now stream->data_item is properly deleted when
session is deleted. We decided not to delete data_item in
nghttp2_stream_free() since we need nghttp2_session to decide whether
data_item should be deleted or not there.
By default, nghttp2 library only handles HTTP/2 frames and does not
recognize first 24 bytes of client connection preface. This design
choice is done due to the fact that server may want to detect the
application protocol based on first few bytes on clear text
communication. But for simple servers which only speak HTTP/2, it is
easier for developers if nghttp2 library takes care of client
connection preface.
If this option is used with nonzero val, nghttp2 library checks first
24 bytes client connection preface. If it is not a valid one,
nghttp2_session_recv() and nghttp2_session_mem_recv() will return
error NGHTTP2_ERR_BAD_PREFACE, which is fatal error.
This commit moves frame_type parameter of
nghttp2_data_soruce_read_length_callback in front of stream_id
parameter. The motivation is that other callback is generally put
frame related parameters first. To make it consistent, we move
frame_type, which is frame ralted parameter, to the left.
Previously we always call on_frame_send_callback before calling
nghttp2_stream_detach_data() after sending DATA frame. As a result,
even if DATA frame has END_STREAM, application cannot call
nghttp2_submit_data() in on_frame_send_callback because previous data
is still attached. This commit makes a change so that
nghttp2_stream_detach_data() is called before on_frame_send_callback
so that application can issue nghttp2_submit_data() in the callback.
Now it returns only stream's available remote window size, without
considering connection level window size. For connection-level window
size, nghttp2_session_get_remote_window_size() is added by this
commit. To get old behavior of
nghttp2_session_get_stream_remote_window_size() is use
min(nghttp2_session_get_stream_remote_window_size(),
nghttp2_session_get_remote_window_size()). The reason of this change
is that it is desirable to know just stream level window size without
taking into connection level window size. This is useful for
debugging purpose.
h2-14 now allows extensions to define new error codes. To allow
application callback to access such error codes, we uses uint32_t as
error_code type for structs and function parameters. Previously we
treated unknown error code as INTERNAL_ERROR, but this change removes
this and unknown error code is passed to application callback as is.
Motivation:
The send window size is currently fixed by a macro at compile time.
In order for users of the library to impact the send window size they
would have to change a macro at compile time. The window size may be dynamic
depending on the environment and deployment scheme. The library users
currently have no way to change this parameter.
Modifications:
Add a new optional callback method which is called before data is sent to
obtain the desired send window size. The callback return value will be
subject to a range check for the current session, stream, and settings
limits defined by flow control.
Result:
Library users have control over their send sizes.
Previously returning NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE from
on_header_callback moves input offset badly and it causes header
decompression error on the subsequent frames. This commit fix this
bug.
This commit makes handling of outgoing HEADERS and PUSH_PROMISE in the
same priority of other frames on the stream, so these frames are
processed in the order they are submitted. This allows application to
submit frames to a stream returned by nghttp2_submit_{request,
headers, push_promise} immediately. The only exception is
WINDOW_UPDATA frame, which requires nghttp2_stream object, which is
not created yet.
The application should be responsible for the size of incoming header
block size. Framing layer just passes everything (we have size limit
for one header/field though) to application.
Reworked no automatic WINDOW_UPDATE feature. We added new API
nghttp2_session_consume() which tells the library how many bytes are
consumed by the application. Instead of submitting WINDOW_UPDATE by
the application, the library is now responsible to submit
WINDOW_UPDATE based on consumed bytes. This is more reliable method,
since it enables us to properly send WINDOW_UPDATE for stream and
connection individually. The previous implementation of nghttpx had
broken connection window management.
Tatsuhiro Tsujikawa
Alexis La Goutte
Scott Mitchell