Previously, in server side, we used closed streams to detect the error
that the misbehaving client sends a frame on the incoming stream it
explicitly closed. With this commit, we make a further step, and
detect one more error case. Since we retain closed streams as long as
the sum of its size and the number of opened streams are equal or less
than max concurrent streams, we can safely say that if we get a frame
which is sent on the stream that is not found in either closed or
opened stream, it is already closed or has not existed. Then we can
send GOAWAY.
The previous code shrinks closed streams when we closed another
stream, but now it is removed. It is enough to adjust closed streams
when new incoming stream is created.
While creating this commit, we noticed that
NGHTTP2_INITIAL_MAX_CONCURRENT_STREAMS is defined as INT32_MAX. But
since SETTINGS can contain value up to UINT32_MAX, it is not enough.
However, since the stream ID space is limited to INT32_MAX, it is high
enough. We could keep this value, but this time we deprecate
NGHTTP2_INITIAL_MAX_CONCURRENT_STREAMS macro. While it is in public
header, the effect of deprecating it is negligible because of the
reason we wrote above, and usually application sets much smaller value
(say, 100) as SETTINGS_MAX_CONCURRENT_STREAMS.
nghttp2_on_invalid_header_callback is similar to
nghttp2_on_header_callback, but the former is only called when the
invalid header field is received which is silently ignored when the
callback is not set. With this callback, application inspects the
incoming invalid field, and it also can reset stream from this
callback by returning NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE, or using
nghttp2_submit_rst_stream() directly with the error code of choice.
We also added nghttp2_on_invalid_header_callback2, which uses
reference counted header fields.
This function sets the maximum length of header block (a set of header
fields per HEADERS frame) to send. The length of given set of header
fields is calculated using nghttp2_hd_deflate_bound(). Previously,
this is hard-coded, and is 64KiB.
This option prevents the nghttp2 library from sending PING frame with
ACK flag set in the reply to incoming PING frame. To allow the
application to send PING with ACK flag set, nghttp2_submit_ping() now
recognizes NGHTTP2_FLAG_PING in its flags parameter.
Disallow request from server, and response from client respectively.
When the violation is detected, return NGHTTP2_ERR_PROTO from
nghttp2_submit_request, nghttp2_submit_response,
nghttp2_submit_headers.
We also did some refactoring, and now self-dependency detection is
placed where it is only required.
Return NGHTTP2_ERR_INVALID_ARGUMENT from nghttp2_submit_headers() if
given stream ID and pri_spec->stream_id are the same (thus trying to
depend on itself).
Also return NGHTTP2_ERR_INVALID_ARGUMENT from nghttp2_submit_request()
and nghttp2_submit_headers() with stream_id == 1, when new stream ID
equals to pri_spec->stream_id.
Previously, these cases are not checked, and just sent to peer.
Previously, stream object for pushed resource was not created during
nghttp2_submit_push_promise(). It was created just before
nghttp2_before_frame_send_callback was called for that PUSH_PROMISE
frame. This means that application could not call
nghttp2_submit_response for the pushed resource before
nghttp2_before_frame_send_callback was called. This could be solved
by callback chaining, but for web server with back pressure from
backend stream, it is a bit unnecessarily hard to use.
This commit changes nghttp2_submit_push_promise() behaviour so that
stream object is created during that call. It makes application call
nghttp2_submit_response right after successful
nghttp2_submit_push_promise call.
The added API is nghttp2_session_change_stream_priority(). This
provides the same functionality to re-prioritize stream when PRIORITY
frame. is received, but we do it without PRIORITY frame. This could
be useful for server to change pushed stream's priority silently.
It has no usecase at the moment. It is most likely that applications
know the flags when it submitted extension frame, no need to modify it
later. Possibly feature bloat.
To validate actual response body length against the value declared in
content-length response header field, we first check request method.
If request method is HEAD, respose body must be 0 regardless of the
value in content-length. nghttp2_session_upgrade() has no parameter
to indicate the request method is HEAD, so we failed to validate
response body if HEAD is used with HTTP Upgrade. New
nghttp2_session_upgrade2() accepts new parameter to indicate that
request method is HEAD or not to fix this issue. Although, this issue
affects client side only, we deprecate nghttp2_session_upgrade() in
favor of nghttp2_session_upgrade2() for both client and server side.
This function is useful for the client application to know that there
is a chance that request can be sent. If this function returns 0,
there is zero chance to make a request.
This commit also set error_code passed to
nghttp2_on_stream_close_callback to NGHTTP2_REFUSED_STREAM if request
is not sent.
This will improve performance since we can avoid indirect call of
internal functions. The downside is we now require libnghttp2 static
library to run unit tests.
Application can utilize this framework to send/receive user defined
extension frames. These frames are expected not to change existing
protocol behaviour.
If application returns NGHTTP2_ERR_PAUSE from send_data_callback, it
means application processed all data, but wants to make
nghttp2_session_mem_send or nghttp2_session_send return immediately.
This is useful if application writes to fixed sized buffers, and there
is no room to write more data.
This change adds new return error code from nghttp2_session_mem_recv
and nghttp2_session_recv functions, namely NGHTTP2_ERR_FLOODED. It is
fatal error, and is returned when flooding was detected.
RFC 7540 does not enforce any limit on the number of incoming reserved
streams (in RFC 7540 terms, streams in reserved (remote) state). This
only affects client side, since only server can push streams.
Malicious server can push arbitrary number of streams, and make
client's memory exhausted. The new option,
nghttp2_set_max_reserved_remote_streams, can set the maximum number of
such incoming streams to avoid possible memory exhaustion. If this
option is set, and pushed streams are automatically closed on
reception, without calling user provided callback, if they exceed the
given limit. The default value is 200. If session is configured as
server side, this option has no effect. Server can control the number
of streams to push.
The intention of this stream API is give server application about
stream dependency information, so that it can utilize it for better
scheduling of stream processing. We have no plan to add object
oriented API based on stream object.
RFC 7541 requires that dynamic table size update must occur at the
beginning of the first header block, and is signaled as SETTINGS
acknowledgement. This commit checks these conditions. If dynamic
table size update appears other than the beginning of the first header
block, it is treated as error. If SETTINGS ACK is received, and next
HEADERS header block does not have dynamic table size update, it is
treated as error.
This commit documents NGHTTP2_ERR_DATA_EXIST also occurs if HEADERS
has been already attached to stream too. This commit also fixes
possible assertion error, and now nghttp2_submit_headers() and
nghttp2_submit_response() may return NGHTTP2_ERR_DATA_EXIST. But we
recommend to use nghttp2_submit_request() and
nghttp2_submit_response(), and using them will avoid this error.
The private global variable nghttp2_enable_strict_preface is also
marked as NGHTTP2_EXTERN, but it is test purpose only (test with
.dll), and not part of public API. It could be removed in the future
release.
From autoconf manual, section 5.6.1 Portability of Headers, says:
"""
The C99 standard says that inttypes.h includes stdint.h, so there's no
need to include stdint.h separately in a standard environment. Some
implementations have inttypes.h but not stdint.h (e.g., Solaris 7),
but we don't know of any implementation that has stdint.h but not
inttypes.h.
"""
Since application most likely allocates the stream object in
nghttp2_on_begin_headers_callback, it is desirable to handle its
failure as stream error. But previously it only signals success or
fatal error. Submitting RST_STREAM does not prevent
nghttp2_on_header_callback from being invoked. This commit improves
this situation by allowing NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE from
nghttp2_on_begin_headers_callback. If that value is returned, library
submits RST_STREAM with error code INTERNAL_ERROR, and
nghttp2_on_header_callback and nghttp2_on_frame_recv_callback for that
frame are not invoked. Note that for PUSH_PROMISE frame, the stream
to be reset is promised stream.
The existing nghttp2_session_consume() affects both connection and
stream level flow control windows. The new functions only affects
either connection or stream. There is some interesting use cases.
For example, we may want to pause a stream by not sending
WINDOW_UPDATE, meanwhile we want to continue to process other streams.
In this case, we use nghttp2_session_consume_connection() to tell
library that only connection level window is recovered. The relevant
discussion: https://code.google.com/p/chromium/issues/detail?id=473259
Previously nghttp2_session_send() and nghttp2_session_mem_send() did
not send 24 bytes client magic byte string (MAGIC). We made
nghttp2_session_recv() and nghttp2_session_mem_recv() process MAGIC by
default, so it is natural to make library send MAGIC as well. This
commit makes nghttp2_session_send() and nghttp2_session_mem_send()
send MAGIC. This commit also replace "connection preface" with
"client magic", since we call MAGIC as "connection preface" but it is
just a part of connection preface. NGHTTP2_CLIENT_CONNECTION_PREFACE
macro was replaced with NGHTTP2_CLIENT_MAGIC. The already deprecated
NGHTTP2_CLIENT_CONNECTION_HEADER macro was removed permanently.
nghttp2_option_set_no_recv_client_preface() was renamed as
nghttp2_option_set_no_recv_client_magic(). NGHTTP2_ERR_BAD_PREFACE
was renamed as NGHTTP2_ERR_BAD_CLIENT_MAGIC.
Since HTTP/2 spec requires for client to send connection preface, it
is reasonable to make this option enabled by default. It is still a
use case to disable this, so replace this option with
nghttp2_option_set_no_recv_client_preface().
HTTP/2 and HPACK are going to be published as RFC, but ALTSVC is still
in draft state. To make our API stable, it would be better to remove
ALTSVC API for 1.0.0 release.
To avoid buffer copy in nghttp2_data_source_read_callback, this commit
introduces NGHTTP2_DATA_FLAG_NO_COPY and nghttp2_send_data_callback.
By using NGHTTP2_DATA_FLAG_NO_COPY in
nghttp2_data_source_read_callback, application can avoid to copy
application data to given buffer. Instead, application has to
implement nghttp2_send_data_callback to send complete DATA frame by
itself. We see noticeable performance increase in nghttpd and
tiny-nghttpd using this new feature. On the other hand, nghttpx does
not show such difference, probably because buffer copy is not
bottleneck. Using nghttp2_send_data_callback adds complexity, so it
is recommended to measure the performance to see whether this extra
complexity worth it.
Previously API reference is gigantic one rst file and it is a bit hard
to use, especially when browsing similar functions. This commit
splits API reference into smaller fine grained files. The macros,
enums, types are now in its own file. Each API function has its own
file now. API reference doc is now index to above documentation
files. The apiref-header.rst is renamed as programmers-guide.rst and
becomes standalone document.
nghttp2_submit_request and nghttp2_submit_response will set
NGHTTP2_FLAG_END_STREAM after all given data is sent (data could be
0). This means we have no way to send trailers. In this commit, we
added NGHTTP2_DATA_FLAG_NO_END_STREAM flag. The application can set
this flag in *data_flags inside nghttp2_data_source_read_callback. If
NGHTTP2_DATA_FLAG_EOF is set, library automatically set
NGHTTP2_FLAG_END_STREAM. But if both NGHTTP2_DATA_FLAG_EOF and
NGHTTP2_DATA_FLAG_NO_END_STREAM are set, NGHTTP2_FLAG_END_STREAM will
not set by library. Then application can use new
nghttp2_submit_trailer() to send trailers. nghttp2_submit_trailer()
will set NGHTTP2_FLAG_END_STREAM and it is actually thing wrapper of
nghttp2_submit_headers().
Tatsuhiro Tsujikawa
Peter Wu
kumagi
Daniel Stenberg