Commit Graph

226 Commits

Author SHA1 Message Date
Tatsuhiro Tsujikawa 814c7e68e0 Ignore regular headers if it includes illegal characters.
This commit only affects the library behaviour unless
nghttp2_option_set_no_http_messaging() is used.

We like strict validation against header field name and value against
RFC 7230, but we have already so much web sites and libraries in
public internet which do not obey these rules.  Simply just
terminating stream because of this may break web sites and it is too
disruptive.  So we decided that we should be conservative here so
those header fields containing illegal characters are just ignored.
But we are conservative only for regular headers.  We are strict for
pseudo headers since it is new to HTTP/2 and new implementations
should know the rules better.
2015-02-22 23:13:27 +09:00
Tatsuhiro Tsujikawa b157d4ebb2 Validate HTTP semantics by default
Previously we did not check HTTP semantics and it is left out for
application.  Although checking is relatively easy, but they are
scattered and error prone.  We have implemented these checks in our
applications and also feel they are tedious.  To make application
development a bit easier, this commit adds basic HTTP semantics
validation to library code.  We do following checks:

server:

* HEADERS is either request header or trailer header.  Other type of
header is disallowed.

client:

* HEADERS is either zero or more non-final response header or final
  response header or trailer header.  Other type of header is
  disallowed.

For both:

* Check mandatory pseudo header fields.
* Make sure that content-length matches the amount of DATA we
  received.

If validation fails, RST_STREAM of type PROTOCOL_ERROR is issued.
2015-02-20 01:01:10 +09:00
Tatsuhiro Tsujikawa 7d4a6aa179 Add test for submission ordering of HEADERS and its RST_STREAM 2015-02-12 21:28:20 +09:00
Tatsuhiro Tsujikawa eec8870ac1 Fix bug that client may send PROTOCOL_ERROR upon canceled push stream
Previously we treat stream in NGHTTP2_STREAM_RESERVED state specially,
that is we don't increment or decrement streams counts if stream is in
that state.  Because of this, we don't change the stream state to
NGHTTP2_STREAM_CLOSING if stream is in NGHTTP2_STREAM_RESERVED.  But
it turns out that it causes a problem.  If client canceled pushed
stream before push response HEADERS, stream is still in
NGHTTP2_STREAM_RESERVED state.  If push response HEADERS arrived in
this state, library happily accepts it and passed to application.

With this commit, this bug was corrected.  We now change stream state
to NGHTTP2_STREAM_CLOSING even if it was in NGHTTP2_STREAM_RESERVED
state.  We now use NGHTTP2_STREAM_FLAG_PUSH to determine whether we
have to increase/decrase stream count.
2015-02-09 22:23:20 +09:00
Tatsuhiro Tsujikawa b685747643 Add nghttp2_submit_shutdown_notice() to start graceful shutdown
nghttp2_submit_shutdown_notice() is used to notify the client that
graceful shutdown is started.  We expect that after this call, the
server application should send another GOAWAY using
nghttp2_submit_goaway() with appropriate last_stream_id.  In this
commit, we also added nghttp2_session_get_last_proc_stream_id(), which
can be used as last_stream_id parameter.

This commit implements graceful shutdown in nghttpx.  The integration
test for graceful shutdown is also added.
2015-01-22 23:21:58 +09:00
Tatsuhiro Tsujikawa 7c0a0c495d Rename nghttp2_stream.data_item to nghttp2_stream.item and related functions
Initially, we use nghttp2_stream.data_item to refer only item with
DATA frame.  But recently we use it to refer HEADERS frame as well.
So it is better to call just item rather than data_item.  This applies
to all related functions.
2014-12-20 21:48:31 +09:00
Tatsuhiro Tsujikawa e8513b3241 Make huffman encoding faster 2014-12-19 23:22:55 +09:00
Tatsuhiro Tsujikawa 280c9dfcf3 Keep idle streams in separate list
Previously we handle idle streams as closed streams.  We only keeps
sum of closed streams and active streams under max concurrent streams
limit, idle streams gets deleted earlier than client expects.

In this change, idle streams are kept in separate list and not handled
as closed streams.  To mitigate possible attack vector to make
unlimited idle streams, we cap the number of idle streams in a half of
max concurrent streams.  This is arbitrary choice.  It may be adjusted
in the future when we have interop experience.
2014-12-13 00:14:52 +09:00
Tatsuhiro Tsujikawa 21b48d24e4 Remove altsvc 2014-12-07 23:11:54 +09:00
Tatsuhiro Tsujikawa b1f807abd1 Reformat lines with clang-format-3.5 2014-11-27 23:56:30 +09:00
Alexis La Goutte bac44d7ffb Fix -Werror=unused-parameter using _U_ macro 2014-11-25 17:08:09 +01:00
Tatsuhiro Tsujikawa 8e94551881 Handle idle stream in priority field 2014-11-24 15:25:19 +09:00
Tatsuhiro Tsujikawa ae93f6345c Allow PRIORITY frame at anytime.
Allowing PRIORITY frame at anytime so that PRIORITY frame to idle
stream can create anchor node in dependency tree.  In this change, we
open stream with new NGHTTP2_STREAM_IDLE state, which is linked in
session->closed_stream_head and is treated as if it is closed stream.
One difference is that if the stream is opened, we remove it from
linked list and change the state to the appropriate one.  To O(1)
removal from linked list, we change session->closed_stream_head to
doubly linked list.
2014-11-24 15:25:19 +09:00
Tatsuhiro Tsujikawa 9a33116526 Fix bug in priority tree
This change fixes the bug that stream is out of dependency tree if the
number of nodes in a dependency tree which we add new node to is
already maximum (NGHTTP2_MAX_DEP_TREE_LENGTH) and the number of
maximum concurrent streams is more than more than
NGHTTP2_MAX_DEP_TREE_LENGTH.
2014-11-05 00:32:16 +09:00
Tatsuhiro Tsujikawa 76800dc8e7 Remove unused functions 2014-10-30 23:31:36 +09:00
Tatsuhiro Tsujikawa 80dcb565eb Check first SETTINGS strictly 2014-10-09 21:37:18 +09:00
Tatsuhiro Tsujikawa b2f88f8fe3 Fix memory leak around stream->data_item
Previously we missed the case where stream->data_item is not deleted
and it caused leak.  Now stream->data_item is properly deleted when
session is deleted.  We decided not to delete data_item in
nghttp2_stream_free() since we need nghttp2_session to decide whether
data_item should be deleted or not there.
2014-09-17 23:16:00 +09:00
Tatsuhiro Tsujikawa 901de5fbce Add nghttp2_option_set_recv_client_preface()
By default, nghttp2 library only handles HTTP/2 frames and does not
recognize first 24 bytes of client connection preface. This design
choice is done due to the fact that server may want to detect the
application protocol based on first few bytes on clear text
communication. But for simple servers which only speak HTTP/2, it is
easier for developers if nghttp2 library takes care of client
connection preface.

If this option is used with nonzero val, nghttp2 library checks first
24 bytes client connection preface. If it is not a valid one,
nghttp2_session_recv() and nghttp2_session_mem_recv() will return
error NGHTTP2_ERR_BAD_PREFACE, which is fatal error.
2014-09-13 19:50:44 +09:00
Tatsuhiro Tsujikawa f34cbf9b45 tests: Include config.h 2014-08-27 23:36:57 +09:00
Tatsuhiro Tsujikawa 4c11cd0671 Add test to submit DATA frame twice
This commit adds test to submit DATA frame twice and fixes the bug
that 2nd DATA is not sent.
2014-08-25 23:46:17 +09:00
Tatsuhiro Tsujikawa 3655090997 Merge branch 'window_size_control' of https://github.com/akamai/nghttp2 into akamai-window_size_control 2014-08-25 21:09:26 +09:00
Scott Mitchell 3cd08251ca Send window size API extension
Motivation:

The send window size is currently fixed by a macro at compile time.
In order for users of the library to impact the send window size they
would have to change a macro at compile time. The window size may be dynamic
depending on the environment and deployment scheme. The library users
currently have no way to change this parameter.

Modifications:

Add a new optional callback method which is called before data is sent to
obtain the desired send window size. The callback return value will be
subject to a range check for the current session, stream, and settings
limits defined by flow control.

Result:
Library users have control over their send sizes.
2014-08-24 11:32:44 -04:00
Tatsuhiro Tsujikawa 9893ae81af Add nghttp2_bufs_realloc 2014-08-24 15:34:55 +09:00
Tatsuhiro Tsujikawa 7bfa276e96 Fix bug that NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE causes session failure
Previously returning NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE from
on_header_callback moves input offset badly and it causes header
decompression error on the subsequent frames.  This commit fix this
bug.
2014-08-24 00:50:55 +09:00
Tatsuhiro Tsujikawa 77374ac6e2 Implement SETTINGS_MAX_FRAME_SIZE and SETTINGS_MAX_HEADER_LIST_SIZE 2014-07-31 23:05:53 +09:00
Tatsuhiro Tsujikawa 38bfbffb1b Remove HPACK reference set 2014-07-31 23:05:52 +09:00
Tatsuhiro Tsujikawa 57e9b94aaa Handle header table size up to UINT32_MAX 2014-07-22 22:38:18 +09:00
Tatsuhiro Tsujikawa 55c697e9f4 Handle multiple SETTINGS_HEADER_TABLE_SIZE in incoming SETTINGS frame
Previously we just assumed that if same settings ID is found in
SETTINGS, it is enough to process last seen entry.  But it turns out
it is not enough for SETTINGS_HEADER_TABLE_SIZE.  If we have 0 and
4096 for SETTINGS_HEADER_TABLE_SIZE in one SETTINGS, we must first
shrink dynamic table to 0 and then enlarge it to 4096.  This means
that we have to remember the minimum value and last value.
2014-07-15 00:25:31 +09:00
Tatsuhiro Tsujikawa e5abc475f1 Don't send more than NGHTTP2_MAX_HEADERSLEN bytes header block 2014-07-02 22:45:38 +09:00
Tatsuhiro Tsujikawa 31de732e3b Allocate header table ringbuffer lazily
Previously in inflater we reserve new ringbuffer when table size is
changed.  This may be potentially a problem if new table size is very
large number.  When inflater is not used directly by application, this
is not a problem because application can choose the buffer size.  On
the other hand, if application uses inflater directly and it does not
have control of new buffer size (e.g., protocol dissector), then we
just fail to allocate large buffer in
nghttp2_hd_inflate_change_table_size() without actually use such huge
buffer.  This change defers the actual allocation of buffer when it is
actually needed so that we will fail when it is absolutely needed.
2014-06-22 13:39:17 +09:00
Tatsuhiro Tsujikawa 8e0fcd3922 Add test for graceful shutdown use case 2014-06-18 12:08:12 +09:00
Tatsuhiro Tsujikawa c46d3dafc6 Remove PAD_HIGH and Pad High field
CONTINUATION now doesn't have padding.
2014-06-07 18:15:36 +09:00
Tatsuhiro Tsujikawa 458ccb3681 Ignore unknown frame types
Unexpected CONTINUATION frame is handled separately as connection
error.
2014-06-07 16:30:55 +09:00
Tatsuhiro Tsujikawa 7b9a8acc22 Add HPACK deflation API 2014-05-13 23:42:55 +09:00
Tatsuhiro Tsujikawa 4f027c1562 libnghttp2: Remove dependency to zlib
We inherited gzip compression API from spdylay codebase.  In spdylay,
the cost of having such API is almost free because spdylay requires
zlib for header compression.  nghttp2 no longer uses gzip to header
compression.  zlib dependency exists just for gzip compression API,
which is not an essential.  So we decided to move gzip code to under
src and remove zlib dependency from libnghttp2 itself.  As nghttp2
package, we depend on zlib to compile tools under src.
2014-05-06 23:10:50 +09:00
Tatsuhiro Tsujikawa 855f39743a Fix crash when indexed repr index=0 2014-05-01 09:06:54 +09:00
Tatsuhiro Tsujikawa 853c9888d9 Distribute effective weight among only streams with marked as top
If stream with dpri value of no_data, we check any its descendant has
stream with dpri value of top.  If so, we have to distribute of its
portion of weight to its descendants.
2014-04-24 23:37:40 +09:00
Tatsuhiro Tsujikawa aa4d43f31e Allow exclusive dependency to stream 0 2014-04-17 21:18:18 +09:00
Tatsuhiro Tsujikawa ac86b51e37 Implement simplified dependency based priority 2014-04-15 22:55:07 +09:00
Tatsuhiro Tsujikawa 7877b676e3 Honor NGHTTP2_NV_FLAG_NO_INDEX in deflater and inflater 2014-04-02 01:25:44 +09:00
Tatsuhiro Tsujikawa f785e56dba Implement ALTSVC frame 2014-04-01 21:47:51 +09:00
Tatsuhiro Tsujikawa ab2dc5967d Replace HTTP/2.0 with HTTP/2 2014-03-30 19:26:37 +09:00
Tatsuhiro Tsujikawa 74daa16a1c Retain incoming closed streams for dependency tree
The number of closed stream to keep is limited by
MAX_CONCURRENT_STREAMS - current active stream.
2014-03-30 17:41:54 +09:00
Tatsuhiro Tsujikawa f7162ab702 Implement dependency based priority 2014-03-30 01:24:16 +09:00
Tatsuhiro Tsujikawa 2ec4b10805 Add nghttp2_buf tests 2014-03-14 21:40:14 +09:00
Tatsuhiro Tsujikawa 0666a73e10 Remove nghttp2_buffer 2014-03-11 21:20:51 +09:00
Tatsuhiro Tsujikawa 358b4386d3 Introduce nghttp2_buf to ease buffer management 2014-03-11 01:47:38 +09:00
Tatsuhiro Tsujikawa 54dab50015 Support END_SEGMENT in nghttp2_submit_data() 2014-03-06 00:19:02 +09:00
Tatsuhiro Tsujikawa 1e95c8b313 Allow always max 1024 padding for HEADERS
We need paddings regardless of payload and frame boundary to mitigate
certain attacks.

Since we handles CONTINUATION internally, we don't show FLAG_PAD_HIGH
and PAD_LOW flags of HEADERS in nghttp/nghttpd. We just show the
total paddings in HEADERS + CONTINUATION.
2014-02-15 01:34:04 +09:00
Tatsuhiro Tsujikawa fd88c6160d HPACK post -05 updates
* Use 1 Huffman code table for both request and response
* Remove complicated deflater side table size management
* Add encoding context update
* Fix memory leak in inflater
2014-02-13 23:22:52 +09:00
Tatsuhiro Tsujikawa 7822bbd7e8 Fix PAD_HIGH and PAD_LOW are not counted in flow control 2014-02-11 21:30:44 +09:00
Tatsuhiro Tsujikawa e78a2100ec Merge branch 'master' into draft-10 2014-02-11 16:03:42 +09:00
Tatsuhiro Tsujikawa cacf4ecf26 Fix premature header block is not treated as connection error 2014-02-11 16:00:59 +09:00
Tatsuhiro Tsujikawa 78d202ac30 Callback based padding from application
Now previous padding options are removed and instead we added
select_padding_callback to select padding length for each frame
by application. If this callback is not implemented by application,
no padding is added.

This change also fixes the broken session_detect_idle_stream()
if stream_id is our side.
2014-02-11 15:28:44 +09:00
Tatsuhiro Tsujikawa 945c57c335 Add test for nghttp2_frame_pack_headers with padding 2014-02-09 15:17:26 +09:00
Tatsuhiro Tsujikawa 814d0f76f3 Implement DATA frame padding 2014-02-08 00:23:18 +09:00
Tatsuhiro Tsujikawa c79adf6997 Remove flow control disabling feature 2014-02-06 00:23:20 +09:00
Tatsuhiro Tsujikawa 8be17f077a Add nghttp2_check_header_name and nghttp2_check_header_value APIs
These are promoted to public API from src/http2.h
2014-02-01 19:31:50 +09:00
Tatsuhiro Tsujikawa ab684a9f30 Remove nghttp2_on_request_recv_callback
It is easy enough to check the end of incoming data by evaluating
frame->hd.flags & NGHTTP2_FLAG_END_STREAM in on_frame_recv_callback
2014-01-29 21:56:01 +09:00
Tatsuhiro Tsujikawa 9314e30987 Support transmission of CONTINUATION, change nghttp2_frame_hd
The maximum frame size including header block is still limited
to NGHTTP2_HD_MAX_BUFFER_LENGTH, which is 32KB.
2014-01-26 23:10:33 +09:00
Tatsuhiro Tsujikawa 91401cfe26 Support CONTINUATION frame reception 2014-01-26 20:31:28 +09:00
Tatsuhiro Tsujikawa 8317559090 nghttp2_hd: Implement stream header inflater
This stream inflater can inflate incoming header block in streaming
fashion. Currently, we buffer up single name/value pair, but we chose
far more smaller buffer size than HTTP/2 frame size.
2014-01-25 18:24:15 +09:00
Tatsuhiro Tsujikawa 707a0b4103 Move name/value validation functions to src
nghttp2 library itself now accept octet header/value pairs,
completely not restricted by HTTP/1 header name/value rule.
The applications may impose restriction about them using
validators.
2014-01-17 02:16:53 +09:00
Tatsuhiro Tsujikawa dfcdea894b Move header name/value pair validation to on_*_received functions 2013-12-25 23:38:55 +09:00
Tatsuhiro Tsujikawa 7c4dbb6ffc nghttp2_hd: Don't malloc if huffman encoded string has 0 length 2013-12-19 23:19:14 +09:00
Tatsuhiro Tsujikawa 6c77cec270 Remove nghttp2_submit_* API functions which has char **nv parameter
The nghttp2_submit_{request,response}2 functions are renamed as
nghttp2_submit_{request, response}.
2013-12-08 21:39:43 +09:00
Tatsuhiro Tsujikawa 40a666e7d1 Add nghttp2_submit_response2 tests 2013-11-28 21:47:10 +09:00
Tatsuhiro Tsujikawa 3d863ed254 nghttp2_hd: Clear reference set with index 0 2013-11-16 17:05:18 +09:00
Tatsuhiro Tsujikawa 8cd2b57f8a tests: Add test for header compression without indexing 2013-11-03 18:30:57 +09:00
Tatsuhiro Tsujikawa 2e7bc014e1 Return 0 if recv_window_size is negative; add tests 2013-10-30 01:58:38 +09:00
Tatsuhiro Tsujikawa 6c99ff12c9 nghttp2_hd: Rename local as deflate 2013-10-29 00:42:08 +09:00
Tatsuhiro Tsujikawa 67ce60544e Change maximum frame length to 16383 2013-10-27 21:17:09 +09:00
Tatsuhiro Tsujikawa cbdd44c4ae nghttp2_hd: Implement local header table size limit for encoder 2013-10-26 18:49:23 +09:00
Tatsuhiro Tsujikawa 8cf3731802 Experiment HPACK with upcoming changes
* remove substitution
* reversed insertion and removal from header table
* unified initial static table
2013-10-12 21:49:01 +09:00
Tatsuhiro Tsujikawa 21d76dcc75 Add nghttp2_session_continue API function
The NGHTTP2_ERR_PAUSE library error code is introduced to pause
the execution of nghttp2_session_mem_recv() when that error code
is returned from nghttp2_on_frame_recv_callback or
nghttp2_on_data_chunk_recv_callback. If this happens, the parameters
available for both callbacks are retained until the application
calls nghttp2_session_continue(). The application must retain
input bytes which was used to produce the frame.
After successful call of nghttp2_session_continue, the application
can continue to call nghttp2_session_mem_recv() to process
additional data.
2013-09-28 17:59:24 +09:00
Tatsuhiro Tsujikawa 94263216fb Add nghttp2_submit_request2
This function is similar to nghttp2_submit_request and the
difference is it takes an array of nghttp2_nv as name/value pairs.
It is useful if name/value pairs is not NULL-terminated in the
application code.
2013-09-11 00:55:35 +09:00
Tatsuhiro Tsujikawa 2c127b5cb6 Remove unused functions/macros 2013-09-07 15:48:16 +09:00
Tatsuhiro Tsujikawa 4dcf9ad4f2 Make hd encoder ordering aware and one-pass
The encoder algorithm is simplified and it now preserves ordering
of the headers. It also becomes one-pass encoder.
2013-09-06 21:53:28 +09:00
Tatsuhiro Tsujikawa 83b0c89e3c Handle indexing entry greater than header table limit without error 2013-08-28 23:16:23 +09:00
Tatsuhiro Tsujikawa b37f99ca03 Connection error if payload size is strictly greater than (1 << 14) - 1
It would be desired to add option to make this limit configurable.
Fix scripted_recv_callback in tests.
2013-08-28 02:13:57 +09:00
Tatsuhiro Tsujikawa 82b9f78fc7 Check incoming header field name strictly
The validation rule is described in
http://tools.ietf.org/html/draft-ietf-httpbis-header-compression-02#section-4.1.2
2013-08-25 17:39:29 +09:00
Tatsuhiro Tsujikawa 005e60a5ed Rework header compression 2013-08-23 23:38:28 +09:00
Tatsuhiro Tsujikawa 346fafde3f Implement draft-ietf-httpbis-header-compression-02 2013-08-23 03:33:05 +09:00
Tatsuhiro Tsujikawa b1ae1c30d8 Allow duplicate settings ID in SETTINGS
If multiple same ID are found, use the last one.
2013-08-23 03:33:05 +09:00
Tatsuhiro Tsujikawa 1155606d5e Rewrite re-prioritization with PRIORITY 2013-08-09 23:40:41 +09:00
Tatsuhiro Tsujikawa b979d2e8d2 Support increment/reduction of local window size by WINDOW_UPDATE 2013-08-09 00:58:52 +09:00
Tatsuhiro Tsujikawa dcfa421d6f Fix connection-level flow control (local)
Fix the bug that connection-level local window is not updated
for the data is the last part of the stream. For the stream
level window may ignore this, connection-level window must
be updated. Also this change fixes the bug that connection-level
window is not updated for the ignored DATA frames.
2013-08-07 22:02:30 +09:00
Tatsuhiro Tsujikawa 737ac01d91 Add API for HTTP Upgrade 2013-08-03 18:05:14 +09:00
Tatsuhiro Tsujikawa 8033152e80 Disable local flow control on FLOW_CONTROL_OPTIONS = 1 2013-07-28 19:05:51 +09:00
Tatsuhiro Tsujikawa f74674aa7f Fix header compression (again) 2013-07-27 19:23:05 +09:00
Tatsuhiro Tsujikawa 1056d8690e Rename functions related to HEADERS 2013-07-25 21:07:38 +09:00
Tatsuhiro Tsujikawa 84cbebf4f7 Implement server push 2013-07-25 01:59:48 +09:00
Tatsuhiro Tsujikawa 0000d3e7f9 Add test for nghttp2_nv_array_from_cstr 2013-07-23 23:03:18 +09:00
Tatsuhiro Tsujikawa b7ff05c4c6 Fail session if header compression failed on send 2013-07-23 22:47:15 +09:00
Tatsuhiro Tsujikawa ef3caffe8b Remove push related members from nghttp2_stream 2013-07-23 00:46:55 +09:00
Tatsuhiro Tsujikawa 0bcf90d32d Support PRIORITY frame send and receive 2013-07-23 00:28:00 +09:00
Tatsuhiro Tsujikawa 30d2c86a7a Remove old zlib compression and other cleanup 2013-07-20 00:20:16 +09:00
Tatsuhiro Tsujikawa 61bf7c6b02 Integrate new header compression 2013-07-20 00:08:14 +09:00
Tatsuhiro Tsujikawa 45c2245bfb Implement header compression draft 01 2013-07-19 16:50:31 +09:00
Tatsuhiro Tsujikawa a3bdda68f8 End flow control by WINDOW_UPDATE 2013-07-16 21:30:36 +09:00