Commit Graph

239 Commits

Author SHA1 Message Date
Tatsuhiro Tsujikawa ee07694783 nghttpx: Add request-header-field-buffer and max-request-header-fields options
This commit adds request-header-field-buffer and
max-request-header-fields, and deprecates header-field-buffer and
max-header-fields options.
2016-02-06 17:22:23 +09:00
Tatsuhiro Tsujikawa 85bb37ab7c Enable ConstructorInitializerAllOnOneLineOrOnePerLine for better diff 2016-01-27 21:14:07 +09:00
Tatsuhiro Tsujikawa 98253b1d0d nghttpx: Use DefaultMemchunks as HTTP/2 and SPDY frontend response buffer 2016-01-27 15:28:01 +09:00
Tatsuhiro Tsujikawa eb8649bf9b nghttpx: Don't greedily read data from backend
This might help throughput, but it interfere stream priority.  The
throughput issue is generally caused by the small buffer size to store
response body, which was 16K.  We increased it to 128K to compensate
this change.
2016-01-20 11:20:15 +09:00
Tatsuhiro Tsujikawa 0402481be4 nghttpx: Organize connection related configuration into struct 2016-01-19 16:56:12 +09:00
Tatsuhiro Tsujikawa 16549bb276 nghttpx: Structured configurations for http and http2 2016-01-18 17:00:20 +09:00
Tatsuhiro Tsujikawa ef5d981ab1 nghttpx: Simplify 2016-01-17 17:04:16 +09:00
Tatsuhiro Tsujikawa d16ff1f519 nghttpx: Use StringAdaptor for Config::server_name 2016-01-17 01:15:11 +09:00
Tatsuhiro Tsujikawa b202e066fd nghttpx: Don't allow certain characters in host and :scheme header field
For HTTP/2, we do this validation in libnghttp2.  http-parser does
this partially, when it parses URI, but it does not do anything for
Host header field.  libspdylay does not perform anything.  So do some
additional validation for HTTP/1 and SPDY cases.  integration tests
were also added to make sure they work.
2016-01-16 16:00:05 +09:00
Tatsuhiro Tsujikawa f8472f4709 nghttpx: Remove backend priority handling code
Currently, this does not do anything useful.
2016-01-15 00:17:40 +09:00
Tatsuhiro Tsujikawa 5a8cf94361 nghttpx: Refactor Downstream::response_sent_bodylen_ 2016-01-14 23:54:28 +09:00
Tatsuhiro Tsujikawa fa31d0940c nghttpx: Refactor Downstream::request_datalen_ 2016-01-14 23:36:47 +09:00
Tatsuhiro Tsujikawa e7e52b11ce nghttpx: Refactor Downstream::response_bodylen_ 2016-01-14 23:20:44 +09:00
Tatsuhiro Tsujikawa bdef0e0b1a nghttpx: Refactor Downstream::request_bodylen_ 2016-01-14 23:14:58 +09:00
Tatsuhiro Tsujikawa 3b8889a2a1 nghttpx: Extract response related fields to Response struct 2016-01-14 01:05:10 +09:00
Tatsuhiro Tsujikawa 919f08eb38 nghttpx: Extract request related fields to Request struct
Header field related functions are now gathered into FieldStore class.
This commit only handles request.  Subsequent commit will do the same
thing for response.
2016-01-14 01:04:32 +09:00
Tatsuhiro Tsujikawa 2f50bc1b3c nghttpx: Take into account of trailers when applying max_header_fields 2016-01-05 16:47:44 +09:00
Tatsuhiro Tsujikawa 1753bea692 nghttpx: Support server push from HTTP/2 backend
This commits enables HTTP/2 server push from HTTP/2 backend to be
relayed to HTTP/2 frontend.  To use this feature, --http2-bridge or
--client is required.  Server push via Link header field contiues to
work.
2015-11-16 23:12:25 +09:00
Tatsuhiro Tsujikawa c6ef1c02b9 Switch to clang-format-3.6 2015-11-13 00:53:29 +09:00
Tatsuhiro Tsujikawa da89f9c150 nghttpx: Refactor client handler write
Move write buffer to Upstream objects
2015-10-03 11:09:42 +09:00
Tatsuhiro Tsujikawa 57aee184ca Update doc 2015-09-12 18:33:07 +09:00
Tatsuhiro Tsujikawa 3ce1c1d39f nghttpx: Add date header field to error_reply and send_reply 2015-09-07 23:11:23 +09:00
Tatsuhiro Tsujikawa ef1595672c nghttpx: Add Request#push in mruby scripting
Refactor Http2Upstream so that we can share code between link header
field based push and mruby push.
2015-09-06 00:05:07 +09:00
Tatsuhiro Tsujikawa a9338f1c0e nghttpx: Fix hang when error page is sent on response phase hook error 2015-09-04 22:34:40 +09:00
Tatsuhiro Tsujikawa 9e685a2734 nghttpx: Various test failures 2015-09-04 02:47:25 +09:00
Tatsuhiro Tsujikawa d044c58558 nghttpx: Detect mruby presence and guard mruby related code with ifdef 2015-09-04 00:54:41 +09:00
Tatsuhiro Tsujikawa 200217d8ea nghttpx: Store empty string to path for server-wide OPTIONS request
This change is required to show path attribute to mruby script.  It is
desirable to construct URI from parts.  Just checking method and path
is "*" is awkward.
2015-09-04 00:14:09 +09:00
Tatsuhiro Tsujikawa 65f2b16132 nghttpx: More freedom for custom response headers 2015-09-03 22:29:16 +09:00
Tatsuhiro Tsujikawa baadec5ef4 nghttpx: Add response mruby hook 2015-09-03 01:33:52 +09:00
Tatsuhiro Tsujikawa e06af02573 nghttpx: Add Response mruby object 2015-09-03 00:40:14 +09:00
Tatsuhiro Tsujikawa 44cbc785fc nghttpx: Don't reuse backend connection if it is not clean 2015-07-22 21:41:16 +09:00
Tatsuhiro Tsujikawa 5dc060c1a2 src: Use C++11 value-initialization, instead of memset-ing 0 2015-07-19 17:55:37 +09:00
Tatsuhiro Tsujikawa ba31b990a2 nghttpx: Remove dead handling of Downstream::STREAM_CLOSED 2015-07-15 23:33:44 +09:00
Tatsuhiro Tsujikawa fa7069a273 nghttpx: Don't rewrite path if http2 proxy or client proxy is enabled
There are many requests which changes its meaning when we rewrite
path.  This is due to bad percent-encoding in URI; reserved characters
are just used without percent encoding.  It seems this is common in ad
services, but I suspect more to come.  For reverse proxying situation,
sane service most likely encodes URI properly, so probably this is not
an issue.
2015-07-11 17:50:58 +09:00
Tatsuhiro Tsujikawa 19e47a1922 nghttpx: Normalize path when setting it to Downstream 2015-07-11 16:12:35 +09:00
Tatsuhiro Tsujikawa 3119fc259c Select backend based on request host and path by extending -b option
-b option syntax is now <HOST>,<PORT>[;<PATTERN>[:...]].  The optional
<PATTERN>s specify the request host and path it is used for.  The
<PATTERN> can contain path, host + path or host.  The matching rule is
closely designed to ServeMux in Go programming language.
2015-07-11 00:15:52 +09:00
Tatsuhiro Tsujikawa 39f89f4a60 src: Use user-defined literals for k, m, and g. 2015-06-21 14:32:47 +09:00
Tatsuhiro Tsujikawa 5b51320dc5 nghttpx: Validate :path on SPDY frontend
Unless method is CONNECT, we require that :path starts with "/",
except for OPTIONS method, which can take "*" as :path (server-wide
OPTIONS request).
2015-06-16 00:09:22 +09:00
Tatsuhiro Tsujikawa 2952706b53 Fix scan-build error 2015-06-10 00:29:03 +09:00
Tatsuhiro Tsujikawa 41dd5f6897 nghttpx: Tokenize request method
We share the same method value with http-parser.  This commit also
returns 501 for unknown request method on HTTP/2 and SPDY frontend.
2015-06-09 23:33:14 +09:00
Tatsuhiro Tsujikawa 8c6f9e899f nghttpx: Enforce header field buffer limit for SPDY frontend 2015-04-29 21:27:36 +09:00
Tatsuhiro Tsujikawa 8c3b379b66 Pool Memchunk per worker 2015-04-08 00:10:48 +09:00
Tatsuhiro Tsujikawa d46e50b112 nghttpx: Refactor DownstreamQueue to avoid expensive std::map 2015-03-12 01:13:55 +09:00
Tatsuhiro Tsujikawa 446de923f3 nghttpx: Support multiple HTTP/2 session per worker
Currently, we use same number of HTTP/2 sessions per worker with given
backend addresses.  New option to specify the number of HTTP/2 session
per worker will follow.
2015-03-10 23:20:21 +09:00
Tatsuhiro Tsujikawa 1c2c5bdd05 Revert "nghttpx: Add missing Downstream::end_upload_data() call in HTTP/2 and SPDY"
This reverts commit ef090d425e.
2015-02-27 00:32:48 +09:00
Tatsuhiro Tsujikawa ef090d425e nghttpx: Add missing Downstream::end_upload_data() call in HTTP/2 and SPDY
This ensures that all frontend code calls
Downstream::end_upload_data() when request was all received.
2015-02-26 23:50:21 +09:00
Tatsuhiro Tsujikawa 1c0d617742 nghttpx: Rename WorkerConfig as LogConfig
This is a sign that we only use thread-local storage for logging only.
2015-02-26 00:02:29 +09:00
Tatsuhiro Tsujikawa 50c4aa061f nghttpx: Response with 503 when re-submission to backend failed 2015-02-20 19:23:52 +09:00
Tatsuhiro Tsujikawa d1a1e882bf nghttpx: Fix request re-submission bug in HTTP/2 backend 2015-02-17 23:15:53 +09:00
Tatsuhiro Tsujikawa 799778af69 Revert "nghttpx: Fix request resubmit bug on HTTP/2 backend connection check"
This reverts commit d45f5a51e4.
2015-02-17 22:28:03 +09:00
Tatsuhiro Tsujikawa d45f5a51e4 nghttpx: Fix request resubmit bug on HTTP/2 backend connection check 2015-02-17 21:50:32 +09:00
Tatsuhiro Tsujikawa 011e3b325d nghttpx: Cancel backend request when frontend HTTP/1 connection is lost 2015-02-13 22:41:50 +09:00
Tatsuhiro Tsujikawa d151759f8a nghttpx: Fix location rewrite, take 2 2015-02-08 21:26:47 +09:00
Tatsuhiro Tsujikawa b14cfaf308 src: Store token in Header object to avoid additional lookups 2015-02-08 16:10:01 +09:00
Tatsuhiro Tsujikawa 4cda09beff src: Prefer std::array 2015-02-06 00:15:43 +09:00
Tatsuhiro Tsujikawa 54851ef7a6 src: Move make_unique to nghttp2 namespace 2015-02-06 00:15:43 +09:00
Tatsuhiro Tsujikawa b2fb888363 Share I/O code with all upstreams/downstream objects 2015-02-05 03:05:34 +09:00
Tatsuhiro Tsujikawa b707cfe986 nghttpx: Fix busy loop when HTTP/2 backend reset after connection established
We have now Downstream retry count to be limited to 5 times.  At 6th
failure, we send 503 message to client.
2015-02-03 01:47:04 +09:00
Tatsuhiro Tsujikawa f1049a66e2 nghttpx: Detach DownstreamConnection early 2015-01-31 01:11:55 +09:00
Tatsuhiro Tsujikawa 6b1ef95d3f nghttpx: Replace RingBuf with sequential Buffer
It turns out that we don't need circular buffer functionality.  We
replaced RingBuf with simple sequential Buffer.
2015-01-29 22:57:56 +09:00
Tatsuhiro Tsujikawa 8ddad1a53d nghttpx: Remove shrinking memchunks for now
It requires more careful optimization.  Remove it for now.
2015-01-28 21:25:22 +09:00
Tatsuhiro Tsujikawa 4e68ca8233 integration: Add tests about via header field manipulation
This test reveals bug in SPDY upstream code, and contains its fix.
2015-01-27 00:19:57 +09:00
Tatsuhiro Tsujikawa 68510f1282 nghttpx: SpdyUpstream: Handle error from error_reply 2015-01-25 15:37:09 +09:00
Tatsuhiro Tsujikawa 5770c6bd04 nghttpx: Return 503 on hard disconnect in HTTP/2 backend 2015-01-21 23:30:48 +09:00
Tatsuhiro Tsujikawa e219d6a94f nghttpx: Clarify error_reply upstream method
There is no application level failure in h1 upstream.  For h2, SPDY
upstreams, don't call DIE(), instead return -1 to delete handler.
2015-01-21 22:55:00 +09:00
Tatsuhiro Tsujikawa 41e72064e0 nghttpx: Log error in INFO level when frame cannot be sent 2015-01-21 22:49:00 +09:00
Tatsuhiro Tsujikawa 041d9863c2 nghttpx: Set request_start_time_ at Downstream ctor
.. so that we can avoid the problem that request_start_time_ is 0
2015-01-21 22:28:15 +09:00
Tatsuhiro Tsujikawa 8997e4369d nghttpx: Adjust backend buffers 2015-01-21 01:47:43 +09:00
Tatsuhiro Tsujikawa 5a6d6ccbd4 nghttpx: Read from backend eagerly in all upstreams 2015-01-21 01:41:17 +09:00
Tatsuhiro Tsujikawa 16e91746d9 nghttpx: Return 400 error if multiple CLs are received in SPDY upstream
This change adds SPDY upstream tests.
2015-01-21 01:03:56 +09:00
Tatsuhiro Tsujikawa 8059380fb0 nghttpx: Don't need to set response state to MSG_COMPLETE after error_reply 2015-01-20 23:33:45 +09:00
Tatsuhiro Tsujikawa a440bdf15e nghttpx: Response 502 if HTTP/2 backend receives invalid Content-Length 2015-01-19 23:44:23 +09:00
Tatsuhiro Tsujikawa af24f8394e nghttpx: Fix nghttp2 error code use in SPDY upstream 2015-01-17 21:35:36 +09:00
Tatsuhiro Tsujikawa 62b9e4bb56 nghttpx: Validate received request body length against content-length 2015-01-17 21:33:23 +09:00
Tatsuhiro Tsujikawa 441f1cc282 nghttpx: Validate received response body length against content-length 2015-01-17 21:33:23 +09:00
Tatsuhiro Tsujikawa 90914b38f1 nghttpx: Do not limit DATA frame length 2015-01-07 01:25:43 +09:00
Tatsuhiro Tsujikawa d80952a2bc nghttpx: Implement stream level timeout using ev_timer 2015-01-06 00:30:57 +09:00
Tatsuhiro Tsujikawa 3ae44ef2f3 nghttpd, nghttpx: Rework incoming header handling 2015-01-05 01:46:41 +09:00
Tatsuhiro Tsujikawa db6eec653b Fix errors reported by scan-build 2015-01-03 00:19:41 +09:00
Tatsuhiro Tsujikawa 7db1864766 nghttpx: Add --backend-http1-connections-per-frontend option 2015-01-03 00:19:41 +09:00
Tatsuhiro Tsujikawa bfac015d61 src: Use libev for rest of the applications 2015-01-03 00:19:41 +09:00
Tatsuhiro Tsujikawa 5fa9dd7cd5 nghttpx: Remove unused variable 2014-12-16 01:10:54 +09:00
Tatsuhiro Tsujikawa c93da867e0 nghttpx: Fix bug SPDY upstream requires content-length if no FIN in SYN_STREAM 2014-12-15 23:42:59 +09:00
Tatsuhiro Tsujikawa dce20c3e6a nghttpx: Check HTTP/2 downstream connection after certain idle time
Previously when requests are issued to HTTP/2 downstream connection,
but it turns out that connection is down, handlers of those requests
are deleted.  In some situations, we only know connection is down when
we write something to network, so we'd like to handle this kind of
situation in more robust manner.  In this change, certain seconds
passed after last network activity, we first issue PING frame to
downstream connection before issuing new HTTP request.  If writing
PING frame is failed, it means connection was lost.  In this case,
instead of deleting handler, pending requests are migrated to new
HTTP2/ downstream connection, so that it can continue without
affecting upstream connection.
2014-12-09 21:41:29 +09:00
Tatsuhiro Tsujikawa 3b03ff626a nghttpx: Don't log access log for blocked streams 2014-12-05 21:29:31 +09:00
Tatsuhiro Tsujikawa 9614611969 nghttpx: Limit # of downstream connections per host when h2 proxy is used
This commit limits the number of concurrent HTTP/1 downstream
connections to same host.  By defualt, it is limited to 8 connections.
--backend-connections-per-frontend option was replaced with
--backend-http1-connections-per-host, which changes the maximum number
of connections per host.  This limitation only kicks in when h2 proxy
is used (-s option).
2014-12-05 01:47:03 +09:00
Tatsuhiro Tsujikawa b1f807abd1 Reformat lines with clang-format-3.5 2014-11-27 23:56:30 +09:00
Lucas Pardue 6e178653a5 fixed conflict 2014-11-23 21:10:51 +00:00
Lucas Pardue 9cf1a0c77c Add features to logging, client and server port,
time_iso8601 and request_time.
2014-11-23 20:37:51 +00:00
Tatsuhiro Tsujikawa 74ec1d3377 nghttpx: Write accesslog when upstream connection is lost for HTTP/2 and SPDY 2014-11-23 17:24:23 +09:00
Tatsuhiro Tsujikawa 27609327ee nghttpx: Fix heap-after-free crash in https upstream
Add Upstream::on_handler_delete() hook to safely write log for
HttpsUpstream.
2014-11-19 01:59:09 +09:00
Tatsuhiro Tsujikawa 958cd0de64 nghttpx: Add configurable access logging format
This commit adds functionality to customize access logging format in
nghttpx.  The format variables are inspired by nginx.  The default
format is combined format.
2014-11-19 01:29:55 +09:00
Tatsuhiro Tsujikawa ce71e65aee nghttpx: Replace WARNING with WARN for consistency 2014-11-08 10:51:56 +09:00
Tatsuhiro Tsujikawa 154876a17b nghttpx: Apply TLS record length limit to DATA frame payload
This is not obvious but it makes intermediaries flush and forward DATA
frame boundary without excessive buffering.  Since we have different
TCP connections frontend and backend, this may not work.  This is
still experimental.
2014-11-06 21:14:14 +09:00
Tatsuhiro Tsujikawa f8c70993c0 nghttpx: Adjust TLS record size dynamically
Use the same behaviour the current Google server does: start with 1300
TLS record size and after transmitting 1MiB, change record size to
16384.  After 1 second idle time, reset to 1300.  Only applies to
HTTP/2 and SPDY upstream connections.
2014-11-06 02:36:53 +09:00
Tatsuhiro Tsujikawa 0d4120ce2c nghttpx: Apply same fix from a225bb2 to spdy upstream 2014-10-13 21:13:45 +09:00
Tatsuhiro Tsujikawa b305495a75 nghttpx: Reset both timeouts when either read or write succeeds
Previously read and write timeouts work independently.  When we are
writing response to the client, read timeout still ticks (e.g., HTTP/2
or tunneled HTTPS connection).  So read timeout may occur during long
download.  This commit fixes this issue.  This commit only fixes the
upstream part.  We need similar fix for the downstream.
2014-09-18 23:03:36 +09:00
Tatsuhiro Tsujikawa 8890e593e6 src: Add util::array_size 2014-08-28 00:45:12 +09:00
Tatsuhiro Tsujikawa 0209b7c083 nghttpx: Fix location rewrite is failed because request headers are empty
Previously we empties request headers after they are sent to
downstream in order to free memory.  But it turns out that we use
request headers when rewriting location header response field.  Also
user reported that request headers are useful to add new features.
This commits defers the deletion of request headers to the point when
response headers are deleted (which is after response headers are sent
to upstream client).
2014-08-27 21:25:25 +09:00
Tatsuhiro Tsujikawa 31528b6267 Use uint32_t for HTTP/2 error_code
h2-14 now allows extensions to define new error codes.  To allow
application callback to access such error codes, we uses uint32_t as
error_code type for structs and function parameters.  Previously we
treated unknown error code as INTERNAL_ERROR, but this change removes
this and unknown error code is passed to application callback as is.
2014-08-25 21:24:04 +09:00