Previously, we use one Http2Session object per DownstreamAddrGroup.
This is not flexible, and we have to provision how many HTTP/2
connection is required in advance. The new strategy is we add
Http2Session object on demand. We measure the number of attached
downstream connection object and server advertised concurrency limit.
As long as former is smaller than the latter, we attach new downstream
connection to it. Once the limit is reached, we create new
Http2Session object. If the number lowers the limit, we start to
share Http2Session object again.
This commit changes nghttpx's mruby script handling. Previously we
have 2 options to specify the mruby script file to be run on request
and on response. Now they are merged into 1 option, namely
--mruby-file. It now must return object. On request, the object's
on_req(env) method is invoked with env object. Similarly, on
response, the object's on_resp(env) method is invoked. The
specification of Env object has not changed.
The control process handles signals, reads configuration, reads
private keys, and bind port (which may be privileged one). It never
drop privileges, so that it can execute new binary with the same
privilege. It forks worker process. The worker process handles all
incoming connections. It drops privilege.
In reverse proxy usage, backend server most likely wants to see the
original header field. So this commit turns off host header rewrite
by default. --no-host-rewrite option is deprecated, and if it is
used, warning message is displayed. --host-rewrite option is added to
enable host rewrite.
-b option syntax is now <HOST>,<PORT>[;<PATTERN>[:...]]. The optional
<PATTERN>s specify the request host and path it is used for. The
<PATTERN> can contain path, host + path or host. The matching rule is
closely designed to ServeMux in Go programming language.
From autoconf manual, section 5.6.1 Portability of Headers, says:
"""
The C99 standard says that inttypes.h includes stdint.h, so there's no
need to include stdint.h separately in a standard environment. Some
implementations have inttypes.h but not stdint.h (e.g., Solaris 7),
but we don't know of any implementation that has stdint.h but not
inttypes.h.
"""
Currently, we use same number of HTTP/2 sessions per worker with given
backend addresses. New option to specify the number of HTTP/2 session
per worker will follow.
It turns out that writing successfully to network is not enough.
After apparently successful network write, read fails and then we
first know network has been lost (at least my android mobile network).
In this change, we say connection check is successful only when
successful read. We already send PING in this case, so we just wait
PING ACK with short timeout. If timeout has expired, drop connection.
Since waiting for PING ACK could degrade performance for fast reliably
connected network, we decided to disable connection check by default.
Use --backend-http2-connection-check to enable it.
When same SSL_CTX is used by multiple thread simultaneously we have to
setup some number of mutex locks for it. We could not check how this
locking affects scalability since we have 4 cores at best in our
development machine. Good side of sharing SSL_CTX across threads is
we can share session ID pool.
If --tls-ctx-per-worker is enabled, SSL_CTX is created per thread
basis and we can eliminate mutex locks. The downside is session ID is
no longer shared, which means if session ID generated by one thread
cannot be acceptable by another thread. But we have now session
ticket enabled and its keys are shared by all threads.
This option specifies files contains 48 random bytes to construct
session ticket key data. This option can be used repeatedly to
specify multiple keys, but only the first one is used to encrypt
tickets.
Update is done by main event loop which is stopped after graceful
shutdown is commenced, which means time is no longer update. To avoid
this situation, we just avoid caching and get time for each logging.
For HTTP/1 backend, -b option can be used several times to specify
multiple backend address. HTTP/2 backend does not support multiple
addresses and only uses first address even if multiple addresses are
specified.
This commit limits the number of concurrent HTTP/1 downstream
connections to same host. By defualt, it is limited to 8 connections.
--backend-connections-per-frontend option was replaced with
--backend-http1-connections-per-host, which changes the maximum number
of connections per host. This limitation only kicks in when h2 proxy
is used (-s option).
This commit adds functionality to customize access logging format in
nghttpx. The format variables are inspired by nginx. The default
format is combined format.
This option limits the number of backend connections per frontend.
This is meaningful for the combination of HTTP/2 and SPDY frontend and
HTTP/1 backend.
nghttpx supports hot deploy feature using signals. The host deploy in
nghttpx is multi step process. First send USR2 signal to nghttpx
process. It will do fork and execute new executable, using same
command-line arguments and environment variables. At this point, both
current and new processes can accept requests. To gracefully shutdown
current process, send QUIT signal to current nghttpx process. When
all existing frontend connections are done, the current process will
exit. At this point, only new nghttpx process exists and serves
incoming requests.
--no-location-rewrite option disallows location header rewrite on
--http2-bridge, --client and default mode. This option is useful when
connecting nghttpx proxy with --http2-bridge to backend nghttpx with
http2-proxy mode.
This change rewrites logging system of nghttpx. Previously access log
and error log are written to stderr or syslog and there was no option
to change stderr to something else. With this change, file path of
access log and error log can be configured separately and logging to
regular file is now added. To support rotating log, if SIGUSR1 signal
is received by nghttpx, it closes the current log files and reopen it
with the same name. The format of access log is changed and has same
look of apache's. But not all columns are not supported yet.
It looks like setting read-rate and read-burst to 0 makes busy loop.
It seems a bug. On the other hand, we most likely want per-thread
rate limit rather than per-connection. So we decided to drop them.
Cipher suites are chosen by DHE and ECDHE ciphers + GCM (AEAD). Now
default cipher list is the one recommended by Mozilla web site. The
--honor-cipher-order option is removed and now it is always assumed.
To make adding new option easier, we decided to make the details of
option struct private and hide it from public API. We provide
functions to set individual option value.
The existing options --{read,write}-{rate,burst} are per connection.
The new options --worker-{read,write}-{rate,burst} are per worker
thread, which is overall rate limit of all connections worker handles.
This commit also changes SPDY's flow control size. Previously,
the size for SPDY is the same amount of bytes with HTTP/2.
For example, --frontend-http2-upstream-window-bits=N,
the window size is 2**N - 1. Now SPDY code uses 2**N.
Using --cacert to load certificate for client certificate authentication
is problematic since, --cacert is also used for client mode.
This commit adds --verify-client-cacert option which specify the CA
certficate file used only for client certificate validation.
This change also removes the default certficate load function for
client certificate validation.
Use --dh-param-file option to specify a file including DH parameters
in PEM format.
For example, you can create DH parameters with 1024 bit key using
following command:
$ openssl dhparam -outform PEM -out dhparam.pem 1024
shrpx:
* Added an option to set the TLS SNI extension between shrpx and the
origin on the command line
spdycat:
* If the user set an explicit host header ( using --headers ) use that
name for the TLS SNI extension.
* Added the handshake completion time to the verbose output
* The gettimeofday call in get_time was using the incorrect structure
( I believe )
* In update_html_parser it was submitting the request regardless of
the return value of add_request.
Patch from Stephen Ludin
Specify proxy URI in the form http://[USER:PASS]PROXY:PORT. USER and
PASS are optional and if they exist they must be properly
percent-encoded. This proxy is used when the backend connection is
SPDY. First, make a CONNECT request to the proxy and it connects to
the backend on behalf of shrpx. This forms tunnel. After that, shrpx
performs SSL/TLS handshake with the downstream through the tunnel. The
timeouts when connecting and making CONNECT request can be specified
by --backend-read-timeout and --backend-write-timeout options.
With --spdy-bridge option, it listens SPDY/HTTPS connections from
front end and forwards them to the backend in SPDY. The usage will be
written later. This change fixes the crash when more than 2
outstanding SpdyDownstreamConnection objects are added to SpdySession
and establishing connection to SPDY backend is failed.
This option specifies additional certificate and private key
file. Shrpx will choose certificates based on the hostname indicated
by client using TLS SNI extension. This option can be used multiple
times.
This avoids the need to provide the password for your
private key interactively.
It can be used via --private-key-passwd-file or private-key-passwd-file
in the given config file. The first line in the file
(without \n) will be treated as the passwd. There isn't
any validation and all lines after the first one (if any)
are ignored.
The security model behind this is a bit simplistic so I
am open to better ideas. Basically your password file
should be root:root (700) and you *should* drop root
and run as an unprivileged user.
If the file exists and a line can be read then a callback
will be set for the SSL ctxt and it'll feed the passwd
when the private key is read (if password is needed).
If the file exists with the wrong permisions it'll be
logged and ignored.
The -k, --insecure option is added to skip this verification. The
system wide trusted CA certificates will be loaded at startup. The
--cacert option is added to specify the trusted CA certificate file.
With --client-mode option, shrpx now accepts unencrypted HTTP
connections and communicates with backend server in SPDY. In short,
this is the "reversed" operation mode against normal mode. This may
be useful for testing purpose because it can sit between HTTP client
and shrpx "normal" mode.
To distinguish the to-be-installed programs and non-installable
example source code, the former programs, spdycat, spdydyd and shrpx,
were moved to src directory. spdynative was removed from Makefile
because it does not appeal to any users much.
Tatsuhiro Tsujikawa
Lucas Pardue
Peeyush Aggarwal
Fabian Möller
moparisthebest
Raul Gutierrez Segales