In reverse proxy usage, backend server most likely wants to see the
original header field. So this commit turns off host header rewrite
by default. --no-host-rewrite option is deprecated, and if it is
used, warning message is displayed. --host-rewrite option is added to
enable host rewrite.
To achieve host-path backend routing, we changed behaviour of
--backend-http2-connections-per-worker. It now sets the number of
HTTP/2 physical connections per pattern group if pattern is used in -b
option.
Fixes GH-292
If pattern ends with '/', and pattern and path matches without that
slash, we consider they match to deal with request to the directory
without trailing slash. That is if pattern is "/foo/" and path is
"/foo", we consider they match.
-b option syntax is now <HOST>,<PORT>[;<PATTERN>[:...]]. The optional
<PATTERN>s specify the request host and path it is used for. The
<PATTERN> can contain path, host + path or host. The matching rule is
closely designed to ServeMux in Go programming language.
This commit add following 3 log variables to SSL/TLS connection:
$ssl_cipher, $ssl_protocol, $ssl_session_id. If no information is
available for them, '-' is produced for each.
From autoconf manual, section 5.6.1 Portability of Headers, says:
"""
The C99 standard says that inttypes.h includes stdint.h, so there's no
need to include stdint.h separately in a standard environment. Some
implementations have inttypes.h but not stdint.h (e.g., Solaris 7),
but we don't know of any implementation that has stdint.h but not
inttypes.h.
"""
Since libev handles SIGCHLD, using waitpid in separate thread to wait
for the completion of fetch-ocsp-response script process is undefined.
This commit rewrite ocsp handling code so that it utilizes libev
ev_child watcher and perform ocsp update without thread.
Previously nghttp2_session_send() and nghttp2_session_mem_send() did
not send 24 bytes client magic byte string (MAGIC). We made
nghttp2_session_recv() and nghttp2_session_mem_recv() process MAGIC by
default, so it is natural to make library send MAGIC as well. This
commit makes nghttp2_session_send() and nghttp2_session_mem_send()
send MAGIC. This commit also replace "connection preface" with
"client magic", since we call MAGIC as "connection preface" but it is
just a part of connection preface. NGHTTP2_CLIENT_CONNECTION_PREFACE
macro was replaced with NGHTTP2_CLIENT_MAGIC. The already deprecated
NGHTTP2_CLIENT_CONNECTION_HEADER macro was removed permanently.
nghttp2_option_set_no_recv_client_preface() was renamed as
nghttp2_option_set_no_recv_client_magic(). NGHTTP2_ERR_BAD_PREFACE
was renamed as NGHTTP2_ERR_BAD_CLIENT_MAGIC.
Since HTTP/2 spec requires for client to send connection preface, it
is reasonable to make this option enabled by default. It is still a
use case to disable this, so replace this option with
nghttp2_option_set_no_recv_client_preface().
HTTP/2 and HPACK are going to be published as RFC, but ALTSVC is still
in draft state. To make our API stable, it would be better to remove
ALTSVC API for 1.0.0 release.
Currently, we use same number of HTTP/2 sessions per worker with given
backend addresses. New option to specify the number of HTTP/2 session
per worker will follow.
It turns out that writing successfully to network is not enough.
After apparently successful network write, read fails and then we
first know network has been lost (at least my android mobile network).
In this change, we say connection check is successful only when
successful read. We already send PING in this case, so we just wait
PING ACK with short timeout. If timeout has expired, drop connection.
Since waiting for PING ACK could degrade performance for fast reliably
connected network, we decided to disable connection check by default.
Use --backend-http2-connection-check to enable it.
When same SSL_CTX is used by multiple thread simultaneously we have to
setup some number of mutex locks for it. We could not check how this
locking affects scalability since we have 4 cores at best in our
development machine. Good side of sharing SSL_CTX across threads is
we can share session ID pool.
If --tls-ctx-per-worker is enabled, SSL_CTX is created per thread
basis and we can eliminate mutex locks. The downside is session ID is
no longer shared, which means if session ID generated by one thread
cannot be acceptable by another thread. But we have now session
ticket enabled and its keys are shared by all threads.
Previously to create manual page for bundled programs, we use help2man
to create man page from program's help output. Then our man2rst.py
script converts man page to rst document. Sphinx generates html from
rst documents.
Now help2rst.py produces rst document from programs output. We use
Sphinx solely to produce both man pages and html files.
This option specifies files contains 48 random bytes to construct
session ticket key data. This option can be used repeatedly to
specify multiple keys, but only the first one is used to encrypt
tickets.
Update is done by main event loop which is stopped after graceful
shutdown is commenced, which means time is no longer update. To avoid
this situation, we just avoid caching and get time for each logging.
The nghttp2 library itself is still h2-14. To experiment with the
implementations to require h2-16 to test new features (e.g.,
prioritization), nghttp, nghttpx, nghttpd and h2load now support h2-16
as well as h2-14. Cleartext HTTP Upgrade is still limited to h2-14
however.
For HTTP/1 backend, -b option can be used several times to specify
multiple backend address. HTTP/2 backend does not support multiple
addresses and only uses first address even if multiple addresses are
specified.
This commit limits the number of concurrent HTTP/1 downstream
connections to same host. By defualt, it is limited to 8 connections.
--backend-connections-per-frontend option was replaced with
--backend-http1-connections-per-host, which changes the maximum number
of connections per host. This limitation only kicks in when h2 proxy
is used (-s option).
This is unfortunate but help2man behaves badly when there is indented
lines in help messages in commnad-line args. We removed indentations
to make help2man happy.
This commit adds functionality to customize access logging format in
nghttpx. The format variables are inspired by nginx. The default
format is combined format.
This is simply programming error, but it is interesting that using
libstdc++ does not reveal this error. With clang++-libc++, we got
std::system_error: mutex lock faild: Invalid argument. This is
because we did not give a name to lock object, so it is immediately
destructed. I think this will fix the reported crash on Mac OSX.
Android lacks /dev/stderr, so directly use /proc/self/fd/2 as default
errorlog-file. Android does not like O_APPEND for /proc/self/fd/1 and
/proc/self/fd/2, so omit the flag for these paths.
This option limits the number of backend connections per frontend.
This is meaningful for the combination of HTTP/2 and SPDY frontend and
HTTP/1 backend.
nghttpx supports hot deploy feature using signals. The host deploy in
nghttpx is multi step process. First send USR2 signal to nghttpx
process. It will do fork and execute new executable, using same
command-line arguments and environment variables. At this point, both
current and new processes can accept requests. To gracefully shutdown
current process, send QUIT signal to current nghttpx process. When
all existing frontend connections are done, the current process will
exit. At this point, only new nghttpx process exists and serves
incoming requests.
--no-location-rewrite option disallows location header rewrite on
--http2-bridge, --client and default mode. This option is useful when
connecting nghttpx proxy with --http2-bridge to backend nghttpx with
http2-proxy mode.
It might be useful to clean the unused stream out to make up the room
for new streams. On the other hand, proxy should maintain the
connection between upstream client and downstream server and they have
the timeout for their own. Proxy just reacts to their decision.
Reworked no automatic WINDOW_UPDATE feature. We added new API
nghttp2_session_consume() which tells the library how many bytes are
consumed by the application. Instead of submitting WINDOW_UPDATE by
the application, the library is now responsible to submit
WINDOW_UPDATE based on consumed bytes. This is more reliable method,
since it enables us to properly send WINDOW_UPDATE for stream and
connection individually. The previous implementation of nghttpx had
broken connection window management.
This change rewrites logging system of nghttpx. Previously access log
and error log are written to stderr or syslog and there was no option
to change stderr to something else. With this change, file path of
access log and error log can be configured separately and logging to
regular file is now added. To support rotating log, if SIGUSR1 signal
is received by nghttpx, it closes the current log files and reopen it
with the same name. The format of access log is changed and has same
look of apache's. But not all columns are not supported yet.
It looks like setting read-rate and read-burst to 0 makes busy loop.
It seems a bug. On the other hand, we most likely want per-thread
rate limit rather than per-connection. So we decided to drop them.
Cipher suites are chosen by DHE and ECDHE ciphers + GCM (AEAD). Now
default cipher list is the one recommended by Mozilla web site. The
--honor-cipher-order option is removed and now it is always assumed.
It seems that specifyig '*' to node parameter in getaddrinfo() is
treated as specifying NULL, but it is not documented. So rather than
relying on this feature, we explicitly treat '*' as "wildcard" address
and specify NULL to node parameter in getaddrinfo().
Now '*,3000' is a default value of --frontend option. Specyfing '*'
binds all addresses including both IPv4 and IPv6.
To make adding new option easier, we decided to make the details of
option struct private and hide it from public API. We provide
functions to set individual option value.
The existing options --{read,write}-{rate,burst} are per connection.
The new options --worker-{read,write}-{rate,burst} are per worker
thread, which is overall rate limit of all connections worker handles.
This commit also changes SPDY's flow control size. Previously,
the size for SPDY is the same amount of bytes with HTTP/2.
For example, --frontend-http2-upstream-window-bits=N,
the window size is 2**N - 1. Now SPDY code uses 2**N.
Using --cacert to load certificate for client certificate authentication
is problematic since, --cacert is also used for client mode.
This commit adds --verify-client-cacert option which specify the CA
certficate file used only for client certificate validation.
This change also removes the default certficate load function for
client certificate validation.
Use --dh-param-file option to specify a file including DH parameters
in PEM format.
For example, you can create DH parameters with 1024 bit key using
following command:
$ openssl dhparam -outform PEM -out dhparam.pem 1024
shrpx:
* Added an option to set the TLS SNI extension between shrpx and the
origin on the command line
spdycat:
* If the user set an explicit host header ( using --headers ) use that
name for the TLS SNI extension.
* Added the handshake completion time to the verbose output
* The gettimeofday call in get_time was using the incorrect structure
( I believe )
* In update_html_parser it was submitting the request regardless of
the return value of add_request.
Patch from Stephen Ludin
Specify proxy URI in the form http://[USER:PASS]PROXY:PORT. USER and
PASS are optional and if they exist they must be properly
percent-encoded. This proxy is used when the backend connection is
SPDY. First, make a CONNECT request to the proxy and it connects to
the backend on behalf of shrpx. This forms tunnel. After that, shrpx
performs SSL/TLS handshake with the downstream through the tunnel. The
timeouts when connecting and making CONNECT request can be specified
by --backend-read-timeout and --backend-write-timeout options.
With --spdy-bridge option, it listens SPDY/HTTPS connections from
front end and forwards them to the backend in SPDY. The usage will be
written later. This change fixes the crash when more than 2
outstanding SpdyDownstreamConnection objects are added to SpdySession
and establishing connection to SPDY backend is failed.
This option specifies additional certificate and private key
file. Shrpx will choose certificates based on the hostname indicated
by client using TLS SNI extension. This option can be used multiple
times.
INFO log and its surrounding code are now guarded by
LOG_ENABLED(SEVERITY) macro so that they don't run if log level
threshold is higher. This increases performance because log formatting
is somewhat expensive.
This avoids the need to provide the password for your
private key interactively.
It can be used via --private-key-passwd-file or private-key-passwd-file
in the given config file. The first line in the file
(without \n) will be treated as the passwd. There isn't
any validation and all lines after the first one (if any)
are ignored.
The security model behind this is a bit simplistic so I
am open to better ideas. Basically your password file
should be root:root (700) and you *should* drop root
and run as an unprivileged user.
If the file exists and a line can be read then a callback
will be set for the SSL ctxt and it'll feed the passwd
when the private key is read (if password is needed).
If the file exists with the wrong permisions it'll be
logged and ignored.
The -k, --insecure option is added to skip this verification. The
system wide trusted CA certificates will be loaded at startup. The
--cacert option is added to specify the trusted CA certificate file.
In client mode, now SPDY connection to the backend server is
established per thread. The frontend connections which belong to the
same thread share the SPDY connection.
With --client-mode option, shrpx now accepts unencrypted HTTP
connections and communicates with backend server in SPDY. In short,
this is the "reversed" operation mode against normal mode. This may
be useful for testing purpose because it can sit between HTTP client
and shrpx "normal" mode.
We should only call daemon() after ListenHandler is
instantiated, where SSL_CTX_use_PrivateKey_file is called,
otherwise we have no stdin/stdout to get the password for
keyfile.
To distinguish the to-be-installed programs and non-installable
example source code, the former programs, spdycat, spdydyd and shrpx,
were moved to src directory. spdynative was removed from Makefile
because it does not appeal to any users much.
Tatsuhiro Tsujikawa
Tomasz Buchert
Stefan Eissing
Fabian Möller
Lucas Pardue
Svante Signell
Nicholas Hurley
moparisthebest
snnn
Raul Gutierrez Segales