walkero
/
nghttp2
1
0
Fork 0
Code Issues Pull Requests Projects Releases 1 Wiki Activity

Merge branch 'nghttpx-rewrite-h2-coalesce'

This commit is contained in:
Tatsuhiro Tsujikawa 2016-02-29 00:52:19 +09:00
parent 9eeac27966 c2ec73302d
commit a59445ccff
47 changed files with 1227 additions and 952 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

84
README.rst
View File

@ -655,46 +655,38 @@ HTTP/2. ``nghttpx`` also offers the functionality to share session
cache and ticket keys among multiple ``nghttpx`` instances via
memcached.
``nghttpx`` has several operational modes:
``nghttpx`` has 2 operation modes:
================== ============================ ============== =============
Mode option Frontend Backend Note
================== ============================ ============== =============
default mode HTTP/2, SPDY, HTTP/1.1 (TLS) HTTP/1.1 Reverse proxy
``--http2-proxy`` HTTP/2, SPDY, HTTP/1.1 (TLS) HTTP/1.1 SPDY proxy
``--http2-bridge`` HTTP/2, SPDY, HTTP/1.1 (TLS) HTTP/2 (TLS)
``--client`` HTTP/2, HTTP/1.1 HTTP/2 (TLS)
``--client-proxy`` HTTP/2, HTTP/1.1 HTTP/2 (TLS) Forward proxy
================== ============================ ============== =============
================== ====================== =================== =============
Mode option Frontend Backend Note
================== ====================== =================== =============
default mode HTTP/2, SPDY, HTTP/1.1 HTTP/1.1, HTTP/2 Reverse proxy
``--http2-proxy`` HTTP/2, SPDY, HTTP/1.1 HTTP/1.1, or HTTP/2 Forward proxy
================== ====================== =================== =============
The interesting mode at the moment is the default mode. It works like
a reverse proxy and listens for HTTP/2, SPDY and HTTP/1.1 and can be
deployed as a SSL/TLS terminator for existing web server.
The default mode, ``--http2-proxy`` and ``--http2-bridge`` modes use
SSL/TLS in the frontend connection by default. To disable SSL/TLS,
use the ``--frontend-no-tls`` option. If that option is used, SPDY is
disabled in the frontend and incoming HTTP/1.1 connections can be
upgraded to HTTP/2 through HTTP Upgrade. In these modes, HTTP/1
backend connections are cleartext by default. To enable TLS, use
``--backend-http1-tls`` opiton.
The ``--http2-bridge``, ``--client`` and ``--client-proxy`` modes use
SSL/TLS in the backend connection by default. To disable SSL/TLS, use
the ``--backend-no-tls`` option.
In all modes, the frontend connections are encrypted by SSL/TLS by
default. To disable encryption, use the ``--frontend-no-tls`` option.
If encryption is disabled, SPDY is disabled in the frontend and
incoming HTTP/1.1 connections can be upgraded to HTTP/2 through HTTP
Upgrade. On the other hard, backend connections are not encrypted by
default. To encrypt backend connections, use ``--backend-tls``
option.
``nghttpx`` supports a configuration file. See the ``--conf`` option and
sample configuration file ``nghttpx.conf.sample``.
In the default mode, (without any of ``--http2-proxy``,
``--http2-bridge``, ``--client-proxy`` and ``--client`` options),
``nghttpx`` works as reverse proxy to the backend server::
In the default mode, ``nghttpx`` works as reverse proxy to the backend
server::
Client <-- (HTTP/2, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/1.1) --> Web Server
Client <-- (HTTP/2, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/1.1, HTTP/2) --> Web Server
[reverse proxy]
With the ``--http2-proxy`` option, it works as a so called secure proxy (aka
SPDY proxy)::
With the ``--http2-proxy`` option, it works as forward proxy, and it
is so called secure HTTP/2 proxy (aka SPDY proxy)::
Client <-- (HTTP/2, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/1.1) --> Proxy
[secure proxy] (e.g., Squid, ATS)
@ -702,9 +694,9 @@ SPDY proxy)::
The ``Client`` in the above example needs to be configured to use
``nghttpx`` as secure proxy.
At the time of this writing, Chrome is the only browser which supports
secure proxy. One way to configure Chrome to use a secure proxy is
to create a proxy.pac script like this:
At the time of this writing, both Chrome and Firefox support secure
HTTP/2 proxy. One way to configure Chrome to use a secure proxy is to
create a proxy.pac script like this:
.. code-block:: javascript
@ -720,37 +712,9 @@ Then run Chrome with the following arguments::
$ google-chrome --proxy-pac-url=file:///path/to/proxy.pac --use-npn
With ``--http2-bridge``, it accepts HTTP/2, SPDY and HTTP/1.1
connections and communicates with the backend in HTTP/2::
Client <-- (HTTP/2, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/2) --> Web or HTTP/2 Proxy etc
(e.g., nghttpx -s)
With ``--client-proxy``, it works as a forward proxy and expects
that the backend is an HTTP/2 proxy::
Client <-- (HTTP/2, HTTP/1.1) --> nghttpx <-- (HTTP/2) --> HTTP/2 Proxy
[forward proxy] (e.g., nghttpx -s)
The ``Client`` needs to be configured to use nghttpx as a forward
proxy. The frontend HTTP/1.1 connection can be upgraded to HTTP/2
through HTTP Upgrade. With the above configuration, one can use
HTTP/1.1 client to access and test their HTTP/2 servers.
With ``--client``, it works as a reverse proxy and expects that
the backend is an HTTP/2 Web server::
Client <-- (HTTP/2, HTTP/1.1) --> nghttpx <-- (HTTP/2) --> Web Server
[reverse proxy]
The frontend HTTP/1.1 connection can be upgraded to HTTP/2
through HTTP Upgrade.
For the operation modes which talk to the backend in HTTP/2 over
SSL/TLS, the backend connections can be tunneled through an HTTP proxy.
The backend HTTP/2 connections can be tunneled through an HTTP proxy.
The proxy is specified using ``--backend-http-proxy-uri``. The
following figure illustrates the example of the ``--http2-bridge`` and
``--backend-http-proxy-uri`` options to talk to the outside HTTP/2
following figure illustrates how nghttpx talks to the outside HTTP/2
proxy through an HTTP proxy::
Client <-- (HTTP/2, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/2) --

10
doc/nghttpx.h2r
View File

@ -98,12 +98,12 @@ Currently, the following restriction is applied for server push:
This limitation may be loosened in the future release.
nghttpx also supports server push if both frontend and backend are
HTTP/2 (which implies :option:`--http2-bridge` or :option:`--client`).
In this case, in addition to server push via Link header field, server
push from backend is relayed to frontend HTTP/2 session.
HTTP/2 in default mode. In this case, in addition to server push via
Link header field, server push from backend is forwarded to frontend
HTTP/2 session.
HTTP/2 server push will be disabled if :option:`--http2-proxy` or
:option:`--client-proxy` is used.
HTTP/2 server push will be disabled if :option:`--http2-proxy` is
used.
UNIX DOMAIN SOCKET
------------------

230
doc/sources/nghttpx-howto.rst
View File

@ -12,37 +12,38 @@ use-cases. It also covers some useful options later.
Default mode
------------
If nghttpx is invoked without any :option:`--http2-proxy`,
:option:`--client`, and :option:`--client-proxy`, it operates in
default mode. In this mode, nghttpx frontend listens for HTTP/2
requests and translates them to HTTP/1 requests. Thus it works as
reverse proxy (gateway) for HTTP/2 clients to HTTP/1 web server. This
is also known as "HTTP/2 router". HTTP/1 requests are also supported
in frontend as a fallback. If nghttpx is linked with spdylay library
and frontend connection is SSL/TLS, the frontend also supports SPDY
If nghttpx is invoked without :option:`--http2-proxy`, it operates in
default mode. In this mode, it works as reverse proxy (gateway) for
both HTTP/2 and HTTP/1 clients to backend servers. This is also known
as "HTTP/2 router". If nghttpx is linked with spdylay library and
frontend connection is SSL/TLS, the frontend also supports SPDY
protocol.
By default, this mode's frontend connection is encrypted using
SSL/TLS. So server's private key and certificate must be supplied to
the command line (or through configuration file). In this case, the
frontend protocol selection will be done via ALPN or NPN.
By default, frontend connection is encrypted using SSL/TLS. So
server's private key and certificate must be supplied to the command
line (or through configuration file). In this case, the frontend
protocol selection will be done via ALPN or NPN.
With :option:`--frontend-no-tls` option, user can turn off SSL/TLS in
frontend connection. In this case, SPDY protocol is not available
even if spdylay library is liked to nghttpx. HTTP/2 and HTTP/1 are
available on the frontend and a HTTP/1 connection can be upgraded to
available on the frontend, and an HTTP/1 connection can be upgraded to
HTTP/2 using HTTP Upgrade. Starting HTTP/2 connection by sending
HTTP/2 connection preface is also supported.
By default, backend HTTP/1 connections are not encrypted. To enable
TLS on HTTP/1 backend connections, use :option:`--backend-http1-tls`
option. This applies to all mode whose backend connections are
HTTP/1.
By default, backend connections are not encrypted. To enable TLS
encryption on backend connections, use :option:`--backend-tls` option.
Using patterns and ``proto`` keyword in :option:`--backend` option,
backend application protocol can be specified per host/request path
pattern. It means that you can use both HTTP/2 and HTTP/1 in backend
connections at the same time. Note that default backend protocol is
HTTP/1.1. To use HTTP/2 in backend, you have to specify ``h2`` in
``proto`` keyword in :option:`--backend` explicitly.
The backend is supposed to be HTTP/1 Web server. For example, to make
The backend is supposed to be Web server. For example, to make
nghttpx listen to encrypted HTTP/2 requests at port 8443, and a
backend HTTP/1 web server is configured to listen to HTTP/1 request at
port 8080 in the same host, run nghttpx command-line like this::
backend Web server is configured to listen to HTTP request at port
8080 in the same host, run nghttpx command-line like this::
$ nghttpx -f0.0.0.0,8443 -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt
@ -58,8 +59,8 @@ If nghttpx is invoked with :option:`--http2-proxy` (or its shorthand
:option:`-s`) option, it operates in HTTP/2 proxy mode. The supported
protocols in frontend and backend connections are the same in `default
mode`_. The difference is that this mode acts like forward proxy and
assumes the backend is HTTP/1 proxy server (e.g., squid, traffic
server). So HTTP/1 request must include absolute URI in request line.
assumes the backend is HTTP proxy server (e.g., Squid, Apache Traffic
Server). HTTP/1 request must include absolute URI in request line.
By default, frontend connection is encrypted. So this mode is also
called secure proxy. If nghttpx is linked with spdylay, it supports
@ -68,16 +69,22 @@ SPDY protocols and it works as so called SPDY proxy.
With :option:`--frontend-no-tls` option, SSL/TLS is turned off in
frontend connection, so the connection gets insecure.
The backend must be HTTP/1 proxy server. nghttpx supports multiple
backend server addresses. It translates incoming requests to HTTP/1
The backend must be HTTP proxy server. nghttpx supports multiple
backend server addresses. It translates incoming requests to HTTP
request to backend server. The backend server performs real proxy
work for each request, for example, dispatching requests to the origin
server and caching contents.
The backend connection is not encrypted by default. To enable
encryption, use :option:`--backend-tls` option. The default backend
protocol is HTTP/1.1. To use HTTP/2 in backend connection, use
:option:`--backend` option, and specify ``h2`` in ``proto`` keyword
explicitly.
For example, to make nghttpx listen to encrypted HTTP/2 requests at
port 8443, and a backend HTTP/1 proxy server is configured to listen
to HTTP/1 request at port 8080 in the same host, run nghttpx
command-line like this::
port 8443, and a backend HTTP proxy server is configured to listen to
HTTP/1 request at port 8080 in the same host, run nghttpx command-line
like this::
$ nghttpx -s -f'*,8443' -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt
@ -122,132 +129,19 @@ Consult Traffic server `documentation
to know how to configure traffic server as forward proxy and its
security implications.
Client mode
-----------
Disable frontend SSL/TLS
------------------------
If nghttpx is invoked with :option:`--client` option, it operates in
client mode. In this mode, nghttpx listens for plain, unencrypted
HTTP/2 and HTTP/1 requests and translates them to encrypted HTTP/2
requests to the backend. User cannot enable SSL/TLS in frontend
connection.
The frontend connections are encrypted with SSL/TLS by default. To
turn off SSL/TLS, use :option:`--frontend-no-tls` option. If this
option is used, the private key and certificate are not required to
run nghttpx.
HTTP/1 frontend connection can be upgraded to HTTP/2 using HTTP
Upgrade. To disable SSL/TLS in backend connection, use
:option:`--backend-no-tls` option.
Enable backend SSL/TLS
----------------------
By default, the number of backend HTTP/2 connections per worker
(thread) is determined by number of :option:`--backend` option. To
adjust this value, use
:option:`--backend-http2-connections-per-worker` option.
The backend server is supporsed to be a HTTP/2 web server (e.g.,
nghttpd). The one use-case of this mode is utilize existing HTTP/1
clients to test HTTP/2 deployment. Suppose that HTTP/2 web server
listens to port 80 without encryption. Then run nghttpx as client
mode to access to that web server::
$ nghttpx --client -f127.0.0.1,8080 -b127.0.0.1,80 --backend-no-tls
.. note::
You may need :option:`--insecure` (or its shorthand :option:`-k`)
option if HTTP/2 server enables SSL/TLS and its certificate is
self-signed. But please note that it is insecure, and you should
know what you are doing.
Then you can use curl to access HTTP/2 server via nghttpx::
$ curl http://localhost:8080/
Client proxy mode
-----------------
If nghttpx is invoked with :option:`--client-proxy` (or its shorthand
:option:`-p`) option, it operates in client proxy mode. This mode
behaves like `client mode`_, but it works like forward proxy. So
HTTP/1 request must include absolute URI in request line.
HTTP/1 frontend connection can be upgraded to HTTP/2 using HTTP
Upgrade. To disable SSL/TLS in backend connection, use
:option:`--backend-no-tls` option.
By default, the number of backend HTTP/2 connections per worker
(thread) is determined by number of :option:`--backend` option. To
adjust this value, use
:option:`--backend-http2-connections-per-worker` option.
The backend server must be a HTTP/2 proxy. You can use nghttpx in
`HTTP/2 proxy mode`_ as backend server. The one use-case of this mode
is utilize existing HTTP/1 clients to test HTTP/2 connections between
2 proxies. The another use-case is use this mode to aggregate local
HTTP/1 connections to one HTTP/2 backend encrypted connection. This
makes HTTP/1 clients which does not support secure proxy can use
secure HTTP/2 proxy via nghttpx client mode.
Suppose that HTTP/2 proxy listens to port 8443, just like we saw in
`HTTP/2 proxy mode`_. To run nghttpx in client proxy mode to access
that server, invoke nghttpx like this::
$ nghttpx -p -f127.0.0.1,8080 -b127.0.0.1,8443
.. note::
You may need :option:`--insecure` (or its shorthand :option:`-k`)
option if HTTP/2 server's certificate is self-signed. But please
note that it is insecure, and you should know what you are doing.
Then you can use curl to issue HTTP request via HTTP/2 proxy::
$ curl --http-proxy=http://localhost:8080 http://www.google.com/
You can configure web browser to use localhost:8080 as forward
proxy.
HTTP/2 bridge mode
------------------
If nghttpx is invoked with :option:`--http2-bridge` option, it
operates in HTTP/2 bridge mode. The supported protocols in frontend
connections are the same in `default mode`_. The protocol in backend
is HTTP/2 only.
With :option:`--frontend-no-tls` option, SSL/TLS is turned off in
frontend connection, so the connection gets insecure. To disable
SSL/TLS in backend connection, use :option:`--backend-no-tls` option.
By default, the number of backend HTTP/2 connections per worker
(thread) is determined by number of :option:`--backend` option. To
adjust this value, use
:option:`--backend-http2-connections-per-worker` option.
The backend server is supporsed to be a HTTP/2 web server or HTTP/2
proxy. If backend server is HTTP/2 proxy, use
:option:`--no-location-rewrite` option to disable rewriting
``Location`` header field.
The use-case of this mode is aggregate the incoming connections to one
HTTP/2 connection. One backend HTTP/2 connection is created per
worker (thread).
Disable SSL/TLS
---------------
In `default mode`_, `HTTP/2 proxy mode`_ and `HTTP/2 bridge mode`_,
frontend connections are encrypted with SSL/TLS by default. To turn
off SSL/TLS, use :option:`--frontend-no-tls` option. If this option
is used, the private key and certificate are not required to run
nghttpx.
In `client mode`_, `client proxy mode`_ and `HTTP/2 bridge mode`_,
backend connections are encrypted with SSL/TLS by default. To turn
off SSL/TLS, use :option:`--backend-no-tls` option.
Enable SSL/TLS on HTTP/1 backend
--------------------------------
In all modes which use HTTP/1 as backend protocol, backend HTTP/1
connection is not encrypted by default. To enable encryption, use
:option:`--backend-http1-tls` option.
The backend connections are not encrypted by default. To enable
SSL/TLS encryption, :option:`--backend-tls` option.
Enable SSL/TLS on memcached connection
--------------------------------------
@ -387,5 +281,37 @@ servers ``serv1:3000`` and ``serv2:3000`` for request host
backend=serv1,3000;example.com/myservice
backend=serv2,3000;example.com/myservice
For HTTP/2 backend, see also
:option:`--backend-http2-connections-per-worker` option.
You can also specify backend application protocol in
:option:`--backend` option using ``proto`` keyword after pattern.
Utilizing this allows ngttpx to route certain request to HTTP/2, other
requests to HTTP/1. For example, to route requests to ``/ws/`` in
backend HTTP/1.1 connection, and use backend HTTP/2 for other
requests, do this:
.. code-block:: text
backend=serv1,3000;/;proto=h2
backend=serv1,3000;/ws/;proto=http/1.1
Note that the backends share the same pattern must have the same
backend protocol. The default backend protocol is HTTP/1.1.
Deprecated modes
----------------
As of nghttpx 1.9.0, ``--http2-bridge``, ``--client`` and
``--client-proxy`` options were removed. These functionality can be
used using combinations of options.
* ``--http2-bridge``: Use
:option:`--backend`\='-b<ADDR>,<PORT>;;proto=h2', and
:option:`--backend-tls`.
* ``--client``: Use :option:`--frontend-no-tls`,
:option:`--backend`\='-b<ADDR>,<PORT>;;proto=h2', and
:option:`--backend-tls`.
* ``--client-proxy``: Use :option:`--http2-proxy`,
:option:`--frontend-no-tls`,
:option:`--backend`\='-b<ADDR>,<PORT>;;proto=h2', and
:option:`--backend-tls`.

7
gennghttpxfun.py
View File

@ -122,7 +122,12 @@ OPTIONS = [
"tls-ticket-key-memcached-cert-file",
"tls-ticket-key-memcached-private-key-file",
"tls-ticket-key-memcached-address-family",
"backend-address-family"
"backend-address-family",
"frontend-http2-max-concurrent-streams",
"backend-http2-max-concurrent-streams",
"backend-connections-per-frontend",
"backend-tls",
"backend-connections-per-host"
]
LOGVARS = [

21
integration-tests/server_tester.go
View File

@ -6,10 +6,10 @@ import (
"crypto/tls"
"errors"
"fmt"
"golang.org/x/net/http2"
"golang.org/x/net/http2/hpack"
"github.com/tatsuhiro-t/go-nghttp2"
"github.com/tatsuhiro-t/spdy"
"golang.org/x/net/http2"
"golang.org/x/net/http2/hpack"
"golang.org/x/net/websocket"
"io"
"io/ioutil"
@ -30,7 +30,7 @@ const (
serverBin = buildDir + "/src/nghttpx"
serverPort = 3009
testDir = sourceDir + "/integration-tests"
logDir = buildDir + "/integration-tests"
logDir = buildDir + "/integration-tests"
)
func pair(name, value string) hpack.HeaderField {
@ -86,14 +86,18 @@ func newServerTesterTLSConfig(args []string, t *testing.T, handler http.HandlerF
// newServerTesterInternal creates test context. If frontendTLS is
// true, set up TLS frontend connection.
func newServerTesterInternal(args []string, t *testing.T, handler http.Handler, frontendTLS bool, clientConfig *tls.Config) *serverTester {
func newServerTesterInternal(src_args []string, t *testing.T, handler http.Handler, frontendTLS bool, clientConfig *tls.Config) *serverTester {
ts := httptest.NewUnstartedServer(handler)
args := []string{}
backendTLS := false
for _, k := range args {
for _, k := range src_args {
switch k {
case "--http2-bridge":
backendTLS = true
default:
args = append(args, k)
}
}
if backendTLS {
@ -102,9 +106,9 @@ func newServerTesterInternal(args []string, t *testing.T, handler http.Handler,
// NextProtos separately for ts.TLS. NextProtos set
// in nghttp2.ConfigureServer is effectively ignored.
ts.TLS = new(tls.Config)
ts.TLS.NextProtos = append(ts.TLS.NextProtos, "h2-14")
ts.TLS.NextProtos = append(ts.TLS.NextProtos, "h2")
ts.StartTLS()
args = append(args, "-k")
args = append(args, "-k", "--backend-tls")
} else {
ts.Start()
}
@ -124,6 +128,9 @@ func newServerTesterInternal(args []string, t *testing.T, handler http.Handler,
// URL.Host looks like "127.0.0.1:8080", but we want
// "127.0.0.1,8080"
b := "-b" + strings.Replace(backendURL.Host, ":", ",", -1)
if backendTLS {
b += ";;proto=h2"
}
args = append(args, fmt.Sprintf("-f127.0.0.1,%v", serverPort), b,
"--errorlog-file="+logDir+"/log.txt", "-LINFO")

1
src/Makefile.am
View File

@ -169,6 +169,7 @@ nghttpx_unittest_SOURCES = shrpx-unittest.cc \
shrpx_ssl_test.cc shrpx_ssl_test.h \
shrpx_downstream_test.cc shrpx_downstream_test.h \
shrpx_config_test.cc shrpx_config_test.h \
shrpx_worker_test.cc shrpx_worker_test.h \
shrpx_http_test.cc shrpx_http_test.h \
http2_test.cc http2_test.h \
util_test.cc util_test.h \

4
src/http2.cc
View File

@ -1416,9 +1416,9 @@ int lookup_method_token(const uint8_t *name, size_t namelen) {
return -1;
}
const char *to_method_string(int method_token) {
StringRef to_method_string(int method_token) {
// we happened to use same value for method with http-parser.
return http_method_str(static_cast<http_method>(method_token));
return StringRef{http_method_str(static_cast<http_method>(method_token))};
}
int get_pure_path_component(const char **base, size_t *baselen,

6
src/http2.h
View File

@ -322,7 +322,11 @@ bool expect_response_body(int status_code);
int lookup_method_token(const uint8_t *name, size_t namelen);
int lookup_method_token(const std::string &name);
const char *to_method_string(int method_token);
// Returns string representation of |method_token|. This is wrapper
// function over http_method_str from http-parser. If |method_token|
// is not known to http-parser, "<unknown>" is returned. The returned
// StringRef is guaranteed to be NULL-terminated.
StringRef to_method_string(int method_token);
template <typename InputIt>
std::string normalize_path(InputIt first, InputIt last) {

5
src/shrpx-unittest.cc
View File

@ -33,6 +33,7 @@
#include "shrpx_ssl_test.h"
#include "shrpx_downstream_test.h"
#include "shrpx_config_test.h"
#include "shrpx_worker_test.h"
#include "http2_test.h"
#include "util_test.h"
#include "nghttp2_gzip_test.h"
@ -118,8 +119,8 @@ int main(int argc, char *argv[]) {
shrpx::test_shrpx_config_read_tls_ticket_key_file) ||
!CU_add_test(pSuite, "config_read_tls_ticket_key_file_aes_256",
shrpx::test_shrpx_config_read_tls_ticket_key_file_aes_256) ||
!CU_add_test(pSuite, "config_match_downstream_addr_group",
shrpx::test_shrpx_config_match_downstream_addr_group) ||
!CU_add_test(pSuite, "worker_match_downstream_addr_group",
shrpx::test_shrpx_worker_match_downstream_addr_group) ||
!CU_add_test(pSuite, "http_create_forwarded",
shrpx::test_shrpx_http_create_forwarded) ||
!CU_add_test(pSuite, "util_streq", shrpx::test_util_streq) ||

265
src/shrpx.cc
View File

@ -656,7 +656,7 @@ int create_tcp_server_socket(UpstreamAddr &faddr,
}
faddr.fd = fd;
faddr.hostport = util::make_http_hostport(host.data(), faddr.port);
faddr.hostport = util::make_http_hostport(StringRef{host.data()}, faddr.port);
LOG(NOTICE) << "Listening on " << faddr.hostport;
@ -1079,7 +1079,8 @@ void fill_default_config() {
tlsconf.session_timeout = std::chrono::hours(12);
auto &httpconf = mod_config()->http;
httpconf.server_name = "nghttpx nghttp2/" NGHTTP2_VERSION;
httpconf.server_name =
StringRef::from_lit("nghttpx nghttp2/" NGHTTP2_VERSION);
httpconf.no_host_rewrite = true;
httpconf.request_header_field_buffer = 64_k;
httpconf.max_request_header_fields = 100;
@ -1096,6 +1097,7 @@ void fill_default_config() {
// HTTP/2 SPDY/3.1 has connection-level flow control. The default
// window size for HTTP/2 is 64KiB - 1. SPDY/3's default is 64KiB
upstreamconf.connection_window_bits = 16;
upstreamconf.max_concurrent_streams = 100;
nghttp2_option_new(&upstreamconf.option);
nghttp2_option_set_no_auto_window_update(upstreamconf.option, 1);
@ -1105,15 +1107,14 @@ void fill_default_config() {
{
auto &downstreamconf = http2conf.downstream;
downstreamconf.window_bits = 16;
downstreamconf.connection_window_bits = 16;
downstreamconf.connection_window_bits = 30;
downstreamconf.max_concurrent_streams = 100;
nghttp2_option_new(&downstreamconf.option);
nghttp2_option_set_no_auto_window_update(downstreamconf.option, 1);
nghttp2_option_set_peer_max_concurrent_streams(downstreamconf.option, 100);
}
http2conf.max_concurrent_streams = 100;
auto &loggingconf = mod_config()->logging;
{
auto &accessconf = loggingconf.access;
@ -1165,6 +1166,7 @@ void fill_default_config() {
downstreamconf.request_buffer_size = 16_k;
downstreamconf.response_buffer_size = 128_k;
downstreamconf.family = AF_UNSPEC;
downstreamconf.no_tls = true;
}
}
@ -1188,48 +1190,49 @@ void print_help(std::ostream &out) {
print_usage(out);
out << R"(
<PRIVATE_KEY>
Set path to server's private key. Required unless -p,
--client or --frontend-no-tls are given.
<CERT> Set path to server's certificate. Required unless -p,
--client or --frontend-no-tls are given. To make OCSP
stapling work, this must be absolute path.
Set path to server's private key. Required unless
--frontend-no-tls are given.
<CERT> Set path to server's certificate. Required unless
--frontend-no-tls are given. To make OCSP stapling
work, this must be an absolute path.
Options:
The options are categorized into several groups.
Connections:
-b, --backend=(<HOST>,<PORT>|unix:<PATH>)[;<PATTERN>[:...]]
-b, --backend=(<HOST>,<PORT>|unix:<PATH>)[;[<PATTERN>[:...]][;proto=<PROTO>]]
Set backend host and port. The multiple backend
addresses are accepted by repeating this option. UNIX
domain socket can be specified by prefixing path name
with "unix:" (e.g., unix:/var/run/backend.sock).
Optionally, if <PATTERN>s are given, the backend address
is only used if request matches the pattern. If -s or
-p is used, <PATTERN>s are ignored. The pattern
matching is closely designed to ServeMux in net/http
package of Go programming language. <PATTERN> consists
of path, host + path or just host. The path must start
with "/". If it ends with "/", it matches all request
path in its subtree. To deal with the request to the
directory without trailing slash, the path which ends
with "/" also matches the request path which only lacks
trailing '/' (e.g., path "/foo/" matches request path
"/foo"). If it does not end with "/", it performs exact
match against the request path. If host is given, it
performs exact match against the request host. If host
alone is given, "/" is appended to it, so that it
matches all request paths under the host (e.g.,
specifying "nghttp2.org" equals to "nghttp2.org/").
is only used if request matches the pattern. If
--http2-proxy is used, <PATTERN>s are ignored. The
pattern matching is closely designed to ServeMux in
net/http package of Go programming language. <PATTERN>
consists of path, host + path or just host. The path
must start with "/". If it ends with "/", it matches
all request path in its subtree. To deal with the
request to the directory without trailing slash, the
path which ends with "/" also matches the request path
which only lacks trailing '/' (e.g., path "/foo/"
matches request path "/foo"). If it does not end with
"/", it performs exact match against the request path.
If host is given, it performs exact match against the
request host. If host alone is given, "/" is appended
to it, so that it matches all request paths under the
host (e.g., specifying "nghttp2.org" equals to
"nghttp2.org/").
Patterns with host take precedence over patterns with
just path. Then, longer patterns take precedence over
shorter ones, breaking a tie by the order of the
appearance in the configuration.
If <PATTERN> is omitted, "/" is used as pattern, which
matches all request paths (catch-all pattern). The
catch-all backend must be given.
If <PATTERN> is omitted or empty string, "/" is used as
pattern, which matches all request paths (catch-all
pattern). The catch-all backend must be given.
When doing a match, nghttpx made some normalization to
pattern, request host and path. For host part, they are
@ -1252,6 +1255,15 @@ Connections:
The backend addresses sharing same <PATTERN> are grouped
together forming load balancing group.
Optionally, backend application protocol can be
specified in <PROTO>. All that share the same <PATTERN>
must have the same <PROTO> value if it is given.
<PROTO> should be one of the following list without
quotes: "h2", "http/1.1". The default value of <PROTO>
is "http/1.1". Note that usually "h2" refers to HTTP/2
over TLS. But in this option, it may mean HTTP/2 over
cleartext TCP unless --backend-tls is used.
Since ";" and ":" are used as delimiter, <PATTERN> must
not contain these characters. Since ";" has special
meaning in shell, the option value must be quoted.
@ -1290,16 +1302,8 @@ Connections:
--backend-write-timeout options.
--accept-proxy-protocol
Accept PROXY protocol version 1 on frontend connection.
--backend-no-tls
Disable SSL/TLS on backend connections. For HTTP/2
backend connections, TLS is enabled by default. For
HTTP/1 backend connections, TLS is disabled by default,
and can be enabled by --backend-http1-tls option. If
both --backend-no-tls and --backend-http1-tls options
are used, --backend-no-tls has the precedence.
--backend-http1-tls
Enable SSL/TLS on backend HTTP/1 connections. See also
--backend-no-tls option.
--backend-tls
Enable SSL/TLS on backend connections.
Performance:
-n, --workers=<N>
@ -1352,31 +1356,24 @@ Performance:
accepts. Setting 0 means unlimited.
Default: )" << get_config()->conn.upstream.worker_connections
<< R"(
--backend-http2-connections-per-worker=<N>
Set maximum number of backend HTTP/2 physical
connections per worker. If pattern is used in -b
option, this limit is applied to each pattern group (in
other words, each pattern group can have maximum <N>
HTTP/2 connections). The default value is 0, which
means that the value is adjusted to the number of
backend addresses. If pattern is used, this adjustment
is done for each pattern group.
--backend-http1-connections-per-host=<N>
Set maximum number of backend concurrent HTTP/1
connections per origin host. This option is meaningful
when -s option is used. The origin host is determined
by authority portion of request URI (or :authority
header field for HTTP/2). To limit the number of
connections per frontend for default mode, use
--backend-http1-connections-per-frontend.
--backend-connections-per-host=<N>
Set maximum number of backend concurrent connections
(and/or streams in case of HTTP/2) per origin host.
This option is meaningful when --http2-proxy option is
used. The origin host is determined by authority
portion of request URI (or :authority header field for
HTTP/2). To limit the number of connections per
frontend for default mode, use
--backend-connections-per-frontend.
Default: )" << get_config()->conn.downstream.connections_per_host
<< R"(
--backend-http1-connections-per-frontend=<N>
Set maximum number of backend concurrent HTTP/1
connections per frontend. This option is only used for
default mode. 0 means unlimited. To limit the number
of connections per host for HTTP/2 or SPDY proxy mode
(-s option), use --backend-http1-connections-per-host.
--backend-connections-per-frontend=<N>
Set maximum number of backend concurrent connections
(and/or streams in case of HTTP/2) per frontend. This
option is only used for default mode. 0 means
unlimited. To limit the number of connections per host
with --http2-proxy option, use
--backend-connections-per-host.
Default: )"
<< get_config()->conn.downstream.connections_per_frontend << R"(
--rlimit-nofile=<N>
@ -1630,10 +1627,18 @@ SSL/TLS:
the complete HTTP/2 cipher suites black list.
HTTP/2 and SPDY:
-c, --http2-max-concurrent-streams=<N>
-c, --frontend-http2-max-concurrent-streams=<N>
Set the maximum number of the concurrent streams in one
HTTP/2 and SPDY session.
Default: )" << get_config()->http2.max_concurrent_streams << R"(
frontend HTTP/2 and SPDY session.
Default: )"
<< get_config()->http2.upstream.max_concurrent_streams << R"(
--backend-http2-max-concurrent-streams=<N>
Set the maximum number of the concurrent streams in one
backend HTTP/2 session. This sets maximum number of
concurrent opened pushed streams. The maximum number of
concurrent requests are set by a remote server.
Default: )"
<< get_config()->http2.downstream.max_concurrent_streams << R"(
--frontend-http2-window-bits=<N>
Sets the per-stream initial window size of HTTP/2 SPDY
frontend connection. For HTTP/2, the size is 2**<N>-1.
@ -1667,37 +1672,20 @@ HTTP/2 and SPDY:
Disable HTTP/2 server push. Server push is supported by
default mode and HTTP/2 frontend via Link header field.
It is also supported if both frontend and backend are
HTTP/2 (which implies --http2-bridge or --client mode).
In this case, server push from backend session is
relayed to frontend, and server push via Link header
field is also supported. HTTP SPDY frontend does not
support server push.
HTTP/2 in default mode. In this case, server push from
backend session is relayed to frontend, and server push
via Link header field is also supported. SPDY frontend
does not support server push.
Mode:
(default mode)
Accept HTTP/2, SPDY and HTTP/1.1 over SSL/TLS. If
--frontend-no-tls is used, accept HTTP/2 and HTTP/1.1.
The incoming HTTP/1.1 connection can be upgraded to
HTTP/2 through HTTP Upgrade. The protocol to the
backend is HTTP/1.1.
HTTP/2 through HTTP Upgrade.
-s, --http2-proxy
Like default mode, but enable secure proxy mode.
--http2-bridge
Like default mode, but communicate with the backend in
HTTP/2 over SSL/TLS. Thus the incoming all connections
are converted to HTTP/2 connection and relayed to the
backend. See --backend-http-proxy-uri option if you are
behind the proxy and want to connect to the outside
HTTP/2 proxy.
--client Accept HTTP/2 and HTTP/1.1 without SSL/TLS. The
incoming HTTP/1.1 connection can be upgraded to HTTP/2
connection through HTTP Upgrade. The protocol to the
backend is HTTP/2. To use nghttpx as a forward proxy,
use -p option instead.
-p, --client-proxy
Like --client option, but it also requires the request
path from frontend must be an absolute URI, suitable for
use as a forward proxy.
Like default mode, but enable forward proxy. This is so
called HTTP/2 proxy mode.
Logging:
-L, --log-level=<LEVEL>
@ -1798,15 +1786,13 @@ HTTP:
--no-via Don't append to Via header field. If Via header field
is received, it is left unaltered.
--no-location-rewrite
Don't rewrite location header field on --http2-bridge,
--client and default mode. For --http2-proxy and
--client-proxy mode, location header field will not be
altered regardless of this option.
Don't rewrite location header field in default mode.
When --http2-proxy is used, location header field will
not be altered regardless of this option.
--host-rewrite
Rewrite host and :authority header fields on
--http2-bridge, --client and default mode. For
--http2-proxy and --client-proxy mode, these headers
will not be altered regardless of this option.
Rewrite host and :authority header fields in default
mode. When --http2-proxy is used, these headers will
not be altered regardless of this option.
--altsvc=<PROTOID,PORT[,HOST,[ORIGIN]]>
Specify protocol ID, port, host and origin of
alternative service. <HOST> and <ORIGIN> are optional.
@ -2055,29 +2041,6 @@ void process_options(
upstreamconf.worker_connections = std::numeric_limits<size_t>::max();
}
if (get_config()->http2_proxy + get_config()->http2_bridge +
get_config()->client_proxy + get_config()->client >
1) {
LOG(FATAL) << "--http2-proxy, --http2-bridge, --client-proxy and --client "
<< "cannot be used at the same time.";
exit(EXIT_FAILURE);
}
if (get_config()->client || get_config()->client_proxy) {
mod_config()->client_mode = true;
upstreamconf.no_tls = true;
}
if (get_config()->client_mode || get_config()->http2_bridge) {
downstreamconf.proto = PROTO_HTTP2;
} else {
downstreamconf.proto = PROTO_HTTP;
}
if (downstreamconf.proto == PROTO_HTTP && !downstreamconf.http1_tls) {
downstreamconf.no_tls = true;
}
if (!upstreamconf.no_tls &&
(tlsconf.private_key_file.empty() || tlsconf.cert_file.empty())) {
print_usage(std::cerr);
@ -2098,23 +2061,34 @@ void process_options(
auto &addr_groups = downstreamconf.addr_groups;
if (addr_groups.empty()) {
DownstreamAddr addr{};
DownstreamAddrConfig addr{};
addr.host = ImmutableString::from_lit(DEFAULT_DOWNSTREAM_HOST);
addr.port = DEFAULT_DOWNSTREAM_PORT;
DownstreamAddrGroup g(StringRef::from_lit("/"));
DownstreamAddrGroupConfig g(StringRef::from_lit("/"));
g.proto = PROTO_HTTP1;
g.addrs.push_back(std::move(addr));
mod_config()->router.add_route(StringRef{g.pattern}, addr_groups.size());
addr_groups.push_back(std::move(g));
} else if (get_config()->http2_proxy || get_config()->client_proxy) {
} else if (get_config()->http2_proxy) {
// We don't support host mapping in these cases. Move all
// non-catch-all patterns to catch-all pattern.
DownstreamAddrGroup catch_all(StringRef::from_lit("/"));
DownstreamAddrGroupConfig catch_all(StringRef::from_lit("/"));
auto proto = PROTO_NONE;
for (auto &g : addr_groups) {
if (proto == PROTO_NONE) {
proto = g.proto;
} else if (proto != g.proto) {
LOG(ERROR) << SHRPX_OPT_BACKEND << ": <PATTERN> was ignored with "
"--http2-proxy, and protocol must "
"be the same for all backends.";
exit(EXIT_FAILURE);
}
std::move(std::begin(g.addrs), std::end(g.addrs),
std::back_inserter(catch_all.addrs));
}
std::vector<DownstreamAddrGroup>().swap(addr_groups);
catch_all.proto = proto;
std::vector<DownstreamAddrGroupConfig>().swap(addr_groups);
// maybe not necessary?
mod_config()->router = Router();
mod_config()->router.add_route(StringRef{catch_all.pattern},
@ -2134,7 +2108,7 @@ void process_options(
}
if (LOG_ENABLED(INFO)) {
LOG(INFO) << "Host-path pattern: group " << i << ": '" << g.pattern
<< "'";
<< "', proto=" << strproto(g.proto);
for (auto &addr : g.addrs) {
LOG(INFO) << "group " << i << " -> " << addr.host.c_str()
<< (addr.host_unix ? "" : ":" + util::utos(addr.port));
@ -2143,7 +2117,7 @@ void process_options(
}
if (catch_all_group == -1) {
LOG(FATAL) << "-b: No catch-all backend address is configured";
LOG(FATAL) << "backend: No catch-all backend address is configured";
exit(EXIT_FAILURE);
}
@ -2187,7 +2161,7 @@ void process_options(
addr.hostport = ImmutableString(
util::make_http_hostport(StringRef(addr.host), addr.port));
auto hostport = util::make_hostport(addr.host.c_str(), addr.port);
auto hostport = util::make_hostport(StringRef{addr.host}, addr.port);
if (resolve_hostname(&addr.addr, addr.host.c_str(), addr.port,
downstreamconf.family) == -1) {
@ -2201,7 +2175,7 @@ void process_options(
auto &proxy = mod_config()->downstream_http_proxy;
if (!proxy.host.empty()) {
auto hostport = util::make_hostport(proxy.host.c_str(), proxy.port);
auto hostport = util::make_hostport(StringRef{proxy.host}, proxy.port);
if (resolve_hostname(&proxy.addr, proxy.host.c_str(), proxy.port,
AF_UNSPEC) == -1) {
LOG(FATAL) << "Resolving backend HTTP proxy address failed: " << hostport;
@ -2460,6 +2434,14 @@ int main(int argc, char **argv) {
{SHRPX_OPT_TLS_SESSION_CACHE_MEMCACHED_ADDRESS_FAMILY,
required_argument, &flag, 115},
{SHRPX_OPT_BACKEND_ADDRESS_FAMILY, required_argument, &flag, 116},
{SHRPX_OPT_FRONTEND_HTTP2_MAX_CONCURRENT_STREAMS, required_argument,
&flag, 117},
{SHRPX_OPT_BACKEND_HTTP2_MAX_CONCURRENT_STREAMS, required_argument,
&flag, 118},
{SHRPX_OPT_BACKEND_CONNECTIONS_PER_FRONTEND, required_argument, &flag,
119},
{SHRPX_OPT_BACKEND_TLS, no_argument, &flag, 120},
{SHRPX_OPT_BACKEND_CONNECTIONS_PER_HOST, required_argument, &flag, 121},
{nullptr, 0, nullptr, 0}};
int option_index = 0;
@ -2955,6 +2937,29 @@ int main(int argc, char **argv) {
// --backend-address-family
cmdcfgs.emplace_back(SHRPX_OPT_BACKEND_ADDRESS_FAMILY, optarg);
break;
case 117:
// --frontend-http2-max-concurrent-streams
cmdcfgs.emplace_back(SHRPX_OPT_FRONTEND_HTTP2_MAX_CONCURRENT_STREAMS,
optarg);
break;
case 118:
// --backend-http2-max-concurrent-streams
cmdcfgs.emplace_back(SHRPX_OPT_BACKEND_HTTP2_MAX_CONCURRENT_STREAMS,
optarg);
break;
case 119:
// --backend-connections-per-frontend
cmdcfgs.emplace_back(SHRPX_OPT_BACKEND_CONNECTIONS_PER_FRONTEND,
optarg);
break;
case 120:
// --backend-tls
cmdcfgs.emplace_back(SHRPX_OPT_BACKEND_TLS, "yes");
break;
case 121:
// --backend-connections-per-host
cmdcfgs.emplace_back(SHRPX_OPT_BACKEND_CONNECTIONS_PER_HOST, optarg);
break;
default:
break;
}

95
src/shrpx_client_handler.cc
View File

@ -385,12 +385,7 @@ ClientHandler::ClientHandler(Worker *worker, int fd, SSL *ssl,
get_config()->conn.upstream.ratelimit.write,
get_config()->conn.upstream.ratelimit.read, writecb, readcb,
timeoutcb, this, get_config()->tls.dyn_rec.warmup_threshold,
get_config()->tls.dyn_rec.idle_timeout),
pinned_http2sessions_(
get_config()->conn.downstream.proto == PROTO_HTTP2
? make_unique<std::vector<ssize_t>>(
worker->get_downstream_addr_groups().size(), -1)
: nullptr),
get_config()->tls.dyn_rec.idle_timeout, PROTO_NONE),
ipaddr_(ipaddr),
port_(port),
faddr_(faddr),
@ -642,13 +637,18 @@ void ClientHandler::pool_downstream_connection(
if (!dconn->poolable()) {
return;
}
dconn->set_client_handler(nullptr);
auto group = dconn->get_downstream_addr_group();
if (LOG_ENABLED(INFO)) {
CLOG(INFO, this) << "Pooling downstream connection DCONN:" << dconn.get()
<< " in group " << dconn->get_group();
<< " in group " << group;
}
dconn->set_client_handler(nullptr);
auto dconn_pool = worker_->get_dconn_pool();
dconn_pool->add_downstream_connection(std::move(dconn));
auto &dconn_pool = group->dconn_pool;
dconn_pool.add_downstream_connection(std::move(dconn));
}
void ClientHandler::remove_downstream_connection(DownstreamConnection *dconn) {
@ -656,13 +656,13 @@ void ClientHandler::remove_downstream_connection(DownstreamConnection *dconn) {
CLOG(INFO, this) << "Removing downstream connection DCONN:" << dconn
<< " from pool";
}
auto dconn_pool = worker_->get_dconn_pool();
dconn_pool->remove_downstream_connection(dconn);
auto &dconn_pool = dconn->get_downstream_addr_group()->dconn_pool;
dconn_pool.remove_downstream_connection(dconn);
}
std::unique_ptr<DownstreamConnection>
ClientHandler::get_downstream_connection(Downstream *downstream) {
size_t group;
size_t group_idx;
auto &downstreamconf = get_config()->conn.downstream;
auto catch_all = downstreamconf.addr_group_catch_all;
auto &groups = worker_->get_downstream_addr_groups();
@ -672,26 +672,26 @@ ClientHandler::get_downstream_connection(Downstream *downstream) {
// Fast path. If we have one group, it must be catch-all group.
// HTTP/2 and client proxy modes fall in this case.
if (groups.size() == 1) {
group = 0;
group_idx = 0;
} else if (req.method == HTTP_CONNECT) {
// We don't know how to treat CONNECT request in host-path
// mapping. It most likely appears in proxy scenario. Since we
// have dealt with proxy case already, just use catch-all group.
group = catch_all;
group_idx = catch_all;
} else {
auto &router = get_config()->router;
if (!req.authority.empty()) {
group =
group_idx =
match_downstream_addr_group(router, StringRef{req.authority},
StringRef{req.path}, groups, catch_all);
} else {
auto h = req.fs.header(http2::HD_HOST);
if (h) {
group =
group_idx =
match_downstream_addr_group(router, StringRef{h->value},
StringRef{req.path}, groups, catch_all);
} else {
group =
group_idx =
match_downstream_addr_group(router, StringRef::from_lit(""),
StringRef{req.path}, groups, catch_all);
}
@ -699,11 +699,12 @@ ClientHandler::get_downstream_connection(Downstream *downstream) {
}
if (LOG_ENABLED(INFO)) {
CLOG(INFO, this) << "Downstream address group: " << group;
CLOG(INFO, this) << "Downstream address group_idx: " << group_idx;
}
auto dconn_pool = worker_->get_dconn_pool();
auto dconn = dconn_pool->pop_downstream_connection(group);
auto &group = worker_->get_downstream_addr_groups()[group_idx];
auto &dconn_pool = group.dconn_pool;
auto dconn = dconn_pool.pop_downstream_connection();
if (!dconn) {
if (LOG_ENABLED(INFO)) {
@ -711,22 +712,34 @@ ClientHandler::get_downstream_connection(Downstream *downstream) {
<< " Create new one";
}
auto dconn_pool = worker_->get_dconn_pool();
if (downstreamconf.proto == PROTO_HTTP2) {
Http2Session *http2session;
auto &pinned = (*pinned_http2sessions_)[group];
if (pinned == -1) {
http2session = worker_->next_http2_session(group);
pinned = http2session->get_index();
} else {
auto dgrp = worker_->get_dgrp(group);
http2session = dgrp->http2sessions[pinned].get();
if (group.proto == PROTO_HTTP2) {
if (group.http2_freelist.empty()) {
if (LOG_ENABLED(INFO)) {
CLOG(INFO, this)
<< "http2_freelist is empty; create new Http2Session";
}
auto session = make_unique<Http2Session>(
conn_.loop, worker_->get_cl_ssl_ctx(), worker_, &group);
group.http2_freelist.append(session.release());
}
dconn = make_unique<Http2DownstreamConnection>(dconn_pool, http2session);
auto http2session = group.http2_freelist.head;
// TODO max_concurrent_streams option must be independent from
// frontend and backend.
if (http2session->max_concurrency_reached(1)) {
if (LOG_ENABLED(INFO)) {
CLOG(INFO, this) << "Maximum streams are reached for Http2Session("
<< http2session
<< "). Remove Http2Session from http2_freelist";
}
group.http2_freelist.remove(http2session);
}
dconn = make_unique<Http2DownstreamConnection>(http2session);
} else {
dconn = make_unique<HttpDownstreamConnection>(dconn_pool, group,
conn_.loop, worker_);
dconn =
make_unique<HttpDownstreamConnection>(&group, conn_.loop, worker_);
}
dconn->set_client_handler(this);
return dconn;
@ -840,11 +853,11 @@ void ClientHandler::write_accesslog(Downstream *downstream) {
upstream_accesslog(
get_config()->logging.access.format,
LogSpec{
downstream, StringRef(ipaddr_), http2::to_method_string(req.method),
downstream, StringRef{ipaddr_}, http2::to_method_string(req.method),
req.method == HTTP_CONNECT
? StringRef(req.authority)
: (get_config()->http2_proxy || get_config()->client_proxy)
: get_config()->http2_proxy
? StringRef(construct_absolute_request_uri(req))
: req.path.empty()
? req.method == HTTP_OPTIONS
@ -1125,18 +1138,18 @@ int ClientHandler::proxy_protocol_read() {
return on_proxy_protocol_finish();
}
StringRef ClientHandler::get_forwarded_by() {
StringRef ClientHandler::get_forwarded_by() const {
auto &fwdconf = get_config()->http.forwarded;
if (fwdconf.by_node_type == FORWARDED_NODE_OBFUSCATED) {
return StringRef(fwdconf.by_obfuscated);
}
return StringRef(faddr_->hostport);
return StringRef{faddr_->hostport};
}
const std::string &ClientHandler::get_forwarded_for() const {
return forwarded_for_;
StringRef ClientHandler::get_forwarded_for() const {
return StringRef{forwarded_for_};
}
} // namespace shrpx

5
src/shrpx_client_handler.h
View File

@ -135,16 +135,15 @@ public:
// Returns string suitable for use in "by" parameter of Forwarded
// header field.
StringRef get_forwarded_by();
StringRef get_forwarded_by() const;
// Returns string suitable for use in "for" parameter of Forwarded
// header field.
const std::string &get_forwarded_for() const;
StringRef get_forwarded_for() const;
private:
Connection conn_;
ev_timer reneg_shutdown_timer_;
std::unique_ptr<Upstream> upstream_;
std::unique_ptr<std::vector<ssize_t>> pinned_http2sessions_;
// IP address of client. If UNIX domain socket is used, this is
// "localhost".
std::string ipaddr_;

285
src/shrpx_config.cc
View File

@ -571,33 +571,70 @@ int parse_duration(ev_tstamp *dest, const char *opt, const char *optarg) {
} // namespace
namespace {
// Parses host-path mapping patterns in |src|, and stores mappings in
// config. We will store each host-path pattern found in |src| with
// |addr|. |addr| will be copied accordingly. Also we make a group
// based on the pattern. The "/" pattern is considered as catch-all.
void parse_mapping(const DownstreamAddr &addr, const char *src) {
// Parses host-path mapping patterns in |src_pattern|, and stores
// mappings in config. We will store each host-path pattern found in
// |src| with |addr|. |addr| will be copied accordingly. Also we
// make a group based on the pattern. The "/" pattern is considered
// as catch-all. We also parse protocol specified in |src_proto|.
//
// This function returns 0 if it succeeds, or -1.
int parse_mapping(const DownstreamAddrConfig &addr,
const StringRef &src_pattern, const StringRef &src_proto) {
// This returns at least 1 element (it could be empty string). We
// will append '/' to all patterns, so it becomes catch-all pattern.
auto mapping = util::split_config_str_list(src, ':');
auto mapping = util::split_str(src_pattern, ':');
assert(!mapping.empty());
auto &addr_groups = mod_config()->conn.downstream.addr_groups;
auto proto = PROTO_HTTP1;
if (!src_proto.empty()) {
if (!util::istarts_with_l(src_proto, "proto=")) {
LOG(ERROR) << "backend: proto keyword not found";
return -1;
}
auto protostr = StringRef{std::begin(src_proto) + str_size("proto="),
std::end(src_proto)};
if (protostr.empty()) {
LOG(ERROR) << "backend: protocol is empty";
return -1;
}
if (util::streq_l("h2", std::begin(protostr), protostr.size())) {
proto = PROTO_HTTP2;
} else if (util::streq_l("http/1.1", std::begin(protostr),
protostr.size())) {
proto = PROTO_HTTP1;
} else {
LOG(ERROR) << "backend: unknown protocol " << protostr;
return -1;
}
}
for (const auto &raw_pattern : mapping) {
auto done = false;
std::string pattern;
auto slash = std::find(raw_pattern.first, raw_pattern.second, '/');
if (slash == raw_pattern.second) {
auto slash = std::find(std::begin(raw_pattern), std::end(raw_pattern), '/');
if (slash == std::end(raw_pattern)) {
// This effectively makes empty pattern to "/".
pattern.assign(raw_pattern.first, raw_pattern.second);
pattern.assign(std::begin(raw_pattern), std::end(raw_pattern));
util::inp_strlower(pattern);
pattern += '/';
} else {
pattern.assign(raw_pattern.first, slash);
pattern.assign(std::begin(raw_pattern), slash);
util::inp_strlower(pattern);
pattern += http2::normalize_path(slash, raw_pattern.second);
pattern += http2::normalize_path(slash, std::end(raw_pattern));
}
for (auto &g : addr_groups) {
if (g.pattern == pattern) {
if (g.proto != proto) {
LOG(ERROR) << "backend: protocol mismatch. We saw protocol "
<< strproto(g.proto) << " for pattern " << g.pattern
<< ", but another protocol " << strproto(proto);
return -1;
}
g.addrs.push_back(addr);
done = true;
break;
@ -606,13 +643,15 @@ void parse_mapping(const DownstreamAddr &addr, const char *src) {
if (done) {
continue;
}
DownstreamAddrGroup g(StringRef{pattern});
DownstreamAddrGroupConfig g(StringRef{pattern});
g.addrs.push_back(addr);
g.proto = proto;
mod_config()->router.add_route(StringRef{g.pattern}, addr_groups.size());
addr_groups.push_back(std::move(g));
}
return 0;
}
} // namespace
@ -654,12 +693,15 @@ enum {
SHRPX_OPTID_ALTSVC,
SHRPX_OPTID_BACKEND,
SHRPX_OPTID_BACKEND_ADDRESS_FAMILY,
SHRPX_OPTID_BACKEND_CONNECTIONS_PER_FRONTEND,
SHRPX_OPTID_BACKEND_CONNECTIONS_PER_HOST,
SHRPX_OPTID_BACKEND_HTTP_PROXY_URI,
SHRPX_OPTID_BACKEND_HTTP1_CONNECTIONS_PER_FRONTEND,
SHRPX_OPTID_BACKEND_HTTP1_CONNECTIONS_PER_HOST,
SHRPX_OPTID_BACKEND_HTTP1_TLS,
SHRPX_OPTID_BACKEND_HTTP2_CONNECTION_WINDOW_BITS,
SHRPX_OPTID_BACKEND_HTTP2_CONNECTIONS_PER_WORKER,
SHRPX_OPTID_BACKEND_HTTP2_MAX_CONCURRENT_STREAMS,
SHRPX_OPTID_BACKEND_HTTP2_WINDOW_BITS,
SHRPX_OPTID_BACKEND_IPV4,
SHRPX_OPTID_BACKEND_IPV6,
@ -668,6 +710,7 @@ enum {
SHRPX_OPTID_BACKEND_READ_TIMEOUT,
SHRPX_OPTID_BACKEND_REQUEST_BUFFER,
SHRPX_OPTID_BACKEND_RESPONSE_BUFFER,
SHRPX_OPTID_BACKEND_TLS,
SHRPX_OPTID_BACKEND_TLS_SNI_FIELD,
SHRPX_OPTID_BACKEND_WRITE_TIMEOUT,
SHRPX_OPTID_BACKLOG,
@ -692,6 +735,7 @@ enum {
SHRPX_OPTID_FRONTEND_HTTP2_CONNECTION_WINDOW_BITS,
SHRPX_OPTID_FRONTEND_HTTP2_DUMP_REQUEST_HEADER,
SHRPX_OPTID_FRONTEND_HTTP2_DUMP_RESPONSE_HEADER,
SHRPX_OPTID_FRONTEND_HTTP2_MAX_CONCURRENT_STREAMS,
SHRPX_OPTID_FRONTEND_HTTP2_READ_TIMEOUT,
SHRPX_OPTID_FRONTEND_HTTP2_WINDOW_BITS,
SHRPX_OPTID_FRONTEND_NO_TLS,
@ -911,6 +955,11 @@ int option_lookup_token(const char *name, size_t namelen) {
break;
case 11:
switch (name[10]) {
case 's':
if (util::strieq_l("backend-tl", name, 10)) {
return SHRPX_OPTID_BACKEND_TLS;
}
break;
case 't':
if (util::strieq_l("write-burs", name, 10)) {
return SHRPX_OPTID_WRITE_BURST;
@ -1322,6 +1371,11 @@ int option_lookup_token(const char *name, size_t namelen) {
return SHRPX_OPTID_TLS_TICKET_KEY_MEMCACHED_TLS;
}
break;
case 't':
if (util::strieq_l("backend-connections-per-hos", name, 27)) {
return SHRPX_OPTID_BACKEND_CONNECTIONS_PER_HOST;
}
break;
}
break;
case 30:
@ -1342,6 +1396,15 @@ int option_lookup_token(const char *name, size_t namelen) {
break;
}
break;
case 32:
switch (name[31]) {
case 'd':
if (util::strieq_l("backend-connections-per-fronten", name, 31)) {
return SHRPX_OPTID_BACKEND_CONNECTIONS_PER_FRONTEND;
}
break;
}
break;
case 33:
switch (name[32]) {
case 'l':
@ -1398,6 +1461,9 @@ int option_lookup_token(const char *name, size_t namelen) {
if (util::strieq_l("backend-http2-connection-window-bit", name, 35)) {
return SHRPX_OPTID_BACKEND_HTTP2_CONNECTION_WINDOW_BITS;
}
if (util::strieq_l("backend-http2-max-concurrent-stream", name, 35)) {
return SHRPX_OPTID_BACKEND_HTTP2_MAX_CONCURRENT_STREAMS;
}
break;
}
break;
@ -1412,6 +1478,9 @@ int option_lookup_token(const char *name, size_t namelen) {
if (util::strieq_l("frontend-http2-connection-window-bit", name, 36)) {
return SHRPX_OPTID_FRONTEND_HTTP2_CONNECTION_WINDOW_BITS;
}
if (util::strieq_l("frontend-http2-max-concurrent-stream", name, 36)) {
return SHRPX_OPTID_FRONTEND_HTTP2_MAX_CONCURRENT_STREAMS;
}
break;
}
break;
@ -1477,19 +1546,17 @@ int parse_config(const char *opt, const char *optarg,
switch (optid) {
case SHRPX_OPTID_BACKEND: {
auto optarglen = strlen(optarg);
const char *pat_delim = strchr(optarg, ';');
if (!pat_delim) {
pat_delim = optarg + optarglen;
}
DownstreamAddr addr{};
auto src = StringRef{optarg};
auto addr_end = std::find(std::begin(src), std::end(src), ';');
DownstreamAddrConfig addr{};
if (util::istarts_with(optarg, SHRPX_UNIX_PATH_PREFIX)) {
auto path = optarg + str_size(SHRPX_UNIX_PATH_PREFIX);
addr.host = ImmutableString(path, pat_delim);
auto path = std::begin(src) + str_size(SHRPX_UNIX_PATH_PREFIX);
addr.host = ImmutableString(path, addr_end);
addr.host_unix = true;
} else {
if (split_host_port(host, sizeof(host), &port, optarg,
pat_delim - optarg) == -1) {
if (split_host_port(host, sizeof(host), &port, &src[0],
addr_end - std::begin(src)) == -1) {
return -1;
}
@ -1497,14 +1564,16 @@ int parse_config(const char *opt, const char *optarg,
addr.port = port;
}
auto mapping = pat_delim < optarg + optarglen ? pat_delim + 1 : pat_delim;
// We may introduce new parameter after additional ';', so don't
// allow extra ';' in pattern for now.
if (strchr(mapping, ';') != nullptr) {
LOG(ERROR) << opt << ": ';' must not be used in pattern";
auto mapping = addr_end == std::end(src) ? addr_end : addr_end + 1;
auto mapping_end = std::find(mapping, std::end(src), ';');
auto proto = mapping_end == std::end(src) ? mapping_end : mapping_end + 1;
auto proto_end = std::find(proto, std::end(src), ';');
if (parse_mapping(addr, StringRef{mapping, mapping_end},
StringRef{proto, proto_end}) != 0) {
return -1;
}
parse_mapping(addr, mapping);
return 0;
}
@ -1559,8 +1628,20 @@ int parse_config(const char *opt, const char *optarg,
#else // !NOTHREADS
return parse_uint(&mod_config()->num_worker, opt, optarg);
#endif // !NOTHREADS
case SHRPX_OPTID_HTTP2_MAX_CONCURRENT_STREAMS:
return parse_uint(&mod_config()->http2.max_concurrent_streams, opt, optarg);
case SHRPX_OPTID_HTTP2_MAX_CONCURRENT_STREAMS: {
LOG(WARN) << opt << ": deprecated. Use "
<< SHRPX_OPT_FRONTEND_HTTP2_MAX_CONCURRENT_STREAMS << " and "
<< SHRPX_OPT_BACKEND_HTTP2_MAX_CONCURRENT_STREAMS << " instead.";
size_t n;
if (parse_uint(&n, opt, optarg) != 0) {
return -1;
}
auto &http2conf = mod_config()->http2;
http2conf.upstream.max_concurrent_streams = n;
http2conf.downstream.max_concurrent_streams = n;
return 0;
}
case SHRPX_OPTID_LOG_LEVEL:
if (Log::set_severity_level_by_name(optarg) == -1) {
LOG(ERROR) << opt << ": Invalid severity level: " << optarg;
@ -1577,13 +1658,13 @@ int parse_config(const char *opt, const char *optarg,
return 0;
case SHRPX_OPTID_HTTP2_BRIDGE:
mod_config()->http2_bridge = util::strieq(optarg, "yes");
return 0;
LOG(ERROR) << opt << ": deprecated. Use backend=<addr>,<port>;;proto=h2 "
"and backend-tls";
return -1;
case SHRPX_OPTID_CLIENT_PROXY:
mod_config()->client_proxy = util::strieq(optarg, "yes");
return 0;
LOG(ERROR) << opt << ": deprecated. Use http2-proxy, frontend-no-tls, "
"backend=<addr>,<port>;;proto=h2 and backend-tls";
return -1;
case SHRPX_OPTID_ADD_X_FORWARDED_FOR:
mod_config()->http.xff.add = util::strieq(optarg, "yes");
@ -1716,8 +1797,8 @@ int parse_config(const char *opt, const char *optarg,
return 0;
case SHRPX_OPTID_BACKEND_NO_TLS:
mod_config()->conn.downstream.no_tls = util::strieq(optarg, "yes");
LOG(WARN) << opt << ": deprecated. backend connection is not encrypted by "
"default. See also " << SHRPX_OPT_BACKEND_TLS;
return 0;
case SHRPX_OPTID_BACKEND_TLS_SNI_FIELD:
mod_config()->tls.backend_sni_name = optarg;
@ -1804,9 +1885,9 @@ int parse_config(const char *opt, const char *optarg,
return 0;
case SHRPX_OPTID_CLIENT:
mod_config()->client = util::strieq(optarg, "yes");
return 0;
LOG(ERROR) << opt << ": deprecated. Use frontend-no-tls, "
"backend=<addr>,<port>;;proto=h2 and backend-tls";
return -1;
case SHRPX_OPTID_INSECURE:
mod_config()->tls.insecure = util::strieq(optarg, "yes");
@ -2007,7 +2088,11 @@ int parse_config(const char *opt, const char *optarg,
"--host-rewrite option.";
return 0;
case SHRPX_OPTID_BACKEND_HTTP1_CONNECTIONS_PER_HOST: {
case SHRPX_OPTID_BACKEND_HTTP1_CONNECTIONS_PER_HOST:
LOG(WARN) << opt
<< ": deprecated. Use backend-connections-per-host instead.";
// fall through
case SHRPX_OPTID_BACKEND_CONNECTIONS_PER_HOST: {
int n;
if (parse_uint(&n, opt, optarg) != 0) {
@ -2025,6 +2110,10 @@ int parse_config(const char *opt, const char *optarg,
return 0;
}
case SHRPX_OPTID_BACKEND_HTTP1_CONNECTIONS_PER_FRONTEND:
LOG(WARN) << opt << ": deprecated. Use "
<< SHRPX_OPT_BACKEND_CONNECTIONS_PER_FRONTEND << " instead.";
// fall through
case SHRPX_OPTID_BACKEND_CONNECTIONS_PER_FRONTEND:
return parse_uint(&mod_config()->conn.downstream.connections_per_frontend,
opt, optarg);
case SHRPX_OPTID_LISTENER_DISABLE_TIMEOUT:
@ -2077,8 +2166,8 @@ int parse_config(const char *opt, const char *optarg,
return 0;
case SHRPX_OPTID_BACKEND_HTTP2_CONNECTIONS_PER_WORKER:
return parse_uint(&mod_config()->http2.downstream.connections_per_worker,
opt, optarg);
LOG(WARN) << opt << ": deprecated.";
return 0;
case SHRPX_OPTID_FETCH_OCSP_RESPONSE_FILE:
mod_config()->tls.ocsp.fetch_ocsp_response_file = optarg;
@ -2278,7 +2367,11 @@ int parse_config(const char *opt, const char *optarg,
return 0;
case SHRPX_OPTID_BACKEND_HTTP1_TLS:
mod_config()->conn.downstream.http1_tls = util::strieq(optarg, "yes");
LOG(WARN) << opt << ": deprecated. Use " << SHRPX_OPT_BACKEND_TLS
<< " instead.";
// fall through
case SHRPX_OPTID_BACKEND_TLS:
mod_config()->conn.downstream.no_tls = !util::strieq(optarg, "yes");
return 0;
case SHRPX_OPTID_TLS_SESSION_CACHE_MEMCACHED_TLS:
@ -2314,6 +2407,12 @@ int parse_config(const char *opt, const char *optarg,
case SHRPX_OPTID_BACKEND_ADDRESS_FAMILY:
return parse_address_family(&mod_config()->conn.downstream.family, opt,
optarg);
case SHRPX_OPTID_FRONTEND_HTTP2_MAX_CONCURRENT_STREAMS:
return parse_uint(&mod_config()->http2.upstream.max_concurrent_streams, opt,
optarg);
case SHRPX_OPTID_BACKEND_HTTP2_MAX_CONCURRENT_STREAMS:
return parse_uint(&mod_config()->http2.downstream.max_concurrent_streams,
opt, optarg);
case SHRPX_OPTID_CONF:
LOG(WARN) << "conf: ignored";
@ -2493,93 +2592,17 @@ int int_syslog_facility(const char *strfacility) {
return -1;
}
namespace {
size_t match_downstream_addr_group_host(
const Router &router, const StringRef &host, const StringRef &path,
const std::vector<DownstreamAddrGroup> &groups, size_t catch_all) {
if (path.empty() || path[0] != '/') {
auto group = router.match(host, StringRef::from_lit("/"));
if (group != -1) {
if (LOG_ENABLED(INFO)) {
LOG(INFO) << "Found pattern with query " << host
<< ", matched pattern=" << groups[group].pattern;
}
return group;
}
return catch_all;
StringRef strproto(shrpx_proto proto) {
switch (proto) {
case PROTO_NONE:
return StringRef::from_lit("none");
case PROTO_HTTP1:
return StringRef::from_lit("http/1.1");
case PROTO_HTTP2:
return StringRef::from_lit("h2");
case PROTO_MEMCACHED:
return StringRef::from_lit("memcached");
}
if (LOG_ENABLED(INFO)) {
LOG(INFO) << "Perform mapping selection, using host=" << host
<< ", path=" << path;
}
auto group = router.match(host, path);
if (group != -1) {
if (LOG_ENABLED(INFO)) {
LOG(INFO) << "Found pattern with query " << host << path
<< ", matched pattern=" << groups[group].pattern;
}
return group;
}
group = router.match("", path);
if (group != -1) {
if (LOG_ENABLED(INFO)) {
LOG(INFO) << "Found pattern with query " << path
<< ", matched pattern=" << groups[group].pattern;
}
return group;
}
if (LOG_ENABLED(INFO)) {
LOG(INFO) << "None match. Use catch-all pattern";
}
return catch_all;
}
} // namespace
size_t match_downstream_addr_group(
const Router &router, const StringRef &hostport, const StringRef &raw_path,
const std::vector<DownstreamAddrGroup> &groups, size_t catch_all) {
if (std::find(std::begin(hostport), std::end(hostport), '/') !=
std::end(hostport)) {
// We use '/' specially, and if '/' is included in host, it breaks
// our code. Select catch-all case.
return catch_all;
}
auto fragment = std::find(std::begin(raw_path), std::end(raw_path), '#');
auto query = std::find(std::begin(raw_path), fragment, '?');
auto path = StringRef{std::begin(raw_path), query};
if (hostport.empty()) {
return match_downstream_addr_group_host(router, hostport, path, groups,
catch_all);
}
std::string host;
if (hostport[0] == '[') {
// assume this is IPv6 numeric address
auto p = std::find(std::begin(hostport), std::end(hostport), ']');
if (p == std::end(hostport)) {
return catch_all;
}
if (p + 1 < std::end(hostport) && *(p + 1) != ':') {
return catch_all;
}
host.assign(std::begin(hostport), p + 1);
} else {
auto p = std::find(std::begin(hostport), std::end(hostport), ':');
if (p == std::begin(hostport)) {
return catch_all;
}
host.assign(std::begin(hostport), p);
}
util::inp_strlower(host);
return match_downstream_addr_group_host(router, StringRef{host}, path, groups,
catch_all);
}
} // namespace shrpx

49
src/shrpx_config.h
View File

@ -61,6 +61,7 @@ namespace shrpx {
struct LogFragment;
class ConnectBlocker;
class Http2Session;
namespace ssl {
@ -227,10 +228,19 @@ constexpr char SHRPX_OPT_TLS_TICKET_KEY_MEMCACHED_PRIVATE_KEY_FILE[] =
constexpr char SHRPX_OPT_TLS_TICKET_KEY_MEMCACHED_ADDRESS_FAMILY[] =
"tls-ticket-key-memcached-address-family";
constexpr char SHRPX_OPT_BACKEND_ADDRESS_FAMILY[] = "backend-address-family";
constexpr char SHRPX_OPT_FRONTEND_HTTP2_MAX_CONCURRENT_STREAMS[] =
"frontend-http2-max-concurrent-streams";
constexpr char SHRPX_OPT_BACKEND_HTTP2_MAX_CONCURRENT_STREAMS[] =
"backend-http2-max-concurrent-streams";
constexpr char SHRPX_OPT_BACKEND_CONNECTIONS_PER_FRONTEND[] =
"backend-connections-per-frontend";
constexpr char SHRPX_OPT_BACKEND_TLS[] = "backend-tls";
constexpr char SHRPX_OPT_BACKEND_CONNECTIONS_PER_HOST[] =
"backend-connections-per-host";
constexpr size_t SHRPX_OBFUSCATED_NODE_LENGTH = 8;
enum shrpx_proto { PROTO_HTTP2, PROTO_HTTP };
enum shrpx_proto { PROTO_NONE, PROTO_HTTP1, PROTO_HTTP2, PROTO_MEMCACHED };
enum shrpx_forwarded_param {
FORWARDED_NONE = 0,
@ -283,27 +293,26 @@ struct TLSSessionCache {
ev_tstamp last_updated;
};
struct DownstreamAddr {
struct DownstreamAddrConfig {
Address addr;
// backend address. If |host_unix| is true, this is UNIX domain
// socket path.
ImmutableString host;
ImmutableString hostport;
ConnectBlocker *connect_blocker;
// Client side TLS session cache
TLSSessionCache tls_session_cache;
// backend port. 0 if |host_unix| is true.
uint16_t port;
// true if |host| contains UNIX domain socket path.
bool host_unix;
};
struct DownstreamAddrGroup {
DownstreamAddrGroup(const StringRef &pattern)
struct DownstreamAddrGroupConfig {
DownstreamAddrGroupConfig(const StringRef &pattern)
: pattern(pattern.c_str(), pattern.size()) {}
ImmutableString pattern;
std::vector<DownstreamAddr> addrs;
std::vector<DownstreamAddrConfig> addrs;
// Application protocol used in this group
shrpx_proto proto;
};
struct TicketKey {
@ -481,19 +490,19 @@ struct Http2Config {
nghttp2_session_callbacks *callbacks;
size_t window_bits;
size_t connection_window_bits;
size_t max_concurrent_streams;
} upstream;
struct {
nghttp2_option *option;
nghttp2_session_callbacks *callbacks;
size_t window_bits;
size_t connection_window_bits;
size_t connections_per_worker;
size_t max_concurrent_streams;
} downstream;
struct {
ev_tstamp stream_read;
ev_tstamp stream_write;
} timeout;
size_t max_concurrent_streams;
bool no_cookie_crumbling;
bool no_server_push;
};
@ -552,15 +561,13 @@ struct ConnectionConfig {
ev_tstamp write;
ev_tstamp idle_read;
} timeout;
std::vector<DownstreamAddrGroup> addr_groups;
std::vector<DownstreamAddrGroupConfig> addr_groups;
// The index of catch-all group in downstream_addr_groups.
size_t addr_group_catch_all;
size_t connections_per_host;
size_t connections_per_frontend;
size_t request_buffer_size;
size_t response_buffer_size;
// downstream protocol; this will be determined by given options.
shrpx_proto proto;
// Address family of backend connection. One of either AF_INET,
// AF_INET6 or AF_UNSPEC. This is ignored if backend connection
// is made via Unix domain socket.
@ -595,11 +602,6 @@ struct Config {
bool verbose;
bool daemon;
bool http2_proxy;
bool http2_bridge;
bool client_proxy;
bool client;
// true if --client or --client-proxy are enabled.
bool client_mode;
};
const Config *get_config();
@ -646,15 +648,8 @@ std::unique_ptr<TicketKeys>
read_tls_ticket_key_file(const std::vector<std::string> &files,
const EVP_CIPHER *cipher, const EVP_MD *hmac);
// Selects group based on request's |hostport| and |path|. |hostport|
// is the value taken from :authority or host header field, and may
// contain port. The |path| may contain query part. We require the
// catch-all pattern in place, so this function always selects one
// group. The catch-all group index is given in |catch_all|. All
// patterns are given in |groups|.
size_t match_downstream_addr_group(
const Router &router, const StringRef &hostport, const StringRef &path,
const std::vector<DownstreamAddrGroup> &groups, size_t catch_all);
// Returns string representation of |proto|.
StringRef strproto(shrpx_proto proto);
} // namespace shrpx

116
src/shrpx_config_test.cc
View File

@ -238,120 +238,4 @@ void test_shrpx_config_read_tls_ticket_key_file_aes_256(void) {
"a..............................b"));
}
void test_shrpx_config_match_downstream_addr_group(void) {
auto groups = std::vector<DownstreamAddrGroup>{
{"nghttp2.org/"},
{"nghttp2.org/alpha/bravo/"},
{"nghttp2.org/alpha/charlie"},
{"nghttp2.org/delta%3A"},
{"www.nghttp2.org/"},
{"[::1]/"},
{"nghttp2.org/alpha/bravo/delta"},
// Check that match is done in the single node
{"example.com/alpha/bravo"},
{"192.168.0.1/alpha/"},
};
Router router;
for (size_t i = 0; i < groups.size(); ++i) {
auto &g = groups[i];
router.add_route(StringRef{g.pattern}, i);
}
CU_ASSERT(0 == match_downstream_addr_group(router, "nghttp2.org", "/", groups,
255));
// port is removed
CU_ASSERT(0 == match_downstream_addr_group(router, "nghttp2.org:8080", "/",
groups, 255));
// host is case-insensitive
CU_ASSERT(4 == match_downstream_addr_group(router, "WWW.nghttp2.org",
"/alpha", groups, 255));
CU_ASSERT(1 == match_downstream_addr_group(router, "nghttp2.org",
"/alpha/bravo/", groups, 255));
// /alpha/bravo also matches /alpha/bravo/
CU_ASSERT(1 == match_downstream_addr_group(router, "nghttp2.org",
"/alpha/bravo", groups, 255));
// path part is case-sensitive
CU_ASSERT(0 == match_downstream_addr_group(router, "nghttp2.org",
"/Alpha/bravo", groups, 255));
CU_ASSERT(1 == match_downstream_addr_group(router, "nghttp2.org",
"/alpha/bravo/charlie", groups,
255));
CU_ASSERT(2 == match_downstream_addr_group(router, "nghttp2.org",
"/alpha/charlie", groups, 255));
// pattern which does not end with '/' must match its entirely. So
// this matches to group 0, not group 2.
CU_ASSERT(0 == match_downstream_addr_group(router, "nghttp2.org",
"/alpha/charlie/", groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(router, "example.org", "/",
groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(router, "", "/", groups, 255));
CU_ASSERT(255 ==
match_downstream_addr_group(router, "", "alpha", groups, 255));
CU_ASSERT(255 ==
match_downstream_addr_group(router, "foo/bar", "/", groups, 255));
// If path is "*", only match with host + "/".
CU_ASSERT(0 == match_downstream_addr_group(router, "nghttp2.org", "*", groups,
255));
CU_ASSERT(5 ==
match_downstream_addr_group(router, "[::1]", "/", groups, 255));
CU_ASSERT(
5 == match_downstream_addr_group(router, "[::1]:8080", "/", groups, 255));
CU_ASSERT(255 ==
match_downstream_addr_group(router, "[::1", "/", groups, 255));
CU_ASSERT(255 ==
match_downstream_addr_group(router, "[::1]8000", "/", groups, 255));
// Check the case where adding route extends tree
CU_ASSERT(6 == match_downstream_addr_group(
router, "nghttp2.org", "/alpha/bravo/delta", groups, 255));
CU_ASSERT(1 == match_downstream_addr_group(router, "nghttp2.org",
"/alpha/bravo/delta/", groups,
255));
// Check the case where query is done in a single node
CU_ASSERT(7 == match_downstream_addr_group(router, "example.com",
"/alpha/bravo", groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(router, "example.com",
"/alpha/bravo/", groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(router, "example.com", "/alpha",
groups, 255));
// Check the case where quey is done in a single node
CU_ASSERT(8 == match_downstream_addr_group(router, "192.168.0.1", "/alpha",
groups, 255));
CU_ASSERT(8 == match_downstream_addr_group(router, "192.168.0.1", "/alpha/",
groups, 255));
CU_ASSERT(8 == match_downstream_addr_group(router, "192.168.0.1",
"/alpha/bravo", groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(router, "192.168.0.1", "/alph",
groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(router, "192.168.0.1", "/",
groups, 255));
router.dump();
}
} // namespace shrpx

5
src/shrpx_connection.cc
View File

@ -47,7 +47,7 @@ Connection::Connection(struct ev_loop *loop, int fd, SSL *ssl,
const RateLimitConfig &read_limit, IOCb writecb,
IOCb readcb, TimerCb timeoutcb, void *data,
size_t tls_dyn_rec_warmup_threshold,
ev_tstamp tls_dyn_rec_idle_timeout)
ev_tstamp tls_dyn_rec_idle_timeout, shrpx_proto proto)
: tls{DefaultMemchunks(mcpool), DefaultPeekMemchunks(mcpool)},
wlimit(loop, &wev, write_limit.rate, write_limit.burst),
rlimit(loop, &rev, read_limit.rate, read_limit.burst, this),
@ -58,7 +58,8 @@ Connection::Connection(struct ev_loop *loop, int fd, SSL *ssl,
data(data),
fd(fd),
tls_dyn_rec_warmup_threshold(tls_dyn_rec_warmup_threshold),
tls_dyn_rec_idle_timeout(tls_dyn_rec_idle_timeout) {
tls_dyn_rec_idle_timeout(tls_dyn_rec_idle_timeout),
proto(proto) {
ev_io_init(&wev, writecb, fd, EV_WRITE);
ev_io_init(&rev, readcb, fd, EV_READ);

6
src/shrpx_connection.h
View File

@ -77,7 +77,7 @@ struct Connection {
const RateLimitConfig &write_limit,
const RateLimitConfig &read_limit, IOCb writecb, IOCb readcb,
TimerCb timeoutcb, void *data, size_t tls_dyn_rec_warmup_threshold,
ev_tstamp tls_dyn_rec_idle_timeout);
ev_tstamp tls_dyn_rec_idle_timeout, shrpx_proto proto);
~Connection();
void disconnect();
@ -133,6 +133,10 @@ struct Connection {
int fd;
size_t tls_dyn_rec_warmup_threshold;
ev_tstamp tls_dyn_rec_idle_timeout;
// Application protocol used over the connection. This field is not
// used in this object at the moment. The rest of the program may
// use this value when it is useful.
shrpx_proto proto;
};
} // namespace shrpx

6
src/shrpx_connection_handler.cc
View File

@ -203,7 +203,7 @@ int ConnectionHandler::create_single_worker() {
nb_.get(),
#endif // HAVE_NEVERBLEED
StringRef{tlsconf.cacert}, StringRef{memcachedconf.cert_file},
StringRef{memcachedconf.private_key_file}, StringRef(), nullptr);
StringRef{memcachedconf.private_key_file}, nullptr);
all_ssl_ctx_.push_back(session_cache_ssl_ctx);
}
@ -253,7 +253,7 @@ int ConnectionHandler::create_worker_thread(size_t num) {
nb_.get(),
#endif // HAVE_NEVERBLEED
StringRef{tlsconf.cacert}, StringRef{memcachedconf.cert_file},
StringRef{memcachedconf.private_key_file}, StringRef{}, nullptr);
StringRef{memcachedconf.private_key_file}, nullptr);
all_ssl_ctx_.push_back(session_cache_ssl_ctx);
}
auto worker =
@ -767,7 +767,7 @@ SSL_CTX *ConnectionHandler::create_tls_ticket_key_memcached_ssl_ctx() {
nb_.get(),
#endif // HAVE_NEVERBLEED
StringRef{tlsconf.cacert}, StringRef{memcachedconf.cert_file},
StringRef{memcachedconf.private_key_file}, StringRef{}, nullptr);
StringRef{memcachedconf.private_key_file}, nullptr);
all_ssl_ctx_.push_back(ssl_ctx);

9
src/shrpx_downstream_connection.cc
View File

@ -26,12 +26,11 @@
#include "shrpx_client_handler.h"
#include "shrpx_downstream.h"
#include "shrpx_downstream_connection_pool.h"
namespace shrpx {
DownstreamConnection::DownstreamConnection(DownstreamConnectionPool *dconn_pool)
: dconn_pool_(dconn_pool), client_handler_(nullptr), downstream_(nullptr) {}
DownstreamConnection::DownstreamConnection()
: client_handler_(nullptr), downstream_(nullptr) {}
DownstreamConnection::~DownstreamConnection() {}
@ -45,8 +44,4 @@ ClientHandler *DownstreamConnection::get_client_handler() {
Downstream *DownstreamConnection::get_downstream() { return downstream_; }
DownstreamConnectionPool *DownstreamConnection::get_dconn_pool() const {
return dconn_pool_;
}
} // namespace shrpx

9
src/shrpx_downstream_connection.h
View File

@ -34,11 +34,11 @@ namespace shrpx {
class ClientHandler;
class Upstream;
class Downstream;
class DownstreamConnectionPool;
struct DownstreamAddrGroup;
class DownstreamConnection {
public:
DownstreamConnection(DownstreamConnectionPool *dconn_pool);
DownstreamConnection();
virtual ~DownstreamConnection();
virtual int attach_downstream(Downstream *downstream) = 0;
virtual void detach_downstream(Downstream *downstream) = 0;
@ -56,18 +56,17 @@ public:
virtual int on_timeout() { return 0; }
virtual void on_upstream_change(Upstream *uptream) = 0;
virtual size_t get_group() const = 0;
// true if this object is poolable.
virtual bool poolable() const = 0;
virtual DownstreamAddrGroup *get_downstream_addr_group() const = 0;
void set_client_handler(ClientHandler *client_handler);
ClientHandler *get_client_handler();
Downstream *get_downstream();
DownstreamConnectionPool *get_dconn_pool() const;
protected:
DownstreamConnectionPool *dconn_pool_;
ClientHandler *client_handler_;
Downstream *downstream_;
};

29
src/shrpx_downstream_connection_pool.cc
View File

@ -27,42 +27,35 @@
namespace shrpx {
DownstreamConnectionPool::DownstreamConnectionPool(size_t num_groups)
: gpool_(num_groups) {}
DownstreamConnectionPool::DownstreamConnectionPool() {}
DownstreamConnectionPool::~DownstreamConnectionPool() {
for (auto &pool : gpool_) {
for (auto dconn : pool) {
delete dconn;
}
for (auto dconn : pool_) {
delete dconn;
}
}
void DownstreamConnectionPool::add_downstream_connection(
std::unique_ptr<DownstreamConnection> dconn) {
auto group = dconn->get_group();
assert(gpool_.size() > group);
gpool_[group].insert(dconn.release());
pool_.insert(dconn.release());
}
std::unique_ptr<DownstreamConnection>
DownstreamConnectionPool::pop_downstream_connection(size_t group) {
assert(gpool_.size() > group);
auto &pool = gpool_[group];
if (pool.empty()) {
DownstreamConnectionPool::pop_downstream_connection() {
if (pool_.empty()) {
return nullptr;
}
auto dconn = std::unique_ptr<DownstreamConnection>(*std::begin(pool));
pool.erase(std::begin(pool));
auto it = std::begin(pool_);
auto dconn = std::unique_ptr<DownstreamConnection>(*it);
pool_.erase(it);
return dconn;
}
void DownstreamConnectionPool::remove_downstream_connection(
DownstreamConnection *dconn) {
auto group = dconn->get_group();
assert(gpool_.size() > group);
gpool_[group].erase(dconn);
pool_.erase(dconn);
delete dconn;
}

6
src/shrpx_downstream_connection_pool.h
View File

@ -36,15 +36,15 @@ class DownstreamConnection;
class DownstreamConnectionPool {
public:
DownstreamConnectionPool(size_t num_groups);
DownstreamConnectionPool();
~DownstreamConnectionPool();
void add_downstream_connection(std::unique_ptr<DownstreamConnection> dconn);
std::unique_ptr<DownstreamConnection> pop_downstream_connection(size_t group);
std::unique_ptr<DownstreamConnection> pop_downstream_connection();
void remove_downstream_connection(DownstreamConnection *dconn);
private:
std::vector<std::set<DownstreamConnection *>> gpool_;
std::set<DownstreamConnection *> pool_;
};
} // namespace shrpx

4
src/shrpx_downstream_test.cc
View File

@ -77,8 +77,8 @@ void test_downstream_field_store_header(void) {
CU_ASSERT(nullptr == fs.header(http2::HD__METHOD));
// By name
CU_ASSERT(Header("alpha", "0") == *fs.header("alpha"));
CU_ASSERT(nullptr == fs.header("bravo"));
CU_ASSERT(Header("alpha", "0") == *fs.header(StringRef::from_lit("alpha")));
CU_ASSERT(nullptr == fs.header(StringRef::from_lit("bravo")));
}
void test_downstream_crumble_request_cookie(void) {

5
src/shrpx_http.cc
View File

@ -62,9 +62,8 @@ std::string create_via_header_value(int major, int minor) {
}
std::string create_forwarded(int params, const StringRef &node_by,
const std::string &node_for,
const std::string &host,
const std::string &proto) {
const StringRef &node_for, const StringRef &host,
const StringRef &proto) {
std::string res;
if ((params & FORWARDED_BY) && !node_by.empty()) {
// This must be quoted-string unless it is obfuscated version

4
src/shrpx_http.h
View File

@ -43,8 +43,8 @@ std::string create_via_header_value(int major, int minor);
// |params| is bitwise-OR of zero or more of shrpx_forwarded_param
// defined in shrpx_config.h.
std::string create_forwarded(int params, const StringRef &node_by,
const std::string &node_for,
const std::string &host, const std::string &proto);
const StringRef &node_for, const StringRef &host,
const StringRef &proto);
// Adds ANSI color codes to HTTP headers |hdrs|.
std::string colorizeHeaders(const char *hdrs);

39
src/shrpx_http2_downstream_connection.cc
View File

@ -37,6 +37,7 @@
#include "shrpx_error.h"
#include "shrpx_http.h"
#include "shrpx_http2_session.h"
#include "shrpx_worker.h"
#include "http2.h"
#include "util.h"
@ -44,10 +45,8 @@ using namespace nghttp2;
namespace shrpx {
Http2DownstreamConnection::Http2DownstreamConnection(
DownstreamConnectionPool *dconn_pool, Http2Session *http2session)
: DownstreamConnection(dconn_pool),
dlnext(nullptr),
Http2DownstreamConnection::Http2DownstreamConnection(Http2Session *http2session)
: dlnext(nullptr),
dlprev(nullptr),
http2session_(http2session),
sd_(nullptr) {}
@ -106,6 +105,11 @@ int Http2DownstreamConnection::attach_downstream(Downstream *downstream) {
downstream_ = downstream;
downstream_->reset_downstream_rtimer();
auto &req = downstream_->request();
// HTTP/2 disables HTTP Upgrade.
req.upgrade_request = false;
return 0;
}
@ -265,9 +269,9 @@ int Http2DownstreamConnection::push_request_headers() {
auto &httpconf = get_config()->http;
auto &http2conf = get_config()->http2;
auto no_host_rewrite =
httpconf.no_host_rewrite || get_config()->http2_proxy ||
get_config()->client_proxy || req.method == HTTP_CONNECT;
auto no_host_rewrite = httpconf.no_host_rewrite ||
get_config()->http2_proxy ||
req.method == HTTP_CONNECT;
// http2session_ has already in CONNECTED state, so we can get
// addr_idx here.
@ -303,7 +307,7 @@ int Http2DownstreamConnection::push_request_headers() {
httpconf.add_request_headers.size());
nva.push_back(
http2::make_nv_lc_nocopy(":method", http2::to_method_string(req.method)));
http2::make_nv_ls_nocopy(":method", http2::to_method_string(req.method)));
if (req.method != HTTP_CONNECT) {
assert(!req.scheme.empty());
@ -351,14 +355,13 @@ int Http2DownstreamConnection::push_request_headers() {
if (fwdconf.params) {
auto params = fwdconf.params;
if (get_config()->http2_proxy || get_config()->client_proxy ||
req.method == HTTP_CONNECT) {
if (get_config()->http2_proxy || req.method == HTTP_CONNECT) {
params &= ~FORWARDED_PROTO;
}
auto value = http::create_forwarded(params, handler->get_forwarded_by(),
handler->get_forwarded_for(),
req.authority, req.scheme);
auto value = http::create_forwarded(
params, handler->get_forwarded_by(), handler->get_forwarded_for(),
StringRef{req.authority}, StringRef{req.scheme});
if (fwd || !value.empty()) {
if (fwd) {
forwarded_value = fwd->value;
@ -395,8 +398,7 @@ int Http2DownstreamConnection::push_request_headers() {
nva.push_back(http2::make_nv_ls_nocopy("x-forwarded-for", (*xff).value));
}
if (!get_config()->http2_proxy && !get_config()->client_proxy &&
req.method != HTTP_CONNECT) {
if (!get_config()->http2_proxy && req.method != HTTP_CONNECT) {
// We use same protocol with :scheme header field
nva.push_back(http2::make_nv_ls_nocopy("x-forwarded-proto", req.scheme));
}
@ -557,10 +559,9 @@ int Http2DownstreamConnection::on_timeout() {
return submit_rst_stream(downstream_, NGHTTP2_NO_ERROR);
}
size_t Http2DownstreamConnection::get_group() const {
// HTTP/2 backend connections are managed by Http2Session object,
// and it stores group index.
return http2session_->get_group();
DownstreamAddrGroup *
Http2DownstreamConnection::get_downstream_addr_group() const {
return http2session_->get_downstream_addr_group();
}
} // namespace shrpx

6
src/shrpx_http2_downstream_connection.h
View File

@ -41,8 +41,7 @@ class DownstreamConnectionPool;
class Http2DownstreamConnection : public DownstreamConnection {
public:
Http2DownstreamConnection(DownstreamConnectionPool *dconn_pool,
Http2Session *http2session);
Http2DownstreamConnection(Http2Session *http2session);
virtual ~Http2DownstreamConnection();
virtual int attach_downstream(Downstream *downstream);
virtual void detach_downstream(Downstream *downstream);
@ -60,12 +59,13 @@ public:
virtual int on_timeout();
virtual void on_upstream_change(Upstream *upstream) {}
virtual size_t get_group() const;
// This object is not poolable because we dont' have facility to
// migrate to another Http2Session object.
virtual bool poolable() const { return false; }
virtual DownstreamAddrGroup *get_downstream_addr_group() const;
int send();
void attach_stream_data(StreamData *sd);

114
src/shrpx_http2_session.cc
View File

@ -74,6 +74,10 @@ void connchk_timeout_cb(struct ev_loop *loop, ev_timer *w, int revents) {
SSLOG(INFO, http2session) << "ping timeout";
}
http2session->disconnect();
if (http2session->get_num_dconns() == 0) {
delete http2session;
}
return;
default:
if (LOG_ENABLED(INFO)) {
@ -92,6 +96,9 @@ void settings_timeout_cb(struct ev_loop *loop, ev_timer *w, int revents) {
SSLOG(INFO, http2session) << "SETTINGS timeout";
if (http2session->terminate_session(NGHTTP2_SETTINGS_TIMEOUT) != 0) {
http2session->disconnect();
if (http2session->get_num_dconns() == 0) {
delete http2session;
}
return;
}
http2session->signal_write();
@ -109,6 +116,9 @@ void timeoutcb(struct ev_loop *loop, ev_timer *w, int revents) {
http2session->disconnect(http2session->get_state() ==
Http2Session::CONNECTING);
if (http2session->get_num_dconns() == 0) {
delete http2session;
}
}
} // namespace
@ -120,6 +130,9 @@ void readcb(struct ev_loop *loop, ev_io *w, int revents) {
rv = http2session->do_read();
if (rv != 0) {
http2session->disconnect(http2session->should_hard_fail());
if (http2session->get_num_dconns() == 0) {
delete http2session;
}
return;
}
http2session->connection_alive();
@ -127,6 +140,9 @@ void readcb(struct ev_loop *loop, ev_io *w, int revents) {
rv = http2session->do_write();
if (rv != 0) {
http2session->disconnect(http2session->should_hard_fail());
if (http2session->get_num_dconns() == 0) {
delete http2session;
}
return;
}
}
@ -140,6 +156,9 @@ void writecb(struct ev_loop *loop, ev_io *w, int revents) {
rv = http2session->do_write();
if (rv != 0) {
http2session->disconnect(http2session->should_hard_fail());
if (http2session->get_num_dconns() == 0) {
delete http2session;
}
return;
}
http2session->reset_connection_check_timer_if_not_checking();
@ -147,23 +166,23 @@ void writecb(struct ev_loop *loop, ev_io *w, int revents) {
} // namespace
Http2Session::Http2Session(struct ev_loop *loop, SSL_CTX *ssl_ctx,
Worker *worker, size_t group, size_t idx)
: conn_(loop, -1, nullptr, worker->get_mcpool(),
Worker *worker, DownstreamAddrGroup *group)
: dlnext(nullptr),
dlprev(nullptr),
conn_(loop, -1, nullptr, worker->get_mcpool(),
get_config()->conn.downstream.timeout.write,
get_config()->conn.downstream.timeout.read, {}, {}, writecb, readcb,
timeoutcb, this, get_config()->tls.dyn_rec.warmup_threshold,
get_config()->tls.dyn_rec.idle_timeout),
get_config()->tls.dyn_rec.idle_timeout, PROTO_HTTP2),
wb_(worker->get_mcpool()),
worker_(worker),
ssl_ctx_(ssl_ctx),
group_(group),
addr_(nullptr),
session_(nullptr),
group_(group),
index_(idx),
state_(DISCONNECTED),
connection_check_state_(CONNECTION_CHECK_NONE),
flow_control_(false) {
read_ = write_ = &Http2Session::noop;
on_read_ = &Http2Session::read_noop;
@ -182,7 +201,16 @@ Http2Session::Http2Session(struct ev_loop *loop, SSL_CTX *ssl_ctx,
settings_timer_.data = this;
}
Http2Session::~Http2Session() { disconnect(); }
Http2Session::~Http2Session() {
disconnect();
if (in_freelist()) {
if (LOG_ENABLED(INFO)) {
SSLOG(INFO, this) << "Removed from http2_freelist";
}
group_->http2_freelist.remove(this);
}
}
int Http2Session::disconnect(bool hard) {
if (LOG_ENABLED(INFO)) {
@ -252,8 +280,7 @@ int Http2Session::disconnect(bool hard) {
int Http2Session::initiate_connection() {
int rv = 0;
auto &groups = worker_->get_downstream_addr_groups();
auto &addrs = groups[group_].addrs;
auto &addrs = group_->addrs;
auto worker_blocker = worker_->get_connect_blocker();
if (state_ == DISCONNECTED) {
@ -265,7 +292,7 @@ int Http2Session::initiate_connection() {
return -1;
}
auto &next_downstream = worker_->get_dgrp(group_)->next;
auto &next_downstream = group_->next;
auto end = next_downstream;
for (;;) {
@ -371,6 +398,8 @@ int Http2Session::initiate_connection() {
return -1;
}
ssl::setup_downstream_http2_alpn(ssl);
conn_.set_ssl(ssl);
}
@ -384,6 +413,13 @@ int Http2Session::initiate_connection() {
// at the time of this writing).
SSL_set_tlsext_host_name(conn_.tls.ssl, sni_name.c_str());
}
auto tls_session = ssl::reuse_tls_session(addr_);
if (tls_session) {
SSL_set_session(conn_.tls.ssl, tls_session);
SSL_SESSION_free(tls_session);
}
// If state_ == PROXY_CONNECTED, we has connected to the proxy
// using conn_.fd and tunnel has been established.
if (state_ == DISCONNECTED) {
@ -598,6 +634,17 @@ void Http2Session::remove_downstream_connection(
Http2DownstreamConnection *dconn) {
dconns_.remove(dconn);
dconn->detach_stream_data();
if (LOG_ENABLED(INFO)) {
SSLOG(INFO, this) << "Remove downstream";
}
if (!in_freelist() && !max_concurrency_reached()) {
if (LOG_ENABLED(INFO)) {
SSLOG(INFO, this) << "Append to Http2Session freelist";
}
group_->http2_freelist.append(this);
}
}
void Http2Session::remove_stream_data(StreamData *sd) {
@ -1408,13 +1455,12 @@ int Http2Session::connection_made() {
std::array<nghttp2_settings_entry, 3> entry;
size_t nentry = 2;
entry[0].settings_id = NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS;
entry[0].value = http2conf.max_concurrent_streams;
entry[0].value = http2conf.downstream.max_concurrent_streams;
entry[1].settings_id = NGHTTP2_SETTINGS_INITIAL_WINDOW_SIZE;
entry[1].value = (1 << http2conf.downstream.window_bits) - 1;
if (http2conf.no_server_push || get_config()->http2_proxy ||
get_config()->client_proxy) {
if (http2conf.no_server_push || get_config()->http2_proxy) {
entry[nentry].settings_id = NGHTTP2_SETTINGS_ENABLE_PUSH;
entry[nentry].value = 0;
++nentry;
@ -1800,6 +1846,13 @@ int Http2Session::tls_handshake() {
return -1;
}
if (!SSL_session_reused(conn_.tls.ssl)) {
auto tls_session = SSL_get0_session(conn_.tls.ssl);
if (tls_session) {
ssl::try_cache_tls_session(addr_, tls_session, ev_now(conn_.loop));
}
}
read_ = &Http2Session::read_tls;
write_ = &Http2Session::write_tls;
@ -1888,11 +1941,7 @@ bool Http2Session::should_hard_fail() const {
}
}
const DownstreamAddr *Http2Session::get_addr() const { return addr_; }
size_t Http2Session::get_group() const { return group_; }
size_t Http2Session::get_index() const { return index_; }
DownstreamAddr *Http2Session::get_addr() const { return addr_; }
int Http2Session::handle_downstream_push_promise(Downstream *downstream,
int32_t promised_stream_id) {
@ -1911,10 +1960,8 @@ int Http2Session::handle_downstream_push_promise(Downstream *downstream,
// promised_downstream->get_stream() still returns 0.
auto handler = upstream->get_client_handler();
auto worker = handler->get_worker();
auto promised_dconn =
make_unique<Http2DownstreamConnection>(worker->get_dconn_pool(), this);
auto promised_dconn = make_unique<Http2DownstreamConnection>(this);
promised_dconn->set_client_handler(handler);
auto ptr = promised_dconn.get();
@ -1986,4 +2033,29 @@ int Http2Session::handle_downstream_push_promise_complete(
return 0;
}
size_t Http2Session::get_num_dconns() const { return dconns_.size(); }
bool Http2Session::in_freelist() const {
return dlnext != nullptr || dlprev != nullptr ||
group_->http2_freelist.head == this ||
group_->http2_freelist.tail == this;
}
bool Http2Session::max_concurrency_reached(size_t extra) const {
if (!session_) {
return dconns_.size() + extra >= 100;
}
// If session does not allow further requests, it effectively means
// that maximum concurrency is reached.
return !nghttp2_session_check_request_allowed(session_) ||
dconns_.size() + extra >=
nghttp2_session_get_remote_settings(
session_, NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS);
}
DownstreamAddrGroup *Http2Session::get_downstream_addr_group() const {
return group_;
}
} // namespace shrpx

35
src/shrpx_http2_session.h
View File

@ -48,6 +48,8 @@ namespace shrpx {
class Http2DownstreamConnection;
class Worker;
struct DownstreamAddrGroup;
struct DownstreamAddr;
struct StreamData {
StreamData *dlnext, *dlprev;
@ -57,7 +59,7 @@ struct StreamData {
class Http2Session {
public:
Http2Session(struct ev_loop *loop, SSL_CTX *ssl_ctx, Worker *worker,
size_t group, size_t idx);
DownstreamAddrGroup *group);
~Http2Session();
// If hard is true, all pending requests are abandoned and
@ -145,17 +147,31 @@ public:
void submit_pending_requests();
const DownstreamAddr *get_addr() const;
DownstreamAddr *get_addr() const;
size_t get_group() const;
size_t get_index() const;
DownstreamAddrGroup *get_downstream_addr_group() const;
int handle_downstream_push_promise(Downstream *downstream,
int32_t promised_stream_id);
int handle_downstream_push_promise_complete(Downstream *downstream,
Downstream *promised_downstream);
// Returns number of downstream connections, including pushed
// streams.
size_t get_num_dconns() const;
// Returns true if this object is included in freelist. See
// DownstreamAddrGroup object.
bool in_freelist() const;
// Returns true if the maximum concurrency is reached. In other
// words, the number of currently participated streams in this
// session is equal or greater than the max concurrent streams limit
// advertised by server. If |extra| is nonzero, it is added to the
// number of current concurrent streams when comparing against
// server initiated concurrency limit.
bool max_concurrency_reached(size_t extra = 0) const;
enum {
// Disconnected
DISCONNECTED,
@ -184,6 +200,8 @@ public:
using ReadBuf = Buffer<8_k>;
Http2Session *dlnext, *dlprev;
private:
Connection conn_;
DefaultMemchunks wb_;
@ -203,13 +221,10 @@ private:
Worker *worker_;
// NULL if no TLS is configured
SSL_CTX *ssl_ctx_;
DownstreamAddrGroup *group_;
// Address of remote endpoint
const DownstreamAddr *addr_;
DownstreamAddr *addr_;
nghttp2_session *session_;
size_t group_;
// index inside group, this is used to pin frontend to certain
// HTTP/2 backend for better throughput.
size_t index_;
int state_;
int connection_check_state_;
bool flow_control_;

20
src/shrpx_http2_upstream.cc
View File

@ -316,7 +316,7 @@ int Http2Upstream::on_request_headers(Downstream *downstream,
if (path) {
if (method_token == HTTP_OPTIONS && path->value == "*") {
// Server-wide OPTIONS request. Path is empty.
} else if (get_config()->http2_proxy || get_config()->client_proxy) {
} else if (get_config()->http2_proxy) {
req.path = http2::value_to_str(path);
} else {
const auto &value = path->value;
@ -824,9 +824,7 @@ Http2Upstream::Http2Upstream(ClientHandler *handler)
downstream_queue_(
get_config()->http2_proxy
? get_config()->conn.downstream.connections_per_host
: get_config()->conn.downstream.proto == PROTO_HTTP
? get_config()->conn.downstream.connections_per_frontend
: 0,
: get_config()->conn.downstream.connections_per_frontend,
!get_config()->http2_proxy),
handler_(handler),
session_(nullptr),
@ -846,7 +844,7 @@ Http2Upstream::Http2Upstream(ClientHandler *handler)
// TODO Maybe call from outside?
std::array<nghttp2_settings_entry, 2> entry;
entry[0].settings_id = NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS;
entry[0].value = http2conf.max_concurrent_streams;
entry[0].value = http2conf.upstream.max_concurrent_streams;
entry[1].settings_id = NGHTTP2_SETTINGS_INITIAL_WINDOW_SIZE;
entry[1].value = (1 << http2conf.upstream.window_bits) - 1;
@ -1348,8 +1346,7 @@ int Http2Upstream::on_downstream_header_complete(Downstream *downstream) {
auto &httpconf = get_config()->http;
if (!get_config()->http2_proxy && !get_config()->client_proxy &&
!httpconf.no_location_rewrite) {
if (!get_config()->http2_proxy && !httpconf.no_location_rewrite) {
downstream->rewrite_location_response_header(req.scheme);
}
@ -1418,7 +1415,7 @@ int Http2Upstream::on_downstream_header_complete(Downstream *downstream) {
http2::copy_headers_to_nva_nocopy(nva, resp.fs.headers());
if (!get_config()->http2_proxy && !get_config()->client_proxy) {
if (!get_config()->http2_proxy) {
nva.push_back(http2::make_nv_ls_nocopy("server", httpconf.server_name));
} else {
auto server = resp.fs.header(http2::HD_SERVER);
@ -1491,9 +1488,8 @@ int Http2Upstream::on_downstream_header_complete(Downstream *downstream) {
if (!http2conf.no_server_push &&
nghttp2_session_get_remote_settings(session_,
NGHTTP2_SETTINGS_ENABLE_PUSH) == 1 &&
!get_config()->http2_proxy && !get_config()->client_proxy &&
(downstream->get_stream_id() % 2) && resp.fs.header(http2::HD_LINK) &&
resp.http_status == 200 &&
!get_config()->http2_proxy && (downstream->get_stream_id() % 2) &&
resp.fs.header(http2::HD_LINK) && resp.http_status == 200 &&
(req.method == HTTP_GET || req.method == HTTP_POST)) {
if (prepare_push_promise(downstream) != 0) {
@ -1854,7 +1850,7 @@ bool Http2Upstream::push_enabled() const {
return !(get_config()->http2.no_server_push ||
nghttp2_session_get_remote_settings(
session_, NGHTTP2_SETTINGS_ENABLE_PUSH) == 0 ||
get_config()->http2_proxy || get_config()->client_proxy);
get_config()->http2_proxy);
}
int Http2Upstream::initiate_push(Downstream *downstream, const char *uri,

60
src/shrpx_http_downstream_connection.cc
View File

@ -111,23 +111,22 @@ void connectcb(struct ev_loop *loop, ev_io *w, int revents) {
}
} // namespace
HttpDownstreamConnection::HttpDownstreamConnection(
DownstreamConnectionPool *dconn_pool, size_t group, struct ev_loop *loop,
Worker *worker)
: DownstreamConnection(dconn_pool),
conn_(loop, -1, nullptr, worker->get_mcpool(),
HttpDownstreamConnection::HttpDownstreamConnection(DownstreamAddrGroup *group,
struct ev_loop *loop,
Worker *worker)
: conn_(loop, -1, nullptr, worker->get_mcpool(),
get_config()->conn.downstream.timeout.write,
get_config()->conn.downstream.timeout.read, {}, {}, connectcb,
readcb, timeoutcb, this, get_config()->tls.dyn_rec.warmup_threshold,
get_config()->tls.dyn_rec.idle_timeout),
get_config()->tls.dyn_rec.idle_timeout, PROTO_HTTP1),
do_read_(&HttpDownstreamConnection::noop),
do_write_(&HttpDownstreamConnection::noop),
worker_(worker),
ssl_ctx_(worker->get_cl_ssl_ctx()),
group_(group),
addr_(nullptr),
ioctrl_(&conn_.rlimit),
response_htp_{0},
group_(group) {}
response_htp_{0} {}
HttpDownstreamConnection::~HttpDownstreamConnection() {}
@ -154,13 +153,14 @@ int HttpDownstreamConnection::attach_downstream(Downstream *downstream) {
return -1;
}
ssl::setup_downstream_http1_alpn(ssl);
conn_.set_ssl(ssl);
}
auto &next_downstream = worker_->get_dgrp(group_)->next;
auto &addrs = group_->addrs;
auto &next_downstream = group_->next;
auto end = next_downstream;
auto &groups = worker_->get_downstream_addr_groups();
auto &addrs = groups[group_].addrs;
for (;;) {
auto &addr = addrs[next_downstream];
@ -279,9 +279,8 @@ int HttpDownstreamConnection::push_request_headers() {
// For HTTP/1.0 request, there is no authority in request. In that
// case, we use backend server's host nonetheless.
auto authority = StringRef(downstream_hostport);
auto no_host_rewrite = httpconf.no_host_rewrite ||
get_config()->http2_proxy ||
get_config()->client_proxy || connect_method;
auto no_host_rewrite =
httpconf.no_host_rewrite || get_config()->http2_proxy || connect_method;
if (no_host_rewrite && !req.authority.empty()) {
authority = StringRef(req.authority);
@ -293,12 +292,12 @@ int HttpDownstreamConnection::push_request_headers() {
// Assume that method and request path do not contain \r\n.
auto meth = http2::to_method_string(req.method);
buf->append(meth, strlen(meth));
buf->append(meth);
buf->append(" ");
if (connect_method) {
buf->append(authority);
} else if (get_config()->http2_proxy || get_config()->client_proxy) {
} else if (get_config()->http2_proxy) {
// Construct absolute-form request target because we are going to
// send a request to a HTTP/1 proxy.
assert(!req.scheme.empty());
@ -363,14 +362,13 @@ int HttpDownstreamConnection::push_request_headers() {
if (fwdconf.params) {
auto params = fwdconf.params;
if (get_config()->http2_proxy || get_config()->client_proxy ||
connect_method) {
if (get_config()->http2_proxy || connect_method) {
params &= ~FORWARDED_PROTO;
}
auto value = http::create_forwarded(params, handler->get_forwarded_by(),
handler->get_forwarded_for(),
req.authority, req.scheme);
auto value = http::create_forwarded(
params, handler->get_forwarded_by(), handler->get_forwarded_for(),
StringRef{req.authority}, StringRef{req.scheme});
if (fwd || !value.empty()) {
buf->append("Forwarded: ");
if (fwd) {
@ -407,8 +405,7 @@ int HttpDownstreamConnection::push_request_headers() {
buf->append((*xff).value);
buf->append("\r\n");
}
if (!get_config()->http2_proxy && !get_config()->client_proxy &&
!connect_method) {
if (!get_config()->http2_proxy && !connect_method) {
buf->append("X-Forwarded-Proto: ");
assert(!req.scheme.empty());
buf->append(req.scheme);
@ -508,8 +505,8 @@ void idle_readcb(struct ev_loop *loop, ev_io *w, int revents) {
if (LOG_ENABLED(INFO)) {
DCLOG(INFO, dconn) << "Idle connection EOF";
}
auto dconn_pool = dconn->get_dconn_pool();
dconn_pool->remove_downstream_connection(dconn);
auto &dconn_pool = dconn->get_downstream_addr_group()->dconn_pool;
dconn_pool.remove_downstream_connection(dconn);
// dconn was deleted
}
} // namespace
@ -521,8 +518,8 @@ void idle_timeoutcb(struct ev_loop *loop, ev_timer *w, int revents) {
if (LOG_ENABLED(INFO)) {
DCLOG(INFO, dconn) << "Idle connection timeout";
}
auto dconn_pool = dconn->get_dconn_pool();
dconn_pool->remove_downstream_connection(dconn);
auto &dconn_pool = dconn->get_downstream_addr_group()->dconn_pool;
dconn_pool.remove_downstream_connection(dconn);
// dconn was deleted
}
} // namespace
@ -1033,7 +1030,7 @@ int HttpDownstreamConnection::process_input(const uint8_t *data,
}
int HttpDownstreamConnection::connected() {
auto connect_blocker = addr_->connect_blocker;
auto &connect_blocker = addr_->connect_blocker;
if (!util::check_socket_connected(conn_.fd)) {
conn_.wlimit.stopw();
@ -1083,8 +1080,11 @@ void HttpDownstreamConnection::signal_write() {
ev_feed_event(conn_.loop, &conn_.wev, EV_WRITE);
}
size_t HttpDownstreamConnection::get_group() const { return group_; }
int HttpDownstreamConnection::noop() { return 0; }
DownstreamAddrGroup *
HttpDownstreamConnection::get_downstream_addr_group() const {
return group_;
}
} // namespace shrpx

11
src/shrpx_http_downstream_connection.h
View File

@ -37,11 +37,13 @@ namespace shrpx {
class DownstreamConnectionPool;
class Worker;
struct DownstreamAddrGroup;
struct DownstreamAddr;
class HttpDownstreamConnection : public DownstreamConnection {
public:
HttpDownstreamConnection(DownstreamConnectionPool *dconn_pool, size_t group,
struct ev_loop *loop, Worker *worker);
HttpDownstreamConnection(DownstreamAddrGroup *group, struct ev_loop *loop,
Worker *worker);
virtual ~HttpDownstreamConnection();
virtual int attach_downstream(Downstream *downstream);
virtual void detach_downstream(Downstream *downstream);
@ -58,10 +60,11 @@ public:
virtual int on_write();
virtual void on_upstream_change(Upstream *upstream);
virtual size_t get_group() const;
virtual bool poolable() const { return true; }
virtual DownstreamAddrGroup *get_downstream_addr_group() const;
int read_clear();
int write_clear();
int read_tls();
@ -81,11 +84,11 @@ private:
Worker *worker_;
// nullptr if TLS is not used.
SSL_CTX *ssl_ctx_;
DownstreamAddrGroup *group_;
// Address of remote endpoint
DownstreamAddr *addr_;
IOControl ioctrl_;
http_parser response_htp_;
size_t group_;
};
} // namespace shrpx

37
src/shrpx_http_test.cc
View File

@ -40,25 +40,36 @@ namespace shrpx {
void test_shrpx_http_create_forwarded(void) {
CU_ASSERT("by=\"example.com:3000\";for=\"[::1]\";host=\"www.example.com\";"
"proto=https" ==
http::create_forwarded(
FORWARDED_BY | FORWARDED_FOR | FORWARDED_HOST | FORWARDED_PROTO,
"example.com:3000", "[::1]", "www.example.com", "https"));
http::create_forwarded(FORWARDED_BY | FORWARDED_FOR |
FORWARDED_HOST | FORWARDED_PROTO,
StringRef::from_lit("example.com:3000"),
StringRef::from_lit("[::1]"),
StringRef::from_lit("www.example.com"),
StringRef::from_lit("https")));
CU_ASSERT("for=192.168.0.1" == http::create_forwarded(FORWARDED_FOR, "alpha",
"192.168.0.1", "bravo",
"charlie"));
CU_ASSERT("for=192.168.0.1" ==
http::create_forwarded(FORWARDED_FOR, StringRef::from_lit("alpha"),
StringRef::from_lit("192.168.0.1"),
StringRef::from_lit("bravo"),
StringRef::from_lit("charlie")));
CU_ASSERT("by=_hidden;for=\"[::1]\"" ==
http::create_forwarded(FORWARDED_BY | FORWARDED_FOR, "_hidden",
"[::1]", "", ""));
http::create_forwarded(
FORWARDED_BY | FORWARDED_FOR, StringRef::from_lit("_hidden"),
StringRef::from_lit("[::1]"), StringRef::from_lit(""),
StringRef::from_lit("")));
CU_ASSERT("by=\"[::1]\";for=_hidden" ==
http::create_forwarded(FORWARDED_BY | FORWARDED_FOR, "[::1]",
"_hidden", "", ""));
http::create_forwarded(
FORWARDED_BY | FORWARDED_FOR, StringRef::from_lit("[::1]"),
StringRef::from_lit("_hidden"), StringRef::from_lit(""),
StringRef::from_lit("")));
CU_ASSERT("" == http::create_forwarded(FORWARDED_BY | FORWARDED_FOR |
FORWARDED_HOST | FORWARDED_PROTO,
"", "", "", ""));
CU_ASSERT("" ==
http::create_forwarded(
FORWARDED_BY | FORWARDED_FOR | FORWARDED_HOST | FORWARDED_PROTO,
StringRef::from_lit(""), StringRef::from_lit(""),
StringRef::from_lit(""), StringRef::from_lit("")));
}
} // namespace shrpx

9
src/shrpx_https_upstream.cc
View File

@ -232,7 +232,7 @@ void rewrite_request_host_path_from_uri(Request &req, const char *uri,
path += '?';
path.append(uri + fdata.off, fdata.len);
}
if (get_config()->http2_proxy || get_config()->client_proxy) {
if (get_config()->http2_proxy) {
req.path = std::move(path);
} else {
req.path = http2::rewrite_clean_path(std::begin(path), std::end(path));
@ -306,7 +306,7 @@ int htp_hdrs_completecb(http_parser *htp) {
}
// checking UF_HOST could be redundant, but just in case ...
if (!(u.field_set & (1 << UF_SCHEMA)) || !(u.field_set & (1 << UF_HOST))) {
if (get_config()->http2_proxy || get_config()->client_proxy) {
if (get_config()->http2_proxy) {
// Request URI should be absolute-form for client proxy mode
return -1;
}
@ -929,8 +929,7 @@ int HttpsUpstream::on_downstream_header_complete(Downstream *downstream) {
auto &httpconf = get_config()->http;
if (!get_config()->http2_proxy && !get_config()->client_proxy &&
!httpconf.no_location_rewrite) {
if (!get_config()->http2_proxy && !httpconf.no_location_rewrite) {
downstream->rewrite_location_response_header(
get_client_handler()->get_upstream_scheme());
}
@ -999,7 +998,7 @@ int HttpsUpstream::on_downstream_header_complete(Downstream *downstream) {
}
}
if (!get_config()->http2_proxy && !get_config()->client_proxy) {
if (!get_config()->http2_proxy) {
buf->append("Server: ");
buf->append(httpconf.server_name);
buf->append("\r\n");

2
src/shrpx_memcached_connection.cc
View File

@ -96,7 +96,7 @@ MemcachedConnection::MemcachedConnection(const Address *addr,
const StringRef &sni_name,
MemchunkPool *mcpool)
: conn_(loop, -1, nullptr, mcpool, write_timeout, read_timeout, {}, {},
connectcb, readcb, timeoutcb, this, 0, 0.),
connectcb, readcb, timeoutcb, this, 0, 0., PROTO_MEMCACHED),
do_read_(&MemcachedConnection::noop),
do_write_(&MemcachedConnection::noop),
sni_name_(sni_name.str()),

2
src/shrpx_mruby_module_request.cc
View File

@ -70,7 +70,7 @@ mrb_value request_get_method(mrb_state *mrb, mrb_value self) {
const auto &req = downstream->request();
auto method = http2::to_method_string(req.method);
return mrb_str_new_cstr(mrb, method);
return mrb_str_new(mrb, method.c_str(), method.size());
}
} // namespace

13
src/shrpx_spdy_upstream.cc
View File

@ -262,7 +262,7 @@ void on_ctrl_recv_callback(spdylay_session *session, spdylay_frame_type type,
} else {
req.scheme = scheme->value;
req.authority = host->value;
if (get_config()->http2_proxy || get_config()->client_proxy) {
if (get_config()->http2_proxy) {
req.path = path->value;
} else if (method_token == HTTP_OPTIONS && path->value == "*") {
// Server-wide OPTIONS request. Path is empty.
@ -503,9 +503,7 @@ SpdyUpstream::SpdyUpstream(uint16_t version, ClientHandler *handler)
downstream_queue_(
get_config()->http2_proxy
? get_config()->conn.downstream.connections_per_host
: get_config()->conn.downstream.proto == PROTO_HTTP
? get_config()->conn.downstream.connections_per_frontend
: 0,
: get_config()->conn.downstream.connections_per_frontend,
!get_config()->http2_proxy),
handler_(handler),
session_(nullptr) {
@ -547,7 +545,7 @@ SpdyUpstream::SpdyUpstream(uint16_t version, ClientHandler *handler)
// TODO Maybe call from outside?
std::array<spdylay_settings_entry, 2> entry;
entry[0].settings_id = SPDYLAY_SETTINGS_MAX_CONCURRENT_STREAMS;
entry[0].value = http2conf.max_concurrent_streams;
entry[0].value = http2conf.upstream.max_concurrent_streams;
entry[0].flags = SPDYLAY_ID_FLAG_SETTINGS_NONE;
entry[1].settings_id = SPDYLAY_SETTINGS_INITIAL_WINDOW_SIZE;
@ -1011,8 +1009,7 @@ int SpdyUpstream::on_downstream_header_complete(Downstream *downstream) {
auto &httpconf = get_config()->http;
if (!get_config()->http2_proxy && !get_config()->client_proxy &&
!httpconf.no_location_rewrite) {
if (!get_config()->http2_proxy && !httpconf.no_location_rewrite) {
downstream->rewrite_location_response_header(req.scheme);
}
@ -1046,7 +1043,7 @@ int SpdyUpstream::on_downstream_header_complete(Downstream *downstream) {
nv[hdidx++] = hd.value.c_str();
}
if (!get_config()->http2_proxy && !get_config()->client_proxy) {
if (!get_config()->http2_proxy) {
nv[hdidx++] = "server";
nv[hdidx++] = httpconf.server_name.c_str();
} else {

60
src/shrpx_ssl.cc
View File

@ -659,12 +659,28 @@ int select_h1_next_proto_cb(SSL *ssl, unsigned char **out,
}
} // namespace
namespace {
int select_next_proto_cb(SSL *ssl, unsigned char **out, unsigned char *outlen,
const unsigned char *in, unsigned int inlen,
void *arg) {
auto conn = static_cast<Connection *>(SSL_get_app_data(ssl));
switch (conn->proto) {
case PROTO_HTTP1:
return select_h1_next_proto_cb(ssl, out, outlen, in, inlen, arg);
case PROTO_HTTP2:
return select_h2_next_proto_cb(ssl, out, outlen, in, inlen, arg);
default:
return SSL_TLSEXT_ERR_NOACK;
}
}
} // namespace
SSL_CTX *create_ssl_client_context(
#ifdef HAVE_NEVERBLEED
neverbleed_t *nb,
#endif // HAVE_NEVERBLEED
const StringRef &cacert, const StringRef &cert_file,
const StringRef &private_key_file, const StringRef &alpn,
const StringRef &private_key_file,
int (*next_proto_select_cb)(SSL *s, unsigned char **out,
unsigned char *outlen, const unsigned char *in,
unsigned int inlen, void *arg)) {
@ -742,14 +758,10 @@ SSL_CTX *create_ssl_client_context(
#endif // HAVE_NEVERBLEED
}
// NPN selection callback
// NPN selection callback. This is required to set SSL_CTX because
// OpenSSL does not offer SSL_set_next_proto_select_cb.
SSL_CTX_set_next_proto_select_cb(ssl_ctx, next_proto_select_cb, nullptr);
#if OPENSSL_VERSION_NUMBER >= 0x10002000L
// ALPN advertisement
SSL_CTX_set_alpn_protos(ssl_ctx, alpn.byte(), alpn.size());
#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
return ssl_ctx;
}
@ -1303,29 +1315,29 @@ SSL_CTX *setup_downstream_client_ssl_context(
}
auto &tlsconf = get_config()->tls;
auto &downstreamconf = get_config()->conn.downstream;
std::vector<unsigned char> h2alpn;
StringRef alpn;
int (*next_proto_select_cb)(SSL *s, unsigned char **out,
unsigned char *outlen, const unsigned char *in,
unsigned int inlen, void *arg);
if (downstreamconf.proto == PROTO_HTTP2) {
h2alpn = util::get_default_alpn();
alpn = StringRef(h2alpn.data(), h2alpn.size());
next_proto_select_cb = select_h2_next_proto_cb;
} else {
alpn = StringRef::from_lit(NGHTTP2_H1_1_ALPN);
next_proto_select_cb = select_h1_next_proto_cb;
}
return ssl::create_ssl_client_context(
#ifdef HAVE_NEVERBLEED
nb,
#endif // HAVE_NEVERBLEED
StringRef{tlsconf.cacert}, StringRef{tlsconf.client.cert_file},
StringRef{tlsconf.client.private_key_file}, alpn, next_proto_select_cb);
StringRef{tlsconf.client.private_key_file}, select_next_proto_cb);
}
void setup_downstream_http2_alpn(SSL *ssl) {
auto alpn = util::get_default_alpn();
#if OPENSSL_VERSION_NUMBER >= 0x10002000L
// ALPN advertisement
SSL_set_alpn_protos(ssl, alpn.data(), alpn.size());
#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
}
void setup_downstream_http1_alpn(SSL *ssl) {
auto alpn = StringRef::from_lit(NGHTTP2_H1_1_ALPN);
#if OPENSSL_VERSION_NUMBER >= 0x10002000L
// ALPN advertisement
SSL_set_alpn_protos(ssl, alpn.byte(), alpn.size());
#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
}
CertLookupTree *create_cert_lookup_tree() {

10
src/shrpx_ssl.h
View File

@ -71,13 +71,14 @@ SSL_CTX *create_ssl_context(const char *private_key_file, const char *cert_file
#endif // HAVE_NEVERBLEED
);
// Create client side SSL_CTX
// Create client side SSL_CTX. This does not configure ALPN settings.
// |next_proto_select_cb| is for NPN.
SSL_CTX *create_ssl_client_context(
#ifdef HAVE_NEVERBLEED
neverbleed_t *nb,
#endif // HAVE_NEVERBLEED
const StringRef &cacert, const StringRef &cert_file,
const StringRef &private_key_file, const StringRef &alpn,
const StringRef &private_key_file,
int (*next_proto_select_cb)(SSL *s, unsigned char **out,
unsigned char *outlen, const unsigned char *in,
unsigned int inlen, void *arg));
@ -201,6 +202,11 @@ SSL_CTX *setup_downstream_client_ssl_context(
#endif // HAVE_NEVERBLEED
);
// Sets ALPN settings in |SSL| suitable for HTTP/2 use.
void setup_downstream_http2_alpn(SSL *ssl);
// Sets ALPN settings in |SSL| suitable for HTTP/1.1 use.
void setup_downstream_http1_alpn(SSL *ssl);
// Creates CertLookupTree. If frontend is configured not to use TLS,
// this function returns nullptr.
CertLookupTree *create_cert_lookup_tree();

160
src/shrpx_worker.cc
View File

@ -71,15 +71,13 @@ Worker::Worker(struct ev_loop *loop, SSL_CTX *sv_ssl_ctx, SSL_CTX *cl_ssl_ctx,
ssl::CertLookupTree *cert_tree,
const std::shared_ptr<TicketKeys> &ticket_keys)
: randgen_(rd()),
dconn_pool_(get_config()->conn.downstream.addr_groups.size()),
worker_stat_(get_config()->conn.downstream.addr_groups.size()),
dgrps_(get_config()->conn.downstream.addr_groups.size()),
worker_stat_{},
loop_(loop),
sv_ssl_ctx_(sv_ssl_ctx),
cl_ssl_ctx_(cl_ssl_ctx),
cert_tree_(cert_tree),
ticket_keys_(ticket_keys),
downstream_addr_groups_(get_config()->conn.downstream.addr_groups),
downstream_addr_groups_(get_config()->conn.downstream.addr_groups.size()),
connect_blocker_(make_unique<ConnectBlocker>(randgen_, loop_)),
graceful_shutdown_(false) {
ev_async_init(&w_, eventcb);
@ -100,25 +98,25 @@ Worker::Worker(struct ev_loop *loop, SSL_CTX *sv_ssl_ctx, SSL_CTX *cl_ssl_ctx,
auto &downstreamconf = get_config()->conn.downstream;
if (downstreamconf.proto == PROTO_HTTP2) {
auto n = get_config()->http2.downstream.connections_per_worker;
size_t group = 0;
for (auto &dgrp : dgrps_) {
auto m = n;
if (m == 0) {
m = downstreamconf.addr_groups[group].addrs.size();
}
for (size_t idx = 0; idx < m; ++idx) {
dgrp.http2sessions.push_back(
make_unique<Http2Session>(loop_, cl_ssl_ctx, this, group, idx));
}
++group;
}
}
for (size_t i = 0; i < downstreamconf.addr_groups.size(); ++i) {
auto &src = downstreamconf.addr_groups[i];
auto &dst = downstream_addr_groups_[i];
for (auto &group : downstream_addr_groups_) {
for (auto &addr : group.addrs) {
addr.connect_blocker = new ConnectBlocker(randgen_, loop_);
dst.pattern = src.pattern;
dst.addrs.resize(src.addrs.size());
dst.proto = src.proto;
for (size_t j = 0; j < src.addrs.size(); ++j) {
auto &src_addr = src.addrs[j];
auto &dst_addr = dst.addrs[j];
dst_addr.addr = src_addr.addr;
dst_addr.host = src_addr.host;
dst_addr.hostport = src_addr.hostport;
dst_addr.port = src_addr.port;
dst_addr.host_unix = src_addr.host_unix;
dst_addr.connect_blocker = make_unique<ConnectBlocker>(randgen_, loop_);
}
}
}
@ -126,12 +124,6 @@ Worker::Worker(struct ev_loop *loop, SSL_CTX *sv_ssl_ctx, SSL_CTX *cl_ssl_ctx,
Worker::~Worker() {
ev_async_stop(loop_, &w_);
ev_timer_stop(loop_, &mcpool_clear_timer_);
for (auto &group : downstream_addr_groups_) {
for (auto &addr : group.addrs) {
delete addr.connect_blocker;
}
}
}
void Worker::schedule_clear_mcpool() {
@ -253,24 +245,6 @@ void Worker::set_ticket_keys(std::shared_ptr<TicketKeys> ticket_keys) {
WorkerStat *Worker::get_worker_stat() { return &worker_stat_; }
DownstreamConnectionPool *Worker::get_dconn_pool() { return &dconn_pool_; }
Http2Session *Worker::next_http2_session(size_t group) {
auto &dgrp = dgrps_[group];
auto &http2sessions = dgrp.http2sessions;
if (http2sessions.empty()) {
return nullptr;
}
auto res = http2sessions[dgrp.next_http2session].get();
++dgrp.next_http2session;
if (dgrp.next_http2session >= http2sessions.size()) {
dgrp.next_http2session = 0;
}
return res;
}
struct ev_loop *Worker::get_loop() const {
return loop_;
}
@ -285,11 +259,6 @@ bool Worker::get_graceful_shutdown() const { return graceful_shutdown_; }
MemchunkPool *Worker::get_mcpool() { return &mcpool_; }
DownstreamGroup *Worker::get_dgrp(size_t group) {
assert(group < dgrps_.size());
return &dgrps_[group];
}
MemcachedDispatcher *Worker::get_session_cache_memcached_dispatcher() {
return session_cache_memcached_dispatcher_.get();
}
@ -319,4 +288,93 @@ ConnectBlocker *Worker::get_connect_blocker() const {
return connect_blocker_.get();
}
namespace {
size_t match_downstream_addr_group_host(
const Router &router, const StringRef &host, const StringRef &path,
const std::vector<DownstreamAddrGroup> &groups, size_t catch_all) {
if (path.empty() || path[0] != '/') {
auto group = router.match(host, StringRef::from_lit("/"));
if (group != -1) {
if (LOG_ENABLED(INFO)) {
LOG(INFO) << "Found pattern with query " << host
<< ", matched pattern=" << groups[group].pattern;
}
return group;
}
return catch_all;
}
if (LOG_ENABLED(INFO)) {
LOG(INFO) << "Perform mapping selection, using host=" << host
<< ", path=" << path;
}
auto group = router.match(host, path);
if (group != -1) {
if (LOG_ENABLED(INFO)) {
LOG(INFO) << "Found pattern with query " << host << path
<< ", matched pattern=" << groups[group].pattern;
}
return group;
}
group = router.match(StringRef::from_lit(""), path);
if (group != -1) {
if (LOG_ENABLED(INFO)) {
LOG(INFO) << "Found pattern with query " << path
<< ", matched pattern=" << groups[group].pattern;
}
return group;
}
if (LOG_ENABLED(INFO)) {
LOG(INFO) << "None match. Use catch-all pattern";
}
return catch_all;
}
} // namespace
size_t match_downstream_addr_group(
const Router &router, const StringRef &hostport, const StringRef &raw_path,
const std::vector<DownstreamAddrGroup> &groups, size_t catch_all) {
if (std::find(std::begin(hostport), std::end(hostport), '/') !=
std::end(hostport)) {
// We use '/' specially, and if '/' is included in host, it breaks
// our code. Select catch-all case.
return catch_all;
}
auto fragment = std::find(std::begin(raw_path), std::end(raw_path), '#');
auto query = std::find(std::begin(raw_path), fragment, '?');
auto path = StringRef{std::begin(raw_path), query};
if (hostport.empty()) {
return match_downstream_addr_group_host(router, hostport, path, groups,
catch_all);
}
std::string host;
if (hostport[0] == '[') {
// assume this is IPv6 numeric address
auto p = std::find(std::begin(hostport), std::end(hostport), ']');
if (p == std::end(hostport)) {
return catch_all;
}
if (p + 1 < std::end(hostport) && *(p + 1) != ':') {
return catch_all;
}
host.assign(std::begin(hostport), p + 1);
} else {
auto p = std::find(std::begin(hostport), std::end(hostport), ':');
if (p == std::begin(hostport)) {
return catch_all;
}
host.assign(std::begin(hostport), p);
}
util::inp_strlower(host);
return match_downstream_addr_group_host(router, StringRef{host}, path, groups,
catch_all);
}
} // namespace shrpx

55
src/shrpx_worker.h
View File

@ -67,20 +67,41 @@ namespace ssl {
class CertLookupTree;
} // namespace ssl
struct DownstreamGroup {
DownstreamGroup() : next_http2session(0), next(0) {}
struct DownstreamAddr {
Address addr;
// backend address. If |host_unix| is true, this is UNIX domain
// socket path.
ImmutableString host;
ImmutableString hostport;
// backend port. 0 if |host_unix| is true.
uint16_t port;
// true if |host| contains UNIX domain socket path.
bool host_unix;
std::vector<std::unique_ptr<Http2Session>> http2sessions;
// Next index in http2sessions.
size_t next_http2session;
// Next downstream address index corresponding to
// Config::downstream_addr_groups[].
std::unique_ptr<ConnectBlocker> connect_blocker;
// Client side TLS session cache
TLSSessionCache tls_session_cache;
};
struct DownstreamAddrGroup {
ImmutableString pattern;
std::vector<DownstreamAddr> addrs;
// Application protocol used in this group
shrpx_proto proto;
// List of Http2Session which is not fully utilized (i.e., the
// server advertized maximum concurrency is not reached). We will
// coalesce as much stream as possible in one Http2Session to fully
// utilize TCP connection.
//
// TODO Verify that this approach performs better in performance
// wise.
DList<Http2Session> http2_freelist;
DownstreamConnectionPool dconn_pool;
// Next downstream address index in addrs.
size_t next;
};
struct WorkerStat {
WorkerStat(size_t num_groups) : num_connections(0) {}
size_t num_connections;
};
@ -121,8 +142,6 @@ public:
void set_ticket_keys(std::shared_ptr<TicketKeys> ticket_keys);
WorkerStat *get_worker_stat();
DownstreamConnectionPool *get_dconn_pool();
Http2Session *next_http2_session(size_t group);
struct ev_loop *get_loop() const;
SSL_CTX *get_sv_ssl_ctx() const;
SSL_CTX *get_cl_ssl_ctx() const;
@ -133,8 +152,6 @@ public:
MemchunkPool *get_mcpool();
void schedule_clear_mcpool();
DownstreamGroup *get_dgrp(size_t group);
MemcachedDispatcher *get_session_cache_memcached_dispatcher();
std::mt19937 &get_randgen();
@ -159,9 +176,7 @@ private:
ev_async w_;
ev_timer mcpool_clear_timer_;
MemchunkPool mcpool_;
DownstreamConnectionPool dconn_pool_;
WorkerStat worker_stat_;
std::vector<DownstreamGroup> dgrps_;
std::unique_ptr<MemcachedDispatcher> session_cache_memcached_dispatcher_;
#ifdef HAVE_MRUBY
@ -184,6 +199,16 @@ private:
bool graceful_shutdown_;
};
// Selects group based on request's |hostport| and |path|. |hostport|
// is the value taken from :authority or host header field, and may
// contain port. The |path| may contain query part. We require the
// catch-all pattern in place, so this function always selects one
// group. The catch-all group index is given in |catch_all|. All
// patterns are given in |groups|.
size_t match_downstream_addr_group(
const Router &router, const StringRef &hostport, const StringRef &path,
const std::vector<DownstreamAddrGroup> &groups, size_t catch_all);
} // namespace shrpx
#endif // SHRPX_WORKER_H

179
src/shrpx_worker_test.cc Normal file
View File

@ -0,0 +1,179 @@
/*
* nghttp2 - HTTP/2 C Library
*
* Copyright (c) 2016 Tatsuhiro Tsujikawa
*
* Permission is hereby granted, free of charge, to any person obtaining
* a copy of this software and associated documentation files (the
* "Software"), to deal in the Software without restriction, including
* without limitation the rights to use, copy, modify, merge, publish,
* distribute, sublicense, and/or sell copies of the Software, and to
* permit persons to whom the Software is furnished to do so, subject to
* the following conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
* LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
* OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
#include "shrpx_worker_test.h"
#ifdef HAVE_UNISTD_H
#include <unistd.h>
#endif // HAVE_UNISTD_H
#include <cstdlib>
#include <CUnit/CUnit.h>
#include "shrpx_worker.h"
#include "shrpx_connect_blocker.h"
namespace shrpx {
void test_shrpx_worker_match_downstream_addr_group(void) {
auto groups = std::vector<DownstreamAddrGroup>();
for (auto &s : {"nghttp2.org/", "nghttp2.org/alpha/bravo/",
"nghttp2.org/alpha/charlie", "nghttp2.org/delta%3A",
"www.nghttp2.org/", "[::1]/", "nghttp2.org/alpha/bravo/delta",
// Check that match is done in the single node
"example.com/alpha/bravo", "192.168.0.1/alpha/"}) {
groups.push_back(DownstreamAddrGroup{ImmutableString(s)});
}
Router router;
for (size_t i = 0; i < groups.size(); ++i) {
auto &g = groups[i];
router.add_route(StringRef{g.pattern}, i);
}
CU_ASSERT(0 == match_downstream_addr_group(
router, StringRef::from_lit("nghttp2.org"),
StringRef::from_lit("/"), groups, 255));
// port is removed
CU_ASSERT(0 == match_downstream_addr_group(
router, StringRef::from_lit("nghttp2.org:8080"),
StringRef::from_lit("/"), groups, 255));
// host is case-insensitive
CU_ASSERT(4 == match_downstream_addr_group(
router, StringRef::from_lit("WWW.nghttp2.org"),
StringRef::from_lit("/alpha"), groups, 255));
CU_ASSERT(1 == match_downstream_addr_group(
router, StringRef::from_lit("nghttp2.org"),
StringRef::from_lit("/alpha/bravo/"), groups, 255));
// /alpha/bravo also matches /alpha/bravo/
CU_ASSERT(1 == match_downstream_addr_group(
router, StringRef::from_lit("nghttp2.org"),
StringRef::from_lit("/alpha/bravo"), groups, 255));
// path part is case-sensitive
CU_ASSERT(0 == match_downstream_addr_group(
router, StringRef::from_lit("nghttp2.org"),
StringRef::from_lit("/Alpha/bravo"), groups, 255));
CU_ASSERT(1 == match_downstream_addr_group(
router, StringRef::from_lit("nghttp2.org"),
StringRef::from_lit("/alpha/bravo/charlie"), groups, 255));
CU_ASSERT(2 == match_downstream_addr_group(
router, StringRef::from_lit("nghttp2.org"),
StringRef::from_lit("/alpha/charlie"), groups, 255));
// pattern which does not end with '/' must match its entirely. So
// this matches to group 0, not group 2.
CU_ASSERT(0 == match_downstream_addr_group(
router, StringRef::from_lit("nghttp2.org"),
StringRef::from_lit("/alpha/charlie/"), groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(
router, StringRef::from_lit("example.org"),
StringRef::from_lit("/"), groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(router, StringRef::from_lit(""),
StringRef::from_lit("/"), groups,
255));
CU_ASSERT(255 == match_downstream_addr_group(router, StringRef::from_lit(""),
StringRef::from_lit("alpha"),
groups, 255));
CU_ASSERT(255 ==
match_downstream_addr_group(router, StringRef::from_lit("foo/bar"),
StringRef::from_lit("/"), groups, 255));
// If path is StringRef::from_lit("*", only match with host + "/").
CU_ASSERT(0 == match_downstream_addr_group(
router, StringRef::from_lit("nghttp2.org"),
StringRef::from_lit("*"), groups, 255));
CU_ASSERT(5 ==
match_downstream_addr_group(router, StringRef::from_lit("[::1]"),
StringRef::from_lit("/"), groups, 255));
CU_ASSERT(5 == match_downstream_addr_group(
router, StringRef::from_lit("[::1]:8080"),
StringRef::from_lit("/"), groups, 255));
CU_ASSERT(255 ==
match_downstream_addr_group(router, StringRef::from_lit("[::1"),
StringRef::from_lit("/"), groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(
router, StringRef::from_lit("[::1]8000"),
StringRef::from_lit("/"), groups, 255));
// Check the case where adding route extends tree
CU_ASSERT(6 == match_downstream_addr_group(
router, StringRef::from_lit("nghttp2.org"),
StringRef::from_lit("/alpha/bravo/delta"), groups, 255));
CU_ASSERT(1 == match_downstream_addr_group(
router, StringRef::from_lit("nghttp2.org"),
StringRef::from_lit("/alpha/bravo/delta/"), groups, 255));
// Check the case where query is done in a single node
CU_ASSERT(7 == match_downstream_addr_group(
router, StringRef::from_lit("example.com"),
StringRef::from_lit("/alpha/bravo"), groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(
router, StringRef::from_lit("example.com"),
StringRef::from_lit("/alpha/bravo/"), groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(
router, StringRef::from_lit("example.com"),
StringRef::from_lit("/alpha"), groups, 255));
// Check the case where quey is done in a single node
CU_ASSERT(8 == match_downstream_addr_group(
router, StringRef::from_lit("192.168.0.1"),
StringRef::from_lit("/alpha"), groups, 255));
CU_ASSERT(8 == match_downstream_addr_group(
router, StringRef::from_lit("192.168.0.1"),
StringRef::from_lit("/alpha/"), groups, 255));
CU_ASSERT(8 == match_downstream_addr_group(
router, StringRef::from_lit("192.168.0.1"),
StringRef::from_lit("/alpha/bravo"), groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(
router, StringRef::from_lit("192.168.0.1"),
StringRef::from_lit("/alph"), groups, 255));
CU_ASSERT(255 == match_downstream_addr_group(
router, StringRef::from_lit("192.168.0.1"),
StringRef::from_lit("/"), groups, 255));
router.dump();
}
} // namespace shrpx

38
src/shrpx_worker_test.h Normal file
View File

@ -0,0 +1,38 @@
/*
* nghttp2 - HTTP/2 C Library
*
* Copyright (c) 2016 Tatsuhiro Tsujikawa
*
* Permission is hereby granted, free of charge, to any person obtaining
* a copy of this software and associated documentation files (the
* "Software"), to deal in the Software without restriction, including
* without limitation the rights to use, copy, modify, merge, publish,
* distribute, sublicense, and/or sell copies of the Software, and to
* permit persons to whom the Software is furnished to do so, subject to
* the following conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
* LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
* OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
#ifndef SHRPX_WORKER_TEST_H
#define SHRPX_WORKER_TEST_H
#ifdef HAVE_CONFIG_H
#include <config.h>
#endif // HAVE_CONFIG_H
namespace shrpx {
void test_shrpx_worker_match_downstream_addr_group(void);
} // namespace shrpx
#endif // SHRPX_WORKER_TEST_H

17
src/template.h
View File

@ -99,14 +99,14 @@ template <typename T, typename F> bool test_flags(T t, F flags) {
// T *dlnext, which point to previous element and next element in the
// list respectively.
template <typename T> struct DList {
DList() : head(nullptr), tail(nullptr) {}
DList() : head(nullptr), tail(nullptr), n(0) {}
DList(const DList &) = delete;
DList &operator=(const DList &) = delete;
DList(DList &&other) : head(other.head), tail(other.tail) {
DList(DList &&other) : head(other.head), tail(other.tail), n(other.n) {
other.head = other.tail = nullptr;
other.n = 0;
}
DList &operator=(DList &&other) {
@ -115,11 +115,16 @@ template <typename T> struct DList {
}
head = other.head;
tail = other.tail;
n = other.n;
other.head = other.tail = nullptr;
other.n = 0;
return *this;
}
void append(T *t) {
++n;
if (tail) {
tail->dlnext = t;
t->dlprev = tail;
@ -130,6 +135,7 @@ template <typename T> struct DList {
}
void remove(T *t) {
--n;
auto p = t->dlprev;
auto n = t->dlnext;
if (p) {
@ -149,7 +155,10 @@ template <typename T> struct DList {
bool empty() const { return head == nullptr; }
size_t size() const { return n; }
T *head, *tail;
size_t n;
};
template <typename T> void dlist_delete_all(DList<T> &dl) {
@ -391,7 +400,7 @@ public:
explicit StringRef(const std::string &s) : base(s.c_str()), len(s.size()) {}
explicit StringRef(const ImmutableString &s)
: base(s.c_str()), len(s.size()) {}
StringRef(const char *s) : base(s), len(strlen(s)) {}
explicit StringRef(const char *s) : base(s), len(strlen(s)) {}
template <typename CharT>
constexpr StringRef(const CharT *s, size_t n)
: base(reinterpret_cast<const char *>(s)), len(n) {}

21
src/util.cc
View File

@ -857,6 +857,27 @@ std::vector<unsigned char> get_default_alpn() {
return res;
}
std::vector<StringRef> split_str(const StringRef &s, char delim) {
size_t len = 1;
auto last = std::end(s);
for (auto first = std::begin(s), d = first;
(d = std::find(first, last, delim)) != last; ++len, first = d + 1)
;
auto list = std::vector<StringRef>(len);
len = 0;
for (auto first = std::begin(s);; ++len) {
auto stop = std::find(first, last, delim);
list[len] = StringRef{first, stop};
if (stop == last) {
break;
}
first = stop + 1;
}
return list;
}
std::vector<Range<const char *>> split_config_str_list(const char *s,
char delim) {
size_t len = 1;

10
src/util.h
View File

@ -225,6 +225,11 @@ bool istarts_with_l(const std::string &a, const CharT(&b)[N]) {
return istarts_with(std::begin(a), std::end(a), b, b + N - 1);
}
template <typename CharT, size_t N>
bool istarts_with_l(const StringRef &a, const CharT(&b)[N]) {
return istarts_with(std::begin(a), std::end(a), b, b + N - 1);
}
template <typename InputIterator1, typename InputIterator2>
bool ends_with(InputIterator1 first1, InputIterator1 last1,
InputIterator2 first2, InputIterator2 last2) {
@ -543,6 +548,11 @@ std::vector<std::string> parse_config_str_list(const char *s, char delim = ',');
std::vector<Range<const char *>> split_config_str_list(const char *s,
char delim);
// Parses delimited strings in |s| and returns Substrings in |s|
// delimited by |delim|. The any white spaces around substring are
// treated as a part of substring.
std::vector<StringRef> split_str(const StringRef &s, char delim);
// Returns given time |tp| in Common Log format (e.g.,
// 03/Jul/2014:00:19:38 +0900)
// Expected type of |tp| is std::chrono::timepoint

15
src/util_test.cc
View File

@ -456,14 +456,19 @@ void test_util_parse_config_str_list(void) {
}
void test_util_make_http_hostport(void) {
CU_ASSERT("localhost" == util::make_http_hostport("localhost", 80));
CU_ASSERT("[::1]" == util::make_http_hostport("::1", 443));
CU_ASSERT("localhost:3000" == util::make_http_hostport("localhost", 3000));
CU_ASSERT("localhost" ==
util::make_http_hostport(StringRef::from_lit("localhost"), 80));
CU_ASSERT("[::1]" ==
util::make_http_hostport(StringRef::from_lit("::1"), 443));
CU_ASSERT("localhost:3000" ==
util::make_http_hostport(StringRef::from_lit("localhost"), 3000));
}
void test_util_make_hostport(void) {
CU_ASSERT("localhost:80" == util::make_hostport("localhost", 80));
CU_ASSERT("[::1]:443" == util::make_hostport("::1", 443));
CU_ASSERT("localhost:80" ==
util::make_hostport(StringRef::from_lit("localhost"), 80));
CU_ASSERT("[::1]:443" ==
util::make_hostport(StringRef::from_lit("::1"), 443));
}
} // namespace shrpx