nghttp2/README.rst

916 lines
27 KiB
ReStructuredText
Raw Normal View History

2013-07-12 17:19:03 +02:00
nghttp2 - HTTP/2.0 C Library
============================
2012-01-26 17:38:35 +01:00
2013-07-12 17:19:03 +02:00
This is an experimental implementation of Hypertext Transfer Protocol
version 2.0.
2012-09-12 15:37:05 +02:00
2012-04-25 14:25:51 +02:00
Development Status
------------------
2012-03-26 16:46:02 +02:00
2014-02-15 09:20:47 +01:00
We started to implement h2-10
2014-02-15 09:19:49 +01:00
(http://tools.ietf.org/html/draft-ietf-httpbis-http2-10) and the
2013-08-22 20:45:55 +02:00
header compression
2014-02-15 09:19:49 +01:00
(http://tools.ietf.org/html/draft-ietf-httpbis-header-compression-06).
2013-08-22 20:45:55 +02:00
The nghttp2 code base was forked from spdylay project.
2013-07-12 17:19:03 +02:00
2014-02-15 09:19:49 +01:00
========================== =====
Features h2-10
========================== =====
HPACK-draft-06 Done
Strict SETTINGS validation Done
Disallow client to push Done
Padding Done
END_SEGMENT
========================== =====
2013-07-16 17:08:05 +02:00
2013-08-10 12:06:33 +02:00
Public Test Server
------------------
2013-08-25 15:04:54 +02:00
The following endpoints are available to try out nghttp2
2013-12-05 15:39:57 +01:00
implementation. These endpoints supports ``HTTP-draft-09/2.0`` and
2013-08-25 15:04:54 +02:00
the earlier draft versions are not supporeted.
2013-08-10 12:06:33 +02:00
2014-02-05 15:05:44 +01:00
* https://106.186.112.116 (TLS + NPN / ALPN)
2013-08-30 17:50:09 +02:00
2014-01-26 15:35:36 +01:00
ALPN and NPN offer ``HTTP-draft-09/2.0``, ``spdy/3.1``, ``spdy/3``,
2013-11-14 16:42:09 +01:00
``spdy/2`` and ``http/1.1``.
2013-08-10 12:06:33 +02:00
Note: certificate is self-signed and a browser will show alert
2013-08-30 17:50:09 +02:00
* http://106.186.112.116 (Upgrade + Direct)
2013-08-10 12:06:33 +02:00
2013-07-27 14:56:46 +02:00
Requirements
------------
2013-07-16 17:08:05 +02:00
2013-07-27 14:56:46 +02:00
The following packages are needed to build the library:
2013-07-19 19:05:07 +02:00
2013-07-27 14:56:46 +02:00
* pkg-config >= 0.20
* zlib >= 1.2.3
2013-07-19 19:05:07 +02:00
2013-07-27 14:56:46 +02:00
To build and run the unit test programs, the following packages are
required:
* cunit >= 2.1
To build the documentation, you need to install:
* sphinx (http://sphinx-doc.org/)
2013-07-27 14:56:46 +02:00
To build and run the application programs (``nghttp``, ``nghttpd`` and
``nghttpx``) in ``src`` directory, the following packages are
required:
* OpenSSL >= 1.0.1
* libevent-openssl >= 2.0.8
2014-01-02 05:32:43 +01:00
ALPN support requires unreleased version OpenSSL >= 1.0.2.
2013-07-27 14:56:46 +02:00
To enable SPDY protocol in the application program ``nghttpx``, the
following packages are required:
2014-01-04 02:44:16 +01:00
* spdylay >= 1.2.3
2013-07-27 14:56:46 +02:00
To enable ``-a`` option (getting linked assets from the downloaded
2013-10-10 18:02:24 +02:00
resource) in ``nghttp``, the following packages are needed:
2013-07-27 14:56:46 +02:00
* libxml2 >= 2.7.7
2014-01-10 13:53:48 +01:00
The HPACK tools require the following package:
2013-10-27 08:23:24 +01:00
2014-01-08 17:48:34 +01:00
* jansson >= 2.5
2013-10-27 08:23:24 +01:00
The Python bindings require the following packages:
* cython >= 0.19
* python >= 2.7
2013-07-27 14:56:46 +02:00
If you are using Ubuntu 12.04, you need the following packages
installed:
* autoconf
* automake
* autotools-dev
* libtool
* pkg-config
* zlib1g-dev
* libcunit1-dev
* libssl-dev
* libxml2-dev
* libevent-dev
2013-10-27 08:23:24 +01:00
* libjansson-dev
2013-07-27 14:56:46 +02:00
spdylay is not packaged in Ubuntu, so you need to build it yourself:
2013-11-21 14:03:18 +01:00
http://tatsuhiro-t.github.io/spdylay/
2013-07-27 14:56:46 +02:00
2013-07-27 15:46:22 +02:00
Build from git
--------------
Building from git is easy, but please be sure that at least autoconf 2.68 is
used::
$ autoreconf -i
$ automake
$ autoconf
$ ./configure
$ make
2013-07-27 14:56:46 +02:00
Building documentation
----------------------
.. note::
Documentation is still incomplete.
To build documentation, run::
$ make html
The documents will be generated under ``doc/manual/html/``.
The generated documents will not be installed with ``make install``.
2013-07-28 16:00:12 +02:00
The online documentation is available at
http://tatsuhiro-t.github.io/nghttp2/
2013-07-27 14:56:46 +02:00
Client, Server and Proxy programs
---------------------------------
The src directory contains HTTP/2.0 client, server and proxy programs.
nghttp - client
+++++++++++++++
2013-08-04 11:29:31 +02:00
``nghttp`` is a HTTP/2.0 client. It can connect to the HTTP/2.0 server
2014-01-01 17:00:11 +01:00
with prior knowledge, HTTP Upgrade and NPN/ALPN TLS extension.
2013-07-27 14:56:46 +02:00
It has verbose output mode for framing information. Here is sample
output from ``nghttp`` client::
2014-02-15 09:19:49 +01:00
$ src/nghttp -nv https://localhost:8443
[ 0.004][NPN] server offers:
* h2-10
* spdy/3.1
2014-01-11 09:06:26 +01:00
* spdy/3
* spdy/2
* http/1.1
2014-02-15 09:19:49 +01:00
The negotiated protocol: h2-10
[ 0.006] send SETTINGS frame <length=10, flags=0x00, stream_id=0>
2014-01-11 09:06:26 +01:00
(niv=2)
2014-02-15 09:19:49 +01:00
[SETTINGS_MAX_CONCURRENT_STREAMS(3):100]
[SETTINGS_INITIAL_WINDOW_SIZE(4):65535]
[ 0.007] send HEADERS frame <length=48, flags=0x05, stream_id=1>
2014-01-11 09:06:26 +01:00
; END_STREAM | END_HEADERS
2014-02-15 09:19:49 +01:00
(padlen=0)
2014-01-11 09:06:26 +01:00
; Open new stream
:authority: localhost:8443
:method: GET
:path: /
:scheme: https
accept: */*
accept-encoding: gzip, deflate
2014-02-15 09:19:49 +01:00
user-agent: nghttp2/0.4.0-DEV
[ 0.007] recv SETTINGS frame <length=15, flags=0x00, stream_id=0>
(niv=3)
[SETTINGS_MAX_CONCURRENT_STREAMS(3):100]
[SETTINGS_INITIAL_WINDOW_SIZE(4):65535]
[SETTINGS_ENABLE_PUSH(2):0]
[ 0.007] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
2014-01-11 09:06:26 +01:00
; ACK
(niv=0)
2014-02-15 09:19:49 +01:00
[ 0.007] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
2014-01-11 09:06:26 +01:00
; ACK
(niv=0)
2014-02-15 09:19:49 +01:00
[ 0.008] (stream_id=1) :status: 200
[ 0.008] (stream_id=1) accept-ranges: bytes
[ 0.008] (stream_id=1) content-encoding: gzip
[ 0.008] (stream_id=1) content-length: 146
[ 0.008] (stream_id=1) content-type: text/html
[ 0.008] (stream_id=1) date: Sat, 15 Feb 2014 08:14:12 GMT
[ 0.008] (stream_id=1) etag: "b1-4e5535a027780-gzip"
[ 0.008] (stream_id=1) last-modified: Sun, 01 Sep 2013 14:34:22 GMT
[ 0.008] (stream_id=1) server: Apache/2.4.6 (Debian)
[ 0.008] (stream_id=1) vary: Accept-Encoding
[ 0.008] (stream_id=1) via: 1.1 nghttpx
[ 0.008] recv HEADERS frame <length=141, flags=0x04, stream_id=1>
2014-01-11 09:06:26 +01:00
; END_HEADERS
2014-02-15 09:19:49 +01:00
(padlen=0)
2014-01-11 09:06:26 +01:00
; First response header
2014-02-15 09:19:49 +01:00
[ 0.008] recv DATA frame <length=146, flags=0x00, stream_id=1>
[ 0.008] recv DATA frame <length=0, flags=0x01, stream_id=1>
2014-01-11 09:06:26 +01:00
; END_STREAM
2014-02-15 09:19:49 +01:00
[ 0.008] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
2014-01-11 09:06:26 +01:00
(last_stream_id=0, error_code=NO_ERROR(0), opaque_data(0)=[])
2013-07-27 14:56:46 +02:00
2013-08-03 12:44:13 +02:00
The HTTP Upgrade is performed like this::
2014-02-15 09:19:49 +01:00
$ src/nghttp -nvu http://localhost:8080
2013-08-03 12:44:13 +02:00
[ 0.000] HTTP Upgrade request
GET / HTTP/1.1
2013-08-22 20:45:55 +02:00
Host: localhost:8080
2013-08-03 12:44:13 +02:00
Connection: Upgrade, HTTP2-Settings
2014-02-15 09:19:49 +01:00
Upgrade: h2-10
HTTP2-Settings: AwAAAGQEAAD__w
2013-08-03 12:44:13 +02:00
Accept: */*
2014-02-15 09:19:49 +01:00
User-Agent: nghttp2/0.4.0-DEV
2013-08-03 12:44:13 +02:00
2014-02-15 09:19:49 +01:00
[ 0.001] HTTP Upgrade response
2013-08-03 12:44:13 +02:00
HTTP/1.1 101 Switching Protocols
Connection: Upgrade
2014-02-15 09:19:49 +01:00
Upgrade: h2-10
2013-08-03 12:44:13 +02:00
2013-10-27 15:29:54 +01:00
[ 0.001] HTTP Upgrade success
2014-02-15 09:19:49 +01:00
[ 0.001] send SETTINGS frame <length=10, flags=0x00, stream_id=0>
2014-01-11 09:06:26 +01:00
(niv=2)
2014-02-15 09:19:49 +01:00
[SETTINGS_MAX_CONCURRENT_STREAMS(3):100]
[SETTINGS_INITIAL_WINDOW_SIZE(4):65535]
[ 0.001] recv SETTINGS frame <length=15, flags=0x00, stream_id=0>
(niv=3)
[SETTINGS_MAX_CONCURRENT_STREAMS(3):100]
[SETTINGS_INITIAL_WINDOW_SIZE(4):65535]
[SETTINGS_ENABLE_PUSH(2):0]
[ 0.001] (stream_id=1) :status: 200
[ 0.001] (stream_id=1) accept-ranges: bytes
[ 0.001] (stream_id=1) content-length: 177
[ 0.001] (stream_id=1) content-type: text/html
[ 0.001] (stream_id=1) date: Sat, 15 Feb 2014 08:16:23 GMT
[ 0.001] (stream_id=1) etag: "b1-4e5535a027780"
[ 0.001] (stream_id=1) last-modified: Sun, 01 Sep 2013 14:34:22 GMT
[ 0.001] (stream_id=1) server: Apache/2.4.6 (Debian)
[ 0.001] (stream_id=1) vary: Accept-Encoding
[ 0.001] (stream_id=1) via: 1.1 nghttpx
[ 0.001] recv HEADERS frame <length=132, flags=0x04, stream_id=1>
2014-01-11 09:06:26 +01:00
; END_HEADERS
2014-02-15 09:19:49 +01:00
(padlen=0)
2014-01-11 09:06:26 +01:00
; First response header
2013-10-27 15:29:54 +01:00
[ 0.001] recv DATA frame <length=177, flags=0x00, stream_id=1>
2013-08-22 20:45:55 +02:00
[ 0.001] recv DATA frame <length=0, flags=0x01, stream_id=1>
2014-01-11 09:06:26 +01:00
; END_STREAM
2014-02-15 09:19:49 +01:00
[ 0.002] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
2014-01-11 09:06:26 +01:00
; ACK
(niv=0)
2014-02-15 09:19:49 +01:00
[ 0.002] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
2014-01-11 09:06:26 +01:00
(last_stream_id=0, error_code=NO_ERROR(0), opaque_data(0)=[])
2014-02-15 09:19:49 +01:00
[ 0.002] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
2014-01-11 09:06:26 +01:00
; ACK
(niv=0)
2013-08-03 12:44:13 +02:00
2013-07-27 14:56:46 +02:00
nghttpd - server
++++++++++++++++
``nghttpd`` is static web server. It is single threaded and
multiplexes connections using non-blocking socket.
2013-07-27 15:40:44 +02:00
By default, it uses SSL/TLS connection. Use ``--no-tls`` option to
disable it.
2014-01-01 17:00:11 +01:00
``nghttpd`` only accept the HTTP/2.0 connection via NPN/ALPN or direct
2013-08-03 12:44:13 +02:00
HTTP/2.0 connection. No HTTP Upgrade is supported.
2013-12-08 16:06:53 +01:00
``-p`` option allows users to configure server push.
2013-07-27 14:56:46 +02:00
Just like ``nghttp``, it has verbose output mode for framing
information. Here is sample output from ``nghttpd`` server::
2013-08-22 20:45:55 +02:00
$ src/nghttpd --no-tls -v 8080
IPv4: listen on port 8080
IPv6: listen on port 8080
2014-02-15 09:19:49 +01:00
[id=1] [ 1.027] send SETTINGS frame <length=10, flags=0x00, stream_id=0>
(niv=2)
[SETTINGS_MAX_CONCURRENT_STREAMS(3):100]
[SETTINGS_ENABLE_PUSH(2):0]
[id=1] [ 1.027] recv SETTINGS frame <length=10, flags=0x00, stream_id=0>
2014-01-11 09:06:26 +01:00
(niv=2)
2014-02-15 09:19:49 +01:00
[SETTINGS_MAX_CONCURRENT_STREAMS(3):100]
[SETTINGS_INITIAL_WINDOW_SIZE(4):65535]
[id=1] [ 1.027] (stream_id=1) :authority: localhost:8080
[id=1] [ 1.027] (stream_id=1) :method: GET
[id=1] [ 1.027] (stream_id=1) :path: /
[id=1] [ 1.027] (stream_id=1) :scheme: http
[id=1] [ 1.027] (stream_id=1) accept: */*
[id=1] [ 1.027] (stream_id=1) accept-encoding: gzip, deflate
[id=1] [ 1.027] (stream_id=1) user-agent: nghttp2/0.4.0-DEV
[id=1] [ 1.027] recv HEADERS frame <length=48, flags=0x05, stream_id=1>
2014-01-11 09:06:26 +01:00
; END_STREAM | END_HEADERS
2014-02-15 09:19:49 +01:00
(padlen=0)
2014-01-11 09:06:26 +01:00
; Open new stream
2014-02-15 09:19:49 +01:00
[id=1] [ 1.027] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
2014-01-11 09:06:26 +01:00
; ACK
(niv=0)
2014-02-15 09:19:49 +01:00
[id=1] [ 1.027] send HEADERS frame <length=72, flags=0x04, stream_id=1>
2014-01-11 09:06:26 +01:00
; END_HEADERS
2014-02-15 09:19:49 +01:00
(padlen=0)
2014-01-11 09:06:26 +01:00
; First response header
:status: 404
content-encoding: gzip
content-type: text/html; charset=UTF-8
2014-02-15 09:19:49 +01:00
date: Sat, 15 Feb 2014 08:18:53 GMT
server: nghttpd nghttp2/0.4.0-DEV
[id=1] [ 1.028] send DATA frame <length=118, flags=0x00, stream_id=1>
[id=1] [ 1.028] send DATA frame <length=0, flags=0x01, stream_id=1>
2014-01-11 09:06:26 +01:00
; END_STREAM
2014-02-15 09:19:49 +01:00
[id=1] [ 1.028] stream_id=1 closed
[id=1] [ 1.028] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
2014-01-11 09:06:26 +01:00
; ACK
(niv=0)
2014-02-15 09:19:49 +01:00
[id=1] [ 1.028] recv GOAWAY frame <length=8, flags=0x00, stream_id=0>
2014-01-11 09:06:26 +01:00
(last_stream_id=0, error_code=NO_ERROR(0), opaque_data(0)=[])
2014-02-15 09:19:49 +01:00
[id=1] [ 1.028] closed
2013-07-27 14:56:46 +02:00
nghttpx - proxy
+++++++++++++++
The ``nghttpx`` is a multi-threaded reverse proxy for
2014-02-15 09:19:49 +01:00
h2-10, SPDY and HTTP/1.1. It has several operation modes:
2013-08-03 12:44:13 +02:00
================== ============================== ============== =============
Mode option Frontend Backend Note
================== ============================== ============== =============
default mode HTTP/2.0, SPDY, HTTP/1.1 (TLS) HTTP/1.1 Reverse proxy
2013-11-04 10:26:08 +01:00
``--http2-proxy`` HTTP/2.0, SPDY, HTTP/1.1 (TLS) HTTP/1.1 SPDY proxy
``--http2-bridge`` HTTP/2.0, SPDY, HTTP/1.1 (TLS) HTTP/2.0 (TLS)
2013-08-03 12:44:13 +02:00
``--client`` HTTP/2.0, HTTP/1.1 HTTP/2.0 (TLS)
``--client-proxy`` HTTP/2.0, HTTP/1.1 HTTP/2.0 (TLS) Forward proxy
================== ============================== ============== =============
2013-07-27 14:56:46 +02:00
The interesting mode at the moment is the default mode. It works like
2014-02-15 09:19:49 +01:00
a reverse proxy and listens h2-10, SPDY and HTTP/1.1 and
2013-08-03 12:44:13 +02:00
can be deployed SSL/TLS terminator for existing web server.
2013-07-27 14:56:46 +02:00
2013-11-04 10:26:08 +01:00
The default mode, ``--http2-proxy`` and ``--http2-bridge`` modes use
SSL/TLS in the frontend connection by default. To disable SSL/TLS, use
2013-08-03 12:44:13 +02:00
``--frontend-no-tls`` option. If that option is used, SPDY is disabled
in the frontend and incoming HTTP/1.1 connection can be upgraded to
HTTP/2.0 through HTTP Upgrade.
2013-11-04 10:26:08 +01:00
The ``--http2-bridge``, ``--client`` and ``--client-proxy`` modes use
2013-08-03 12:44:13 +02:00
SSL/TLS in the backend connection by deafult. To disable SSL/TLS, use
``--backend-no-tls`` option.
2013-07-27 15:40:44 +02:00
2013-07-27 14:56:46 +02:00
The ``nghttpx`` supports configuration file. See ``--conf`` option and
sample configuration file ``nghttpx.conf.sample``.
2013-12-18 15:33:20 +01:00
The ``nghttpx`` does not support server push.
2013-11-04 10:26:08 +01:00
In the default mode, (without any of ``--http2-proxy``,
``--http2-bridge``, ``--client-proxy`` and ``--client`` options),
``nghttpx`` works as reverse proxy to the backend server::
2013-07-27 14:56:46 +02:00
2013-08-03 12:44:13 +02:00
Client <-- (HTTP/2.0, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/1.1) --> Web Server
[reverse proxy]
2013-07-27 14:56:46 +02:00
2013-11-04 10:26:08 +01:00
With ``--http2-proxy`` option, it works as so called secure proxy (aka
SPDY proxy)::
2013-07-27 14:56:46 +02:00
2013-08-03 12:44:13 +02:00
Client <-- (HTTP/2.0, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/1.1) --> Proxy
[secure proxy] (e.g., Squid)
2013-07-27 14:56:46 +02:00
2013-08-03 12:44:13 +02:00
The ``Client`` in the above is needs to be configured to use
``nghttpx`` as secure proxy.
2013-07-27 14:56:46 +02:00
At the time of this writing, Chrome is the only browser which supports
2013-08-03 12:44:13 +02:00
secure proxy. The one way to configure Chrome to use secure proxy is
2014-01-11 09:13:24 +01:00
create proxy.pac script like this:
.. code-block:: javascript
2013-07-27 14:56:46 +02:00
function FindProxyForURL(url, host) {
return "HTTPS SERVERADDR:PORT";
}
``SERVERADDR`` and ``PORT`` is the hostname/address and port of the
machine nghttpx is running. Please note that Chrome requires valid
2013-08-03 12:44:13 +02:00
certificate for secure proxy.
2013-07-27 14:56:46 +02:00
Then run chrome with the following arguments::
$ google-chrome --proxy-pac-url=file:///path/to/proxy.pac --use-npn
2013-11-04 10:26:08 +01:00
With ``--http2-bridge``, it accepts HTTP/2.0, SPDY and HTTP/1.1
2013-07-27 14:56:46 +02:00
connections and communicates with backend in HTTP/2.0::
2013-08-03 12:44:13 +02:00
Client <-- (HTTP/2.0, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/2.0) --> Web or HTTP/2.0 Proxy etc
(e.g., nghttpx -s)
2013-07-27 14:56:46 +02:00
2013-08-03 12:44:13 +02:00
With ``--client-proxy`` option, it works as forward proxy and expects
that the backend is HTTP/2.0 proxy::
2013-07-27 14:56:46 +02:00
2013-08-03 12:44:13 +02:00
Client <-- (HTTP/2.0, HTTP/1.1) --> nghttpx <-- (HTTP/2.0) --> HTTP/2.0 Proxy
[forward proxy] (e.g., nghttpx -s)
2013-07-27 14:56:46 +02:00
2013-08-03 12:44:13 +02:00
The ``Client`` is needs to be configured to use nghttpx as forward
proxy. The frontend HTTP/1.1 connection can be upgraded to HTTP/2.0
through HTTP Upgrade. With the above configuration, one can use
HTTP/1.1 client to access and test their HTTP/2.0 servers.
2013-07-27 14:56:46 +02:00
With ``--client`` option, it works as reverse proxy and expects that
the backend is HTTP/2.0 Web server::
2013-08-03 12:44:13 +02:00
Client <-- (HTTP/2.0, HTTP/1.1) --> nghttpx <-- (HTTP/2.0) --> Web Server
[reverse proxy]
The frontend HTTP/1.1 connection can be upgraded to HTTP/2.0
through HTTP Upgrade.
2013-07-27 14:56:46 +02:00
2013-08-03 12:44:13 +02:00
For the operation modes which talk to the backend in HTTP/2.0 over
SSL/TLS, the backend connections can be tunneled though HTTP
proxy. The proxy is specified using ``--backend-http-proxy-uri``
option. The following figure illustrates the example of
2013-11-04 10:26:08 +01:00
``--http2-bridge`` and ``--backend-http-proxy-uri`` option to talk to
2013-08-03 12:44:13 +02:00
the outside HTTP/2.0 proxy through HTTP proxy::
2013-07-27 14:56:46 +02:00
2013-08-03 12:44:13 +02:00
Client <-- (HTTP/2.0, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/2.0) --
2013-07-27 14:56:46 +02:00
--===================---> HTTP/2.0 Proxy
(HTTP proxy tunnel) (e.g., nghttpx -s)
2013-10-28 16:22:18 +01:00
2014-01-10 13:53:48 +01:00
HPACK tools
-----------
2013-10-28 16:22:18 +01:00
2014-01-10 13:53:48 +01:00
The ``src`` directory contains HPACK tools. The ``deflatehd`` is
command-line header compression tool. The ``inflatehd`` is
command-line header decompression tool. Both tools read input from
stdin and write output to stdout. The errors are written to
2014-01-11 09:06:26 +01:00
stderr. They take JSON as input and output. We use the same JSON data
format used in https://github.com/Jxck/hpack-test-case
2013-10-28 16:22:18 +01:00
deflatehd - header compressor
+++++++++++++++++++++++++++++
2014-01-11 09:06:26 +01:00
The ``deflatehd`` reads JSON data or HTTP/1-style header fields from
stdin and outputs compressed header block in JSON.
2013-10-28 16:22:18 +01:00
2014-01-11 09:06:26 +01:00
For the JSON input, the root JSON object must contain ``context`` key,
which indicates which compression context is used. If it is
``request``, request compression context is used. Otherwise, response
compression context is used. The value of ``cases`` key contains the
sequence of input header set. They share the same compression context
and are processed in the order they appear. Each item in the sequence
is a JSON object and it must have at least ``headers`` key. Its value
is an array of a JSON object containing exactly one name/value pair.
2013-10-28 16:22:18 +01:00
2014-01-11 09:13:24 +01:00
Example:
.. code-block:: json
2013-10-28 16:22:18 +01:00
2014-01-11 09:06:26 +01:00
{
"context": "request",
"cases":
[
{
"headers": [
{ ":method": "GET" },
{ ":path": "/" }
]
},
{
"headers": [
{ ":method": "POST" },
{ ":path": "/" }
]
}
]
}
2013-10-28 16:22:18 +01:00
With ``-t`` option, the program can accept more familiar HTTP/1 style
header field block. Each header set is delimited by empty line:
Example::
:method: GET
:scheme: https
:path: /
:method: POST
user-agent: nghttp2
2014-01-11 09:06:26 +01:00
The output is JSON object. It contains ``context`` key and its value
is ``request`` if the compression context is request, otherwise
``response``. The root JSON object also contains ``cases`` key and its
value is an array of JSON object, which has at least following keys:
2013-10-28 16:22:18 +01:00
seq
The index of header set in the input.
2014-01-11 09:06:26 +01:00
input_length
2013-10-28 16:22:18 +01:00
The sum of length of name/value pair in the input.
2014-01-11 09:06:26 +01:00
output_length
2013-10-28 16:22:18 +01:00
The length of compressed header block.
2014-01-11 09:06:26 +01:00
percentage_of_original_size
``input_length`` / ``output_length`` * 100
2013-10-28 16:22:18 +01:00
2014-01-11 09:06:26 +01:00
wire
2013-10-28 16:22:18 +01:00
The compressed header block in hex string.
2014-01-11 09:06:26 +01:00
headers
The input header set.
header_table_size
The header table size adjsuted before deflating header set.
2014-01-11 09:13:24 +01:00
Examples:
.. code-block:: json
2013-10-28 16:22:18 +01:00
2014-01-11 09:06:26 +01:00
{
"context": "request",
"cases":
[
{
"seq": 0,
"input_length": 66,
"output_length": 20,
"percentage_of_original_size": 30.303030303030305,
"wire": "01881f3468e5891afcbf83868a3d856659c62e3f",
"headers": [
{
":authority": "example.org"
},
{
":method": "GET"
},
{
":path": "/"
},
{
":scheme": "https"
},
{
"user-agent": "nghttp2"
}
],
"header_table_size": 4096
}
,
{
"seq": 1,
"input_length": 74,
"output_length": 10,
"percentage_of_original_size": 13.513513513513514,
"wire": "88448504252dd5918485",
"headers": [
{
":authority": "example.org"
},
{
":method": "POST"
},
{
":path": "/account"
},
{
":scheme": "https"
},
{
"user-agent": "nghttp2"
}
],
"header_table_size": 4096
}
]
}
The output can be used as the input for ``inflatehd`` and
``deflatehd``.
With ``-d`` option, the extra ``header_table`` key is added and its
2013-10-28 16:22:18 +01:00
associated value contains the state of dyanmic header table after the
corresponding header set was processed. The value contains following
keys:
entries
The entry in the header table. If ``referenced`` is ``true``, it
is in the reference set. The ``size`` includes the overhead (32
bytes). The ``index`` corresponds to the index of header table.
The ``name`` is the header field name and the ``value`` is the
header field value. They may be displayed as ``**DEALLOCATED**``,
which means that the memory for that string is freed and not
2013-12-21 15:04:57 +01:00
available. This will happen when the specifying smaller value in
``-S`` than ``-s``.
2013-10-28 16:22:18 +01:00
size
The sum of the spaces entries occupied, this includes the
entry overhead.
2014-01-11 09:06:26 +01:00
max_size
2013-10-28 16:22:18 +01:00
The maximum header table size.
2014-01-11 09:06:26 +01:00
deflate_size
The sum of the spaces entries occupied within
``max_deflate_size``.
2013-10-28 16:22:18 +01:00
2014-01-11 09:06:26 +01:00
max_deflate_size
2013-10-28 16:22:18 +01:00
The maximum header table size encoder uses. This can be smaller
2014-01-11 09:06:26 +01:00
than ``max_size``. In this case, encoder only uses up to first
``max_deflate_size`` buffer. Since the header table size is still
``max_size``, the encoder has to keep track of entries ouside the
``max_deflate_size`` but inside the ``max_size`` and make sure
that they are no longer referenced.
2013-10-28 16:22:18 +01:00
2014-01-11 09:13:24 +01:00
Example:
.. code-block:: json
2013-10-28 16:22:18 +01:00
2014-01-11 09:06:26 +01:00
{
"context": "request",
"cases":
[
{
"seq": 0,
"input_length": 66,
"output_length": 20,
"percentage_of_original_size": 30.303030303030305,
"wire": "01881f3468e5891afcbf83868a3d856659c62e3f",
"headers": [
{
":authority": "example.org"
},
{
":method": "GET"
},
{
":path": "/"
},
{
":scheme": "https"
},
{
"user-agent": "nghttp2"
}
],
"header_table_size": 4096,
"header_table": {
"entries": [
{
"index": 1,
"name": "user-agent",
"value": "nghttp2",
"referenced": true,
"size": 49
},
{
"index": 2,
"name": ":scheme",
"value": "https",
"referenced": true,
"size": 44
},
{
"index": 3,
"name": ":path",
"value": "/",
"referenced": true,
"size": 38
},
{
"index": 4,
"name": ":method",
"value": "GET",
"referenced": true,
"size": 42
},
{
"index": 5,
"name": ":authority",
"value": "example.org",
"referenced": true,
"size": 53
}
],
"size": 226,
"max_size": 4096,
"deflate_size": 226,
"max_deflate_size": 4096
}
}
,
{
"seq": 1,
"input_length": 74,
"output_length": 10,
"percentage_of_original_size": 13.513513513513514,
"wire": "88448504252dd5918485",
"headers": [
{
":authority": "example.org"
},
{
":method": "POST"
},
{
":path": "/account"
},
{
":scheme": "https"
},
{
"user-agent": "nghttp2"
}
],
"header_table_size": 4096,
"header_table": {
"entries": [
{
"index": 1,
"name": ":method",
"value": "POST",
"referenced": true,
"size": 43
},
{
"index": 2,
"name": "user-agent",
"value": "nghttp2",
"referenced": true,
"size": 49
},
{
"index": 3,
"name": ":scheme",
"value": "https",
"referenced": true,
"size": 44
},
{
"index": 4,
"name": ":path",
"value": "/",
"referenced": false,
"size": 38
},
{
"index": 5,
"name": ":method",
"value": "GET",
"referenced": false,
"size": 42
},
{
"index": 6,
"name": ":authority",
"value": "example.org",
"referenced": true,
"size": 53
}
],
"size": 269,
"max_size": 4096,
"deflate_size": 269,
"max_deflate_size": 4096
}
}
]
}
2013-10-28 16:22:18 +01:00
inflatehd - header decompressor
+++++++++++++++++++++++++++++++
2014-01-11 09:06:26 +01:00
The ``inflatehd`` reads JSON data from stdin and outputs decompressed
name/value pairs in JSON.
2013-10-28 16:22:18 +01:00
2014-01-11 09:06:26 +01:00
The root JSON object must contain ``context`` key, which indicates
which compression context is used. If it is ``request``, request
compression context is used. Otherwise, response compression context
is used. The value of ``cases`` key contains the sequence of
compressed header block. They share the same compression context and
are processed in the order they appear. Each item in the sequence is a
JSON object and it must have at least ``wire`` key. Its value is a
string containing compressed header block in hex string.
2013-10-28 16:22:18 +01:00
2014-01-11 09:13:24 +01:00
Example:
.. code-block:: json
2013-10-28 16:22:18 +01:00
2014-01-11 09:06:26 +01:00
{
"context": "request",
"cases":
[
{ "wire": "8285" },
{ "wire": "8583" }
]
}
2013-10-28 16:22:18 +01:00
2014-01-11 09:06:26 +01:00
The output is JSON object. It contains ``context`` key and its value
is ``request`` if the compression context is request, otherwise
``response``. The root JSON object also contains ``cases`` key and its
value is an array of JSON object, which has at least following keys:
2013-10-28 16:22:18 +01:00
seq
The index of header set in the input.
headers
2014-01-11 09:06:26 +01:00
The JSON array contains decompressed name/value pairs.
wire
The compressed header block in hex string.
header_table_size
The header table size adjsuted before inflating compressed header
block.
2013-10-28 16:22:18 +01:00
2014-01-11 09:13:24 +01:00
Example:
.. code-block:: json
2013-10-28 16:22:18 +01:00
2014-01-11 09:06:26 +01:00
{
"context": "request",
"cases":
[
{
"seq": 0,
"wire": "01881f3468e5891afcbf83868a3d856659c62e3f",
"headers": [
{
":authority": "example.org"
},
{
":method": "GET"
},
{
":path": "/"
},
{
":scheme": "https"
},
{
"user-agent": "nghttp2"
}
],
"header_table_size": 4096
}
,
{
"seq": 1,
"wire": "88448504252dd5918485",
"headers": [
{
":method": "POST"
},
{
":path": "/account"
},
{
"user-agent": "nghttp2"
},
{
":scheme": "https"
},
{
":authority": "example.org"
}
],
"header_table_size": 4096
}
]
}
The output can be used as the input for ``deflatehd`` and
``inflatehd``.
With ``-d`` option, the extra ``header_table`` key is added and its
2013-10-28 16:22:18 +01:00
associated value contains the state of dyanmic header table after the
corresponding header set was processed. The format is the same as
``deflatehd``.
Python bindings
---------------
This ``python`` directory contains nghttp2 Python bindings. The
bindings currently only provide HPACK compressor and decompressor
classes.
The extension module is called ``nghttp2``.
``make`` will build the bindings and target Python version is
determined by configure script. If the detected Python version is not
what you expect, specify a path to Python executable in ``PYTHON``
variable as an argument to configure script (e.g., ``./configure
PYTHON=/usr/bin/python3.3``).
Example
+++++++
The following example code illustrates basic usage of HPACK compressor
2014-01-10 18:02:53 +01:00
and decompressor in Python:
.. code-block:: python
import binascii
import nghttp2
deflater = nghttp2.HDDeflater(nghttp2.HD_SIDE_REQUEST)
inflater = nghttp2.HDInflater(nghttp2.HD_SIDE_REQUEST)
data = deflater.deflate([(b'foo', b'bar'),
2014-01-11 09:06:26 +01:00
(b'baz', b'buz')])
print(binascii.b2a_hex(data))
hdrs = inflater.inflate(data)
print(hdrs)