walkero
/
nghttp2
1
0
Fork 0
Code Issues Pull Requests Projects Releases 1 Wiki Activity

Merge branch 'bagder-server-tutorial'

This commit is contained in:
Tatsuhiro Tsujikawa 2014-08-01 20:55:45 +09:00
parent 99ceb7df33 ecb7e7db74
commit a234166fc4
1 changed files with 142 additions and 152 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

294
doc/sources/tutorial-server.rst
View File

@ -3,29 +3,28 @@ Tutorial: HTTP/2 server
In this tutorial, we are going to write single-threaded, event-based In this tutorial, we are going to write single-threaded, event-based
HTTP/2 web server, which supports HTTPS only. It can handle HTTP/2 web server, which supports HTTPS only. It can handle
concurrent multiple requests, but only GET method is supported. The concurrent multiple requests, but only the GET method is supported. The
complete source code, `libevent-server.c`_, is attached at the end of complete source code, `libevent-server.c`_, is attached at the end of
this page. It also resides in examples directory in the archive or this page. It also resides in examples directory in the archive or
repository. repository.
This simple server takes 3 arguments, a port number to listen to, a This simple server takes 3 arguments, a port number to listen to, a path to
path to SSL/TLS private key file and certificate file. Its synopsis your SSL/TLS private key file and a path to your certificate file. Its
is like this:: synopsis is like this::
$ libevent-server PORT /path/to/server.key /path/to/server.crt $ libevent-server PORT /path/to/server.key /path/to/server.crt
We use libevent in this tutorial to handle networking I/O. Please We use libevent in this tutorial to handle networking I/O. Please
note that nghttp2 itself does not depend on libevent. note that nghttp2 itself does not depend on libevent.
First we do some setup routine for libevent and OpenSSL library in First we create a setup routine for libevent and OpenSSL in the functions
function ``main()`` and ``run()``, which is not so relevant to nghttp2 ``main()`` and ``run()``. One thing in there you should look at, is the setup
library use. The one thing you should look at is setup NPN callback. of the NPN callback. The NPN callback is used for the server to advertise
The NPN callback is used for the server to advertise the application which application protocols the server supports to a client. In this example
protocols the server supports to a client. In this example program, program, when creating ``SSL_CTX`` object, we store the application protocol
when creating ``SSL_CTX`` object, we stores the application protocol name in the wire format of NPN in a statically allocated buffer. This is safe
name in the wire format of NPN in statically allocated buffer. This is because we only create one ``SSL_CTX`` object in the program's entire life
safe because we only create 1 ``SSL_CTX`` object in the entire program time::
life time::
static unsigned char next_proto_list[256]; static unsigned char next_proto_list[256];
static size_t next_proto_list_len; static size_t next_proto_list_len;
@ -54,25 +53,25 @@ life time::
return ssl_ctx; return ssl_ctx;
} }
The wire format of NPN is a sequence of length prefixed string. The The wire format of NPN is a sequence of length prefixed string. Exactly one
exactly one byte is used to specify the length of each protocol byte is used to specify the length of each protocol identifier. In this
identifier. In this tutorial, we advertise the HTTP/2 protocol the tutorial, we advertise the specific HTTP/2 protocol version the current
nghttp2 library supports. The nghttp2 library exports its identifier nghttp2 library supports. The nghttp2 library exports its identifier in
in :macro:`NGHTTP2_PROTO_VERSION_ID`. The ``next_proto_cb()`` function :macro:`NGHTTP2_PROTO_VERSION_ID`. The ``next_proto_cb()`` function is the
is the server-side NPN callback. In OpenSSL implementation, we just server-side NPN callback. In the OpenSSL implementation, we just assign the
assign the pointer to the NPN buffers we filled earlier. The NPN pointer to the NPN buffers we filled in earlier. The NPN callback function is
callback function is set to ``SSL_CTX`` object using set to the ``SSL_CTX`` object using
``SSL_CTX_set_next_protos_advertised_cb()``. ``SSL_CTX_set_next_protos_advertised_cb()``.
We use ``app_content`` structure to store the application-wide data:: We use the ``app_content`` structure to store application-wide data::
struct app_context { struct app_context {
SSL_CTX *ssl_ctx; SSL_CTX *ssl_ctx;
struct event_base *evbase; struct event_base *evbase;
}; };
We use ``http2_session_data`` structure to store the session-level We use the ``http2_session_data`` structure to store session-level
(which corresponds to 1 HTTP/2 connection) data:: (which corresponds to one HTTP/2 connection) data::
typedef struct http2_session_data { typedef struct http2_session_data {
struct http2_stream_data root; struct http2_stream_data root;
@ -83,8 +82,7 @@ We use ``http2_session_data`` structure to store the session-level
size_t handshake_leftlen; size_t handshake_leftlen;
} http2_session_data; } http2_session_data;
We use ``http2_stream_data`` structure to store the stream-level We use the ``http2_stream_data`` structure to store stream-level data::
data::
typedef struct http2_stream_data { typedef struct http2_stream_data {
struct http2_stream_data *prev, *next; struct http2_stream_data *prev, *next;
@ -93,23 +91,20 @@ data::
int fd; int fd;
} http2_stream_data; } http2_stream_data;
1 HTTP/2 session can have multiple streams. We manage these multiple A single HTTP/2 session can have multiple streams. We manage these multiple
streams by intrusive doubly linked list to add and remove the object streams with a doubly linked list. The first element of this list is pointed
in O(1). The first element of this list is pointed by the to by the ``root->next`` in ``http2_session_data``. Initially, ``root->next``
``root->next`` in ``http2_session_data``. Initially, ``root->next`` is ``NULL``. The ``handshake_leftlen`` member of ``http2_session_data`` is
is ``NULL``. The ``handshake_leftlen`` member of used to track the number of bytes remaining when receiving the first client
``http2_session_data`` is used to track the number of bytes remaining connection preface (:macro:`NGHTTP2_CLIENT_CONNECTION_PREFACE`), which is a 24
when receiving first client connection preface bytes long magic string from the client. We use libevent's bufferevent
(:macro:`NGHTTP2_CLIENT_CONNECTION_PREFACE`), which is 24 bytes magic structure to perform network I/O. Note that the bufferevent object is kept in
byte string, from the client. We use libevent's bufferevent structure ``http2_session_data`` and not in ``http2_stream_data``. This is because
to perform network I/O. Notice that bufferevent object is in ``http2_stream_data`` is just a logical stream multiplexed over the single
``http2_session_data`` and not in ``http2_stream_data``. This is connection managed by bufferevent in ``http2_session_data``.
because ``http2_stream_data`` is just a logical stream multiplexed
over the single connection managed by bufferevent in
``http2_session_data``.
We first create listener object to accept incoming connections. We first create a listener object to accept incoming connections. We use
We use libevent's ``struct evconnlistener`` for this purpose:: libevent's ``struct evconnlistener`` for this purpose::
static void start_listen(struct event_base *evbase, const char *service, static void start_listen(struct event_base *evbase, const char *service,
app_context *app_ctx) app_context *app_ctx)
@ -143,8 +138,8 @@ We use libevent's ``struct evconnlistener`` for this purpose::
errx(1, "Could not start listener"); errx(1, "Could not start listener");
} }
We specify ``acceptcb`` callback which is called when a new connection We specify the ``acceptcb`` callback which is called when a new connection is
is accepted:: accepted::
static void acceptcb(struct evconnlistener *listener, int fd, static void acceptcb(struct evconnlistener *listener, int fd,
struct sockaddr *addr, int addrlen, void *arg) struct sockaddr *addr, int addrlen, void *arg)
@ -157,11 +152,11 @@ is accepted::
session_data); session_data);
} }
Here we create ``http2_session_data`` object. The bufferevent for this Here we create the ``http2_session_data`` object. The bufferevent for this
connection is also initialized at this time. We specify 2 callbacks connection is also initialized at this time. We specify two callbacks for the
for the bufferevent: ``handshake_readcb`` and ``eventcb``. bufferevent: ``handshake_readcb`` and ``eventcb``.
The ``eventcb()`` is invoked by libevent event loop when an event The ``eventcb()`` callback is invoked by the libevent event loop when an event
(e.g., connection has been established, timeout, etc) happens on the (e.g., connection has been established, timeout, etc) happens on the
underlying network socket:: underlying network socket::
@ -182,17 +177,15 @@ underlying network socket::
delete_http2_session_data(session_data); delete_http2_session_data(session_data);
} }
For ``BEV_EVENT_EOF``, ``BEV_EVENT_ERROR`` and ``BEV_EVENT_TIMEOUT`` For the ``BEV_EVENT_EOF``, ``BEV_EVENT_ERROR`` and ``BEV_EVENT_TIMEOUT``
event, we just simply tear down the connection. The events, we just simply tear down the connection. The
``delete_http2_session_data()`` function destroys ``delete_http2_session_data()`` function destroys the ``http2_session_data``
``http2_session_data`` object and thus its bufferevent member. As a object and thus also its bufferevent member. As a result, the underlying
result, the underlying connection is closed. The connection is closed. The ``BEV_EVENT_CONNECTED`` event is invoked when
``BEV_EVENT_CONNECTED`` event is invoked when SSL/TLS handshake is SSL/TLS handshake is finished successfully.
finished successfully.
The ``handshake_readcb()`` is a callback function to handle 24 bytes ``handshake_readcb()`` is a callback function to handle a 24 bytes magic byte
magic byte string from a client, since nghttp2 library does not handle string coming from a client, since the nghttp2 library does not handle it::
it::
static void handshake_readcb(struct bufferevent *bev, void *ptr) static void handshake_readcb(struct bufferevent *bev, void *ptr)
{ {
@ -225,14 +218,13 @@ it::
} }
We check that the received byte string matches We check that the received byte string matches
:macro:`NGHTTP2_CLIENT_CONNECTION_PREFACE`. When they match, the :macro:`NGHTTP2_CLIENT_CONNECTION_PREFACE`. When they match, the connection
connection state is ready for starting HTTP/2 communication. First state is ready to start the HTTP/2 communication. First we change the callback
we change the callback functions for the bufferevent object. We use functions for the bufferevent object. We use the same ``eventcb`` callback as
same ``eventcb`` as before. But we specify new ``readcb`` and before, but we specify new ``readcb`` and ``writecb`` functions to handle the
``writecb`` function to handle HTTP/2 communication. We describe HTTP/2 communication. These two functions are described later.
these 2 functions later.
We initialize nghttp2 session object which is done in We initialize a nghttp2 session object which is done in
``initialize_nghttp2_session()``:: ``initialize_nghttp2_session()``::
static void initialize_nghttp2_session(http2_session_data *session_data) static void initialize_nghttp2_session(http2_session_data *session_data)
@ -247,12 +239,12 @@ We initialize nghttp2 session object which is done in
nghttp2_session_server_new(&session_data->session, &callbacks, session_data); nghttp2_session_server_new(&session_data->session, &callbacks, session_data);
} }
Since we are creating server, nghttp2 session object is created using Since we are creating a server, the nghttp2 session object is created using
`nghttp2_session_server_new()` function. We registers 5 callbacks to `nghttp2_session_server_new()` function. We registers five callbacks for
nghttp2 session object. We'll talk about these callbacks later. nghttp2 session object. We'll talk about these callbacks later.
After initialization of nghttp2 session object, we are going to send After initialization of the nghttp2 session object, we are going to send
server connection header in ``send_server_connection_header()``:: a server connection header in ``send_server_connection_header()``::
static int send_server_connection_header(http2_session_data *session_data) static int send_server_connection_header(http2_session_data *session_data)
{ {
@ -270,18 +262,18 @@ server connection header in ``send_server_connection_header()``::
return 0; return 0;
} }
The server connection header is SETTINGS frame. We specify The server connection header is a SETTINGS frame. We specify
SETTINGS_MAX_CONCURRENT_STREAMS to 100 in SETTINGS frame. To queue SETTINGS_MAX_CONCURRENT_STREAMS to 100 in the SETTINGS frame. To queue
the SETTINGS frame for the transmission, we use the SETTINGS frame for the transmission, we use
`nghttp2_submit_settings()`. Note that `nghttp2_submit_settings()` `nghttp2_submit_settings()`. Note that `nghttp2_submit_settings()`
function only queues the frame and not actually send it. All function only queues the frame and it does not actually send it. All
``nghttp2_submit_*()`` family functions have this property. To functions in the ``nghttp2_submit_*()`` family have this property. To
actually send the frame, `nghttp2_session_send()` is used, which is actually send the frame, `nghttp2_session_send()` should be used, as
described about later. described later.
Since bufferevent may buffer more than first 24 bytes from the client, Since bufferevent may buffer more than the first 24 bytes from the client, we
we have to process them here since libevent won't invoke callback have to process them here since libevent won't invoke callback functions for
functions for these pending data. To process received data, we call this pending data. To process the received data, we call the
``session_recv()`` function:: ``session_recv()`` function::
static int session_recv(http2_session_data *session_data) static int session_recv(http2_session_data *session_data)
@ -306,12 +298,12 @@ functions for these pending data. To process received data, we call
return 0; return 0;
} }
In this function, we feed all unprocessed, received data to nghttp2 In this function, we feed all unprocessed but already received data to the
session object using `nghttp2_session_mem_recv()` function. The nghttp2 session object using the `nghttp2_session_mem_recv()` function. The
`nghttp2_session_mem_recv()` processes the received data and may `nghttp2_session_mem_recv()` function processes the data and may invoke the
invoke nghttp2 callbacks and also queue outgoing frames. Since there nghttp2 callbacks and also queue outgoing frames. Since there may be pending
may be pending frames, we call ``session_send()`` function to send outgoing frames, we call ``session_send()`` function to send off those
those frames. The ``session_send()`` function is defined as follows:: frames. The ``session_send()`` function is defined as follows::
static int session_send(http2_session_data *session_data) static int session_send(http2_session_data *session_data)
{ {
@ -325,8 +317,8 @@ those frames. The ``session_send()`` function is defined as follows::
} }
The `nghttp2_session_send()` function serializes the frame into wire The `nghttp2_session_send()` function serializes the frame into wire
format and call :member:`nghttp2_session_callbacks.send_callback` with format and calls :member:`nghttp2_session_callbacks.send_callback` with
it. We set ``send_callback()`` function to it. We set the ``send_callback()`` function to
:member:`nghttp2_session_callbacks.send_callback` in :member:`nghttp2_session_callbacks.send_callback` in
``initialize_nghttp2_session()`` function described earlier. It is ``initialize_nghttp2_session()`` function described earlier. It is
defined as follows:: defined as follows::
@ -346,18 +338,17 @@ defined as follows::
return length; return length;
} }
Since we use bufferevent to abstract network I/O, we just write the Since we use bufferevent to abstract network I/O, we just write the data to
data to the bufferevent object. Note that `nghttp2_session_send()` the bufferevent object. Note that `nghttp2_session_send()` continues to write
continues to write all frames queued so far. If we were writing the all frames queued so far. If we were writing the data to a non-blocking socket
data to the non-blocking socket directly using ``write()`` system call directly using ``write()`` system call in the
in the :member:`nghttp2_session_callbacks.send_callback`, we will :member:`nghttp2_session_callbacks.send_callback`, we would surely get
surely get ``EAGAIN`` or ``EWOULDBLOCK`` since the socket has limited ``EAGAIN`` or ``EWOULDBLOCK`` back since the socket has limited send
send buffer. If that happens, we can return buffer. If that happens, we can return :macro:`NGHTTP2_ERR_WOULDBLOCK` to
:macro:`NGHTTP2_ERR_WOULDBLOCK` to signal the nghttp2 library to stop signal the nghttp2 library to stop sending further data. But when writing to
sending further data. But writing to the bufferevent, we have to the bufferevent, we have to regulate the amount data to get buffered ourselves
regulate the amount data to be buffered by ourselves to avoid possible to avoid using huge amounts of memory. To achieve this, we check the size of
huge memory consumption. To achieve this, we check the size of output the output buffer and if it reaches more than or equal to
buffer and if it is more than or equal to
``OUTPUT_WOULDBLOCK_THRESHOLD`` bytes, we stop writing data and return ``OUTPUT_WOULDBLOCK_THRESHOLD`` bytes, we stop writing data and return
:macro:`NGHTTP2_ERR_WOULDBLOCK` to tell the library to stop calling :macro:`NGHTTP2_ERR_WOULDBLOCK` to tell the library to stop calling
send_callback. send_callback.
@ -377,8 +368,8 @@ data is available to read in the bufferevent input buffer::
In this function, we just call ``session_recv()`` to process incoming In this function, we just call ``session_recv()`` to process incoming
data. data.
The third bufferevent callback is ``writecb()``, which is invoked when The third bufferevent callback is ``writecb()``, which is invoked when all
all data written in the bufferevent output buffer have been sent:: data in the bufferevent output buffer has been sent::
static void writecb(struct bufferevent *bev, void *ptr) static void writecb(struct bufferevent *bev, void *ptr)
{ {
@ -397,29 +388,28 @@ all data written in the bufferevent output buffer have been sent::
} }
} }
First we check whether we should drop connection or not. The nghttp2 First we check whether we should drop the connection or not. The nghttp2
session object keeps track of reception and transmission of GOAWAY session object keeps track of reception and transmission of GOAWAY frames and
frame and other error conditions as well. Using these information, other error conditions as well. Using this information, the nghttp2 session
nghttp2 session object will tell whether the connection should be object will tell whether the connection should be dropped or not. More
dropped or not. More specifically, both `nghttp2_session_want_read()` specifically, if both `nghttp2_session_want_read()` and
and `nghttp2_session_want_write()` return 0, we have no business in `nghttp2_session_want_write()` return 0, we have no business left in the
the connection. But since we are using bufferevent and its deferred connection. But since we are using bufferevent and its deferred callback
callback option, the bufferevent output buffer may contain the pending option, the bufferevent output buffer may contain pending data when the
data when the ``writecb()`` is called. To handle this situation, we ``writecb()`` is called. To handle this, we check whether the output buffer is
also check whether the output buffer is empty or not. If these empty or not. If all these conditions are met, we drop connection.
conditions are met, we drop connection.
Otherwise, we call ``session_send()`` to process pending output Otherwise, we call ``session_send()`` to process the pending output
data. Remember that in ``send_callback()``, we may not write all data data. Remember that in ``send_callback()``, we must not write all data to
to bufferevent to avoid excessive buffering. We continue process bufferevent to avoid excessive buffering. We continue processing pending data
pending data when output buffer becomes empty. when the output buffer becomes empty.
We have already described about nghttp2 callback ``send_callback()``. We have already described the nghttp2 callback ``send_callback()``. Let's
Let's describe remaining nghttp2 callbacks we setup in learn about the remaining nghttp2 callbacks we setup in
``initialize_nghttp2_setup()`` function. ``initialize_nghttp2_setup()`` function.
The ``on_begin_headers_callback()`` function is invoked when reception The ``on_begin_headers_callback()`` function is invoked when the reception of
of header block in HEADERS or PUSH_PROMISE frame is started:: a header block in HEADERS or PUSH_PROMISE frame is started::
static int on_begin_headers_callback(nghttp2_session *session, static int on_begin_headers_callback(nghttp2_session *session,
const nghttp2_frame *frame, const nghttp2_frame *frame,
@ -438,17 +428,17 @@ of header block in HEADERS or PUSH_PROMISE frame is started::
return 0; return 0;
} }
We only interested in HEADERS frame in this function. Since HEADERS We are only interested in the HEADERS frame in this function. Since the
frame has several roles in HTTP/2 protocol, we check that it is a HEADERS frame has several roles in the HTTP/2 protocol, we check that it is a
request HEADERS, which opens new stream. If frame is request HEADERS, request HEADERS, which opens new stream. If the frame is a request HEADERS, we
then we create ``http2_stream_data`` object to store stream related create a ``http2_stream_data`` object to store the stream related data. We
data. We associate created ``http2_stream_data`` object to the stream associate the created ``http2_stream_data`` object with the stream in the
in nghttp2 session object using `nghttp2_set_stream_user_data()` in nghttp2 session object using `nghttp2_set_stream_user_data()` to get the
order to get the object without searching through doubly linked list. object without searching through the doubly linked list.
In this example server, we want to serve files relative to the current In this example server, we want to serve files relative to the current working
working directory the program was invoked. Each header name/value pair directory in which the program was invoked. Each header name/value pair is
is emitted via ``on_header_callback`` function, which is called after emitted via ``on_header_callback`` function, which is called after
``on_begin_headers_callback()``:: ``on_begin_headers_callback()``::
static int on_header_callback(nghttp2_session *session, static int on_header_callback(nghttp2_session *session,
@ -479,10 +469,10 @@ is emitted via ``on_header_callback`` function, which is called after
return 0; return 0;
} }
We search ``:path`` header field in request headers and keep the We search for the ``:path`` header field among the request headers and store
requested path in ``http2_stream_data`` object. In this example the requested path in the ``http2_stream_data`` object. In this example
program, we ignore ``:method`` header field and always treat the program, we ignore ``:method`` header field and always treat the request as a
request as GET request. GET request.
The ``on_frame_recv_callback()`` function is invoked when a frame is The ``on_frame_recv_callback()`` function is invoked when a frame is
fully received:: fully received::
@ -513,15 +503,15 @@ fully received::
return 0; return 0;
} }
First we retrieve ``http2_stream_data`` object associated to the First we retrieve the ``http2_stream_data`` object associated with the stream
stream in ``on_begin_headers_callback()``. It is done using in ``on_begin_headers_callback()``. It is done using
`nghttp2_session_get_stream_user_data()`. If the requested path cannot `nghttp2_session_get_stream_user_data()`. If the requested path cannot be
be served for some reasons (e.g., file is not found), we send 404 served for some reason (e.g., file is not found), we send a 404 response,
response, which is done in ``error_reply()``. Otherwise, we open which is done in ``error_reply()``. Otherwise, we open the requested file and
requested file and send its content. We send 1 header field send its content. We send the header field ``:status`` as a single response
``:status`` as a response header. header.
Sending content of a file is done in ``send_response()`` function:: Sending the content of the file is done in ``send_response()`` function::
static int send_response(nghttp2_session *session, int32_t stream_id, static int send_response(nghttp2_session *session, int32_t stream_id,
nghttp2_nv *nva, size_t nvlen, int fd) nghttp2_nv *nva, size_t nvlen, int fd)
@ -539,12 +529,12 @@ Sending content of a file is done in ``send_response()`` function::
return 0; return 0;
} }
The nghttp2 library uses :type:`nghttp2_data_provider` structure to The nghttp2 library uses the :type:`nghttp2_data_provider` structure to
send entity body to the remote peer. The ``source`` member of this send entity body to the remote peer. The ``source`` member of this
structure is a union and it can be either void pointer or int which is structure is a union and it can be either void pointer or int which is
intended to be used as file descriptor. In this example server, we use intended to be used as file descriptor. In this example server, we use
file descriptor. We also set ``file_read_callback()`` callback the file descriptor. We also set the ``file_read_callback()`` callback
function to read content of the file:: function to read the contents of the file::
static ssize_t file_read_callback static ssize_t file_read_callback
(nghttp2_session *session, int32_t stream_id, (nghttp2_session *session, int32_t stream_id,
@ -563,14 +553,14 @@ function to read content of the file::
return r; return r;
} }
If error happens while reading file, we return If an error happens while reading the file, we return
:macro:`NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE`. This tells the :macro:`NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE`. This tells the
library to send RST_STREAM to the stream. When all data are read, set library to send RST_STREAM to the stream. When all data has been read, set
:macro:`NGHTTP2_DATA_FLAG_EOF` flag to ``*data_flags`` to tell the the :macro:`NGHTTP2_DATA_FLAG_EOF` flag to ``*data_flags`` to tell the
nghttp2 library that we have finished reading file. nghttp2 library that we have finished reading the file.
The `nghttp2_submit_response()` is used to send response to the remote The `nghttp2_submit_response()` function is used to send the response to the
peer. remote peer.
The ``on_stream_close_callback()`` function is invoked when the stream The ``on_stream_close_callback()`` function is invoked when the stream
is about to close:: is about to close::
@ -592,5 +582,5 @@ is about to close::
return 0; return 0;
} }
We destroy ``http2_stream_data`` object in this function since the We destroy the ``http2_stream_data`` object in this function since the stream
stream is about to close and we no longer use that object. is about to close and we no longer use that object.