diff --git a/doc/sources/tutorial-server.rst b/doc/sources/tutorial-server.rst index 4da8a667..8b4a36e8 100644 --- a/doc/sources/tutorial-server.rst +++ b/doc/sources/tutorial-server.rst @@ -3,29 +3,28 @@ Tutorial: HTTP/2 server In this tutorial, we are going to write single-threaded, event-based HTTP/2 web server, which supports HTTPS only. It can handle -concurrent multiple requests, but only GET method is supported. The +concurrent multiple requests, but only the GET method is supported. The complete source code, `libevent-server.c`_, is attached at the end of this page. It also resides in examples directory in the archive or repository. -This simple server takes 3 arguments, a port number to listen to, a -path to SSL/TLS private key file and certificate file. Its synopsis -is like this:: +This simple server takes 3 arguments, a port number to listen to, a path to +your SSL/TLS private key file and a path to your certificate file. Its +synopsis is like this:: $ libevent-server PORT /path/to/server.key /path/to/server.crt We use libevent in this tutorial to handle networking I/O. Please note that nghttp2 itself does not depend on libevent. -First we do some setup routine for libevent and OpenSSL library in -function ``main()`` and ``run()``, which is not so relevant to nghttp2 -library use. The one thing you should look at is setup NPN callback. -The NPN callback is used for the server to advertise the application -protocols the server supports to a client. In this example program, -when creating ``SSL_CTX`` object, we stores the application protocol -name in the wire format of NPN in statically allocated buffer. This is -safe because we only create 1 ``SSL_CTX`` object in the entire program -life time:: +First we create a setup routine for libevent and OpenSSL in the functions +``main()`` and ``run()``. One thing in there you should look at, is the setup +of the NPN callback. The NPN callback is used for the server to advertise +which application protocols the server supports to a client. In this example +program, when creating ``SSL_CTX`` object, we store the application protocol +name in the wire format of NPN in a statically allocated buffer. This is safe +because we only create one ``SSL_CTX`` object in the program's entire life +time:: static unsigned char next_proto_list[256]; static size_t next_proto_list_len; @@ -54,25 +53,25 @@ life time:: return ssl_ctx; } -The wire format of NPN is a sequence of length prefixed string. The -exactly one byte is used to specify the length of each protocol -identifier. In this tutorial, we advertise the HTTP/2 protocol the -nghttp2 library supports. The nghttp2 library exports its identifier -in :macro:`NGHTTP2_PROTO_VERSION_ID`. The ``next_proto_cb()`` function -is the server-side NPN callback. In OpenSSL implementation, we just -assign the pointer to the NPN buffers we filled earlier. The NPN -callback function is set to ``SSL_CTX`` object using +The wire format of NPN is a sequence of length prefixed string. Exactly one +byte is used to specify the length of each protocol identifier. In this +tutorial, we advertise the specific HTTP/2 protocol version the current +nghttp2 library supports. The nghttp2 library exports its identifier in +:macro:`NGHTTP2_PROTO_VERSION_ID`. The ``next_proto_cb()`` function is the +server-side NPN callback. In the OpenSSL implementation, we just assign the +pointer to the NPN buffers we filled in earlier. The NPN callback function is +set to the ``SSL_CTX`` object using ``SSL_CTX_set_next_protos_advertised_cb()``. -We use ``app_content`` structure to store the application-wide data:: +We use the ``app_content`` structure to store application-wide data:: struct app_context { SSL_CTX *ssl_ctx; struct event_base *evbase; }; -We use ``http2_session_data`` structure to store the session-level -(which corresponds to 1 HTTP/2 connection) data:: +We use the ``http2_session_data`` structure to store session-level +(which corresponds to one HTTP/2 connection) data:: typedef struct http2_session_data { struct http2_stream_data root; @@ -83,8 +82,7 @@ We use ``http2_session_data`` structure to store the session-level size_t handshake_leftlen; } http2_session_data; -We use ``http2_stream_data`` structure to store the stream-level -data:: +We use the ``http2_stream_data`` structure to store stream-level data:: typedef struct http2_stream_data { struct http2_stream_data *prev, *next; @@ -93,23 +91,20 @@ data:: int fd; } http2_stream_data; -1 HTTP/2 session can have multiple streams. We manage these multiple -streams by intrusive doubly linked list to add and remove the object -in O(1). The first element of this list is pointed by the -``root->next`` in ``http2_session_data``. Initially, ``root->next`` -is ``NULL``. The ``handshake_leftlen`` member of -``http2_session_data`` is used to track the number of bytes remaining -when receiving first client connection preface -(:macro:`NGHTTP2_CLIENT_CONNECTION_PREFACE`), which is 24 bytes magic -byte string, from the client. We use libevent's bufferevent structure -to perform network I/O. Notice that bufferevent object is in -``http2_session_data`` and not in ``http2_stream_data``. This is -because ``http2_stream_data`` is just a logical stream multiplexed -over the single connection managed by bufferevent in -``http2_session_data``. +A single HTTP/2 session can have multiple streams. We manage these multiple +streams with a doubly linked list. The first element of this list is pointed +to by the ``root->next`` in ``http2_session_data``. Initially, ``root->next`` +is ``NULL``. The ``handshake_leftlen`` member of ``http2_session_data`` is +used to track the number of bytes remaining when receiving the first client +connection preface (:macro:`NGHTTP2_CLIENT_CONNECTION_PREFACE`), which is a 24 +bytes long magic string from the client. We use libevent's bufferevent +structure to perform network I/O. Note that the bufferevent object is kept in +``http2_session_data`` and not in ``http2_stream_data``. This is because +``http2_stream_data`` is just a logical stream multiplexed over the single +connection managed by bufferevent in ``http2_session_data``. -We first create listener object to accept incoming connections. -We use libevent's ``struct evconnlistener`` for this purpose:: +We first create a listener object to accept incoming connections. We use +libevent's ``struct evconnlistener`` for this purpose:: static void start_listen(struct event_base *evbase, const char *service, app_context *app_ctx) @@ -143,8 +138,8 @@ We use libevent's ``struct evconnlistener`` for this purpose:: errx(1, "Could not start listener"); } -We specify ``acceptcb`` callback which is called when a new connection -is accepted:: +We specify the ``acceptcb`` callback which is called when a new connection is +accepted:: static void acceptcb(struct evconnlistener *listener, int fd, struct sockaddr *addr, int addrlen, void *arg) @@ -157,11 +152,11 @@ is accepted:: session_data); } -Here we create ``http2_session_data`` object. The bufferevent for this -connection is also initialized at this time. We specify 2 callbacks -for the bufferevent: ``handshake_readcb`` and ``eventcb``. +Here we create the ``http2_session_data`` object. The bufferevent for this +connection is also initialized at this time. We specify two callbacks for the +bufferevent: ``handshake_readcb`` and ``eventcb``. -The ``eventcb()`` is invoked by libevent event loop when an event +The ``eventcb()`` callback is invoked by the libevent event loop when an event (e.g., connection has been established, timeout, etc) happens on the underlying network socket:: @@ -182,17 +177,15 @@ underlying network socket:: delete_http2_session_data(session_data); } -For ``BEV_EVENT_EOF``, ``BEV_EVENT_ERROR`` and ``BEV_EVENT_TIMEOUT`` -event, we just simply tear down the connection. The -``delete_http2_session_data()`` function destroys -``http2_session_data`` object and thus its bufferevent member. As a -result, the underlying connection is closed. The -``BEV_EVENT_CONNECTED`` event is invoked when SSL/TLS handshake is -finished successfully. +For the ``BEV_EVENT_EOF``, ``BEV_EVENT_ERROR`` and ``BEV_EVENT_TIMEOUT`` +events, we just simply tear down the connection. The +``delete_http2_session_data()`` function destroys the ``http2_session_data`` +object and thus also its bufferevent member. As a result, the underlying +connection is closed. The ``BEV_EVENT_CONNECTED`` event is invoked when +SSL/TLS handshake is finished successfully. -The ``handshake_readcb()`` is a callback function to handle 24 bytes -magic byte string from a client, since nghttp2 library does not handle -it:: +``handshake_readcb()`` is a callback function to handle a 24 bytes magic byte +string coming from a client, since the nghttp2 library does not handle it:: static void handshake_readcb(struct bufferevent *bev, void *ptr) { @@ -225,14 +218,13 @@ it:: } We check that the received byte string matches -:macro:`NGHTTP2_CLIENT_CONNECTION_PREFACE`. When they match, the -connection state is ready for starting HTTP/2 communication. First -we change the callback functions for the bufferevent object. We use -same ``eventcb`` as before. But we specify new ``readcb`` and -``writecb`` function to handle HTTP/2 communication. We describe -these 2 functions later. +:macro:`NGHTTP2_CLIENT_CONNECTION_PREFACE`. When they match, the connection +state is ready to start the HTTP/2 communication. First we change the callback +functions for the bufferevent object. We use the same ``eventcb`` callback as +before, but we specify new ``readcb`` and ``writecb`` functions to handle the +HTTP/2 communication. These two functions are described later. -We initialize nghttp2 session object which is done in +We initialize a nghttp2 session object which is done in ``initialize_nghttp2_session()``:: static void initialize_nghttp2_session(http2_session_data *session_data) @@ -247,12 +239,12 @@ We initialize nghttp2 session object which is done in nghttp2_session_server_new(&session_data->session, &callbacks, session_data); } -Since we are creating server, nghttp2 session object is created using -`nghttp2_session_server_new()` function. We registers 5 callbacks to +Since we are creating a server, the nghttp2 session object is created using +`nghttp2_session_server_new()` function. We registers five callbacks for nghttp2 session object. We'll talk about these callbacks later. -After initialization of nghttp2 session object, we are going to send -server connection header in ``send_server_connection_header()``:: +After initialization of the nghttp2 session object, we are going to send +a server connection header in ``send_server_connection_header()``:: static int send_server_connection_header(http2_session_data *session_data) { @@ -270,18 +262,18 @@ server connection header in ``send_server_connection_header()``:: return 0; } -The server connection header is SETTINGS frame. We specify -SETTINGS_MAX_CONCURRENT_STREAMS to 100 in SETTINGS frame. To queue +The server connection header is a SETTINGS frame. We specify +SETTINGS_MAX_CONCURRENT_STREAMS to 100 in the SETTINGS frame. To queue the SETTINGS frame for the transmission, we use `nghttp2_submit_settings()`. Note that `nghttp2_submit_settings()` -function only queues the frame and not actually send it. All -``nghttp2_submit_*()`` family functions have this property. To -actually send the frame, `nghttp2_session_send()` is used, which is -described about later. +function only queues the frame and it does not actually send it. All +functions in the ``nghttp2_submit_*()`` family have this property. To +actually send the frame, `nghttp2_session_send()` should be used, as +described later. -Since bufferevent may buffer more than first 24 bytes from the client, -we have to process them here since libevent won't invoke callback -functions for these pending data. To process received data, we call +Since bufferevent may buffer more than the first 24 bytes from the client, we +have to process them here since libevent won't invoke callback functions for +this pending data. To process the received data, we call the ``session_recv()`` function:: static int session_recv(http2_session_data *session_data) @@ -306,12 +298,12 @@ functions for these pending data. To process received data, we call return 0; } -In this function, we feed all unprocessed, received data to nghttp2 -session object using `nghttp2_session_mem_recv()` function. The -`nghttp2_session_mem_recv()` processes the received data and may -invoke nghttp2 callbacks and also queue outgoing frames. Since there -may be pending frames, we call ``session_send()`` function to send -those frames. The ``session_send()`` function is defined as follows:: +In this function, we feed all unprocessed but already received data to the +nghttp2 session object using the `nghttp2_session_mem_recv()` function. The +`nghttp2_session_mem_recv()` function processes the data and may invoke the +nghttp2 callbacks and also queue outgoing frames. Since there may be pending +outgoing frames, we call ``session_send()`` function to send off those +frames. The ``session_send()`` function is defined as follows:: static int session_send(http2_session_data *session_data) { @@ -325,8 +317,8 @@ those frames. The ``session_send()`` function is defined as follows:: } The `nghttp2_session_send()` function serializes the frame into wire -format and call :member:`nghttp2_session_callbacks.send_callback` with -it. We set ``send_callback()`` function to +format and calls :member:`nghttp2_session_callbacks.send_callback` with +it. We set the ``send_callback()`` function to :member:`nghttp2_session_callbacks.send_callback` in ``initialize_nghttp2_session()`` function described earlier. It is defined as follows:: @@ -346,18 +338,17 @@ defined as follows:: return length; } -Since we use bufferevent to abstract network I/O, we just write the -data to the bufferevent object. Note that `nghttp2_session_send()` -continues to write all frames queued so far. If we were writing the -data to the non-blocking socket directly using ``write()`` system call -in the :member:`nghttp2_session_callbacks.send_callback`, we will -surely get ``EAGAIN`` or ``EWOULDBLOCK`` since the socket has limited -send buffer. If that happens, we can return -:macro:`NGHTTP2_ERR_WOULDBLOCK` to signal the nghttp2 library to stop -sending further data. But writing to the bufferevent, we have to -regulate the amount data to be buffered by ourselves to avoid possible -huge memory consumption. To achieve this, we check the size of output -buffer and if it is more than or equal to +Since we use bufferevent to abstract network I/O, we just write the data to +the bufferevent object. Note that `nghttp2_session_send()` continues to write +all frames queued so far. If we were writing the data to a non-blocking socket +directly using ``write()`` system call in the +:member:`nghttp2_session_callbacks.send_callback`, we would surely get +``EAGAIN`` or ``EWOULDBLOCK`` back since the socket has limited send +buffer. If that happens, we can return :macro:`NGHTTP2_ERR_WOULDBLOCK` to +signal the nghttp2 library to stop sending further data. But when writing to +the bufferevent, we have to regulate the amount data to get buffered ourselves +to avoid using huge amounts of memory. To achieve this, we check the size of +the output buffer and if it reaches more than or equal to ``OUTPUT_WOULDBLOCK_THRESHOLD`` bytes, we stop writing data and return :macro:`NGHTTP2_ERR_WOULDBLOCK` to tell the library to stop calling send_callback. @@ -377,8 +368,8 @@ data is available to read in the bufferevent input buffer:: In this function, we just call ``session_recv()`` to process incoming data. -The third bufferevent callback is ``writecb()``, which is invoked when -all data written in the bufferevent output buffer have been sent:: +The third bufferevent callback is ``writecb()``, which is invoked when all +data in the bufferevent output buffer has been sent:: static void writecb(struct bufferevent *bev, void *ptr) { @@ -397,29 +388,28 @@ all data written in the bufferevent output buffer have been sent:: } } -First we check whether we should drop connection or not. The nghttp2 -session object keeps track of reception and transmission of GOAWAY -frame and other error conditions as well. Using these information, -nghttp2 session object will tell whether the connection should be -dropped or not. More specifically, both `nghttp2_session_want_read()` -and `nghttp2_session_want_write()` return 0, we have no business in -the connection. But since we are using bufferevent and its deferred -callback option, the bufferevent output buffer may contain the pending -data when the ``writecb()`` is called. To handle this situation, we -also check whether the output buffer is empty or not. If these -conditions are met, we drop connection. +First we check whether we should drop the connection or not. The nghttp2 +session object keeps track of reception and transmission of GOAWAY frames and +other error conditions as well. Using this information, the nghttp2 session +object will tell whether the connection should be dropped or not. More +specifically, if both `nghttp2_session_want_read()` and +`nghttp2_session_want_write()` return 0, we have no business left in the +connection. But since we are using bufferevent and its deferred callback +option, the bufferevent output buffer may contain pending data when the +``writecb()`` is called. To handle this, we check whether the output buffer is +empty or not. If all these conditions are met, we drop connection. -Otherwise, we call ``session_send()`` to process pending output -data. Remember that in ``send_callback()``, we may not write all data -to bufferevent to avoid excessive buffering. We continue process -pending data when output buffer becomes empty. +Otherwise, we call ``session_send()`` to process the pending output +data. Remember that in ``send_callback()``, we must not write all data to +bufferevent to avoid excessive buffering. We continue processing pending data +when the output buffer becomes empty. -We have already described about nghttp2 callback ``send_callback()``. -Let's describe remaining nghttp2 callbacks we setup in +We have already described the nghttp2 callback ``send_callback()``. Let's +learn about the remaining nghttp2 callbacks we setup in ``initialize_nghttp2_setup()`` function. -The ``on_begin_headers_callback()`` function is invoked when reception -of header block in HEADERS or PUSH_PROMISE frame is started:: +The ``on_begin_headers_callback()`` function is invoked when the reception of +a header block in HEADERS or PUSH_PROMISE frame is started:: static int on_begin_headers_callback(nghttp2_session *session, const nghttp2_frame *frame, @@ -438,17 +428,17 @@ of header block in HEADERS or PUSH_PROMISE frame is started:: return 0; } -We only interested in HEADERS frame in this function. Since HEADERS -frame has several roles in HTTP/2 protocol, we check that it is a -request HEADERS, which opens new stream. If frame is request HEADERS, -then we create ``http2_stream_data`` object to store stream related -data. We associate created ``http2_stream_data`` object to the stream -in nghttp2 session object using `nghttp2_set_stream_user_data()` in -order to get the object without searching through doubly linked list. +We are only interested in the HEADERS frame in this function. Since the +HEADERS frame has several roles in the HTTP/2 protocol, we check that it is a +request HEADERS, which opens new stream. If the frame is a request HEADERS, we +create a ``http2_stream_data`` object to store the stream related data. We +associate the created ``http2_stream_data`` object with the stream in the +nghttp2 session object using `nghttp2_set_stream_user_data()` to get the +object without searching through the doubly linked list. -In this example server, we want to serve files relative to the current -working directory the program was invoked. Each header name/value pair -is emitted via ``on_header_callback`` function, which is called after +In this example server, we want to serve files relative to the current working +directory in which the program was invoked. Each header name/value pair is +emitted via ``on_header_callback`` function, which is called after ``on_begin_headers_callback()``:: static int on_header_callback(nghttp2_session *session, @@ -479,10 +469,10 @@ is emitted via ``on_header_callback`` function, which is called after return 0; } -We search ``:path`` header field in request headers and keep the -requested path in ``http2_stream_data`` object. In this example -program, we ignore ``:method`` header field and always treat the -request as GET request. +We search for the ``:path`` header field among the request headers and store +the requested path in the ``http2_stream_data`` object. In this example +program, we ignore ``:method`` header field and always treat the request as a +GET request. The ``on_frame_recv_callback()`` function is invoked when a frame is fully received:: @@ -513,15 +503,15 @@ fully received:: return 0; } -First we retrieve ``http2_stream_data`` object associated to the -stream in ``on_begin_headers_callback()``. It is done using -`nghttp2_session_get_stream_user_data()`. If the requested path cannot -be served for some reasons (e.g., file is not found), we send 404 -response, which is done in ``error_reply()``. Otherwise, we open -requested file and send its content. We send 1 header field -``:status`` as a response header. +First we retrieve the ``http2_stream_data`` object associated with the stream +in ``on_begin_headers_callback()``. It is done using +`nghttp2_session_get_stream_user_data()`. If the requested path cannot be +served for some reason (e.g., file is not found), we send a 404 response, +which is done in ``error_reply()``. Otherwise, we open the requested file and +send its content. We send the header field ``:status`` as a single response +header. -Sending content of a file is done in ``send_response()`` function:: +Sending the content of the file is done in ``send_response()`` function:: static int send_response(nghttp2_session *session, int32_t stream_id, nghttp2_nv *nva, size_t nvlen, int fd) @@ -539,12 +529,12 @@ Sending content of a file is done in ``send_response()`` function:: return 0; } -The nghttp2 library uses :type:`nghttp2_data_provider` structure to +The nghttp2 library uses the :type:`nghttp2_data_provider` structure to send entity body to the remote peer. The ``source`` member of this structure is a union and it can be either void pointer or int which is intended to be used as file descriptor. In this example server, we use -file descriptor. We also set ``file_read_callback()`` callback -function to read content of the file:: +the file descriptor. We also set the ``file_read_callback()`` callback +function to read the contents of the file:: static ssize_t file_read_callback (nghttp2_session *session, int32_t stream_id, @@ -563,14 +553,14 @@ function to read content of the file:: return r; } -If error happens while reading file, we return +If an error happens while reading the file, we return :macro:`NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE`. This tells the -library to send RST_STREAM to the stream. When all data are read, set -:macro:`NGHTTP2_DATA_FLAG_EOF` flag to ``*data_flags`` to tell the -nghttp2 library that we have finished reading file. +library to send RST_STREAM to the stream. When all data has been read, set +the :macro:`NGHTTP2_DATA_FLAG_EOF` flag to ``*data_flags`` to tell the +nghttp2 library that we have finished reading the file. -The `nghttp2_submit_response()` is used to send response to the remote -peer. +The `nghttp2_submit_response()` function is used to send the response to the +remote peer. The ``on_stream_close_callback()`` function is invoked when the stream is about to close:: @@ -592,5 +582,5 @@ is about to close:: return 0; } -We destroy ``http2_stream_data`` object in this function since the -stream is about to close and we no longer use that object. +We destroy the ``http2_stream_data`` object in this function since the stream +is about to close and we no longer use that object.