Tutorial: HPACK API¶
In this tutorial, we describe basic use of HPACK API in nghttp2 library. We briefly describe APIs for deflating and inflating header fields. The example of using these APIs are presented as complete source code deflate.c.
Deflating (encoding) headers¶
First we need to initialize nghttp2_hd_deflater object using nghttp2_hd_deflate_new() function:
int nghttp2_hd_deflate_new(nghttp2_hd_deflater **deflater_ptr,
size_t deflate_hd_table_bufsize_max);
This function allocates nghttp2_hd_deflater object and initializes it and assigns its pointer to *deflater_ptr passed by parameter. The deflate_hd_table_bufsize_max is the upper bound of header table size the deflater will use. This will limit the memory usage in deflater object for dynamic header table. If you doubt, just specify 4096 here, which is the default upper bound of dynamic header table buffer size.
To encode header fields, nghttp2_hd_deflate_hd() function:
ssize_t nghttp2_hd_deflate_hd(nghttp2_hd_deflater *deflater,
uint8_t *buf, size_t buflen,
const nghttp2_nv *nva, size_t nvlen);
The deflater is the deflater object initialized by nghttp2_hd_deflate_new() function described above. The buf is a pointer to buffer to store encoded byte string. The buflen is capacity of buf. The nva is a pointer to nghttp2_nv, which is an array of header fields to deflate. The nvlen is the number of header fields which nva contains.
It is important to initialize and assign all members of nghttp2_nv. If a header field should not be inserted in dynamic header table for a security reason, set NGHTTP2_NV_FLAG_NO_INDEX flag in nghttp2_nv.flags.
nghttp2_hd_deflate_hd() processes all headers given in nva. The nva must include all request or response header fields to be sent in one HEADERS (or optionally following (multiple) CONTINUATION frame(s)). The buf must have enough space to store the encoded result. Otherwise, the function will fail. To estimate the upper bound of encoded result, use nghttp2_hd_deflate_bound() function:
size_t nghttp2_hd_deflate_bound(nghttp2_hd_deflater *deflater,
const nghttp2_nv *nva, size_t nvlen);
Pass this function with the same paramters deflater, nva and nvlen which will be passed to nghttp2_hd_deflate_hd().
The subsequent call of nghttp2_hd_deflate_hd() will use current encoder state and perform differential encoding which is the fundamental compression gain for HPACK.
Once nghttp2_hd_deflate_hd() fails, it cannot be undone and its further call with the same deflater object shall fail. So it is very important to use nghttp2_hd_deflate_bound() to know the required size of buffer.
To delete nghttp2_hd_deflater object, use nghttp2_hd_deflate_del() function.
Note
Generally, the order of header fields passed to nghttp2_hd_deflate_hd() function is not preserved. It is known that the relative ordering of header fields which do not share the same name is insignificant. But some header fields sharing the same name require the explicit ordering. To preserve this ordering, those header values are concatenated into single header field value using NULL (0x00) as delimiter. This is transparent to HPACK API. Therefore, the application should examine the inflated header values and split into multiple header field values by NULL.
Inflating (decoding) headers¶
We use nghttp2_hd_inflater object to inflate compressed header data. To initialize the object, use nghttp2_hd_inflate_new():
int nghttp2_hd_inflate_new(nghttp2_hd_inflater **inflater_ptr);
To inflate header data, use nghttp2_hd_inflate_hd() function:
ssize_t nghttp2_hd_inflate_hd(nghttp2_hd_inflater *inflater,
nghttp2_nv *nv_out, int *inflate_flags,
uint8_t *in, size_t inlen, int in_final);
The inflater is the inflater object initialized above. The nv_out is a pointer to nghttp2_nv to store the result. The in is a pointer to input data and inlen is its length. The caller is not required to specify whole deflated header data to in at once. It can call this function multiple times for portion of the data in streaming way. If in_final is nonzero, it tells the function that the passed data is the final sequence of deflated header data. The inflate_flags is output parameter and successful call of this function stores a set of flags in it. It will be described later.
This function returns when each header field is inflated. When this happens, the function sets NGHTTP2_HD_INFLATE_EMIT flag to inflate_flag parameter and header field is stored in nv_out. The return value indicates the number of data read from in to processed so far. It may be less than inlen. The caller should call the function repeatedly until all data are processed by adjusting in and inlen with the processed bytes.
If in_final is nonzero and all given data was processed, the function sets NGHTTP2_HD_INFLATE_FINAL flag to inflate_flag. If the caller sees this flag set, call nghttp2_hd_inflate_end_headers() function.
If in_final is zero and NGHTTP2_HD_INFLATE_EMIT flag is not set, it indicates that all given data was processed. The caller is required to pass subsequent data.
It is important to note that the function may produce one or more header fields even if inlen is 0 when in_final is nonzero, due to differential encoding.
The example use of nghttp2_hd_inflate_hd() is shown in inflate_header_block() function in deflate.c.
To delete nghttp2_hd_inflater object, use nghttp2_hd_inflate_del() function.
deflate.c¶
/*
* nghttp2 - HTTP/2 C Library
*
* Copyright (c) 2014 Tatsuhiro Tsujikawa
*
* Permission is hereby granted, free of charge, to any person obtaining
* a copy of this software and associated documentation files (the
* "Software"), to deal in the Software without restriction, including
* without limitation the rights to use, copy, modify, merge, publish,
* distribute, sublicense, and/or sell copies of the Software, and to
* permit persons to whom the Software is furnished to do so, subject to
* the following conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
* LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
* OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
#include <stdio.h>
#include <string.h>
#include <nghttp2/nghttp2.h>
#define MAKE_NV(K, V) \
{ (uint8_t*)K, (uint8_t*)V, sizeof(K) - 1, sizeof(V) - 1, \
NGHTTP2_NV_FLAG_NONE }
static void deflate(nghttp2_hd_deflater *deflater,
nghttp2_hd_inflater *inflater,
const nghttp2_nv * const nva, size_t nvlen);
static int inflate_header_block(nghttp2_hd_inflater *inflater,
uint8_t *in, size_t inlen, int final);
int main(int argc, char **argv)
{
int rv;
nghttp2_hd_deflater *deflater;
nghttp2_hd_inflater *inflater;
/* Define 1st header set. This is looks like a HTTP request. */
nghttp2_nv nva1[] = {
MAKE_NV(":scheme", "https"),
MAKE_NV(":authority", "example.org"),
MAKE_NV(":path", "/"),
MAKE_NV("user-agent", "libnghttp2"),
MAKE_NV("accept-encoding", "gzip, deflate")
};
/* Define 2nd header set */
nghttp2_nv nva2[] = {
MAKE_NV(":scheme", "https"),
MAKE_NV(":authority", "example.org"),
MAKE_NV(":path", "/stylesheet/style.css"),
MAKE_NV("user-agent", "libnghttp2"),
MAKE_NV("accept-encoding", "gzip, deflate"),
MAKE_NV("referer", "https://example.org")
};
rv = nghttp2_hd_deflate_new(&deflater, 4096);
if(rv != 0) {
fprintf(stderr, "nghttp2_hd_deflate_init failed with error: %s\n",
nghttp2_strerror(rv));
exit(EXIT_FAILURE);
}
rv = nghttp2_hd_inflate_new(&inflater);
if(rv != 0) {
fprintf(stderr, "nghttp2_hd_inflate_init failed with error: %s\n",
nghttp2_strerror(rv));
exit(EXIT_FAILURE);
}
/* Encode and decode 1st header set */
deflate(deflater, inflater, nva1, sizeof(nva1) / sizeof(nva1[0]));
/* Encode and decode 2nd header set, using differential encoding
using state after encoding 1st header set. */
deflate(deflater, inflater, nva2, sizeof(nva2) / sizeof(nva2[0]));
nghttp2_hd_inflate_del(inflater);
nghttp2_hd_deflate_del(deflater);
return 0;
}
static void deflate(nghttp2_hd_deflater *deflater,
nghttp2_hd_inflater *inflater,
const nghttp2_nv * const nva, size_t nvlen)
{
ssize_t rv;
uint8_t *buf;
size_t buflen;
size_t outlen;
size_t i;
size_t sum;
sum = 0;
for(i = 0; i < nvlen; ++i) {
sum += nva[i].namelen + nva[i].valuelen;
}
printf("Input (%zu byte(s)):\n\n", sum);
for(i = 0; i < nvlen; ++i) {
fwrite(nva[i].name, nva[i].namelen, 1, stdout);
printf(": ");
fwrite(nva[i].value, nva[i].valuelen, 1, stdout);
printf("\n");
}
buflen = nghttp2_hd_deflate_bound(deflater, nva, nvlen);
buf = malloc(buflen);
rv = nghttp2_hd_deflate_hd(deflater, buf, buflen, nva, nvlen);
if(rv < 0) {
fprintf(stderr, "nghttp2_hd_deflate_hd() failed with error: %s\n",
nghttp2_strerror((int)rv));
free(buf);
exit(EXIT_FAILURE);
}
outlen = rv;
printf("\nDeflate (%zu byte(s), ratio %.02f):\n\n",
outlen, sum == 0 ? 0 : (double)outlen / sum);
for(i = 0; i < outlen; ++i) {
if((i & 0x0fu) == 0) {
printf("%08zX: ", i);
}
printf("%02X ", buf[i]);
if(((i + 1) & 0x0fu) == 0) {
printf("\n");
}
}
printf("\n\nInflate:\n\n");
/* We pass 1 to final parameter, because buf contains whole deflated
header data. */
rv = inflate_header_block(inflater, buf, outlen, 1);
if(rv != 0) {
free(buf);
exit(EXIT_FAILURE);
}
printf("\n-----------------------------------------------------------"
"--------------------\n");
free(buf);
}
int inflate_header_block(nghttp2_hd_inflater *inflater,
uint8_t *in, size_t inlen, int final)
{
ssize_t rv;
for(;;) {
nghttp2_nv nv;
int inflate_flags = 0;
size_t proclen;
rv = nghttp2_hd_inflate_hd(inflater, &nv, &inflate_flags,
in, inlen, final);
if(rv < 0) {
fprintf(stderr, "inflate failed with error code %zd", rv);
return -1;
}
proclen = rv;
in += proclen;
inlen -= proclen;
if(inflate_flags & NGHTTP2_HD_INFLATE_EMIT) {
fwrite(nv.name, nv.namelen, 1, stderr);
fprintf(stderr, ": ");
fwrite(nv.value, nv.valuelen, 1, stderr);
fprintf(stderr, "\n");
}
if(inflate_flags & NGHTTP2_HD_INFLATE_FINAL) {
nghttp2_hd_inflate_end_headers(inflater);
break;
}
if((inflate_flags & NGHTTP2_HD_INFLATE_EMIT) == 0 &&
inlen == 0) {
break;
}
}
return 0;
}