walkero
/
nghttp2
1
0
Fork 0
Code Issues Pull Requests Projects Releases 1 Wiki Activity

python: Add experimental python extension module 

To build extension module, cython is required.
The module is not built with `make` in the top directory.
A user has to run `make build_ext` in python directory.
Currently header compression objects are available for testing.
This commit is contained in:
Tatsuhiro Tsujikawa 2013-12-14 23:50:30 +09:00
parent 65e54ca7b0
commit 4e0ca71ef0
7 changed files with 379 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Makefile.am
View File

@ -20,7 +20,7 @@
# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
SUBDIRS = lib src examples hdtest tests doc SUBDIRS = lib src examples hdtest python tests doc
ACLOCAL_AMFLAGS = -I m4 ACLOCAL_AMFLAGS = -I m4

1
configure.ac
View File

@ -324,6 +324,7 @@ AC_CONFIG_FILES([
src/Makefile src/Makefile
examples/Makefile examples/Makefile
hdtest/Makefile hdtest/Makefile
python/Makefile
doc/Makefile doc/Makefile
doc/conf.py doc/conf.py
]) ])

34
python/Makefile.am Normal file
View File

@ -0,0 +1,34 @@
# nghttp2 - HTTP/2.0 C Library
# Copyright (c) 2013 Tatsuhiro Tsujikawa
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
PYSETUP_INCLUDE_DIRS=$(top_srcdir)/lib/includes:$(top_srcdir)/lib
PYSETUP_LIBDIRS=$(top_builddir)/lib/.libs
nghttp2.c: nghttp2.pyx cnghttp2.pxd
cython nghttp2.pyx
.PHONY: build_ext
build_ext: nghttp2.c
python setup.py build_ext --include-dirs=$(PYSETUP_INCLUDE_DIRS) \
--library-dirs=$(PYSETUP_LIBDIRS)

39
python/README.rst Normal file
View File

@ -0,0 +1,39 @@
nghttp2 Python C extension module
=================================
This directory contains nghttp2 Python C extension module. Currently,
header compressor and decompressor are implemented in extension using
cython.
This is experimental and adds some dependencies which is a bit hard to
check, so this extension module does not built with usual ``make
install`` in the top directory. Instead, a user has to run ``make
build_ext`` in this directory.
The build extension module is called ``nghttp2``.
The module refers to the libnghttp2.so. If nghttp2 is installed using
``make install``, then importing nghttp2 module should work. If a
user does not want to install nghttp2, then use ``LD_LIBRARY_PATH``
pointing to the location of libnghttp2.so, which is usually in
``lib/.libs``. If a user also does not want to install nghttp2 module,
use PYTHONPATH to point the location of extension module. This depends
on the architecture and Python version. For example, x86_64
architecture and Python 2.7 series, a module will be located at
``build/lib.linux-x86_64-2.7``.
Header compression
------------------
The following example code illustrates basic usage of compressor and
decompressor::
import binascii
import nghttp2
deflater = nghttp2.HDDeflater(nghttp2.HD_SIDE_REQUEST)
inflater = nghttp2.HDInflater(nghttp2.HD_SIDE_REQUEST)
data = deflater.deflate([(b'foo', b'bar'),
(b'baz', b'buz')])
print(binascii.b2a_hex(data))

79
python/cnghttp2.pxd Normal file
View File

@ -0,0 +1,79 @@
# nghttp2 - HTTP/2.0 C Library
# Copyright (c) 2013 Tatsuhiro Tsujikawa
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
from libc.stdint cimport uint8_t, uint16_t, uint32_t, int32_t
cdef extern from 'nghttp2/nghttp2.h':
ctypedef struct nghttp2_nv:
uint8_t *name
uint8_t *value
uint16_t namelen
uint16_t valuelen
const char* nghttp2_strerror(int lib_error_code)
cdef extern from 'nghttp2_helper.h':
void nghttp2_free(void *ptr)
cdef extern from 'nghttp2_frame.h':
void nghttp2_nv_array_del(nghttp2_nv *nva)
cdef extern from 'nghttp2_hd.h':
ctypedef enum nghttp2_hd_side:
NGHTTP2_HD_SIDE_REQUEST
NGHTTP2_HD_SIDE_RESPONSE
ctypedef struct nghttp2_hd_context:
pass
int nghttp2_hd_deflate_init2(nghttp2_hd_context *deflater,
nghttp2_hd_side side,
size_t deflate_hd_table_bufsize_max)
int nghttp2_hd_inflate_init(nghttp2_hd_context *inflater,
nghttp2_hd_side side)
void nghttp2_hd_deflate_free(nghttp2_hd_context *deflater)
void nghttp2_hd_inflate_free(nghttp2_hd_context *inflater)
void nghttp2_hd_deflate_set_no_refset(nghttp2_hd_context *deflater,
uint8_t no_refset)
int nghttp2_hd_change_table_size(nghttp2_hd_context *context,
size_t hd_table_bufsize_max)
ssize_t nghttp2_hd_deflate_hd(nghttp2_hd_context *deflater,
uint8_t **buf_ptr, size_t *buflen_ptr,
size_t nv_offset,
nghttp2_nv *nva, size_t nvlen)
ssize_t nghttp2_hd_inflate_hd(nghttp2_hd_context *inflater,
nghttp2_nv **nva_ptr,
uint8_t *input, size_t inlen)
int nghttp2_hd_end_headers(nghttp2_hd_context *deflater_or_inflater)

188
python/nghttp2.pyx Normal file
View File

@ -0,0 +1,188 @@
# nghttp2 - HTTP/2.0 C Library
# Copyright (c) 2013 Tatsuhiro Tsujikawa
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
cimport cnghttp2
from libc.stdlib cimport malloc, free
from libc.string cimport memcpy, memset
from libc.stdint cimport uint8_t, uint16_t, uint32_t, int32_t
HD_SIDE_REQUEST = cnghttp2.NGHTTP2_HD_SIDE_REQUEST
HD_SIDE_RESPONSE = cnghttp2.NGHTTP2_HD_SIDE_RESPONSE
HD_DEFLATE_HD_TABLE_BUFSIZE_MAX = 4096
cdef class HDDeflater:
'''Performs header compression. The header compression algorithm has
to know the header set to be compressed is request headers or
response headers. It is indicated by |side| parameter in the
constructor. The constructor also takes |hd_table_bufsize_max|
parameter, which limits the usage of header table in the given
amount of bytes. This is necessary because the header compressor
and decompressor has to share the same amount of header table and
the decompressor decides that number. The compressor may not want
to use all header table size because of limited memory
availability. In that case, the |hd_table_bufsize_max| can be used
to cap the upper limit of talbe size whatever the header table
size is chosen. The default value of |hd_table_bufsize_max| is
4096 bytes.
The following example shows how to compress request header sets:
import binascii, nghttp2
deflater = nghttp2.HDDeflater(nghttp2.HD_SIDE_REQUEST)
res = deflater.deflate([(b'foo', b'bar'),
(b'baz', b'buz')])
print(binascii.b2a_hex(res))
'''
cdef cnghttp2.nghttp2_hd_context _deflater
def __cinit__(self, side,
hd_table_bufsize_max = HD_DEFLATE_HD_TABLE_BUFSIZE_MAX):
rv = cnghttp2.nghttp2_hd_deflate_init2(&self._deflater, side,
hd_table_bufsize_max)
if rv != 0:
raise Exception(_strerror(rv))
def __init__(self, side,
hd_table_bufsize_max = HD_DEFLATE_HD_TABLE_BUFSIZE_MAX):
pass
def __dealloc__(self):
cnghttp2.nghttp2_hd_deflate_free(&self._deflater)
def deflate(self, headers):
'''Compresses the |headers|. The |headers| must be sequence of tuple
of name/value pair, which are sequence of bytes (not unicode
string).
This function returns the encoded header block in byte string.
An exception will be raised on error.
'''
cdef cnghttp2.nghttp2_nv *nva = <cnghttp2.nghttp2_nv*>\
malloc(sizeof(cnghttp2.nghttp2_nv)*\
len(headers))
cdef cnghttp2.nghttp2_nv *nvap = nva
for k, v in headers:
nvap[0].name = k
nvap[0].namelen = len(k)
nvap[0].value = v
nvap[0].valuelen = len(v)
nvap += 1
cdef uint8_t *out = NULL
cdef size_t outcap = 0
cdef ssize_t rv
rv = cnghttp2.nghttp2_hd_deflate_hd(&self._deflater, &out, &outcap,
0, nva, len(headers))
free(nva)
if rv < 0:
raise Exception(_strerror(rv))
cdef bytes res
try:
res = out[:rv]
finally:
cnghttp2.nghttp2_free(out)
return res
def change_table_size(self, hd_table_bufsize_max):
'''Changes header table size to |hd_table_bufsize_max| byte.
An exception will be raised on error.
'''
_hd_change_table_size(&self._deflater, hd_table_bufsize_max)
def set_no_refset(self, no_refset):
'''Tells the compressor not to use reference set if |no_refset| is
nonzero. If |no_refset| is nonzero, on each invocation of
deflate(), compressor first emits index=0 to clear up
reference set.
'''
cnghttp2.nghttp2_hd_deflate_set_no_refset(&self._deflater, no_refset)
cdef class HDInflater:
'''Performs header decompression.
The following example shows how to compress request header sets:
data = b'0082c5ad82bd0f000362617a0362757a'
inflater = nghttp2.HDInflater(nghttp2.HD_SIDE_REQUEST)
hdrs = inflater.inflate(data)
print(hdrs)
'''
cdef cnghttp2.nghttp2_hd_context _inflater
def __cinit__(self, side):
rv = cnghttp2.nghttp2_hd_inflate_init(&self._inflater, side)
if rv != 0:
raise Exception(_strerror(rv))
def __init__(self, side):
pass
def __dealloc__(self):
cnghttp2.nghttp2_hd_inflate_free(&self._inflater)
def inflate(self, data):
'''Decompresses the compressed header block |data|. The |data| must be
byte string (not unicode string).
'''
cdef cnghttp2.nghttp2_nv *nva
cdef ssize_t rv
rv = cnghttp2.nghttp2_hd_inflate_hd(&self._inflater, &nva,
data, len(data))
if rv < 0:
raise Exception(_strerror(rv))
try:
res = [(nva[i].name[:nva[i].namelen],
nva[i].value[:nva[i].valuelen]) for i in range(rv)]
finally:
cnghttp2.nghttp2_nv_array_del(nva)
cnghttp2.nghttp2_hd_end_headers(&self._inflater)
return res
def change_table_size(self, hd_table_bufsize_max):
'''Changes header table size to |hd_table_bufsize_max| byte.
An exception will be raised on error.
'''
_hd_change_table_size(&self._inflater, hd_table_bufsize_max)
cdef _hd_change_table_size(cnghttp2.nghttp2_hd_context *context,
size_t hd_table_bufsize_max):
cdef int rv
rv = cnghttp2.nghttp2_hd_change_table_size(context, hd_table_bufsize_max)
if rv != 0:
raise Exception(_strerror(rv))
cdef _strerror(int liberror_code):
return cnghttp2.nghttp2_strerror(liberror_code).decode('utf-8')

37
python/setup.py Normal file
View File

@ -0,0 +1,37 @@
# nghttp2 - HTTP/2.0 C Library
# Copyright (c) 2013 Tatsuhiro Tsujikawa
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
from distutils.core import setup
from distutils.extension import Extension
setup(
name = 'python-nghttp2',
description = 'Python HTTP/2.0 library on top of nghttp2',
author = 'Tatsuhiro Tsujikawa',
author_email = 'tatsuhiro.t@gmail.com',
url = 'http://tatsuhiro-t.github.io/nghttp2/',
keywords = [],
ext_modules = [Extension("nghttp2",
["nghttp2.c"],
libraries=['nghttp2'])],
long_description='TBD'
)