fix: tornado HTTP 599 issue / gnutls_handshake() failed

After I moved a Tornado application into docker. I noticed lots of HTTP 599 errors were popping out during async http client's fetching method. And these errors only occurs on HTTPS requests.

The HTTP code 599 itself is not specified in any RFCs, but in Tornado, it's an error stands for SSL error.

Here's the unit test code I found in Tornado source code: https://github.com/tornadoweb/tornado/blob/master/tornado/test/httpserver_test.py#L87


Here's the server log with error message

[ERROR] 2015-01-15 18:07:40,933 url fetch error: xxx.xxxx.com, HTTP 599: gnutls_handshake() failed: Illegal parameter, 599
...

This is the code I used in my application, since the curl and pycurl are installed in advance, the code will use libcurl to fetching content underneath.

try:
    import pycurl
    logging.info("py curl version:%s" % pycurl.version)

    httpclient.AsyncHTTPClient.configure(
        "tornado.curl_httpclient.CurlAsyncHTTPClient", 
        max_clients=1000
        )
    logging.info("curl found, use curl client")
except:
    logging.warning("run pip install pycurl for better performance")

The first impression I got is, the target website has a invalid SSL certificate.
Anyway, I tried to avoid SSL certificate verification by adding parameter

validate_cert=False,

in http client's fetch method.

But the problem is still there, also, I noticed that I can get correct response on my dev machine where the application is not running in docker.
So, the most suspicious cause is the curl/pycurl library comes with the docker image.



And this is the pycurl version in the docker container.

python
>>> import pycurl
>>> pycurl.version_info()
(3, '7.35.0', 467712, 'x86_64-pc-linux-gnu', 50877, 'GnuTLS/2.12.23', 0, '1.2.8', ('dict', 'file', 'ftp', 'ftps', 'gopher', 'http', 'https', 'imap', 'imaps', 'ldap', 'ldaps', 'pop3', 'pop3s', 'rtmp', 'rtsp', 'smtp', 'smtps', 'telnet', 'tftp'), None, 0, '1.28')

On my dev machine, it's linked with OpenSSL instead of GnuTLS. So, I reinstalled pycurl with openSSL.

apt-get update
apt-get install -y curl libcurl3-openssl-dev
pip uninstall -y pycurl

export PYCURL_SSL_LIBRARY=openssl
pip install pycurl

Here's the information after I replace the SSL library used by pycurl

python
>>> import pycurl
>>> pycurl.version_info()
(3, '7.35.0', 467712, 'x86_64-pc-linux-gnu', 50877, 'OpenSSL/1.0.1f', 0, '1.2.8', ('dict', 'file', 'ftp', 'ftps', 'gopher', 'http', 'https', 'imap', 'imaps', 'ldap', 'ldaps', 'pop3', 'pop3s', 'rtmp', 'rtsp', 'smtp', 'smtps', 'telnet', 'tftp'), None, 0, '1.28')

Finally, the HTTP 599 error code gone in my Tornado application.
Happy 2015~