Thursday, January 15, 2015

fix: tornado HTTP 599 issue / gnutls_handshake() failed

After I moved a Tornado application into docker. I noticed lots of HTTP 599 errors were popping out during async http client's fetching method. And these errors only occurs on HTTPS requests.

The HTTP code 509 itself is not specified in any RFCs, but in Tornado, it's an error stands for SSL error.

Here's the unit test code I found in Tornado source code:

Here's the server log with error message
[ERROR] 2015-01-15 18:07:40,933 url fetch error:, HTTP 599: gnutls_handshake() failed: Illegal parameter, 599

This is the code I used in my application, since the curl and pycurl are installed in advance, the code will use libcurl to fetching content underneath.
    import pycurl"py curl version:%s" % pycurl.version)

        )"curl found, use curl client")
    logging.warning("run pip install pycurl for better performance")

The first impression I got is, the target website has a invalid SSL certificate.
Anyway, I tried to avoid SSL certificate verification by adding parameter
in http client's fetch method.

But the problem is still there, also, I noticed that I can get correct response on my dev machine where the application is not running in docker.
So, the most suspicious cause is the curl/pycurl library comes with the docker image.

And this is the pycurl version in the docker container.
>>> import pycurl
>>> pycurl.version_info()
(3, '7.35.0', 467712, 'x86_64-pc-linux-gnu', 50877, 'GnuTLS/2.12.23', 0, '1.2.8', ('dict', 'file', 'ftp', 'ftps', 'gopher', 'http', 'https', 'imap', 'imaps', 'ldap', 'ldaps', 'pop3', 'pop3s', 'rtmp', 'rtsp', 'smtp', 'smtps', 'telnet', 'tftp'), None, 0, '1.28')
On my dev machine, it's linked with OpenSSL instead of GnuTLS. So, I reinstalled pycurl with openSSL.
apt-get update
apt-get install -y curl libcurl3-openssl-dev
pip uninstall -y pycurl

export PYCURL_SSL_LIBRARY=openssl
pip install pycurl

Here's the information after I replace the SSL library used by pycurl
>>> import pycurl
>>> pycurl.version_info()
(3, '7.35.0', 467712, 'x86_64-pc-linux-gnu', 50877, 'OpenSSL/1.0.1f', 0, '1.2.8', ('dict', 'file', 'ftp', 'ftps', 'gopher', 'http', 'https', 'imap', 'imaps', 'ldap', 'ldaps', 'pop3', 'pop3s', 'rtmp', 'rtsp', 'smtp', 'smtps', 'telnet', 'tftp'), None, 0, '1.28')

Finally, the HTTP 599 error code gone in my Tornado application.
Happy 2015~

Saturday, November 29, 2014

Error occurred while loading visualforce page.错误:加载 Visualforce 页面时出现错误

While using the visa interview appointment system (,
keep running into "Error occurred while loading visualforce page." after the time window selection page.
And the error page shows up no matter what OS/browser you use.

总是在选择完时间的页面之后遇到"错误:加载 Visualforce 页面时出现错误"的报错页面。

Someone added you as their dependent(family member who're going to take the interview together) in their unfinished appointment.

- Delete you from that(theirs) unfinished appointment draft at step 6.
- Come back to your appointment, you may find many fields in your form are reseted.
- Finish your appointment, you will see appointment confirmation and appointment invoice after the time window selection page.

Samsung S5 can not get new Gmail or Hangout message while using mobile data

Problem description: In Mobile network(3G/4G data), you cannot receive "server pushed message", applications such as Gmail, Google hangouts whose who heavily relay on this "server push" technology(Google Cloud Message) can not work properly, no new message can be received.
- but, you can manually pull email by "refresh" button in Gmail app.
- after switch to Wi-Fi, no no problem at all.

Fix: Settings -> Data usage -> upper right corner(a 3-dotted button) ->
- make sure the "restrict background data" is unchecked,
- if it's already unchecked when you seen it, check it, then uncheck it. (UI and underneath setting may mismatched)

Wednesday, November 19, 2014

private docker registry push issue: Invalid registry endpoint

I ran into the problem during pushing an image to my private docker registry. It's the first time pushing after I upgrade my docker client to v1.3.1

2014/11/19 15:55:06 Error: Invalid registry endpoint Get dial tcp i/o timeout. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/

Since I'm running docker via boot2docker, docker instance actually is managed by boot2docker. I have to add parameter --insecure-registry to somewhere in the boot2docker virtual machine.

Here's the official solution: (

Insecure Registry
As of Docker version 1.3.1, if your registry doesn't support HTTPS, you must add it as an insecure registry.
$ boot2docker init
$ boot2docker up
$ boot2docker ssh
$ echo 'EXTRA_ARGS="--insecure-registry <YOUR INSECURE HOST>"' | sudo tee -a /var/lib/boot2docker/profile
$ sudo /etc/init.d/docker restart

--- update 1: ---

To add HTTPS support for your private docker hub,
1, install nginx
apt-get install nginx

rm  /etc/nginx/sites-enabled/default 

3, add nginx profile `docker` under /etc/nginx/sites-enabled/
suppose your docker hub application listening on 80
server {
    listen 443 ssl;
    ssl on;
    ssl_certificate YOURCERT.crt;
    ssl_certificate_key YOUR_PRIVATE_KEY.pem;

    client_max_body_size 0;
    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Scheme $scheme;
        proxy_pass_header Server;
        proxy_set_header X-Forwarded-Proto https;
        proxy_redirect http:// https://;
make sure you added
client_max_body_size 0;
, this setting will allow you to upload images(POST data) >1M.

4, restart nginx
service restart nginx

5, try to browse your docker hub by both HTTP and HTTPS. No error or warning should be seen.

--- update 2: still see certificate warning ---
you're probably using intermediate CA, you need to put root CA and intermediate CA together(concat them) in the .crt file.


Wednesday, November 5, 2014

USCIS case monitoring app

I created an USCIS case tracking application on Google App Engine recently.

For those who are waiting for their USCIS case decisions, whatever it's a visa case or an immigration case, the application can track it as well.

Due to the resource quota on Google App Engine, the tracking app refreshes your case status every 6 hours and each user is able to track up to 2 cases.
Once your case got an update, you will get an Email notification.

You will be asked to login with Google account, no private data will be stored except your email address which is used for notification purpose.

Wish all your USCIS cases get approved ASAP

Tuesday, April 8, 2014

patch for openssl heartbleed bug on Ubuntu

Install update

apt-get update
apt-get install openssl libssl1.0.0

Check libssl version

root@hydrausdev:~# dpkg -l|grep libssl
ii libssl-dev 1.0.1-4ubuntu5.12 SSL development libraries, header files and documentation
ii libssl-doc 1.0.1-4ubuntu5.11 SSL development documentation documentation
ii libssl1.0.0 1.0.1-4ubuntu5.12 SSL shared libraries

Restart Nginx/Apache


by online tool: