Friday, May 27, 2016

add insecure-registry flag in docker for Mac client


  1. get current setting by command pinata get daemon > daemon.json.
  2. add key value pair insecure-registry:<ip/domain:port> or inecure-registries:[<ip/domain:prot>,<ip/domain:prot>]
  3. write new setting back pinata set daemon @daemon.json
  4. restart the client pinata restart

Tuesday, April 26, 2016

Save large amount of data in Redis

  1. make sure the Redis work directory is in a partition own free space larger than memory

  2. to avoid BGSAVE fail with a fork() error even with a lot of availbe memory

    • run echo 1 > /proc/sys/vm/overcommit_memory as temporary solution.
    • add vm.overcommit_memory=1 to /etc/sysctl.conf and reboot as a long term solution
  3. keep writing when BGSAVE error
    in /etc/redis.conf, add or modify stop-writes-on-bgsave-error no

  4. disable Transparent Huge Pages(THP)
    This will create latency and memory usage issue with redis.
    Run the following command, and add it to /etc/rc.local to retain the setting after reboot

    sudo echo never > /sys/kernel/mm/transparent_hugepage/enabled
  5. enlarge TCP backlog setting
    Run the following command, and add it to /etc/rc.local to retain the setting after reboot

    sysctl -w net.core.somaxconn=65535

Monday, April 25, 2016

Run multiple Redis instances on CentOS7

Install 1st Redis instance

By installing the first Redis instance, you can use the configuration as template for your 2nd one.

yum install redis
service redis start

This redis instance listens on localhost:6379 by default.

Install 2nd Redis instance

create a new working directory

The default redis instance uses /var/lib/redis as its working directory, dumped memory content is saved under this directory with name dump.rdb if you did not change it manually.

to avoid runtime conflict, we need to create a new working directory

mkdir -p /var/lib/redis2/
chown redis /var/lib/redis2/
chgrp redis /var/lib/redis2/

generate configurations

Create a new configuration file by copying /etc/redis.conf

cp /etc/redis.conf /etc/redis2.conf

Edit following settings to avoid conflicts

logfile "/var/log/redis/redis2.log"
dir "/var/lib/redis2"
pidfile "/var/run/redis/"
port 6380

add startup script

Create service file

cp /usr/lib/systemd/system/redis.service /usr/lib/systemd/system/redis2.service

Modify the settings under Service section

ExecStart=/usr/bin/redis-server /etc/redis2.conf --daemonize no
ExecStop=/usr/bin/redis-shutdown redis2

set to start with boot

systemctl enable redis2

start 2nd redis

service redis2 start

check status

lsof -i:6379
lsof -i:6380

Thursday, October 29, 2015

Install docker 1.8 on CentOS 7

Remember to install docker-selinux manually due to the issue(?)
yum install docker-selinux
Docker 1.9 has fixed this issue

The full list of commands
sudo yum update
curl -sSL | sh
service docker start

chkconfig docker on

Friday, September 18, 2015

Start with No?

Here's one chapter in the book "Getting Real" (37 signal)
full book:

Start With No


Don't be a yes-man

Make each feature work hard to be implemented. Make each feature prove itself and show that it's a survivor. It's like "Fight Club." You should only consider features if they're willing to stand on the porch for three days waiting to be let in.

That's why you start with no. Every new feature request that comes to us — or from us — meets a no. We listen but don't act. The initial response is "not now." If a request for a feature keeps coming back, that's when we know it's time to take a deeper look. Then, and only then, do we start considering the feature for real.

And what do you say to people who complain when you won't adopt their feature idea? Remind them why they like the app in the first place. "You like it because we say no. You like it because it doesn't do 100 other things. You like it because it doesn't try to please everyone all the time."

Don't say so in an interview, interviewer may not understand the point without sufficient context >.<
They may misunderstand you to be a lazy, always-no man.

I've burned the book....

USCIS case tracking service upgraded


Now, you can get notification if adjacent case status changed.

Thursday, March 12, 2015

dj-sso-server and dj-sso-client: Django applications provide SSO feature



dj-sso-server is a Django application that provides Single Sign-on feature for your project.

The dj-sso-server application works as a SSO provider , you can use dj-sso-client ( as the SSO client in other projects need SSO.


Install by command pip install dj-sso-server

The dependent package dj-api-auth ( will be installed automatically.

How it works

  • Based on the dj-api-auth module, we can create an API key with SSO related APIs initially included. All the API communications between dj-sso-server and dj-sso-client are protected by dj-api-auth

  • The API key will also be bind with a host which is used to limit the origin of SSO requests.

  • SSO work flow with dj-sso-client

    1. Firstly, dj-sso-client applies a request key via API reqeusttoken/ on dj-sso-server

    2. The request key in dj-sso-server side will be kept in cache for 5 minutes, so the whole SSO login process should be done in 5 minutes.

    3. With the request key, dj-sso-client redirects user to SSO login page on SSO provider, and get auth token if login success. dj-sso-server will

      • verify the request origin
      • verify request key validity (expired?)
      • save user information in cache
    4. dj-sso-client verifies the auth token with dj-sso-server via API authtoken/, and get a SSOUser object.

    5. dj-sso-server delete the request key from cache once the authtoken/ is called.

  • If there's an already logged-in account on dj-sso-server (say, the project where SSO provider is placed also provides other features, and there's a valid cookies in browser side and valid session on server side), user can select to continue with that logged account.

  • SSO login through dj-sso-server with not affect the login status on dj-sso-server.


Since request keys are stored in cache waiting for verification or expiration. If you have multiple application process running in your deployment (gunicorn etc.), please use proper cache system that can be shared between processes.

Memcached and Redis are both great for caching, be aware, the Local-memory caching (django.core.cache.backends.locmem.LocMemCache) is a toy for local debugging.

Add dj-sso-server to project

  1. Add djapiauth and djssoserver to INSTALLED_APPS in
  2. Assign an URL to the module
# add auth for a browser-oriented view
url(r'^sso/', include("djssoserver.urls"))


    • optional, a path to function receives an user object and return a json string.

    • the default SSO_SERVER_USER_TO_JSON_FUNC function is djssoserver.utility.default_user_to_json

      def default_user_to_json(user):
          return json.dumps(model_to_dict(user, exclude=["password", "user_permissions"]),

Scan API

In order to discover and manage APIs, after dj-sso-server is added in an accessible, run command python reloadentrypoints to collect APIs to database.

Create API key for SSO

  1. From your admin site, create an API key at Single sign-on/SSO credential. All SSO related APIs will assigned to this API Key automatically.
  2. After the API key for SSO is ready, you can assign more APIs for this API key at API Auth/Credential from admin site

Customize SSO login page

You can add styles to your own SSO login page. simply create djsso/ssologin.html under the templates folder. Revamp it by imitating the
original page

SSOUser object

dj-sso-client gets a SSOUser object whatever the User model is used in SSO provider project.

See detail in README file of dj-sso-client (


We have a SSO provider application running on Heroku (

Source code: under example folder

To try the demo out, check out the README file of dj-sso-client (


dj-sso-client is the a Django application works as SSO client side of dj-sso-server (


pip install dj-sso-client

Add to your project

  • Modify following settings in of your project

    • AUTHENTICATION_BACKENDS, add djssoclient.authbackend.SSOAuthBackend as the backends
    • AUTH_USER_MODEL, set djssoclient.SSOUser as user model
AUTHENTICATION_BACKENDS = ('djssoclient.authbackend.SSOAuthBackend',)
AUTH_USER_MODEL = 'djssoclient.SSOUser'
  • Add following dj-sso-client settings base on your demand

    • SSO_API_AUTH_SETTING: set API key, SEC key and remote SSO provider URL. This setting is used by underneath dj-api-auth module to proejct API accessing.

          "apikey": "f4a05287",
          "seckey": "6a4eeaea54d54f51af703e79c6096d51",
          "url": "",
    • SSO_REMOTE_URL_PREFIX (optional): SSO path in remote SSO provider. default /sso/

    • SSO_USER_STORAGE``(optional):  SSOUser storage solution, there are 2 storage backends in ``dj-sso-client already. default: SSOUserDBStorage

      • djssoclient.userstorage.SSOUserDBStorage: store user data in database
      • djssoclient.userstorage.SSOUserCacheStorage: store user data in cache. You will get better performance.
    • SSO_SETTING_CACHE (optional): if you selected SSOUserCacheStorage as your user storage backend, and you have more than one cache in, you can pick up the cache name here. default: default

Attention: SSOUserCacheStorage

The default django.core.cache.backends.locmem.LocMemCache stores data per process. In multi-process production environment (gunicorn on multi-core server), it may cause problem while using SSOUserCacheStorage as your user storage engine.

Please use dedicate cache system (Memcached or Redis) as cache backend to avoid this problem.


SSOUser is the user model to store user data. It can be used as database model class if you selected SSOUserDBStorage to be your user storage engine.

class SSOUser(AbstractBaseUser):
    username = models.CharField(unique=True, max_length=50)
    extras = models.TextField(default="{}")

extra user attributes : attributes not exists in the SSOUser class. (attributes except username, password, last_login etc.)

All extra user attributes can be access by getattr method or . operator. And they are stored in class member extras in JSON format.

Get your hands dirty

We already have a SSO provider (dj-sso-server) application running on Heroku: . Run the example application in folder example/ssoclient/ locally.

The API key using in the example application is binding with localhost:8000, so make sure you're accessing local application by localhost:8000 rather than the

fresh login

  1. make sure you're not logged in on, you should see please log in with ...
  2. click sso login on local application, you will be redirected to
  3. you will see user information after be redirected to local application.
  4. on , your status is still not logged-in

login with existing logged account

  1. log in with user name user1 and password 123 on
  2. click sso login on local application, you will be redirected to, but you will see a different login page with fresh login.
  3. select continue with current user? user1
  4. you will be logged in as user1 at local application

switch account

  1. if you select switch account and login with user2/456 from step 3 in previous sample
  2. you will be logged in as user2 at local application
  3. your login status on will NOT be changed (still logged in as user1)


  • example: work as an extra auth method