Deploying with Docker

This page details how to deploy a production-ready instance of this application using Docker.

Configuration

Read more about how this application is configured in the wiki. It is imperative that the backend is served via TLS (i.e. https://) to protect clear-text credentials used for authentication.

Deployment

A Dockerfile is placed at the root of this repository. An easy way to use this is via docker-compose. The following docker-compose script will launch an efficient and secure stack to use in production.

The stack includes

a chemcurator_django instance running the API
- exposed at port 8000
- 8 Gunicorn gevent based processes
a chemcurator_django instance running the admin interface
- exposed at port 8001
- 2 Gunicorn gevent based processes
a non-persistent Redis instance
- private to the Docker network
- used for cache
a pgbouncer instance
- private to the Docker network
- used to pool connections to a PostgreSQL server
- Each request/response cycle to Django will open/close a database connection. This overhead is significant and especially problematic in an API where many small requests are expected. A connection pool removes this overhead.

The following variables in the docker-compose script must be replaced or set via environment variables in order for the script to work:

${ADMIN_BRANCH}: the branch (or commit) of chemcurator_django you want to base the admin container on.
${ADMIN_DOMAIN}: the domain name the chemcurator_django admin instance will be reached at.
${ADMIN_SECRET_KEY}: a long secret to use for the chemcurator_django admin instance.
${API_BRANCH}: the branch (or commit) of chemcurator_django you want to base the API container on.
${API_DOMAIN}: the domain name the chemcurator_django API container will be reached at.
${API_SECRET_KEY}: a long secret to use for the chemcurator_django API instance.
${SPA_DOMAIN}: the domain name where a website will make use of the API.
${SQL_DATABASE}: the PostgreSQL database to use for the application.
${SQL_HOST}: the hostname where the PostgreSQL instance runs at.
${SQL_PASSWORD}: the password to login to the PostgreSQL instance.
${SQL_PORT}: the port of the PostgreSQL instance.
${SQL_USER}: the username to login to the PostgreSQL instance.

docker-compose.yaml

version: "3.1"

services:
    chemreg-api:
        build:
            context: https://github.com/Chemical-Curation/chemcurator_django.git#${API_BRANCH}
        image: chemreg-django
        restart: unless-stopped
        env_file:
          - .env
        environment:
            CACHE_URL: redis://redis:6379/0
            DATABASE_URL: postgres://pgbouncer:5432/${SQL_DATABASE}
            SECRET_KEY: ${API_SECRET_KEY}
            URL_CONF: api
            WEB_CONCURRENCY: 8
            WHITELIST_CORS: ${SPA_DOMAIN}
            WHITELIST_HOST: ${API_DOMAIN}
        ports:
            - 8000:8000
        depends_on:
          - pgbouncer

    chemreg-admin:
        build:
            context: https://github.com/Chemical-Curation/chemcurator_django.git#${ADMIN_BRANCH}
        image: chemreg-django
        restart: unless-stopped
        env_file:
          - .env
        environment:
            CACHE_URL: redis://redis:6379/1
            DATABASE_URL: postgres://pgbouncer:5432/${SQL_DATABASE}
            SECRET_KEY: ${ADMIN_SECRET_KEY}
            URL_CONF: admin
            WEB_CONCURRENCY: 2
            WHITELIST_HOST: ${ADMIN_DOMAIN}
        ports:
            - 8001:8000
        depends_on:
          - pgbouncer

    redis:
        image: redis:5-alpine
        restart: unless-stopped

    pgbouncer:
        image: pgbouncer/pgbouncer:1.12.0
        restart: unless-stopped
        environment:
            DATABASES_HOST: ${SQL_HOST}
            DATABASES_PASSWORD: ${SQL_PASSWORD}
            DATABASES_PORT: ${SQL_PORT}
            DATABASES_USER: ${SQL_USER}
            PGBOUNCER_LISTEN_PORT: 5432

Docker Cross Network Communications (Developer Setup)

With the introduction of resolver, we now have two networks of docker containers that are expected to run on different servers. Communication between docker containers on different docker networks on the same machine can be a little tricky. The docker network design intentionally prevents this from happening to help enforce encapsulation. However many developers will be working with a single machine, and without the help of an orchestration software like Kubernetes.

To get around this, containers on a different network will need to be referred to by their external container IP address. For instance, to find the resolver_web container's IP address

To find your `RESOLVER_URL` for docker deployments

Build your resolver docker-compose using the docker-compose commands. Verify it's working and publicly available by navigating to http://127.0.0.1:5000/swagger-ui where 5000 is your resolver_web port number.

docker-compose build
docker-compose up

List your docker bridge networks. This can be done by using the command docker network ls. Locate the network your container belongs to. In our case this will be resolver_default
Inspect the network. This can be done using the command docker network inspect resolver_default. From here we find the container's IPv4Address. This will be the IP address that containers on other networks will refer to this pod by.