Published on

My Docker-Compose Setup

Authors

My Docker-compose Setup

I have been using docker-compose quite a bit lately. This post details the setup I end up using.

Dockerfile and Project Setup

Typically my projects will have a Dockerfile in the root of the project. If it is a monorepo then each subrepo will have the Dockerfile in the root of that project for example:

<rootOfTheMonorepo>/
                     --> .env.local
                     --> .env.example
                     --> makefile
                     --> ...
                     --> infrastructure
                                        --> compose-1.yaml
                                        --> compose-2.yaml
                     --> subProject-1
                                      --> Dockerfile
                                      --> ...
                     --> subProject-2
                                      --> Dockerfile
                                      --> ...

This structure allows me to have multiple compose's for different situations. It uses a makefile or any other cross tech build file to unify the projects. The scripts folder is used to hold any helper build scripts used by the makefile.

Docker-compose

My compose files are fairly standard. An example of one is:

version: '3.7'

services:
  service-1:
    image: some-org/service-1:0.1
    build:
      context: ./service-1
    container_name: some-org-service-1
    depends_on:
      - 'queue-1'

  service-2:
    image: another/image
    container_name: some-org-service-2
    restart: unless-stopped
    ports:
      - '8080:8080'
    volumes:
      - '${DATA_VOLUME_1:?NO_DATA_1_VOLUME_PROVIDED}:/data/'
      - '${COMMON_VOLUME:?NO_COMMON_VOLUME}:/common/'
    command: ['--config-file=/common/config.yaml']

  queue:
    image: ibmcom/mq
    restart: unless-stopped
    container_name: swiftir-ibm-mq
    ports:
      - '1414:1414'
      - '9443:9443'
    volumes:
      - 'qm1data:/mnt/mqm'
    environment:
      - LICENSE=accept
      - MQ_QMGR_NAME=QM1

volumes:
  qm1data:

Environmental

Certain things in compose files can be environment-specific. To cater for this I use a .env.local in the root of the project which each dev has to configure for their environment and is not version controlled. An .env.example is version controlled. Typically when setting this project up the dev will copy the .env.example as .env.local and configure the local version accordingly.

My .env.local looks something like the below:

# this is specific to the dev's machine
# this is where docker-compose will mount the host volumes in the file above
# this is needed as ~ cannot be used, compose does not seem to expand this
DATA_VOLUME_1=/path/to/some/local/dir/
COMMON_VOLUME=/path/to/some/other/local/dir/

Bringing it All Together

From the root of the monorepo, I use a makefile to call the following bash command to startup a specific compose file:

#!/usr/bin/env bash
set -ex

echo "--------------------------------"
echo "WORKING DIRECTORY: $(PWD)"
echo "--------------------------------"

env $(cat .env.local | grep "#" -v) docker-compose --project-directory . -f ./infrastructure/compose-1.yaml --compatibility up --build -d

In the above there are a few important things to note:

  • This command is housed in a script under scripts/ and called by the makefile using: ./scripts/startScriptScenariox.sh
    • We need to do it this way as env $(cat .env.local | grep "#" -v) does not work properly from a makefile target directly.
    • The cat portion of the script is due to a limitation in compose where you can not override the default .env file name.
    • Externalizing build steps to bash scripts allows us to have the script be as long as we need and it is also easier to debug
  • --project-directory .: The script is run from the root of the monorepo. This flag tells docker-compose that it must use the root of the mono repo when referring to any relative paths in the compose file under build:
    • In the compose file when we refer to context: under the build: tag it is relative to the --project-directory
    • If --project-directory is /my/path then when context: in a service is ./foo the path compose will use is: /my/path/foo.
    • This is where compose will look for the Dockerfile for service-1 here
    • We can specify a custom Dockerfile location too if it is not the same as the context. This is also relative to the --project-directory
  • --build: As described in this issue this flag is needed to get compose to build images as configured with the build: tag. This behaves the same as running docker build . -t some-image-name:0.1. It will cache layers and only rebuild layers that change.
  • --compatibility: This allows us to use compose 3.x wihtout being forced to use swarm for some things. For example if we want to constrain resources of a container this was doable in 2 without swarm but not 3.
    • To limit resources do so under the deploy section (with the compatibility flag this will be converted to the compose 2 equivalent).
    • docker stats should reflect your resource constraints if all went well. docker stats containerName can show you just the container you're interested in.

The associated stop script can be found below:

env $(cat .env.local | grep "#" -v) docker-compose --project-directory . -f ./infrastructure/compose-1.yaml down || true
docker volume prune -f || true
docker network prune -f || true