Automation
Docker
Docker is the open container platform we run self-hosted services on: images, volumes and networks, described per stack in a single Compose file.
Docker is an open platform for developing, shipping and running applications. It packages an application into a loosely isolated environment called a container, and the container carries its own dependencies, so the host does not have to provide them. Docker uses a client server architecture: the client sends commands to the daemon, which manages images, containers, networks and volumes. The two communicate over a REST API, on a UNIX socket or over a network interface, and a client can also connect to a remote daemon.
A container is a runnable instance of an image, and an image is a read only template built from a Dockerfile. Because an image is made of layers, a rebuild only redoes the layers that changed. That is what lets a service behave the same way on a Windows workstation and on a Linux server.
Compose: one file, one stack
A service rarely consists of a single container. Docker Compose is the tool for defining and running multi-container applications: services, networks and volumes live in one YAML file, a single command starts the whole stack, and the same file works for development, testing and production. Lifecycle commands belong to it as well: start, stop and rebuild services, check status, stream logs, or run a one-off command inside a service.
- Every self-hosted service we run is its own Compose stack: n8n, PostgreSQL, ClickHouse, Qdrant, the nginx reverse proxy and the rest, each with its own compose.yaml.
- A stack is a directory with the Compose file and its environment variables, so backup and restore stay file level.
- Compose caches the configuration used to create a container, so an unchanged service is reused on restart.
- The Linux server runs the same layout on Docker Engine, while the Windows machine uses Docker Desktop.
Data, network, updates
The writable layer of a container disappears together with the container, so durable data belongs in a volume or a bind mount. Volumes are created and managed by Docker, survive the removal of the container and are easier to back up or migrate than bind mounts; a bind mount is the right choice when the files must also be reachable from the host. Mounting an empty volume over a directory that holds files copies them into the volume by default, while a non-empty volume obscures the existing content.
Every container gets networking: by default on the default bridge, and on a user defined network containers also resolve each other by name. Only published ports are visible outside the host, so a database port does not have to be exposed, and a container can join several networks at once.
An update follows the documented order: pull the new image, then rebuild and recreate the affected service. In production an override file carries the restart policy and the host ports.
What to watch
- By default a container has no resource constraints. On Linux the kernel can kill any process when memory runs out, so memory and CPU limits are worth setting.
- The memory limit and swap are read together, and disabling the OOM killer is only safe when a memory limit is set.
- Some limits need kernel support; docker info warns when a capability is missing.
- A volume is not removed on its own, and unused ones are cleaned up with docker volume prune.
- Backing up a volume means reading its content from a separate container, usually into a tar archive, and the restore path needs testing.
Further reading
At CyberElectro Docker is the base our own services run on: every stack is a Compose file kept in version control and can be brought up on a clean machine with one command.