2026/07/06

Securing services with Rootless Containers

#devops #security #podman #systemd #software-architecture

Introduction

There is a common belief among many backend developers stating that Docker/OCI containers are a good security layer for their deployed services and applications. However, this is not the case.

While it is true that their underlying mechanisms (cgroups for resource limiting, and Linux namespaces for isolation on how resources are viewed/accessed) can be used as building blocks to implement some security mechanisms, these are not enough to offer an in-depth protection against many types of attack.

There are some relatively good guides out there that one can follow to harden their Docker deployments, such as the OWASP’s Docker Security Cheat Sheet.

In this post I don’t intend to replicate these other articles, but to explain in sufficient detail one of the many security measures that ideally every containerized application should apply: running in rootless containers.

I prefer using Podman over Docker, so that’s what I’ll use for the rest of the article.

A Very Basic Threat Model

When we run a container with the “classic” Docker daemon (or rootful Podman), the container runtime runs as root. The processes inside our container might be configured to run as an unprivileged user, but the boundary around them is enforced by a privileged daemon.

If an attacker breaks out of the container, through a kernel bug, a runtime bug, a careless bind mount, or a misconfigured capability, they can land on the host as root. This is not hypothetical, we have well known examples, such as CVE-2025-9074.

If the same happened with a rootless container, the attacker would be able to act in the host system as well, but with an unprivileged user, limiting the scope and impact of their “expedition”.

How does `rootless` work?

Behind rootless containers there are 2 main pieces of Linux machinery:

The first mechanism is user namespaces. Unprivileged users are granted a range of subordinate UIDs and GIDs, which are listed in /etc/subuid and /etc/subgid. For example, if I run the command cat /etc/subuid in my laptop, I get:

andreu:100000:65536
jocs:165536:65536
sandbox:231072:6553

We can read the first line as: the user andreu has assigned a range of subordinate UIDs starting at the number 100000 (inclusive), and spanning up to 65536 different UIDs.

Rootless podman uses them to build a mapping: uid 0 inside the container maps to our real unprivilieged host user (in this case it would be andreu), and every other container UID maps to some other UIDs in our subordinate UIDs range.

The second detail that contributes to the rootless behavior is the fact that Podman is daemon-less by default. We use the init system (systemd) to start and supervise our containerized processes, and there is no root-owned socket that could be used to compromise our system.

Now… I assume that you are reading this article not just for the theory, don’t fret, I’ll jump into a practical case right now.

I could have organized this article as an “adventure” in which I keep finding myself in trouble and showing you how I found the solutions… but I’ll save us time by getting ahead of these situations before we go through the pain.

A “Simple” Case: PostgreSQL

For the sake of this example, let’s assume that we have an unprivileged user for our database service: librarian.

Lingering Users

By default, if we do nothing to prevent it, a service managed by a non-privileged user will only run while said user has an active login session. In most servers this won’t be the case, we won’t have anything nor anyone logging in as these users.

To skip the need to have active login sessions, we’ll run the following command (just once):

loginctl enable-linger librarian

Systemd Service Unit Files

A service unit file is a simple configuration INI file that will be picked up by Systemd to decide how to run a given service. For our particular case of rootless services, we’ll place them in /home/[our_service_user]/.config/containers/systemd/ (in this case we would use the user librarian).

An example (db.container file):

Description=PostgreSQL (db-1), rootless

[Container]
ContainerName=db-1
Image=registry.example.com/postgres:18

# Send container logs to journald so the host's log agent collects them from
# the persistent journal. As a rootless container these entries land in the
# service user's journal.
LogDriver=journald

# Bind only to the private IP, never 0.0.0.0, so Postgres stays off the public
# interface.
PublishPort=10.10.0.17:5432:5432

# Map the in-container postgres uid/gid (999) onto this rootless user, so the
# on-disk data files are owned by the service user and stay legible for backups.
#
# `keep-id` is what ensures that the container's UID and GID are not mapped to
# arbitrarily high UIDs/GIDs on the host side.
UserNS=keep-id:uid=999,gid=999

# Secrets live in a 0600 file owned by the service user instead of inline, so
# they do not land in provisioning logs or in the unit text.
EnvironmentFile=/data/db/env
Environment=PGDATA=/var/lib/postgresql/data/pgdata

# ":U" chowns the bind mount to the mapped container user on start, so the
# freshly initialised data dir ends up owned by the in-container postgres uid.
Volume=/data/db/data:/var/lib/postgresql/data/pgdata:U

[Service]
Restart=always

# This is what decides what will trigger the start of this service, by using
# `default.target` we ensure that it's always started.
[Install]
WantedBy=default.target

Starting the service

I typically use Ansible to simplify how I manage these services, but I think it is a good idea to talk about some of the individual commands we can run directly in the host, as we’ll have to debug our services sooner or later.

Assuming that we have logged as root, we must first become the unpriviliged user:

sudo -u librarian -s

But that is not enough, we’ll have to set a couple of environment variables for our commands to work properly. Assuming that our UID is 1024, run:

export XDG_RUNTIME_DIR="/run/user/1024"
export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1024/bus"

Now, we can see the status of our rootless container (we must use the --user flag even if we are not root if we want to check rootless services):

systemctl --user status db

⚠️ Notice that we didn’t write db-1, nor db.container. We just strip the .container part from our unit file name. We could also write db.service.

The other commands we can run are also pretty standard, and their names are quite self-descriptive for the most part:

# Enable & Disable persist across machine restarts
systemctl --user enable db
systemctl --user disable db

# The basic ones everyone expects
systemctl --user start db
systemctl --user restart db
systemctl --user stop db

# To reload the unit config
systemctl --user reload db

# And much more, just read the help...
systemctl --help

Another Example: Anubis

Before I said that I would focus only on running rootless services, however it would be a pitty to miss the opportunity of explaining other interesting bits. For this example I’ll use my Anubis unit file:

[Container]
ContainerName=tarpit-1
Image=registry.example.com/anubis:latest
LogDriver=journald
PublishPort=10.10.0.5:3333:3333

Environment=BIND=:3333
Environment=TARGET=http://10.10.0.5:3000
Environment=POLICY_FNAME=/data/cfg/botPolicy.yaml

# The policy file is root-owned but world-readable, mounted read-only; a rootless
# process reads it without a chown.
Volume=/data/volumes/anubis/botPolicies.yaml:/data/cfg/botPolicy.yaml:ro

#######################################
# The new interesting details to check:
#######################################

# We ensure that the container's filesystem is readonly
ReadOnly=true

# No processes in the container can gain new privileges through setuid binaries
NoNewPrivileges=true

# We strip all Linux capabilities because we don't need them (we use "high"
# ports, above 1024). This is more relevant when we have to use root as the
# user inside the container.
DropCapability=ALL

What About Networking?

It turns out that having rootless containers also affects how we must configure their networking details. This is not a big deal for containers that are not directly exposed to the Internet, but care must be placed on those that are.

There are 3 interesting points to take into account:

dealing with privileged ports (below the number 1024), given that some capabilities have to be enabled for that to work when using unprivileged users.
preserving the source IP address of the clients reaching our service through the Internet. The way to handle this detail might differ depending on how modern is our system (for example, Podman switched to a different component to manage its network configurations after v5.0, introducing some interesting changes).
firewall rules have some relevant differences depending on whether our containers are rootful or rootless (different rules chains).

These are interesting enough to deserve their own article, so I’ll stop here for today and bring you the next part some time soon in the comming days.

Thank you for staying with me until the end 🤓!