Podman Task Driver
Name: podman
The Podman task driver plugin for Nomad uses the Pod Manager (podman) daemonless container runtime for executing Nomad tasks. Podman supports OCI containers and its command line tool is meant to be a drop-in replacement for docker.
Source is on GitHub
Installation
You can download a precompiled binary and verify the
binary using the available SHA-256 sums. After downloading nomad-driver-podman
driver, unzip the package. Make sure that the nomad-driver-podman
binary is
available on your plugin_dir path,
specified by the client's config file, before continuing with the other guides.
Usage
The example job created by nomad init -short
is easily adapted
to use Podman instead:
Refer to the project's homepage for details.
Client Requirements
The Podman task driver is not built into Nomad. It must be downloaded onto the client host in the configured plugin directory.
- Linux host with
podman
installed - For rootless containers you need a system supporting cgroups v2 and a few other things, follow this tutorial.
You need a v3.x or higher podman binary and a system socket activation unit. It is recommended to install podman via your system's package manager, which will configure systemd for you.
Ensure that Nomad can find the plugin, refer to plugin_dir
.
Capabilities
The podman
driver implements the following capabilities.
Feature | Implementation |
---|---|
nomad alloc signal | true |
nomad alloc exec | true |
filesystem isolation | image |
network isolation | host, group, task, none |
volume mounting | true |
Task Configuration
args
- (Optional) A list of arguments to the optional command. If nocommand
is specified, the arguments are passed directly to the container.auth
- (Optional) Authenticate to the image registry using a static credential. By setting tlsVerify to false the driver will allow using self- signed certificates or plain HTTP connections to the registry.cap_add
- (Optional) A list of Linux capabilities as strings to pass to--cap-add
.cap_drop
- (Optional) A list of Linux capabilities as strings to pass to--cap-drop
.command
- (Optional) The command to run when starting the container.devices
- (Optional) A list ofhost-device[:container-device][:permissions]
definitions. Each entry adds a host device to the container. Optional permissions can be used to specify device permissions, it is a combination ofr
for read,w
for write, andm
formknod(2)
. Refer to Podman's documentation for more details.entrypoint
- (Optional) The entrypoint for the container. Defaults to theentrypoint
set in the image.force_pull
- (Optional)true
orfalse
(default). Always pull the latest image on container start.hostname
- (Optional) The hostname to assign to the container. When launching more than one of a task (usingcount
) with this option set, every container the task starts will have the same hostname.image
- The image to run. Accepted transports aredocker
(default if missing),oci-archive
anddocker-archive
. Images referenced as short-names will be treated according to user-configured preferences.extra_hosts
- (Optional) Set additional hosts in the containerimage_pull_timeout
- (Optional) Time duration for your pull timeout (default to"5m"
). Cannot be longer than theclient_http_timeout
.init
- (Optional) Run aninit
inside the container that forwards signals and reaps processes.init_path
- (Optional) Path to thecontainer-init
binary.labels
- (Optional) Set labels on the container.logging
- (Optional) Configure logging. Also refer to the plugin optiondisable_log_collection
.driver = "nomad"
- (Default) Podman redirects its combinedstdout/stderr
logstream directly to a Nomadfifo
. Benefits of this mode are: zero overhead, don't have to worry about log rotation at system or Podman level. Downside: you cannot easily ship the logstream to a log aggregator plusstdout/stderr
is multiplexed into a single stream.driver = "journald"
- The container log is forwarded from Podman to thejournald
on your host. Next, it's pulled by the Podman API back from the journal into the Nomadfifo
(controllable bydisable_log_collection
). Benefits: all containers can log into the host journal, you can ship a structured stream including metadata to your log aggregator. No log rotation at Podman level. You can add additional tags to the journal. Drawbacks: a bit more overhead, depends on Journal (will not work on WSL2). You should configure some rotation policy for your Journal. Ensure you're running Podman 3.1.0 or higher because of bugs in older versions.
memory_reservation
- (Optional) Memory soft limit (units =b
(bytes),k
(kilobytes),m
(megabytes), org
(gigabytes)).After setting memory reservation, when the system detects memory contention or low memory, containers are forced to restrict their consumption to their reservation. So you should always set the value below
--memory
, otherwise the hard limit will take precedence. By default, memory reservation will be the same as memory limit.memory_swap
- (Optional) A limit value equal to memory plus swap. The swap limit should always be larger than the memory value. Unit can beb
(bytes),k
(kilobytes),m
(megabytes), org
(gigabytes). If you don't specify a unit,b
is used. SetLIMIT
to-1
to enable unlimited swap.memory_swappiness
- Tune a container's memory swappiness behavior. Accepts an integer between0
and100
.network_mode
- (Optional) Set the network mode for the container. By default the task uses the network stack defined in the task groupnetwork
block. If the groups network behavior is also undefined, it will fallback tobridge
in rootful mode orslirp4netns
for rootless containers.bridge
- (Default for rootful) Create a network stack on the default Podman bridge.container:id
- Reuse another container's network stack.host
- Use the Podman host network stack. Note: the host mode gives the container full access to local system services such as D-bus and is therefore considered insecure.none
- No networking.slirp4netns
- (Default for rootless) Useslirp4netns
to create a user network stack. Podman currently does not support this option for rootful containers (issue).task:name-of-other-task
: Join the network of another task in the same allocation.
ports
- (Optional) Forward and expose ports. Refer to Docker driver configuration for details.privileged
- (Optional)true
orfalse
(default). A privileged container turns off the security features that isolate the container from the host. Dropped Capabilities, limited devices, read-only mount points, Apparmor/SELinux separation, and Seccomp filters are all disabled.readonly_rootfs
- (Optional)true
orfalse
(default). Mount the rootfs as read-only.sysctl
- (Optional) A key-value map ofsysctl
configurations to set to the containers on start.tmpfs
- (Optional) A list of/container_path
strings fortmpfs
mount points. Refer topodman run --tmpfs
options for details.tty
- (Optional)true
orfalse
(default). Allocate a pseudo-TTY for the container.volumes
- (Optional) A list ofhost_path:container_path:options
strings to bind host paths to container paths. Named volumes are not supported.working_dir
- (Optional) The working directory for the container. Defaults to the default set in the image.ulimit
- (Optional) A key-value map of ulimit configurations to set to the containers to start.
Additionally, the Podman driver supports customization of the container's user
through the task's user
option.
Network Configuration
Nomad lifecycle hooks combined with the drivers
network_mode
allows very flexible network namespace definitions. This
feature does not build upon the native Podman pod structure but simply reuses
the networking namespace of one container for other tasks in the same group.
A typical example is a network server and a metric exporter or log shipping
sidecar. The metric exporter needs access to a private monitoring port which
should not be exposed to the network and thus is usually bound to localhost
.
The nomad-driver-podman
repository includes three different
examples jobs for such a setup. All of them will start a
nats server and a
prometheus-nats-exporter
using different approaches.
You can use curl
to prove that the job is working correctly and that you can
get Prometheus metrics:
2 Task setup, server defines the network
Reference examples/jobs/nats_simple_pod.nomad
.
Here, the server
task is started as main workload and the exporter
runs as
a poststart
sidecar. Because of that, Nomad guarantees that the server is
started first and thus the exporter can easily join the servers network
namespace via network_mode = "task:server"
.
Note, that the server
configuration file binds the http_port
to
localhost
.
Be aware that ports must be defined in the parent network namespace, here
server
.
3 Task setup, a pause container defines the network
Reference examples/jobs/nats_pod.nomad
.
A slightly different setup is demonstrated in this job. It reassembles more
closesly the idea of a pod
by starting a pause
task, named pod
via a
prestart
sidecar hook.
Next, the main workload, server
is started and joins the network namespace by
using the network_mode = "task:pod"
block. Finally, Nomad starts the
poststart
sidecar exporter
which also joins the network.
Note that all ports must be defined on the pod
level.
2 Task setup, shared Nomad network namespace
Reference examples/jobs/nats_group.nomad
.
This example is very different. Both server
and exporter
join a network
namespace which is created and managed by Nomad itself. Refer to Nomad's
network
block to get started with this generic approach.
Plugin Options
The Podman plugin has options which may be customized in the agent's configuration file.
gc
block:container
- Defaults totrue
. This option can be used to disable Nomad from removing a container when the task exits.
recover_stopped
- (Deprecated) Defaults tofalse
. Allows the driver to start and reuse a previously stopped container after a Nomad client restart. Consider a simple single node system and a complete reboot. All previously managed containers will be reused instead of disposed and recreated.This option may cause Nomad client to hang on startup. It now defaults to being disabled and may be removed in a future release.
socket_path
(string)
- Defaults tounix://run/podman/io.podman
when running asroot
or a cgroup V1 system, andunix://run/user/<USER_ID>/podman/io.podman
for rootless cgroup V2 systems.disable_log_collection
(bool: false)
- Setting this totrue
will disable Nomad logs collection of Podman tasks. If you don't rely on Nomad log capabilities and exclusively use host based log aggregation, you may consider this option to disable Nomad log collection overhead. Beware to you also lose automatic log rotation.client_http_timeout
(string: "60s")
- Default timeout used byhttp.Client
requests.volumes
block:enabled
- Defaults totrue
. Allows tasks to bind host paths (volumes) inside their container.selinuxlabel
- Allows the operator to set a SELinux label to the allocation and task local bind-mounts to containers. If used withvolumes.enabled
set to false, the labels will still be applied to the standard binds in the container.
If you need to configure authorization to an external registry, you can write a
file containing the details in a location such as
/etc/nomad.d/registry/auth.json
:
You can then instruct Nomad and the Podman driver to use this information by
adding the following to the /etc/nomad.d/nomad.env
file: