Tools

Tools: Stop Shipping Broken systemd Units: Practical `systemd-analyze verify` for Linux Services - 2025 Update

2026-05-01 0 views admin
Tools: Stop Shipping Broken systemd Units: Practical `systemd-analyze verify` for Linux Services - 2025 Update

What systemd-analyze verify actually checks

A broken service example

A clean service and timer pair

A practical local workflow before install

Make warnings fail CI with --recursive-errors=

Verifying staged files in a package or image root

What verify does not replace

A simple repo-friendly check script

Final take

References If you write or package systemd units regularly, you have probably hit this pattern at least once. You edit a service file, run systemctl daemon-reload, try to start it, and only then discover a typo, a missing binary path, or a dependency name you misspelled half asleep. systemd-analyze verify is a simple way to catch a lot of that before the unit ever reaches production. In this guide, I will show a practical workflow for: According to the systemd-analyze(1) manual, systemd-analyze verify FILE... loads the specified unit files and also loads units referenced by them. The manual says it currently detects at least these classes of problems: That makes it a very good lint step for systemd unit authoring. Here is a deliberately bad unit: On a current Debian system, this produces errors like: That is exactly the kind of breakage you want to catch before a reload. A more realistic pattern is a service plus a timer. Verify both together: If verification succeeds cleanly, the command prints nothing and exits successfully. I like verifying related units in one command because a timer that points at the wrong service name is just as broken as a bad service file. When I am editing units by hand, this is the order I prefer: Then confirm both the unit state and recent logs: One subtle detail from the manual matters a lot for automation. If you do not pass --recursive-errors=, systemd-analyze verify may still print warnings while returning a zero exit status. For CI or packaging checks, use one of these: For most CI checks, I would choose yes if the build environment contains the full dependency set, or one if you want a stricter signal on the files you directly touched without turning unrelated environment noise into failures. systemd-analyze also supports --root=PATH for verification against a different filesystem tree. That is useful when you build packages, chroots, or machine images and want to validate units before they land on the live host. A practical warning here: this works best when the alternate root actually contains the unit dependencies and executable paths your unit references. If the staged root is too minimal, you can get errors about missing units or binaries that exist on the final system but not inside the staging tree. So --root= is excellent for representative chroots and image roots, but less useful on a skeletal directory tree that only contains one unit file. systemd-analyze verify is valuable, but it is not the whole test plan. It does not prove that: After a clean verify, I still recommend testing the real activation path. For timer units, this is especially useful: That way you validate both the unit syntax and the real runtime behavior. If you keep your units in Git, add a small verifier script: Then run it locally before commits, or in CI before packaging and deployment. For a GitHub Actions step, the core check is as simple as: That one step catches a surprising number of avoidable mistakes. If you work with systemd, systemd-analyze verify is one of those small tools that pays for itself fast. It will not replace actually starting the service, but it is excellent at catching the boring, expensive mistakes early: typos, wrong dependency names, and broken command paths. My rule of thumb is simple: That turns unit-file edits from guesswork into a repeatable workflow. Templates let you quickly answer FAQs or store snippets for re-use. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse

Command

Copy

# bad-demo.-weight: 500;">service [Unit] Description=Bad demo After=network-online.targt [Service] Typ=oneshot ExecStart=/usr/bin/not-a-real-binary Restart=on-failure # bad-demo.-weight: 500;">service [Unit] Description=Bad demo After=network-online.targt [Service] Typ=oneshot ExecStart=/usr/bin/not-a-real-binary Restart=on-failure # bad-demo.-weight: 500;">service [Unit] Description=Bad demo After=network-online.targt [Service] Typ=oneshot ExecStart=/usr/bin/not-a-real-binary Restart=on-failure systemd-analyze verify ./bad-demo.-weight: 500;">service systemd-analyze verify ./bad-demo.-weight: 500;">service systemd-analyze verify ./bad-demo.-weight: 500;">service ./bad-demo.-weight: 500;">service:3: Failed to add dependency on network-online.targt, ignoring: Invalid argument ./bad-demo.-weight: 500;">service:6: Unknown key 'Typ' in section [Service], ignoring. bad-demo.-weight: 500;">service: Command /usr/bin/not-a-real-binary is not executable: No such file or directory ./bad-demo.-weight: 500;">service:3: Failed to add dependency on network-online.targt, ignoring: Invalid argument ./bad-demo.-weight: 500;">service:6: Unknown key 'Typ' in section [Service], ignoring. bad-demo.-weight: 500;">service: Command /usr/bin/not-a-real-binary is not executable: No such file or directory ./bad-demo.-weight: 500;">service:3: Failed to add dependency on network-online.targt, ignoring: Invalid argument ./bad-demo.-weight: 500;">service:6: Unknown key 'Typ' in section [Service], ignoring. bad-demo.-weight: 500;">service: Command /usr/bin/not-a-real-binary is not executable: No such file or directory # demo-backup.-weight: 500;">service [Unit] Description=Demo backup job Wants=network-online.target After=network-online.target [Service] Type=oneshot ExecStart=/usr/bin/env bash -lc 'echo backing up; exit 0' ProtectSystem=strict ReadWritePaths=/var/backups NoNewPrivileges=yes # demo-backup.-weight: 500;">service [Unit] Description=Demo backup job Wants=network-online.target After=network-online.target [Service] Type=oneshot ExecStart=/usr/bin/env bash -lc 'echo backing up; exit 0' ProtectSystem=strict ReadWritePaths=/var/backups NoNewPrivileges=yes # demo-backup.-weight: 500;">service [Unit] Description=Demo backup job Wants=network-online.target After=network-online.target [Service] Type=oneshot ExecStart=/usr/bin/env bash -lc 'echo backing up; exit 0' ProtectSystem=strict ReadWritePaths=/var/backups NoNewPrivileges=yes # demo-backup.timer [Unit] Description=Run demo backup every night [Timer] OnCalendar=03:15 Persistent=true Unit=demo-backup.-weight: 500;">service [Install] WantedBy=timers.target # demo-backup.timer [Unit] Description=Run demo backup every night [Timer] OnCalendar=03:15 Persistent=true Unit=demo-backup.-weight: 500;">service [Install] WantedBy=timers.target # demo-backup.timer [Unit] Description=Run demo backup every night [Timer] OnCalendar=03:15 Persistent=true Unit=demo-backup.-weight: 500;">service [Install] WantedBy=timers.target systemd-analyze verify ./demo-backup.-weight: 500;">service ./demo-backup.timer systemd-analyze verify ./demo-backup.-weight: 500;">service ./demo-backup.timer systemd-analyze verify ./demo-backup.-weight: 500;">service ./demo-backup.timer systemd-analyze verify ./myjob.-weight: 500;">service ./myjob.timer && \ -weight: 600;">sudo -weight: 500;">install -m 0644 ./myjob.-weight: 500;">service ./myjob.timer /etc/systemd/system/ && \ -weight: 600;">sudo -weight: 500;">systemctl daemon-reload && \ -weight: 600;">sudo -weight: 500;">systemctl -weight: 500;">enable --now myjob.timer systemd-analyze verify ./myjob.-weight: 500;">service ./myjob.timer && \ -weight: 600;">sudo -weight: 500;">install -m 0644 ./myjob.-weight: 500;">service ./myjob.timer /etc/systemd/system/ && \ -weight: 600;">sudo -weight: 500;">systemctl daemon-reload && \ -weight: 600;">sudo -weight: 500;">systemctl -weight: 500;">enable --now myjob.timer systemd-analyze verify ./myjob.-weight: 500;">service ./myjob.timer && \ -weight: 600;">sudo -weight: 500;">install -m 0644 ./myjob.-weight: 500;">service ./myjob.timer /etc/systemd/system/ && \ -weight: 600;">sudo -weight: 500;">systemctl daemon-reload && \ -weight: 600;">sudo -weight: 500;">systemctl -weight: 500;">enable --now myjob.timer -weight: 500;">systemctl -weight: 500;">status myjob.timer myjob.-weight: 500;">service --no-pager journalctl -u myjob.-weight: 500;">service -u myjob.timer -b --no-pager -weight: 500;">systemctl -weight: 500;">status myjob.timer myjob.-weight: 500;">service --no-pager journalctl -u myjob.-weight: 500;">service -u myjob.timer -b --no-pager -weight: 500;">systemctl -weight: 500;">status myjob.timer myjob.-weight: 500;">service --no-pager journalctl -u myjob.-weight: 500;">service -u myjob.timer -b --no-pager systemd-analyze verify --recursive-errors=yes ./myjob.-weight: 500;">service ./myjob.timer systemd-analyze verify --recursive-errors=yes ./myjob.-weight: 500;">service ./myjob.timer systemd-analyze verify --recursive-errors=yes ./myjob.-weight: 500;">service ./myjob.timer pkgroot/ └── etc/systemd/system/ └── app.-weight: 500;">service pkgroot/ └── etc/systemd/system/ └── app.-weight: 500;">service pkgroot/ └── etc/systemd/system/ └── app.-weight: 500;">service systemd-analyze verify --root="$PWD/pkgroot" app.-weight: 500;">service systemd-analyze verify --root="$PWD/pkgroot" app.-weight: 500;">service systemd-analyze verify --root="$PWD/pkgroot" app.-weight: 500;">service systemd-analyze calendar '03:15' -weight: 500;">systemctl -weight: 500;">start myjob.-weight: 500;">service journalctl -u myjob.-weight: 500;">service -n 50 --no-pager systemd-analyze calendar '03:15' -weight: 500;">systemctl -weight: 500;">start myjob.-weight: 500;">service journalctl -u myjob.-weight: 500;">service -n 50 --no-pager systemd-analyze calendar '03:15' -weight: 500;">systemctl -weight: 500;">start myjob.-weight: 500;">service journalctl -u myjob.-weight: 500;">service -n 50 --no-pager #!/usr/bin/env bash set -euo pipefail units=( systemd/myjob.-weight: 500;">service systemd/myjob.timer ) systemd-analyze verify --recursive-errors=one "${units[@]}" #!/usr/bin/env bash set -euo pipefail units=( systemd/myjob.-weight: 500;">service systemd/myjob.timer ) systemd-analyze verify --recursive-errors=one "${units[@]}" #!/usr/bin/env bash set -euo pipefail units=( systemd/myjob.-weight: 500;">service systemd/myjob.timer ) systemd-analyze verify --recursive-errors=one "${units[@]}" - name: Verify systemd units run: | systemd-analyze verify --recursive-errors=one \ systemd/myjob.-weight: 500;">service \ systemd/myjob.timer - name: Verify systemd units run: | systemd-analyze verify --recursive-errors=one \ systemd/myjob.-weight: 500;">service \ systemd/myjob.timer - name: Verify systemd units run: | systemd-analyze verify --recursive-errors=one \ systemd/myjob.-weight: 500;">service \ systemd/myjob.timer - validating unit files before reload or deploy - catching unknown directives and bad dependency names - verifying a -weight: 500;">service and its timer together - making verification fail your CI job when warnings appear - understanding what verify catches, and what it does not - unknown sections and directives - missing dependencies required to -weight: 500;">start the unit - Documentation= man pages that are not present - commands in ExecStart= and similar directives that are missing or not executable - Write or -weight: 500;">update the unit files in a working directory. - Run systemd-analyze verify against the -weight: 500;">service, timer, socket, or path units involved. - Copy them into /etc/systemd/system/ only after they verify cleanly. - Run -weight: 500;">systemctl daemon-reload. - Start the unit and inspect logs. - yes: fail on warnings in the unit or any associated dependencies - one: fail on warnings in the unit or its immediate dependencies - no: fail only on warnings in the explicitly specified unit - your -weight: 500;">service logic is correct - the command behaves correctly with real environment variables or credentials - the -weight: 500;">service has all required runtime permissions - the timer schedule is what you intended - the -weight: 500;">service will stay healthy after startup - verify before -weight: 500;">install - reload only after verify passes - -weight: 500;">start the unit and inspect logs before calling it done - systemd-analyze(1) manual: https://www.freedesktop.org/software/systemd/man/latest/systemd-analyze.html - Linux man page mirror for systemd-analyze(1): https://man7.org/linux/man-pages/man1/systemd-analyze.1.html - systemd.unit(5) manual: https://www.freedesktop.org/software/systemd/man/latest/systemd.unit.html - systemd.timer(5) manual: https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html - -weight: 500;">systemctl(1) manual: https://www.freedesktop.org/software/systemd/man/latest/-weight: 500;">systemctl.html

🏷️ Tags

toolsutilitiessecurity toolsshippingbrokensystemdunitspracticalanalyzeverify

More from Tools

Tools: Report: How I Built a Self-Improving AI Election Assistant

Tools: Report: How I Built a Self-Improving AI Election Assistant

2026-05-01 0
Tools: Latest: How to Deploy Llama 3.2 with Speculative Decoding on a $10/Month DigitalOcean Droplet: 3x Faster Inference at 1/100th API Cost

Tools: Latest: How to Deploy Llama 3.2 with Speculative Decoding on a $10/Month DigitalOcean Droplet: 3x Faster Inference at 1/100th API Cost

2026-05-01 0
Tools: 10 Free Online Tools Every Developer Should Bookmark in 2026

Tools: 10 Free Online Tools Every Developer Should Bookmark in 2026

2026-05-01 0
Tools: Essential Guide: 10 CLAUDE.md Rules Every Python Developer Needs in 2026

Tools: Essential Guide: 10 CLAUDE.md Rules Every Python Developer Needs in 2026

2026-05-01 0

Trending

1

CVE-2025-61481: Critical Remote Code Execution Vulnerability in MikroTik RouterOS & SwitchOS

2025-10-27 • 189 views
2

CVE-2025-43939: Dell Unity OS Command Injection (High)

2025-10-30 • 148 views
3

Google disputes false claims of massive Gmail data breach

2025-10-30 • 130 views
4

Microsoft: DNS outage impacts Azure and Microsoft 365 services

2025-10-30 • 88 views
5

3.5B Accounts, 1 Critical Flaw: Meta Closes WhatsApp Data-Harvesting

2025-11-25 • 81 views