To create a systemd service, write a unit file at /etc/systemd/system/myapp.service with [Unit], [Service] and [Install] sections, run sudo systemctl daemon-reload, then sudo systemctl enable --now myapp. systemd then starts your app at every boot, restarts it after a crash when you set Restart=on-failure, and sends everything it prints to the journal, where journalctl -u myapp reads it.
The guide builds one service step by step, then explains every line. It targets Ubuntu 24.04 LTS, which ships systemd 255; Debian 12 ships systemd 252 and Debian 13 ships systemd 257, and the few lines that need a newer version are marked. Every directive was checked against the official manuals on October 3, 2026.
Key takeaways
- A systemd service is a unit file in /etc/systemd/system/NAME.service; after sudo systemctl daemon-reload, sudo systemctl enable --now NAME starts it now and at every boot.
- Use Type=exec for long-running apps, as the systemd manual recommends, so systemctl start fails loudly when the binary or the user is missing; use Type=notify only if the app reports readiness.
- With only Restart=on-failure, a crashing app burns the default 5 starts in 10 seconds in about half a second; StartLimitIntervalSec= and StartLimitBurst= belong in [Unit] and trip only when the burst fits inside the interval.
- NoNewPrivileges=yes, ProtectSystem=strict, ProtectHome=yes and PrivateTmp=yes cost four lines; give the app a writable StateDirectory= and score the result with systemd-analyze security.
- A .timer unit replaces cron with per-unit logs, catch-up runs (Persistent=true) and no overlapping runs; check any schedule with systemd-analyze calendar before you enable it.
You need a Linux server you can reach as a sudo user. If this is a fresh VPS, connect over SSH and run through the VPS security checklist first; this guide does not repeat SSH keys, UFW or updates.
How to create a systemd service in 5 steps
The example is a small Python web app with no dependencies, so every step has a check you can run. Swap in your own program at step 2; the templates below cover a Python virtual environment, Node.js and a Go binary.
Step 1: Create a user for the service
A service should not run as root. Create a system account with no login shell and no home directory:
sudo useradd --system --user-group --no-create-home --shell /usr/sbin/nologin myapp
If you would rather not manage an account at all, DynamicUser=yes lets systemd allocate one each time the service starts; see the hardening section for the trade-offs.
Step 2: Put the program where the service can read it
Keep the code in /opt/myapp, owned by root, so the service can read it but not rewrite it. Create the file with sudo nano /opt/myapp/app.py after creating the folder:
sudo install -d -m 755 /opt/myapp
sudo nano /opt/myapp/app.py
Paste this demo app. It answers on 127.0.0.1:8000 and keeps a hit counter in the state folder that systemd will create for it:
#!/usr/bin/env python3
"""Demo app for the systemd guide: a hit counter on 127.0.0.1."""
import os
from http.server import BaseHTTPRequestHandler, HTTPServer
from pathlib import Path
STATE = Path(os.environ.get("STATE_DIRECTORY", "/tmp")) / "hits.txt"
PORT = int(os.environ.get("PORT", "8000"))
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
hits = int(STATE.read_text()) + 1 if STATE.exists() else 1
STATE.write_text(str(hits))
body = f"hello from systemd, hit {hits}\n".encode()
self.send_response(200)
self.send_header("Content-Type", "text/plain")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
if __name__ == "__main__":
print(f"listening on 127.0.0.1:{PORT}, state in {STATE}", flush=True)
HTTPServer(("127.0.0.1", PORT), Handler).serve_forever()
Settings go in a separate environment file. Make it readable by root only: systemd reads it as root before it switches to the myapp user, so the app never needs access to the file itself:
sudo install -d -m 755 /etc/myapp
sudo install -m 600 /dev/null /etc/myapp/myapp.env
echo 'PORT=8000' | sudo tee /etc/myapp/myapp.env > /dev/null
Step 3: Write the unit file
Create /etc/systemd/system/myapp.service with sudo nano. The file name, minus .service, becomes the service name:
[Unit]
Description=My app (demo hit counter)
StartLimitIntervalSec=5min
StartLimitBurst=10
[Service]
Type=exec
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
EnvironmentFile=/etc/myapp/myapp.env
ExecStart=/usr/bin/python3 /opt/myapp/app.py
Restart=on-failure
RestartSec=5s
StateDirectory=myapp
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
Every line is explained in the anatomy section. One shortcut: sudo systemctl edit --force --full myapp.service creates the same file in your editor and reloads systemd when you save, as the systemctl(1) manual describes.
Step 4: Check the file, then enable and start it
systemd-analyze verify reports unknown sections, misspelled directives and missing executables before anything runs:
sudo systemd-analyze verify /etc/systemd/system/myapp.service
No output means no problems found. Tell systemd to reread its unit files, then enable the service for boot and start it now:
sudo systemctl daemon-reload
sudo systemctl enable --now myapp.service
Step 5: Prove it works, restarts and survives a reboot
systemctl status myapp.service
curl -s http://127.0.0.1:8000/
sudo journalctl -u myapp.service -n 20 --no-pager
Look for active (running) in the status, a hello from systemd, hit 1 reply, and the listening on line plus one request line in the journal. Now kill the process the hard way. SIGKILL counts as a failure, so Restart=on-failure brings the app back after the 5-second RestartSec=:
sudo systemctl kill --signal=SIGKILL myapp.service
sleep 7
systemctl status myapp.service --no-pager | head -n 3
Finally run sudo reboot, reconnect and curl the app again. The counter keeps counting, because /var/lib/myapp/hits.txt lives in the state directory that survives restarts and reboots.
Start, stop, restart and reload the service
A handful of systemctl verbs cover daily work. You can leave off .service, because systemctl assumes it. Do not confuse reload with daemon-reload: the first asks your app to reread its own config, the second makes systemd reread unit files, a distinction the systemctl(1) manual spells out.
| Task | Command | Good to know |
|---|---|---|
| Start or stop it now | sudo systemctl start myapp, sudo systemctl stop myapp | A manual stop is never undone by Restart=; an enabled unit still starts at the next boot |
| Restart after a code or env-file change | sudo systemctl restart myapp | Stop, then start; starts it if it was not running |
| Reload the app’s own config without a restart | sudo systemctl reload myapp | Works only if the unit has ExecReload= or Type= |
| Apply edits to the unit file | sudo systemctl daemon-reload, then restart | A plain restart keeps running the old unit definition |
| Start at boot, or stop starting at boot | sudo systemctl enable myapp, sudo systemctl disable myapp | Add --now to also start or stop it |
| Check from a script | systemctl is-active myapp, systemctl is-enabled myapp | Exit code 0 means active or enabled |
Systemd service file anatomy: what each line does
A unit file is an ini-style text file. [Unit] and [Install] are common to every unit type and documented in systemd.unit(5); [Service] options come from systemd.service(5) and, for the process environment and sandboxing, systemd.exec(5).
| Line | Section | What it does |
|---|---|---|
Description= | [Unit] | The name shown in systemctl status and in log lines. |
StartLimitIntervalSec= | StartLimitBurst= | Rate limit on starts: no more than Burst starts per interval. Both belong in [Unit] (details). |
Type= | [Service] | Treats the service as started once the program has actually been executed, so a missing binary or user makes systemctl start fail. |
User=, Group= | [Service] | The account the process runs as. The default for system services is root. |
WorkingDirectory= | [Service] | The current directory for the process. The default is / for system services, which breaks apps that open files by relative path. |
EnvironmentFile= | [Service] | Reads KEY= lines from a file. A missing file stops the start unless the path is prefixed with -. |
ExecStart= | [Service] | The command that starts the app. Not a shell: no pipes, redirects or &&. |
Restart= | [Service] | Restarts after a crash, a non-zero exit, a timeout or a watchdog failure, never after systemctl stop. |
RestartSec= | [Service] | Wait before each restart. The default is 100 ms. |
StateDirectory= | [Service] | Creates /var/, owned by User=, writable even under ProtectSystem=, and passes its path as $STATE_. |
NoNewPrivileges=, ProtectSystem=, ProtectHome=, PrivateTmp= | [Service] | The four-line sandbox explained in the hardening section. |
WantedBy= | [Install] | Read only by systemctl enable, which links the service into the normal boot target so it starts at boot. |
Where do systemd service files go?
| Path | What lives there | Edit it? |
|---|---|---|
/etc/ | Units you write as the administrator. They override same-named units below. | Yes: your own services go here |
/etc/ | Drop-ins: partial overrides for any unit, created by systemctl edit NAME | Yes: the safe way to change a packaged unit |
/usr/ | Units installed by packages (PostgreSQL, nginx, Docker) | No: package upgrades overwrite it |
~/.config/ | A user’s own user services | Yes, as that user |
/etc/ | User units the administrator provides for all users | Rarely |
Which Type= should you use?
Type= tells systemd when the service counts as started, which decides when systemctl start returns and when units ordered after it may start. The systemd.service manual is direct about the usual choice:
It is recommended to use Type=exec for long-running services, as it ensures that process setup errors (e.g. errors such as a missing service executable, or missing user) are properly tracked.
systemd.service(5), systemd 255
| Type | Started when… | Use it for |
|---|---|---|
simple | the process has been forked, before the binary even runs (the default when ExecStart= is set) | Legacy units. systemctl start reports success even if the binary is missing. |
exec | the binary has been executed (systemd 240+) | Any long-running Python, Node.js or Go app. The default choice in this guide. |
notify | the app sends READY= to systemd | Apps that other units depend on, so they start only once it accepts connections (example). |
notify-reload | like notify; systemctl reload sends SIGHUP and waits for the app to confirm (systemd 253+) | Apps that reload config on SIGHUP and implement the reload notification. Not on Debian 12. |
oneshot | the process has exited | Scripts and jobs, including timer jobs. Several ExecStart= lines are allowed. |
forking | the parent process exits after forking a daemon | Old daemons that background themselves. The manual discourages it in favor of notify or dbus. |
ExecStart= is not a shell command
systemd splits ExecStart= into words itself and runs the program directly. The manual lists what that means in practice:
- The program must be an absolute path, or a bare name found in systemd’s fixed search path, which includes
/usr/local/binand/usr/bin. A relative path such as./appis rejected. - Pipes, redirection (
>,>>),&&and background&are not supported. Wrap them explicitly:ExecStart=/bin/sh -c 'mycmd | tee -a /var/log/myapp/out.log'. ${VAR}expands to one argument and$VARsplits at whitespace; the variable comes fromEnvironment=orEnvironmentFile=, not from your login shell. Write$$for a literal dollar sign.- A
-prefix (ExecStartPre=-/usr/bin/false) turns a failure of that command into success. Do not useExecStartPre=for long-running processes; systemd kills them before the next command runs.
Environment variables and secrets
A service does not read ~/.bashrc or your login environment, so a program that works in your shell can fail under systemd for lack of one variable. Set what it needs explicitly:
[Service]
Environment=NODE_ENV=production "GREETING=hello world"
EnvironmentFile=/etc/myapp/myapp.env
EnvironmentFile=-/etc/myapp/optional.env
In the file, one KEY=value per line; lines starting with # or ; are comments, and single- or double-quoted values follow POSIX shell quoting rules. Values from EnvironmentFile= override Environment=. Unit files are normally world-readable, so never put a password on an Environment= line. systemd.exec(5) goes further and says environment variables are not suitable for secrets at all, because they are exposed over D-Bus and inherited by child processes; it recommends LoadCredential=. Our Discord bot guide shows how to pass a token with LoadCredential= and read it from $CREDENTIALS_DIRECTORY.
Wait for the network only if you need it
An app that only listens on 0.0.0.0, 127.0.0.1 or [::] needs no network ordering: systemd’s network-online guide notes those addresses are always available. An app that must reach the internet the moment it starts, such as a bot that logs in to an API, should wait until the network is up:
[Unit]
Wants=network-online.target
After=network-online.target
Both lines are needed: Wants= pulls the target in and After= orders the start after it. network.target alone says almost nothing about connectivity at boot.
Systemd service file examples for Python, Node.js and Go
Each template assumes a myapp user (step 1), code in /opt/myapp and settings in /etc/myapp/myapp.env. Real-world versions of these units run the Discord bot, the Telegram bot, the Minecraft server and the Valheim dedicated server in our other guides.
Python in a virtual environment
Point ExecStart= at the virtual environment’s own interpreter; you never activate a venv in a unit file. On Ubuntu 24.04, python3 -m venv needs the python3-venv package first.
sudo apt install python3-venv
sudo python3 -m venv /opt/myapp/venv
sudo /opt/myapp/venv/bin/pip install -r /opt/myapp/requirements.txt
[Unit]
Description=My Python app
[Service]
Type=exec
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
Environment=PYTHONUNBUFFERED=1
EnvironmentFile=/etc/myapp/myapp.env
ExecStart=/opt/myapp/venv/bin/python /opt/myapp/app.py
Restart=on-failure
RestartSec=5s
StateDirectory=myapp
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
PYTHONUNBUFFERED=1 has the same effect as python -u: output reaches the journal line by line instead of in delayed blocks. For a WSGI app served by Gunicorn, use ExecStart=/opt/myapp/venv/bin/gunicorn --bind 127.0.0.1:8000 app:app with Type=notify, which Gunicorn’s deployment docs use because Gunicorn reports readiness itself.
Node.js
Find the real path of node with command -v node. Ubuntu’s and NodeSource’s packages install /usr/bin/node; a version manager such as nvm installs under your home directory, which ProtectHome=yes hides from the service, so the start fails with status=203/EXEC. Install Node.js system-wide for services, or copy the binary to /usr/local/bin.
[Unit]
Description=My Node.js app
[Service]
Type=exec
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
Environment=NODE_ENV=production
EnvironmentFile=/etc/myapp/myapp.env
ExecStart=/usr/bin/node /opt/myapp/server.js
Restart=on-failure
RestartSec=5s
StateDirectory=myapp
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
Install dependencies as root inside /opt/myapp (sudo npm ci --omit=dev), so the running app cannot modify its own node_modules. Do not add MemoryDenyWriteExecute=yes to a Node.js unit: systemd.exec(5) says it is incompatible with JIT engines, and V8 is one.
A Go (or any compiled) binary
A compiled program is the simplest case. Copy the binary to /usr/local/bin with root ownership; the unit then needs no interpreter path:
sudo install -m 755 ./myapp /usr/local/bin/myapp
Start from the Python template, delete the PYTHONUNBUFFERED line, and change these lines:
[Unit]
Description=My Go app
[Service]
WorkingDirectory=/var/lib/myapp
ExecStart=/usr/local/bin/myapp
The working directory is now the state directory itself, so files the program writes by relative path land somewhere writable. systemd creates /var/lib/myapp before it changes into it.
Type=notify without a library (Python)
With Type=notify, systemd waits for the app to say READY=1 before it marks the service started. The protocol is one datagram to the Unix socket named in $NOTIFY_SOCKET. This function is trimmed from the standalone Python implementation in the sd_notify(3) manual (published there under MIT-0) and uses only the standard library:
import os
import socket
def sd_notify(message: bytes) -> None:
"""Send a state change to systemd; do nothing outside systemd."""
path = os.environ.get("NOTIFY_SOCKET")
if not path:
return
if path[0] == "@": # abstract socket namespace
path = "\0" + path[1:]
with socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_CLOEXEC) as sock:
sock.connect(path)
sock.sendall(message)
# ...open the database, bind the port, load the config...
sd_notify(b"READY=1")
Then set Type=notify in the unit; systemd sets NotifyAccess=main by itself. Add WatchdogSec=30s and call sd_notify(b"WATCHDOG=1") more often than that, and systemd kills and restarts an app that hangs without crashing. Node.js has no built-in way to send this datagram, so Node.js services usually stay on Type=exec.
Restart=on-failure, RestartSec and start limits explained
Restart= decides which kinds of exit trigger a restart. This table is the one in systemd.service(5), condensed:
| Exit cause | no | always | on-success | on-failure | on-abnormal | on-abort | on-watchdog |
|---|---|---|---|---|---|---|---|
Clean exit: code 0, or SIGHUP, SIGINT, SIGTERM, SIGPIPE | No | Yes | Yes | No | No | No | No |
| Unclean exit code (1 to 255) | No | Yes | No | Yes | No | No | No |
Unclean signal (SIGKILL, segfault) | No | Yes | No | Yes | Yes | Yes | No |
| Timeout (start, stop, reload) | No | Yes | No | Yes | Yes | No | No |
| Watchdog timeout | No | Yes | No | Yes | Yes | No | Yes |
The manual calls on-failure “the recommended choice for long-running services”. Use always only for programs that should never exit on their own, such as a bot. Two details trip people up: a clean SIGTERM from outside is not a failure, and Type=oneshot services reject always and on-success.
Why your service stops restarting: the start-limit arithmetic
Every start counts against a rate limit, including restarts and your own systemctl start. The defaults in systemd-system.conf(5) are 5 starts per 10 seconds, and RestartSec= defaults to 100 ms. So a unit with only Restart=on-failure and an app that crashes at once burns its 5 starts in about half a second, and systemd gives up:
myapp.service: Start request repeated too quickly.
myapp.service: Failed with result 'start-limit-hit'.
The limit can only trip if the burst fits inside the interval. Our arithmetic: with a crash after R seconds of running, consecutive starts are RestartSec + R apart, so the start that exceeds the burst comes StartLimitBurst × (RestartSec + R) seconds after the first one. The limit trips only if that is shorter than StartLimitIntervalSec. That gives four useful profiles:
| Goal | Settings | Crash loop (app dies at once) | Occasional crashes |
|---|---|---|---|
| systemd defaults | Restart= only | 5 starts in about 0.4 s; the 6th, at about 0.5 s, is refused and the unit stays failed | Restarted |
| Retry forever | RestartSec=, default limits | Never trips (5 × 5 s = 25 s, longer than 10 s): about 17,280 attempts a day | Restarted |
| Give up on a tight loop | RestartSec= plus StartLimitIntervalSec= and StartLimitBurst= in [Unit] | 10 starts in 45 s; the 11th, at about 50 s, is refused | Restarted while each run lasts more than about 25 s (10 × 30 s = 300 s) |
| Back off, keep trying (systemd 254+) | RestartSec=, RestartSteps=, RestartMaxDelaySec= | Waits of 2, 3.9, 7.8, 15.4 and 30.4 s, then 60 s per attempt: at most 1,440 a day | Restarted, but each crash climbs one step: from the sixth automatic restart on, every wait is 60 s |
The quick-start unit uses the third profile: a broken config fails fast and visibly, while a crash every few minutes still gets restarted. The fourth needs RestartSteps= and RestartMaxDelaySec=, added in systemd 254: fine on Ubuntu 24.04 (255) and Debian 13 (257), ignored with a warning on Debian 12 (252). Backoff counts every automatic restart since the last manual start, not only the latest crash loop: systemd’s service code (v255) resets the counter, which systemctl show myapp -p NRestarts prints, only on a manual start or systemctl reset-failed.
The [Service] trap: StartLimitIntervalSec= and StartLimitBurst= are [Unit] options. systemd’s own parser (load-fragment-gperf.gperf.in, v255) still accepts the legacy spellings StartLimitInterval= and StartLimitBurst= under [Service], but not StartLimitIntervalSec=. Put both under [Service] and you get your burst with the default 10-second interval, plus one line in the journal: Unknown key name 'StartLimitIntervalSec' in section 'Service', ignoring. systemd-analyze verify prints the same warning, so run it.
After you fix whatever caused a crash loop, clear the counter and start again. StartLimitIntervalSec=0 turns the limit off entirely:
sudo systemctl reset-failed myapp.service
sudo systemctl start myapp.service
How to harden a systemd service
The four sandbox lines in every template cost nothing at runtime and shrink what a bug or a compromised dependency can touch. Each comes from systemd.exec(5), which recommends ProtectSystem= and ProtectHome= for all long-running services:
| Option | What it does | What it can break | Fix |
|---|---|---|---|
NoNewPrivileges= | The process and its children can never gain privileges through setuid, setgid or file capabilities | An app that shells out to sudo or another setuid helper | Do that step outside the service |
ProtectSystem= | Mounts the whole file system read-only for the service, except /dev, /proc and /sys | Read-only file system (Python OSError: [Errno 30], Node.js EROFS) when it writes | StateDirectory=, CacheDirectory=, LogsDirectory= or ReadWritePaths= |
ProtectHome= | Makes /home, /root and /run/ empty and inaccessible | Code, a venv or a Node.js install under /home: 203/ or 200/ | Move it to /opt, or use ProtectHome= |
PrivateTmp= | Gives the service its own /tmp and /var/, deleted when it stops | Sharing a socket or file in /tmp with another program | RuntimeDirectory= for /run/ |
ReadWritePaths= | Allow-lists one writable path under ProtectSystem= | status= if the path does not exist | Create it first, or write -/srv/ to skip it when missing |
StateDirectory= | Creates /var/ owned by User= and sets $STATE_ | If the folder exists with another owner, systemd changes ownership recursively | Point it at a folder used only by this service |
Then measure. systemd-analyze security scores a service from 0.0 (tight) to 10.0 (wide open) and lists every setting it checked. The manual warns the number is an estimate of systemd’s own sandboxing, not proof that a service is or is not vulnerable:
systemd-analyze security myapp.service
To tighten further, add the options below one or two at a time with sudo systemctl edit myapp.service, restart, exercise the app, and read the journal. Test each step: an empty CapabilityBoundingSet= removes every capability:
[Service]
PrivateDevices=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictSUIDSGID=yes
RestrictNamespaces=yes
LockPersonality=yes
CapabilityBoundingSet=
SystemCallArchitectures=native
Two related options belong in the same review. DynamicUser=yes allocates a throwaway user each start and implies ProtectSystem=strict, ProtectHome=read-only, a private /tmp and NoNewPrivileges=yes, so any state must live in StateDirectory=. MemoryMax= and CPUQuota= cap resources so a runaway process cannot starve SSH; the AI agent sandbox shows both with sizing for a small VPS.
How to read systemd service logs with journalctl
Everything a service writes to stdout and stderr goes to the journal by default, with timestamps, across restarts. Use sudo, or add your user to the adm or systemd-journal group, to read system services’ logs:
| Goal | Command |
|---|---|
Follow live, like tail -f | sudo journalctl -u myapp -f |
| Last 50 lines, no pager | sudo journalctl -u myapp -n 50 --no-pager |
| Jump to the end with explanations (the usual first look at a failure) | sudo journalctl -xeu myapp |
| A time window | sudo journalctl -u myapp --since "1 hour ago" or --since today --until "10: |
| This boot, or the previous one | sudo journalctl -u myapp -b, -b -1 |
| Only the message text | sudo journalctl -u myapp -o cat |
| Only warnings and errors | sudo journalctl -u myapp -p warning |
| A user service | journalctl --user -u myapp |
| Disk used by the journal, and trimming it | journalctl --disk-usage, sudo journalctl --vacuum-time= |
A -p filter only sees priorities the app set. Plain print() or console.log() output is stored at the info level, so a Python traceback does not count as an error; search the text with grep instead. Logs that never appear usually mean output buffering (see PYTHONUNBUFFERED above). The Discord bot guide walks through reading a real bot’s tracebacks.
Systemd timers instead of cron: OnCalendar examples
A timer is a second unit that starts a service on a schedule. Use it for anything you would put in a crontab: cleanups, reports, a nightly restic backup or a daily game-server restart. The job is a Type=oneshot service with no [Install] section. This one deletes the demo app’s temporary files older than 7 days:
# /etc/systemd/system/myapp-cleanup.service
[Unit]
Description=Delete myapp temp files older than 7 days
[Service]
Type=oneshot
User=myapp
Group=myapp
ExecStart=/usr/bin/find /var/lib/myapp -name '*.tmp' -mtime +7 -delete
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
ReadWritePaths=/var/lib/myapp
systemd removes the single quotes around '*.tmp' and does no globbing, so find receives the pattern itself, as it should. The timer has the same name with .timer, which is how it knows which service to start:
# /etc/systemd/system/myapp-cleanup.timer
[Unit]
Description=Run myapp-cleanup every day at 03:30
[Timer]
OnCalendar=*-*-* 03:30:00
Persistent=true
RandomizedDelaySec=15min
[Install]
WantedBy=timers.target
Check the schedule, enable the timer (not the service), and run the job once by hand to test it:
systemd-analyze calendar --iterations=3 '*-*-* 03:30:00'
sudo systemctl daemon-reload
sudo systemctl enable --now myapp-cleanup.timer
systemctl list-timers myapp-cleanup.timer
sudo systemctl start myapp-cleanup.service
sudo journalctl -u myapp-cleanup.service -n 20 --no-pager
Persistent=true runs a missed job at the next boot if the server was off at 03:30; it only works with OnCalendar=. RandomizedDelaySec= spreads starts over 15 minutes so several jobs do not hit the disk at once. Without it, AccuracySec= still lets systemd shift the start by up to 1 minute (the default) to batch wake-ups.
Cron to OnCalendar translations
| crontab line | OnCalendar= value | Meaning |
|---|---|---|
*/5 * * * * | *:0/ | Every 5 minutes, on the clock (:00, :05, …) |
0 * * * * | hourly | Every hour on the hour (*-*-* *:00:) |
30 3 * * * | *-*-* 03: | Every day at 03:30 |
0 0 * * * | daily | Every day at midnight |
0 4 * * 1 | Mon *-*-* 04: | Mondays at 04:00 |
0 9 * * 1-5 | Mon..Fri *-*-* 09: | Weekdays at 09:00 |
0 2 1 * * | *-*-01 02: | The 1st of each month at 02:00 |
@reboot | OnBootSec= (not OnCalendar=) | Once, 2 minutes after boot |
Times are in the server’s time zone, which timedatectl shows; many VPS images default to UTC. Append a zone to pin a job to a fixed local time, for example OnCalendar=*-*-* 03:30:00 Europe/Istanbul for a server in our Istanbul location; timedatectl list-timezones prints every valid name.
Is a systemd timer better than cron?
| cron | systemd timer | |
|---|---|---|
| Setup | One line in a crontab | Two small files |
| Server was off at run time | The run is skipped | Persistent= runs it at the next boot |
| Previous run still going | Starts another copy | Leaves the running service alone; no overlap |
| Output | Mailed to the crontab owner, or lost without a mail setup | In the journal, per unit: journalctl -u NAME |
| Sandbox, user, memory cap | Not built in | Every [Service] option: User=, ProtectSystem=, MemoryMax= |
| Run it now to test | Copy the command into a shell, with a different environment | systemctl start NAME., same environment as the schedule |
| Spread start times | Not built in | RandomizedDelaySec= |
Cron is the better fit for a quick one-liner, for crontabs you already maintain, and for scripts that must also run on systems without systemd. For anything you need to debug later, the journal and the no-overlap rule usually win. Lab 7 of our Linux labs schedules a health-check script with a timer instead of cron, on a throwaway server.
User services vs system services
Everything above is a system service, managed by PID 1 with sudo systemctl. Each logged-in user also gets a personal service manager that runs units from ~/.config/systemd/user/ without root. That suits a developer’s own tools, but it changes several rules:
| System service | User service | |
|---|---|---|
| Unit location | /etc/ | ~/.config/ |
| Managed with | sudo systemctl … | systemctl --user …, no sudo |
| Runs as | Any account via User= (root by default) | Always that user; User= cannot switch |
WantedBy= | multi-user. | default. |
| Starts at boot | Yes, once enabled | Only with lingering enabled; otherwise it starts at login and stops after logout |
| Logs | sudo journalctl -u NAME | journalctl --user -u NAME |
| Sandboxing | All options | ProtectHome=, PrivateTmp= and similar need unprivileged user namespaces; some options are system-only |
To keep a user service running without an open SSH session, enable lingering, which starts the user’s manager at boot and keeps it after logout (loginctl(1)):
mkdir -p ~/.config/systemd/user
nano ~/.config/systemd/user/myapp.service
systemctl --user daemon-reload
systemctl --user enable --now myapp.service
sudo loginctl enable-linger "$USER"
From root, sudo -iu alice systemctl --user status fails with Failed to connect to bus, because that shell has no user session. Since systemd 248, connect to her manager directly: sudo systemctl --user -M alice@ status myapp. For anything that must survive reboots and be sandboxed, a system service with User=alice is simpler.
How to debug a failed systemd service
Work from the outside in. These five commands find the cause of almost every failed unit:
systemctl status myapp: theActive:line, theMain PIDline withcode=exited, status=…, and the last 10 log lines.sudo journalctl -xeu myapp: the full error, with systemd’s explanation of the result.sudo systemd-analyze verify /etc/systemd/system/myapp.service: typos, unknown keys and missing executables.systemctl cat myapp: the unit and drop-ins systemd actually loaded. It warns if the file changed on disk since the lastdaemon-reload.systemctl show myapp -p ExecStart -p User -p Restart -p NRestarts: the effective values, including how often it has restarted.
The status= number points at the step that failed. Codes 200 to 245 come from systemd itself, before or while it starts your program; the full list is the Process Exit Codes table in systemd.exec(5).
| What you see | Likely cause | Fix |
|---|---|---|
status= | The ExecStart= program is missing, not executable, lacks a #! line, or sits under /home with ProtectHome= | Check the path with ls -l; chmod +x; move the program to /opt or /usr/ |
status= | WorkingDirectory= is missing or unreadable for User= | Create it, fix ownership, or prefix the path with - |
status= | The User= account does not exist | Create it (step 1) or use DynamicUser= |
status= | A path in ReadWritePaths= or ReadOnlyPaths= does not exist | Create the path, or prefix it with - |
status= (or another small number) | Your app started and exited with an error | Read the journal lines just above the exit |
Failed to load environment files | The EnvironmentFile= path is wrong | Fix the path, or prefix it with - if the file is optional |
Read-only file system, EROFS | ProtectSystem= is doing its job | Write to $STATE_ or add a ReadWritePaths= entry |
ModuleNotFoundError (Python) | ExecStart= uses the system Python, not the venv | Use /opt/ |
Start request repeated too quickly | Crash loop hit the start limit | Fix the cause, then systemctl reset-failed (arithmetic) |
Loaded: bad-setting | A line systemd cannot accept, such as a relative ExecStart= path | Run systemd-analyze verify and fix the line it names |
changed on disk. Run 'systemctl daemon-reload' | You edited the file but systemd still uses the old version | sudo systemctl daemon-reload, then restart |
| Works in your shell, fails as a service | A variable from your shell profile, or a file only your user can read | Set it with Environment=; run the command as sudo -u myapp to compare |
| Not running after a reboot | Started but never enabled, or no [Install] section | systemctl is-enabled myapp; sudo systemctl enable myapp |
| Timer never fires | You enabled the .service instead of the .timer | sudo systemctl enable --now NAME.; check systemctl list-timers |
Edit, override or remove a service
To change a unit, prefer a drop-in over editing the file. sudo systemctl edit myapp opens a drop-in file, override.conf under /etc/systemd/system/myapp.service.d/, in your editor and reloads systemd when you save. This is the safe way to change units that packages or installers own, such as the one the Ollama install script writes, because a package upgrade or a rerun of the installer replaces the main file. List-type settings add up across files, so clear ExecStart= before you set a new one:
[Service]
ExecStart=
ExecStart=/opt/myapp/venv/bin/python /opt/myapp/app.py --workers 2
Environment=LOG_LEVEL=debug
sudo systemctl revert myapp deletes all drop-ins and returns to the main file. To remove a service completely:
sudo systemctl disable --now myapp.service
sudo systemctl clean --what=state myapp.service
sudo systemctl reset-failed myapp.service
sudo rm -f /etc/systemd/system/myapp.service
sudo rm -rf /etc/systemd/system/myapp.service.d
sudo systemctl daemon-reload
systemctl clean --what=state deletes the StateDirectory= data, so skip that line if you want to keep /var/lib/myapp. It only works on a stopped unit, which is why disable --now comes first. reset-failed runs while the unit is still loaded, so a failed state does not linger in systemctl --failed.
Practice on a throwaway server, then run it 24/7
Unit files are easiest to learn where breaking things costs nothing: kill the process, misspell a path, reboot, and watch what systemd does. An hourly VPS is built for that. Deploy one, run the five steps and the troubleshooting rows on purpose, then delete it.
Cost: a Quartz Q1 (1 vCPU, 1 GB RAM) is billed at $0.01/hour, so a two-hour systemd lab costs $0.02. Every server is billed by the hour: the plan’s hourly rate is deducted from the server’s prepaid balance for every hour it exists, powered on or off, until you delete it. A stopped server is still billed, because its vCPU, memory, disk and IP addresses stay reserved for you; only deleting the server stops billing. Ordering a server also takes an initial credit, prepaid and used for that server’s hours; the hourly billing guide shows how the balance and top-ups work. Before you delete, copy anything you want to keep with the delete VPS checklist.
For the real deployment, leave the server running; there is no plan to switch to. It is still billed by the hour, and charges stop once they reach the plan’s monthly price, so a Q1 never costs more than $5.00 in a billing period (one month from your order date). That is what a monthly VPS is here: an always-on server with an automatic monthly cap, no contract and no monthly prepayment. The hourly vs monthly guide explains why a capped hourly bill never costs more than a monthly one, and the cost calculator prices your own schedule.
| What the service runs | Starting plan | Monthly cap | Reasoning |
|---|---|---|---|
| One small API, bot or worker (Python, Node.js or Go) | Quartz Q1 | $5.00 | One app process plus Ubuntu fits in 1 GB for small apps; check the Memory: line in systemctl status |
| App plus Caddy for HTTPS and a database | Quartz Q2 | $10.00 | Three services share RAM; a PostgreSQL server wants its own headroom |
| Several services, or a game server | Quartz Q4 | $15.00 | 2 vCPUs keep one busy service from slowing the rest |
If your app already ships as a container, Docker’s restart policies replace most of this guide; see how to install Docker on a VPS. For one or two programs, a unit file is one layer fewer.
Deploy this setup
Run your Python, Node.js or Go app 24/7 under systemd
Quartz Q1 · 1 shared vCPU · 1 GB RAM · 25 GB NVMe · Istanbul
- Per hour$0.01/hour
- Per day (24 h)$0.24/day
- Monthly cap$5.00/monthFor this job
Starts with a $5 initial credit, which goes into the server’s balance and pays for its hours.
Billed by the hour, never more than $5.00 per billing period. Delete the server and billing stops.



