---
title: Backup
url: 'https://wiki.teletypegames.org/infrastructure/backup'
markdown: 'https://wiki.teletypegames.org/infrastructure/backup.md'
lang: en
date: '2026-09-14'
description: 'Two chains: a Makefile-based backup for database dumps and Gitea repository mirrors, followed by an rsync-based mirror of the whole file tree to an off-site NAS. This has been the setup since 2026-10-16. Before that the second chain was restic; we retired it: the backup is simpler now, at the cost…'
taxonomy:
  tag:
    - howto
---

# Backup

> Two chains: a Makefile-based backup for database dumps and Gitea repository mirrors, followed by an rsync-based mirror of the whole file tree to an off-site NAS. This has been the setup since 2026-10-16. Before that the second chain was restic; we retired it: the backup is simpler now, at the cost of having no versioning.

# Daily schedule

| Time | What | Tool |
|---|---|---|
| 04:00 | Redmine (MariaDB), catalog and Gitea (MySQL) dumps → mirror every Gitea repository → delete dumps older than 7 days | `make backup` (cron, `infra/backup-scripts`) |
| 05:00 | Matomo, ResourceSpace and Uptime Kuma dumps → Woodpecker and Grafana SQLite copies → package list → full file mirror to the NAS | `vps-backup-sync` (systemd timer, `infra/host-config`) |

The mirror runs an hour later so it carries the fresh dumps and repository mirrors too. The 04:00 chain does not sync on its own; everything reaches the NAS with the 05:00 mirror.

# Chain 1: dumps and repository mirrors (Makefile)

Environment variables in the `Makefile`:

| Variable | Meaning |
|---|---|
| `BACKUP_BASE` | Root of all backups (default `/srv/backup`) |
| `BACKUP_KEEP_DAYS` | Retention for DB dumps (default `7`) |
| `DATABASE_BACKUP_BASE` | Where SQL dumps go |
| `REPOSITORY_BACKUP_BASE` | Where mirrored repos go |
| `GITEA_SERVER`, `GITEA_TOKEN` | Gitea API for listing repositories; the token comes from an `.envrc` file the Makefile reads itself (`-include`), so it works under cron too |

## Targets

```bash
make backup                          # db_redmine -> db_catalog -> db_gitea -> repos -> db_cleanup
make db_redmine db_catalog db_gitea  # databases only
make repos                           # update the git mirrors
make db_cleanup                      # drop expired dumps
```

## Database backups

Each dump runs inside the service's container. Dumps are timestamped into `/db_backup` in the container, which is mounted to the host's `DATABASE_BACKUP_BASE`.

| Service | How | Note |
|---|---|---|
| Redmine (MariaDB) | `database/mariadb_backup.sh` | `MARIADB_USER`, `MARIADB_PASSWORD`, `MARIADB_DATABASE` in the container |
| Catalog (MySQL, `softwares`) | `database/mysql_backup.sh` | `MYSQL_ROOT_PASSWORD`, `MYSQL_DATABASE` in the container |
| Gitea (MySQL, `gitea`) | `db_gitea` in the Makefile: `mysqldump --single-transaction --routines --triggers` | **Since 2026-10-16.** Before that the Gitea database (users, permissions, issues, PRs, OAuth apps, registry metadata) was not backed up anywhere. |
| Wiki (PostgreSQL) | `database/pg_backup.sh` | **Retired 2026-09-11** – the Wiki.js database is gone; the script is legacy. |

## Repository backups

`make repos` lists every private and public repository from the Gitea API, then per repo: `fetch --all --prune` + `pull` when it exists locally, `git clone` when it does not.

## Cleanup

`make db_cleanup` groups dumps by database prefix and deletes files older than `BACKUP_KEEP_DAYS`. A safety check keeps at least one backup per database, even when it is older than the retention window.

# Chain 2: file mirror (vps-backup-sync)

The `/usr/local/sbin/vps-backup-sync` script (source in the `infra/host-config` repository) runs as root, started by `vps-backup-sync.timer`.

**Dumps at the start of the script:** Matomo (`mariadb-dump`), ResourceSpace (`--all-databases`), Uptime Kuma (`sqlite3 .backup`). For the live Woodpecker and Grafana SQLite databases, `sqlite3 .backup` writes a consistent copy next to the file (`woodpecker-backup.sqlite`, `grafana-backup.db`). The list of installed packages goes to `/srv/backup/system/` (`dpkg --get-selections`, `apt-mark showmanual`).

**What it mirrors:** `/srv/stacks`, `/srv/stacks_disabled`, `/srv/backup`, `/srv/host-config`, `/srv/presentation`, `/srv/dockge`, the host's `/etc`, `/root`, `/home`, `/usr/local/sbin`, `/var/spool/cron` and `/var/www`, plus the ResourceSpace filestore and Matomo Docker volumes. About 27 GB in total.

**What it skips:** live database directories (the dumps stand in for them, since a running DB's files are not consistent), `node_modules`, `tmp`, `log`, Gitea search indexes, caches.

**How:** `rsync -aH --numeric-ids --relative --delete` into a mode-700 directory on the NAS, with absolute paths. A non-root user receives on the NAS, so owner, permissions and device data are stored in xattrs (`-M--fake-super`). Copy back with the same option, otherwise every file ends up owned by the NAS user.

**No versioning.** The NAS always holds the latest state only. If something is deleted or corrupted on the server, it is deleted or corrupted on the NAS the next night too. For databases, the 7-day dump rotation gives some history.

# Off-site mirror

The target is an external Synology NAS (DS212j, DSM 6.2, ext4). It only accepts `ssh-rsa` keys, has no SFTP subsystem and cannot do Btrfs snapshots, so NAS-side versioning would only be possible with hardlinked copies.

**Open question:** the mirror is writable from the server, so a compromised host could delete the backup too. Secrets (`/root`, `/etc/shadow`, `.env` files, certificate keys) sit unencrypted on the NAS.

# Restoring to a new server

The exact commands are in the `infra/host-config` README, section „Mentés és visszaállítás”. In short:

1. Copy the whole tree back as root, with `-M--fake-super`, into a temporary directory.
2. Install packages from `/srv/backup/system/apt-manual.txt`, then put `/etc` back selectively and restore `/srv`, `/home`, `/root`, `/var/www`, `/var/spool/cron`, `/usr/local/sbin`.
3. `sudo /srv/host-config/install.sh` for the host's own files.
4. Start the stacks with empty databases, then load the latest dumps (`redmine_production`, `softwares`, `gitea`, `matomo`, `resourcespace`).
5. SQLite: move the `*-backup.*` copies into place with the stack stopped.
6. Docker volumes: restore the ResourceSpace filestore and Matomo volume contents.

# File structure

The backup repository (`infra/backup-scripts`):

```text
.
├── Makefile                # every chain-1 task
└── database/
    ├── mariadb_backup.sh
    ├── mysql_backup.sh
    └── pg_backup.sh        # legacy, no longer runs
```

Chain 2 lives in the `infra/host-config` repository: `files/usr/local/sbin/vps-backup-sync`, `files/etc/systemd/system/vps-backup-sync.{service,timer}`, `files/etc/logrotate.d/vps-backup-sync`. Install with `sudo ./install.sh`.

On the server (`BACKUP_BASE=/srv/backup`):

```text
/srv/backup/
├── database/                     # dumps, 7 days
│   ├── gitea-2026-10-16-02-00.sql
│   ├── redmine_production-2026-10-16-02-00.sql
│   ├── softwares-2026-10-16-02-00.sql
│   ├── matomo-2026-10-16-05-03.sql
│   └── resourcespace-2026-10-16-05-03.sql
├── repositories/                 # mirrors of every Gitea repository
├── system/                       # package list (dpkg, apt-mark)
└── archive-2026-10-15/           # tar.gz archives of volumes removed during cleanup
```

# Verification

- The mirror log on the host (`/var/log/vps-backup-sync.log`): every step ends with an `OK` or `HIBA` (error) line.
- After an error-free run the script sends an Uptime Kuma push; if none arrives for 26 hours, an alert goes to the Discord Alert channel.
- Grafana: the **VPS: Backup** dashboard shows the last run state, dump age and size, mirror size and the backup log lines. The script writes the metrics for node-exporter's textfile collector (`/srv/stacks/observability/textfile/vps_backup.prom`); the log lines are in Loki (`{service_name="cron-jobs", program="vps-backup-sync"}`). Grafana: https://grafana.teletypegames.org (sign in with your Gitea account).
- If the newest dump of a database gets old, Grafana's **Database dump stale** alert goes to Discord.
- Every database must have a fresh file for every day in the dump directory; if a prefix disappears or stops, the chain is stuck.

# Repositories

- https://git.teletypegames.org/infra/backup-scripts
- https://git.teletypegames.org/infra/host-config

'; var prevLevel = 2; for (var i = 0; i < headings.length; i++) { var h = headings\[i\]; var text = h.textContent.trim(); var slug = text.toLowerCase() .replace(/\[áàâä\]/g,'a').replace(/\[éèêë\]/g,'e').replace(/\[íìîï\]/g,'i') .replace(/\[óòôöő\]/g,'o').replace(/\[úùûüű\]/g,'u') .replace(/\[^a-z0-9\]+/g, '-').replace(/^-|-$/g, ''); if (!slug) slug = 'heading-' + i; if (used\[slug\]) { slug += '-' + i; } used\[slug\] = true; h.id = slug; var level = parseInt(h.tagName.charAt(1)); var cls = level > 2 ? ' class="toc-child"' : ''; html += '- [' + text + '](#'%20+%20slug%20+%20')
'; } html += '
'; toc.innerHTML = html; // Scroll-spy: highlight current heading var links = toc.querySelectorAll('a'); var ids = \[\]; for (var j = 0; j < links.length; j++) ids.push(links\[j\].getAttribute('href').slice(1)); function updateActive() { var current = ''; for (var k = 0; k < ids.length; k++) { var el = document.getElementById(ids\[k\]); if (el && el.getBoundingClientRect().top <= 80) current = ids\[k\]; } for (var m = 0; m < links.length; m++) { links\[m\].parentNode.classList.toggle('active', links\[m\].getAttribute('href') === '#' + current); } } window.addEventListener('scroll', updateActive, { passive: true }); updateActive(); })();

---

## Navigation

- Parent: [Infrastructure](https://wiki.teletypegames.org/infrastructure.md)
- Next: [Deployment](https://wiki.teletypegames.org/infrastructure/deployment.md)