WarpEngine Batocera Store

warp-engine-batocera-store pulls games from a WarpEngine catalog onto a Batocera box, with EmulationStation metadata and box art, and gives the store its own entry in the EmulationStation menu. This is the engine only: it knows the WarpEngine API and nothing about any particular site. The host, the store name and where games land come from a config.json in a separate store repository. The half that is not about Batocera is warpstore, shared with the RetroArch engine.

warpstore                           the shared core — warpstore.py
        ▲
warp-engine-batocera-store          this engine — store.py, install.sh
        ▲
        │ config.json
        │
  ┌─────┴─────┬───────────────┐
  │           │               │
ttg-…       my-…            other-…       store repositories
  │           │               │
  ▼           ▼               ▼
teletypegames.org   my.example   other.example

The reference store is ttg-batocera-store.

Repository: https://git.teletypegames.org/stores/warp-engine-batocera-store Catalog side: WarpEngine Shared core: warpstore Sibling engine: RetroArch store engine Batocera: https://batocera.org

What you get

• Its own menu entry — one carousel entry named after the store, holding a folder per platform. Written as an es_systems_<id>.cfg overlay; the launch command and emulator list are copied off the box's own systems, so nothing about emulators is hardcoded.
• Site-agnostic — any host mounting WarpEngine can have a store; only the config changes.
• Several stores per box — each owns a ROM folder, a menu entry and its own state, so one store's prune cannot reach another's games.
• The box is left alone — the games live in the store's folder, not in the box's systems. Nothing outside it can be touched, and inside it only what state.json records is ever deleted.
• Safe next to EmulationStation — the in-ES sync defers the gamelist write until ES exits, and the tags ES owns (favourites, play counts) are carried over on every rewrite.
• Zero dependencies — Python 3 standard library only. Batocera ships python3 and no pip.

How it works

"<store name>" ▸ "Update <store name>"
        │
        ├─ GET /api/software                        the whole catalog
        ├─ keep platforms this box can run          c64 → c64, tic80 → tic80
        ├─ pick the newest non-dev release          that carries the right asset
        ├─ GET /api/download?path=<asset>           .prg / .tic into the store's folder
        ├─ GET <software.imageUrl>                  box art
        ├─ write gamelist.xml                       title, desc, author, image
        └─ write es_systems_<id>.cfg                the store's menu entry

The store's own menu entry

Everything the store installs lives in one folder of its own, /userdata/roms/<subfolder>, with a directory per platform inside it. An es_systems_<id>.cfg in EmulationStation's user config declares one ES system per platform, all carrying the same <group> — and ES turns a group into a single carousel entry holding a folder per system:

main carousel:  …  ▸  Teletype Games  ▸  ┌ Commodore 64
                                         ├ TIC-80
                                         ├ Ports
                                         └ Update Teletype Games

Nothing about emulators is hardcoded. The engine reads the box's own /usr/share/emulationstation/es_systems.cfg and copies the launch command, the extensions, the platform, the theme and the emulator list off the system it borrows — a file Batocera generated for this image, so it lists exactly the emulators the box has. The one thing rewritten is %SYSTEM%: ES replaces it with the name of the system being launched, which for us would be ttg-c64, a name configgen has never heard of. The box's own system name goes in instead, so a game of ours is launched exactly as the box would launch it.

Five details, all of them EmulationStation's, and two of them cost an afternoon on a real box:

• The menu entry is declared, not implied. ES would invent the group parent itself, but its theme folder would then be the group's name, which no theme has — and HideUniqueGroups (on by default) dissolves a group with a single system in it unless a system by that name exists. The declared parent fixes both: it carries a <theme> (ports by default, present in every theme) and keeps the entry in place on a box that only has c64 games.
• A system with no games of its own is dropped, not merely hidden: loadSystem logs System "..." has no games! Ignoring it. and throws it away. So the entry cannot be an empty shelf — it is the store/ folder, and the updater in it is the game that keeps it alive. That is also why Update … sits directly in the entry rather than in a subfolder.
• The entry must not be the directory above the systems. ES marks an enumerated directory with a <dir>/* entry in its file cache, and adds the mark before reading the contents; from then on any path under it that is not itself cached answers "does not exist" (FileSystemUtil.cpp, getCacheEntry). Systems load in a thread pool, so an entry sitting above the systems loses that race for a random couple of them, and ES drops those with System "..." path does not exist ! — a different couple on every start, which looks exactly like a theme problem. store/ is their sibling, so there is no race to lose.
• An empty system does not show, so the entry never has a dead end in it.
• A new system needs an ES restart, which the engine does after a sync that added one.

If the box already has a system named like the store id — a store called nes, say — the entry is called <id>-store instead and the log says so: an overlay whose <name> matches an existing system modifies that system rather than adding one.

Its logo

EmulationStation resolves a system's carousel logo through the theme (SystemData::getProperty("image") asks the theme for system/logo, and the theme builds the path from the system's theme folder — carbon tries art/logos/${system.theme}.png, then .svg). There is no user-level override, so a custom system either borrows a theme folder that already has artwork, or the file goes where the themes look. Both are supported:

• without menu.logo the entry borrows ports, which every theme has, so it has an icon — just not the store's;
• with menu.logo (a path or URL, or a list of them for several formats) the engine installs it into each theme's logo folder. It finds that folder by looking for the logos of systems every theme has (ports, snes, nes, …) rather than by knowing the layouts, writes the file as <menu.theme>.<ext> — menu.theme then defaults to the store id, so nothing of the theme's own is overwritten — and records every path in state.json, so an uninstall takes exactly those back out and a name clash is reported and skipped;
• a theme it cannot reach shows the store's name as text, which is EmulationStation's own fallback when a logo image is missing.

Batocera wrinkle: themes that ship with the system image are under /usr/share, and Batocera's root is a RAM overlay — the copy shows up immediately but is gone after a reboot. The next sync puts it back; batocera-save-overlay makes it permanent. Themes in /userdata/themes keep it.

For the old behaviour — games inside the box's own systems, merged into its gamelists — set emulationstation.menu.mode to merge.

Upgrading a store installed before the menu entry

Engine 4.0 is the layout change. The first sync after the upgrade takes the old install off the box — ROMs, box art, port payloads, our nodes in the box's gamelists, the empty folders, the Ports entry — and then downloads everything again into the store's own folder. Files are not carried across on purpose: the catalog is small, and a half-moved install is worse than a slightly longer sync. -n shows the whole thing without doing any of it.

Installing a store

The installer takes the store as a parameter, so it works with any config:

curl -fsSL https://git.teletypegames.org/stores/warp-engine-batocera-store/raw/branch/master/install.sh |
  STORE_CONFIG=https://git.example.org/tools/my-store/raw/branch/master/config.json sh

From a checkout, with a local config:

scp -r warp-engine-batocera-store root@batocera:/tmp/
ssh root@batocera 'STORE_CONFIG=/tmp/my-config.json /tmp/warp-engine-batocera-store/install.sh'

It reads store.id from the config, installs the engine and the config into /userdata/system/batocera-store/<id>/, writes a <id>-store launcher, and runs the first sync — which is what creates the menu entry. EmulationStation has to restart to see it.

Writing a store repository

Three files:

my-batocera-store/
├── config.json     the store: URL, name, subfolder, platform mapping
├── install.sh      a wrapper that hands that config to the engine installer
└── README.md

install.sh in full:

#!/bin/bash
set -euo pipefail
ENGINE_RAW_BASE="${ENGINE_RAW_BASE:-https://git.teletypegames.org/stores/warp-engine-batocera-store/raw/branch/master}"
export STORE_CONFIG="${STORE_CONFIG:-https://git.example.org/tools/my-batocera-store/raw/branch/master/config.json}"
curl -fsSL "$ENGINE_RAW_BASE/install.sh" | sh

Pick a store.id and a paths.subfolder nobody else uses — they are what keeps two stores on the same box out of each other's way, and the id is what names the menu entry and its es_systems file.

Use

On the device. "\<store name>" ▸ "Update \<store name>". The script downloads anything new, then restarts EmulationStation. A script launched by ES gets no console, so the output goes to store.log in the store home.

Over SSH, through the launcher the installer wrote:

S=/userdata/system/batocera-store/example-store

$S list                 # compatible catalog entries; * installed, ^ update available
$S sync                 # download everything new, refresh gamelists and the menu
$S sync blessingofra    # just one title (never prunes)
$S -n sync              # dry run
$S remove c64:demo      # uninstall (bare name works too)
$S purge                # uninstall everything this store installed
$S config               # effective configuration, and where the menu entry lives

Global flags go before the subcommand: $S --roms-root /tmp/roms sync.

Uninstalling

ssh root@batocera 'curl -fsSL https://git.teletypegames.org/stores/warp-engine-batocera-store/raw/branch/master/uninstall.sh | sh'

The engine does the removing, not the shell script: purge first, then any Ports entry, the launcher and the store home. Only state.json knows which ROMs, box art, port payloads, gamelist entries and ES systems were ours — and if it cannot finish (an unmounted ROMs root, say) nothing else is touched, so you are never left with the files but without the engine that knows them.

In the store's own layout the uninstall is total: the ROM folder, the gamelists and the es_systems_<id>.cfg all go, and the box is as it was. In merge mode these stay behind on purpose:

• ROMs the user put in the store's subfolder — a directory goes only when empty, so one stray file keeps it. ports/.data also stays: another store may keep a payload there.
• the gamelists — they belong to the box. Our <game> and <folder> nodes go; the user's own games, play counts and favourites are written back untouched. A gamelist we created and left empty is deleted, since it was never theirs.
• gamelist.xml.<store id>-backup, listed at the end of the run: the gamelists as we first found them.

A dry run lists files but not directory removals: it cannot know which directories will end up empty.

To take every store off a machine — both engines, the shared core, the launchers — use the script in warpstore:

curl -fsSL https://git.teletypegames.org/engines/warpstore/raw/branch/master/uninstall.sh | sh

Where things land

/userdata/system/batocera-store/
├── example-store                       launcher for the store below
└── example/
    ├── store.py, warpstore.py          the engine and its shared core
    ├── config.json                     the store that selected it
    └── state.json, catalog.json, store.log

/userdata/system/configs/emulationstation/es_systems_example.cfg   the menu entry

/userdata/roms/example/                 the store's own ROM folder
├── c64/
│   ├── blessingofra-2.0.0.prg, rabbit-1.0.0.prg, …
│   ├── images/blessingofra.png, …
│   └── gamelist.xml
├── ports/
│   ├── <name>.sh, images/<name>.png, .data/<name>/
│   └── gamelist.xml
└── store/                             the menu entry itself
    ├── update.sh                       the sync, as the menu starts it
    └── gamelist.xml

The box's own system folders are not touched at all. In merge mode — the old behaviour, still available — the same content goes to <roms_root>/<system>/<subfolder>/, the first gamelist.xml touched is copied to gamelist.xml.<store id>-backup, and every merge rewrites only the nodes under that store's subfolder, so two stores can share one gamelist.

Configuration

config.json in the store home; the template is config.example.json in the repository.

Platform mapping

A platform is installable when its release asset is a file a system this box has can boot directly — "has" meaning declared in the box's own es_systems.cfg, which is an emulator that got built into this image (a ROM folder can be there with nothing behind it):

The others (ebitengine, love, godot, bevy, phaser) ship html and per-OS zips, not a ROM a system launches, so they are not mapped by default. Adding one is a config edit:

"platforms": { "godot": { "system": "godot", "kind": "linux_x64", "ext": ".zip", "enabled": true } }

How a sync stays safe next to EmulationStation

EmulationStation keeps gamelists in memory and writes them back on exit, so a merge done while it runs can be clobbered. Launched from the menu (sync-from-es) the engine downloads first, then hands the merge to a detached apply-gamelists --wait-pid <es-pid> child that waits for the old ES process to die. From SSH, with no ES running, the merge is inline. The es_systems overlay needs none of that: ES reads it and never writes it.

A re-run is idempotent: existing files are left alone, a release bump (1.1 → 2.0.0) deletes the old asset before fetching the new one, and a gamelist or es_systems file whose content has not changed is not rewritten at all. Downloads go through GET /api/download rather than /file/, so they count towards the catalog statistics.

A sync aborts when it cannot tell which systems the box has — an unreadable es_systems.cfg with an unmounted /userdata/roms behind it would otherwise look like "nothing is compatible any more" and prune every installed game.

State

state.json records what the store installed, keyed <system>:<name>, so the same software name on two systems stays two entries. layout says which directory layout those records belong to: that is how the upgrade to the store's own folder knows to run once — it removes the old install, and the sync downloads it again into the new place. A version: 1 file from the pre-split ttg-store is migrated on first run.