Data Management

How app data is stored, backed up, and protected.

Where Data Lives

All app data is stored in a single root directory. By default this is the folder where the app was installed, but during development it is the source code directory. Everything under this root is self-contained — moving the root folder moves all your data.

Folder / File	Contents
data/pipeline.json	All pipeline entries (business info, step statuses, URLs, notes).
data/queue.json	Businesses discovered but not yet evaluated in the Swiper.
data/rejected.json	Businesses you pressed Pass on. Never shown again.
data/saved_searches.json	Saved Discovery presets (niche, coordinates, radius, chunks).
data/temp/	Temporary prompt files created during AI operations. Auto-cleaned after each run.
downloaded_sites/	Raw website copies downloaded by Cyotek WebCopy. One subfolder per slug.
edited_sites/	AI-rebuilt React sites. One subfolder per slug. This is the working directory for all AI operations.
example_sites/	Sites downloaded via the ★ EXAMPLE action, organized by niche subfolder.
Site Backups/	Automatic backups created after each Form/Implement step. Organized as Site Backups/<slug>/Edition N/.

JSON State Store

Hippopotamoose does not use a database. All pipeline, queue, and rejected data is stored as JSON arrays in flat text files. This design offers:

• Human-readable state — you can open pipeline.json in any text editor
• Easy portability — copy the folder, move the data
• Simple backup and restore

Crash Safety & Atomic Writes

To prevent data corruption if the app crashes during a save, all JSON writes use an atomic process:

1. New content is written to a .tmp file beside the real file.
2. The OS flushes the file to disk (fsync).
3. The .tmp file replaces the real file atomically using os.replace().

This means you will never have a half-written, corrupted JSON file. If the process is interrupted between steps, the previous version of the file is preserved intact.

Stuck Step Recovery

If the app exits while a pipeline step shows in_progress (e.g. the AI terminal was open), that step would be permanently stuck. On every startup, the app scans for any in_progress steps and resets them to not_started. You can safely re-run any step after a crash.

Automatic Backups

A site backup is automatically created every time the Form / Implement step completes successfully. Backups are stored in Site Backups/<slug>/Edition N/ where N increments with each revision.

Backups are full folder copies — they include all site files at that point in time. You can restore a previous edition by manually copying the folder back to edited_sites/<slug>/.

Corrupted Files

If a JSON data file cannot be parsed (e.g. due to disk error or manual editing), the app renames the corrupt file with a .corrupt suffix and creates a new empty file in its place. This prevents a startup crash. You can inspect the .corrupt file with a text editor to attempt manual recovery.