Hermes agent state.db too large: shrink it without losing history

Stop the gateway and run hermes sessions optimize-storage. On a database created before Hermes's compact search layout, most of state.db is duplicated search index, and this command removes the duplication without touching your messages. The catch is free space: the command needs about as much free disk as the database already uses. If your disk is nearly full, it refuses. When a hosted Hermes instance of ours reached 97% of its volume on 2026-09-29, we ran the same migration on a copy on another disk, checked it, and swapped it back. state.db went from 14.77 GiB to 3.83 GiB, and all 441,789 messages were kept.

At a glance

Why Hermes state.db gets this big

state.db is a single SQLite file in your Hermes home (~/.hermes/state.db by default). It holds session metadata, every message, and the full-text indexes behind session_search. The Hermes docs say that without pruning it "grows without bound — multi-GB files within weeks were reported on gateway + cron installs."

On a database created before the compact layout, most of that growth is the search index, not the conversations. In the old layout, messages_fts and messages_fts_trigram each store their own copy of the text they index, and the trigram index also covers tool output. On tool-heavy agents, tool output is often the bulkiest text in the database. Pull request #65798, merged on 2026-07-22, introduced the compact v23 layout. Its indexes refer back to the messages table instead of copying it, and the trigram index skips tool rows. The PR reports a real 25 GB database in which 18.9 GB was index data over 3.4 GB of actual text.

The important detail is that the migration is opt-in. New installs start on the compact layout. An existing database stays on the large layout, however many times you upgrade Hermes, until you run the command. That is why long-running agents are the ones that hit this. Upstream reports describe the same shape: a 9.4 GBstate.db with about 447,000 messages saturating disk I/O on v0.19 (#68858), and a database that bloated to 7.66 GB three times in 72 hours (#100108).

Check whether this is your problem

ls -lh ~/.hermes/state.db*
df -h ~/.hermes
hermes sessions optimize-storage

The first line shows the database and its -wal/-shm sidecars. The second shows how much room you have. The third is safe to run as a check. It inspects the real index structure. If there is nothing to migrate, it prints Search index is already on the compact layout — nothing to do. and exits. Otherwise it prints the current size, the free disk, and the space it needs, then asksProceed? [y/N]. Answer N if you are only looking.

Don't wait for hermes doctor to suggest it. Issue #129927, opened on 2026-10-01, reports that doctor never recommends optimize-storage when the database's fts_storage_version marker was written ahead of the real layout. You get generic auto-prune advice instead. The command itself checks the layout directly, so running it is the reliable test.

The fix when you have enough free disk

hermes gateway stop
hermes sessions optimize-storage --yes
hermes gateway start

Stop the gateway first. Current Hermes refuses optimize, optimize-storage andprune while another process holds state.db, and stops withRefusing `hermes sessions optimize-storage`: another process is using … followed by the process holding it. That includes the gateway, the Desktop app, the dashboard and cron. The source explains why it refuses: running the rebuild under live gateways once put every agent into a write refusal until all writers were stopped.--force skips the check. Don't use it to avoid stopping your gateway.

The run is resumable. On a database over 500 MB, the command warns that it may take a while and says it is "safe to Ctrl-C and re-run (it resumes)". Ours was interrupted by a network drop partway through and picked up from its checkpoint. At the end it runs VACUUM, which is the step that actually gives disk space back to the filesystem.

When the disk is already full

This is where the documented fix fails. Before starting, the command checks for free space roughly equal to the current file size. Its source gives the reason: "the new index is built before the old is torn down, and VACUUM needs a full second copy". If it can't get that space, it stops with:

⚠ Not enough free disk to complete safely. Free up space, or run with --no-vacuum
(rebuilds the index but doesn't reclaim space until a later VACUUM).

Our instance had a 20 GiB volume, a 14.77 GiB database and about 0.8 GiB free. It failed both checks.--no-vacuum needs about 30% of the file size, which was about 4.4 GiB here, and even then the file would not have shrunk until a later VACUUM, which also needs room to write a full copy. You have four honest options:

The copy-and-swap route keeps every message. Here is the procedure, with /mnt/big standing for any disk with free space of at least three times the database size (the backup, the working copy, and the migration's own headroom):

# 1. Stop every writer, then take a consistent backup (includes WAL contents)
hermes gateway stop
sqlite3 ~/.hermes/state.db ".backup '/mnt/big/state-before.db'"

# 2. Migrate a working copy, using the SAME Hermes version you run in production
mkdir -p /mnt/big/hermes-maint
cp /mnt/big/state-before.db /mnt/big/hermes-maint/state.db
HERMES_HOME=/mnt/big/hermes-maint hermes sessions optimize-storage --yes

# 3. Verify the copy (next section), then swap it in
rm -f ~/.hermes/state.db ~/.hermes/state.db-wal ~/.hermes/state.db-shm
cp /mnt/big/hermes-maint/state.db ~/.hermes/state.db
hermes gateway start

Three details matter. First, HERMES_HOME controls which state.db the CLI opens, so pointing it at the maintenance directory makes the command work on the copy only. Second, use the same Hermes build you run in production. We ran the migration in a maintenance container using our Hermes image, so the migration code and SQLite build matched what the instance runs. Third, delete the old -wal and-shm files along with the old database. They belong to the old file. The .backup in step 1 already contains anything they held, and SQLite should not find them next to the new database. If Hermes runs as a service user, restore that owner and file mode on the new file before starting the gateway.

Verify before you swap it back

sqlite3 /mnt/big/hermes-maint/state.db "PRAGMA integrity_check;"
for db in /mnt/big/state-before.db /mnt/big/hermes-maint/state.db; do
  sqlite3 "$db" "SELECT (SELECT COUNT(*) FROM messages), (SELECT COUNT(*) FROM sessions);"
done

integrity_check should print ok, and the two count lines should match. We went further, because this was a customer's history. We compared row counts and a content hash for all 26 application tables, excluding only the search tables and internal bookkeeping. We also ran the external-content integrity check on both rebuilt indexes. All of them matched: 441,789 messages, 12,432 sessions, plus routing, usage, prompt and delivery records.

Keep the backup until the agent has run normally for a while. Ours compressed to about 5.5 GB. A copy on another disk is what makes this whole procedure safe to undo.

optimize, optimize-storage and prune are different tools

Auto-prune is already on by default (sessions.auto_prune: true, retention_days: 90). It only removes ended sessions, and it only VACUUMs when at least 30 days have passed since the last one and more than a quarter of the file is reclaimable. So an agent can be pruning correctly and still carry a large index. The index layout is a separate problem from retention, and optimize-storage is the command that fixes it.

How this works on managed Hermes hosting

On Lobsterland you don't get a shell on the instance, so this is not something you run yourself. It was a support operation. The 2026-09-29 run above was a hosted Hermes instance, and we did it with the owner's approval. We stopped the deployment, backed up the database in a temporary maintenance pod on the same node, ran Hermes's own migration against an isolated copy, verified it table by table, and restarted with the same image. No history was pruned and the runtime was not upgraded. Afterwards the volume was at 40% with 11.7 GiB free.

The managed part is limited, and this is what it covers. Someone who has done this before handles the stop, backup, verification and restore. Disk size is part of the plan: Light has 20 GB, Medium 80 GB and Large 160 GB, and moving to a larger plan expands the existing volume. Hermes itself does not change: the opt-in layout and the headroom rule above apply to a hosted agent too.

If you would rather not run this kind of maintenance yourself, seemanaged Hermes runtime hosting and the plan storage sizes. If you are still choosing a runtime, compare the OpenClaw and Hermes runtimes on what actually differs, or start with what the Hermes agent runtime is. Other Hermes problems we have run into in production include thememory character-limit deadlock andTelegram going silent while the gateway stays up. A large state.db also makes recovery slower after a crash, because thepost-crash integrity check that can trap the gateway in a restart loop has to read the whole file. Moving over from OpenClaw? See what carries over when you migrate an OpenClaw instance to Hermes.

FAQ

Sources

• NousResearch/hermes-agent source, hermes_cli/sessions_cmd.py (main, last changed 2026-09-27): the disk preflight, the "Not enough free disk" and "already on the compact layout" messages, and the refusal while another process holds the store.
• NousResearch/hermes-agent pull request #65798, "compact v23 FTS layout … opt-inhermes sessions optimize-storage", merged 2026-07-22: the layout change and the 18.9 GB of 25 GB measurement.
• Hermes Agent documentation: the Sessions guide (state.db location, unbounded growth, auto-prune defaults) and the CLI reference (optimize, optimize-storage, prune).
• NousResearch/hermes-agent issues #68858 (9.4 GB state.db saturating disk I/O on v0.19, opened 2026-07-21), #100108 (state.db bloated to 7.6 GB, opened 2026-09-01) and #129927 (doctor never suggests optimize-storage when the version marker is ahead of the real layout, opened 2026-10-01).
• SQLite documentation, "VACUUM": why deleted rows do not shrink the file until it is rebuilt.
• Lobsterland production maintenance, 2026-09-29: one hosted Hermes instance, measured before and after sizes, and table-by-table verification.