Hermes agent state.db too large: shrink it without losing history

Stop the gateway and run hermes sessions optimize-storage. On a database created before Hermes's compact search layout, most of state.db is duplicated search index, and this command removes the duplication without touching your messages. The catch is free space: the command needs about as much free disk as the database already uses. If your disk is nearly full, it refuses. When a hosted Hermes instance of ours reached 97% of its volume on 2026-09-29, we ran the same migration on a copy on another disk, checked it, and swapped it back. state.db went from 14.77 GiB to 3.83 GiB, and all 441,789 messages were kept.

At a glance

Why Hermes state.db gets this big

state.db is a single SQLite file in your Hermes home (~/.hermes/state.db by default). It holds session metadata, every message, and the full-text indexes behind session_search. The Hermes docs say that without pruning it "grows without bound — multi-GB files within weeks were reported on gateway + cron installs."

On a database created before the compact layout, most of that growth is the search index, not the conversations. In the old layout, messages_fts and messages_fts_trigram each store their own copy of the text they index, and the trigram index also covers tool output. On tool-heavy agents, tool output is often the bulkiest text in the database. Pull request #65798, merged on 2026-07-22, introduced the compact v23 layout. Its indexes refer back to the messages table instead of copying it, and the trigram index skips tool rows. The PR reports a real 25 GB database in which 18.9 GB was index data over 3.4 GB of actual text.

The important detail is that the migration is opt-in. New installs start on the compact layout. An existing database stays on the large layout, however many times you upgrade Hermes, until you run the command. That is why long-running agents are the ones that hit this. Upstream reports describe the same shape: a 9.4 GB state.db with about 447,000 messages saturating disk I/O on v0.19 (#68858), and a database that bloated to 7.66 GB three times in 72 hours (#100108).

Check whether this is your problem

ls -lh ~/.hermes/state.db*
df -h ~/.hermes
hermes sessions optimize-storage

The first line shows the database and its -wal/-shm sidecars. The second shows how much room you have. The third is safe to run as a check. It inspects the real index structure. If there is nothing to migrate, it prints Search index is already on the compact layout — nothing to do. and exits. Otherwise it prints the current size, the free disk, and the space it needs, then asks Proceed? [y/N]. Answer N if you are only looking.

Don't wait for hermes doctor to suggest it. Issue #129927, opened on 2026-10-01, reports that doctor never recommends optimize-storage when the database's fts_storage_version marker was written ahead of the real layout. You get generic auto-prune advice instead. The command itself checks the layout directly, so running it is the reliable test.

The fix when you have enough free disk

hermes gateway stop
hermes sessions optimize-storage --yes
hermes gateway start

Stop the gateway first. Current Hermes refuses optimize, optimize-storage and prune while another process holds state.db, and stops with Refusing `hermes sessions optimize-storage`: another process is using … followed by the process holding it. That includes the gateway, the Desktop app, the dashboard and cron. The source explains why it refuses: running the rebuild under live gateways once put every agent into a write refusal until all writers were stopped. --force skips the check. Don't use it to avoid stopping your gateway.

The run is resumable. On a database over 500 MB, the command warns that it may take a while and says it is "safe to Ctrl-C and re-run (it resumes)". Ours was interrupted by a network drop partway through and picked up from its checkpoint. At the end it runs VACUUM, which is the step that actually gives disk space back to the filesystem.

When the disk is already full

This is where the documented fix fails. Before starting, the command checks for free space roughly equal to the current file size. Its source gives the reason: "the new index is built before the old is torn down, and VACUUM needs a full second copy". If it can't get that space, it stops with:

⚠ Not enough free disk to complete safely. Free up space, or run with --no-vacuum
(rebuilds the index but doesn't reclaim space until a later VACUUM).

Our instance had a 20 GiB volume, a 14.77 GiB database and about 0.8 GiB free. It failed both checks. --no-vacuum needs about 30% of the file size, which was about 4.4 GiB here, and even then the file would not have shrunk until a later VACUUM, which also needs room to write a full copy. You have four honest options:

The copy-and-swap route keeps every message. Here is the procedure, with /mnt/big standing for any disk with free space of at least three times the database size (the backup, the working copy, and the migration's own headroom):

# 1. Stop every writer, then take a consistent backup (includes WAL contents)
hermes gateway stop
sqlite3 ~/.hermes/state.db ".backup '/mnt/big/state-before.db'"

# 2. Migrate a working copy, using the SAME Hermes version you run in production
mkdir -p /mnt/big/hermes-maint
cp /mnt/big/state-before.db /mnt/big/hermes-maint/state.db
HERMES_HOME=/mnt/big/hermes-maint hermes sessions optimize-storage --yes

# 3. Verify the copy (next section), then swap it in
rm -f ~/.hermes/state.db ~/.hermes/state.db-wal ~/.hermes/state.db-shm
cp /mnt/big/hermes-maint/state.db ~/.hermes/state.db
hermes gateway start

Three details matter. First, HERMES_HOME controls which state.db the CLI opens, so pointing it at the maintenance directory makes the command work on the copy only. Second, use the same Hermes build you run in production. We ran the migration in a maintenance container using our Hermes image, so the migration code and SQLite build matched what the instance runs. Third, delete the old -wal and -shm files along with the old database. They belong to the old file. The .backup in step 1 already contains anything they held, and SQLite should not find them next to the new database. If Hermes runs as a service user, restore that owner and file mode on the new file before starting the gateway.

Verify before you swap it back

sqlite3 /mnt/big/hermes-maint/state.db "PRAGMA integrity_check;"
for db in /mnt/big/state-before.db /mnt/big/hermes-maint/state.db; do
  sqlite3 "$db" "SELECT (SELECT COUNT(*) FROM messages), (SELECT COUNT(*) FROM sessions);"
done

integrity_check should print ok, and the two count lines should match. We went further, because this was a customer's history. We compared row counts and a content hash for all 26 application tables, excluding only the search tables and internal bookkeeping. We also ran the external-content integrity check on both rebuilt indexes. All of them matched: 441,789 messages, 12,432 sessions, plus routing, usage, prompt and delivery records.

Keep the backup until the agent has run normally for a while. Ours compressed to about 5.5 GB. A copy on another disk is what makes this whole procedure safe to undo.

optimize, optimize-storage and prune are different tools

Auto-prune is already on by default (sessions.auto_prune: true, retention_days: 90). It only removes ended sessions, and it only VACUUMs when at least 30 days have passed since the last one and more than a quarter of the file is reclaimable. So an agent can be pruning correctly and still carry a large index. The index layout is a separate problem from retention, and optimize-storage is the command that fixes it.

Stop babysitting your Hermes box

Fix it once — or stop fixing it for good.

Apply the fix above and keep self-hosting, or skip the maintenance entirely: run Hermes on managed hosting from $6.90/mo, starting with a 7-day free trial. We handle gateway restarts, Hermes version upgrades, and uptime, and you bring your own model keys. Cancel anytime.

Managed hosting — from $6.90/mo Your own hosted Hermes instance with Telegram, Slack and browser chat, automatic restarts and version upgrades. Starts with a 7-day free trial; cancel anytime.

$199 managed setup — optional Prefer we do it for you? One workspace configured end-to-end: first-run config, one 30-minute onboarding/debug session, and a 7-day follow-up. Limited weekly slots.

• Managed hosting handles gateway restarts and Hermes version upgrades for you
• Telegram, Slack, browser chat, skills and an optional Hosted Browser, with your own model keys
• Already running OpenClaw on Lobsterland? Switch that instance to Hermes from the dashboard
• The optional $199 setup is scoped: no custom development, enterprise/SRE support, or unsupported self-hosting repair

If you would rather compare options first, see managed Hermes runtime hosting or the plan sizes and pricing before deciding.

Run it on managed hosting — from $6.90/mo Prefer we set it up? $199 managed setup Compare the OpenClaw and Hermes runtimes

How this works on managed Hermes hosting

On Lobsterland you don't get a shell on the instance, so this is not something you run yourself. It was a support operation. The 2026-09-29 run above was a hosted Hermes instance, and we did it with the owner's approval. We stopped the deployment, backed up the database in a temporary maintenance pod on the same node, ran Hermes's own migration against an isolated copy, verified it table by table, and restarted with the same image. No history was pruned and the runtime was not upgraded. Afterwards the volume was at 40% with 11.7 GiB free.

The managed part is limited, and this is what it covers. Someone who has done this before handles the stop, backup, verification and restore. Disk size is part of the plan: Light has 20 GB, Medium 80 GB and Large 160 GB, and moving to a larger plan expands the existing volume. Hermes itself does not change: the opt-in layout and the headroom rule above apply to a hosted agent too.

If you would rather not run this kind of maintenance yourself, see managed Hermes runtime hosting and the plan storage sizes. If you are still choosing a runtime, compare the OpenClaw and Hermes runtimes on what actually differs, or start with what the Hermes agent runtime is. Other Hermes problems we have run into in production include the memory character-limit deadlock and Telegram going silent while the gateway stays up. A large state.db also makes recovery slower after a crash, because the post-crash integrity check that can trap the gateway in a restart loop has to read the whole file. Moving over from OpenClaw? See what carries over when you migrate an OpenClaw instance to Hermes.

FAQ

Why is my Hermes state.db so large?

On most large installs it is the search index, not your conversations. Databases created before Hermes's compact v23 layout keep the legacy inline FTS5 layout: both full-text indexes store their own copy of every message, and the trigram index also covers tool output. The upstream pull request that introduced the compact layout reports a 25 GB database in which 18.9 GB was index data over 3.4 GB of actual text. The migration is opt-in, so an older database stays on the large layout until you run hermes sessions optimize-storage.

Does hermes sessions optimize-storage delete my conversation history?

No. It rebuilds the two full-text search indexes in the compact external-content layout and leaves the messages table unchanged. On our 2026-09-29 run, all 26 application tables matched their pre-run row counts and row-content hashes afterwards, including 441,789 messages and 12,432 sessions. The command that deletes history is hermes sessions prune, a different tool.

What does 'Not enough free disk to complete safely' mean?

optimize-storage checks free space before it starts. It builds the new index before it removes the old one, and its final VACUUM writes a full second copy of the database, so it needs free space roughly equal to the current file size. With --no-vacuum it needs about 30 percent of the file size, but the file does not get smaller until a later VACUUM. If you cannot meet either number, run the migration on a copy on a disk with room, or grow the disk first.

What is the difference between hermes sessions optimize and optimize-storage?

hermes sessions optimize merges FTS5 index segments and runs VACUUM; it is routine maintenance and does not change the index layout. hermes sessions optimize-storage migrates a legacy database to the compact v23 index layout, which is where the large savings come from. If optimize-storage reports 'Search index is already on the compact layout — nothing to do', your database is already compact and its size is real history; at that point pruning or a larger disk are the remaining options.

Can I run optimize-storage while the Hermes gateway is running?

Not by default. Current Hermes refuses optimize, optimize-storage and prune while another process (gateway, Desktop, dashboard or cron) holds state.db, and prints 'Refusing hermes sessions optimize-storage: another process is using' the database. The source cites an incident where running the rebuild under live gateways put every agent into a write refusal until all writers were stopped. Stop the gateway with hermes gateway stop, run it, then start the gateway again. --force exists, but don't use it to skip stopping the gateway.

Sources

• NousResearch/hermes-agent source, hermes_cli/sessions_cmd.py (main, last changed 2026-09-27): the disk preflight, the "Not enough free disk" and "already on the compact layout" messages, and the refusal while another process holds the store.
• NousResearch/hermes-agent pull request #65798, "compact v23 FTS layout … opt-in hermes sessions optimize-storage", merged 2026-07-22: the layout change and the 18.9 GB of 25 GB measurement.
• Hermes Agent documentation: the Sessions guide (state.db location, unbounded growth, auto-prune defaults) and the CLI reference (optimize, optimize-storage, prune).
• NousResearch/hermes-agent issues #68858 (9.4 GB state.db saturating disk I/O on v0.19, opened 2026-07-21), #100108 (state.db bloated to 7.6 GB, opened 2026-09-01) and #129927 (doctor never suggests optimize-storage when the version marker is ahead of the real layout, opened 2026-10-01).
• SQLite documentation, "VACUUM": why deleted rows do not shrink the file until it is rebuilt.
• Lobsterland production maintenance, 2026-09-29: one hosted Hermes instance, measured before and after sizes, and table-by-table verification.