Blog
OpenClaw scheduler and lane isolation: how the concurrency model works
OpenClaw’s scheduler is a lane-aware, in-process FIFO queue: every agent run is admitted first through a
per-session lane that allows one active run per session, then through a global lane (main for
chat, cron-nested for cron and heartbeats, subagent for spawned children) whose width
caps how many runs execute at once. This page describes the model as it ships in OpenClaw 2026.9.4,
read from the source at that tag, including what changed since spring 2026 and which isolation gaps are still
open on September 13, 2026.
The model at a glance, OpenClaw 2026.9.4 - Two-level admission:
session:<key> (width 1, always) → a global lane. main is min(16, max(8, CPU parallelism)) unless agents.defaults.maxConcurrent is set. cron, cron-nested and hook-dispatch are fixed at 8; cron.maxConcurrentRuns is a retired key. - Inside a lane, foreground turns (user, manual) start before normal, and normal before background (cron, heartbeat, memory, overflow).
- A hung run holds its global slot for the run timeout plus 30 seconds; the upstream default timeout is 48 hours.
- Still open: per-agent lanes (#43235) and a
lane parameter on sessions_spawn (#10467). All agents share main.
Mid-incident right now?
If a cron job says enqueued and never runs, start with the
cron enqueued-but-idle runbook.
This page explains the machinery that runbook is poking at.
How does OpenClaw admit an agent run?
Every embedded agent run goes through two queues in sequence
(src/agents/embedded-agent-runner/run-orchestrator.ts):
- The session lane. The runner resolves
session:<sessionKey> and enqueues
there. Lanes that nobody configured are created with maxConcurrent: 1, so a session can never
have two runs executing at once. This is not a setting; it is how the lane is constructed. In 2026.9.4, CLI,
embedded and Codex runs share the same session-key lane, and an idle session lane is retired from the lane map
once it is empty.
- The global lane. Inside the session-lane task, the runner enqueues the actual run into a
global lane. Chat turns go to
main. A caller that passes cron is remapped to
cron-nested, because the outer cron job already holds a cron slot and letting its
inner agent turn wait on the same lane would deadlock. Spawned children go to subagent.
The ordering matters for how stalls look. A run waiting in the session lane is waiting for its own
previous turn. A run waiting in the global lane is waiting for everyone else’s. The runner marks
both phases on the reply operation (markWaitingForGlobalLane, markGlobalLaneWaitEnded),
and the second marker is the one you will see in watchdog output as global_lane:wait_ended.
Which command lanes exist in OpenClaw 2026.9.4 and how wide are they?
| Lane | Width | Configurable? | What runs there |
main | min(16, max(8, os.availableParallelism())) | agents.defaults.maxConcurrent | inbound chat turns from every channel |
session:<key> | 1 | no | the per-session serializer in front of every global lane |
cron | 8 | no (fixed constant) | the outer wrapper of each cron job |
cron-nested | 8, shared budget with hook-dispatch | no | isolated cron agent turns and heartbeat runs |
hook-dispatch | 8 when hooks are enabled, 1 slot reserved inside the cron budget | no | external hook agent runs |
subagent | 8 | agents.defaults.subagents.maxConcurrent | every sessions_spawn child |
nested | 1 | no | sessions.send and other nested agent work |
background | 3 in total | no | Skill Workshop reviews (at most 1) and plugin background work such as dreaming |
Two things in that table are new since mid-2026. First, main is sized from CPU parallelism; the
2026.8.1 release notes call it
“CPU-scaled foreground concurrency … bounded between 8 and 16 simultaneous runs, while preserving
explicit operator limits”. The 2026.6.2 source still hard-coded 4. Second, lanes can now belong to a
capacity group: a hard aggregate budget across several lanes with per-member reservations.
cron-nested and hook-dispatch share one budget of 8, and hooks reserve one slot inside
it, so enabling hooks costs cron nothing in aggregate concurrency but guarantees hooks cannot be starved.
main, cron, subagent, nested and every session lane are
deliberately ineligible for groups, because other lanes wait on them synchronously and a group wait on top of
that could deadlock.
What order does work run in inside a lane?
Each lane keeps three FIFO rings and always drains foreground, then normal,
then background. The run’s trigger decides the ring:
| Trigger | Ring |
user, manual | foreground |
cron, heartbeat, memory, overflow | background |
| anything else | normal |
| restart-recovery input, inter-session (agent-to-agent) input | forced to background |
This landed in 2026.5.18 (“prioritize manual user turns ahead of queued cron and maintenance work in the
same session lane, so visible follow-ups no longer wait behind background runs”). Before it, a human reply
queued behind a heartbeat in the same session waited its turn. It does not preempt: a background run that is
already executing keeps its slot; the ring only decides who starts next.
What releases a lane slot when a run hangs?
The lane queue has had a task-level timeout since 2026.4.29. The budget for a global-lane task
is the run’s own timeout plus a 30-second grace (EMBEDDED_RUN_LANE_TIMEOUT_GRACE_MS). The run
timeout comes from agents.defaults.timeoutSeconds, and the upstream default is
172,800 seconds — 48 hours. On an unconfigured self-host, a run whose promise never
settles holds its main slot for 48 hours and 30 seconds before the lane reclaims it.
When the lane does reclaim a slot, the error names the reason. 2026.9.4 distinguishes five:
task-budget — elapsed time reached the budget. owner-deadline — the attempt handed the lane its own deadline and that deadline passed. progress-idle — no progress signal for the budget window. abort-grace — the run was aborted but did not unwind within 30 seconds, so the slot was forced free. release-signal — the runtime asked for the lane back after an attempt timeout.
Command lane "<lane>" task timed out: <cause> (task budget <n>ms, elapsed <n>ms)
On timeout the runner also aborts the attempt’s action signal, so a released slot always means the run is
being cancelled, not orphaned.
Two other mechanisms free lanes without a timeout:
- Stall recovery. With diagnostics on, a session stuck in
processing is
classified session.long_running, session.stalled or session.stuck;
session.stuck always runs a recovery that can release the session lane, and
session.stalled past the abort threshold can trigger an active abort. The old
diagnostics.stuckSessionWarnMs and stuckSessionAbortMs keys were retired in the same 2026.8.1 change; the
thresholds are built in.
- In-process restart. After a
SIGUSR1 restart, resetAllLanes() bumps
every lane’s generation, clears active task ids, keeps queued entries and re-pumps them, so stale
completions from the previous generation are ignored instead of blocking new work. This has been in place
since 2026.2.13.
What is fixed and what is configurable in 2026.9.4?
-
cron.maxConcurrentRuns is a retired config key. openclaw doctor --fix
strips it, and the cron cap is a fixed 8 for both the outer cron lane and cron-nested.
It was retired in 2026.8.1 (#111382, “purge numeric tuning knobs behind built-in defaults”); guides that tell you to raise it describe an older contract.
agents.defaults.maxConcurrent still sets main; leave it unset to get the CPU-scaled 8–16. agents.defaults.subagents.maxConcurrent still sets subagent (default 8). -
nested is 1 and has no knob. The 2026.9.4 source says so in one comment:
“sessions.send work uses a shared nested lane with no config knob.”
-
hooks.enabled is the only switch that changes lane layout at runtime; it creates the
cron-hooks group with the one-slot reservation.
Where are the lane isolation gaps as of September 13, 2026?
- No per-agent lanes. Every agent’s inbound turns share
main.
Issue #43235 (“Support per-agent command
lanes for multi-agent deployments”) is still open on September 13, 2026; the maintainers’ triage bot
parked it on April 30, 2026 as an idea with no sponsor, and its last activity is September 9, 2026. Its
predecessor #42686 was closed as a duplicate,
and the workaround upstream wrote into it is blunt: “Deploy a separate openclaw gateway process for agents
that run long tasks.”
- No
lane parameter on sessions_spawn. Issue #10467 is open; the maintainer review of
September 9, 2026 confirms the tool schema still has no lane parameter and every spawn path
dispatches to the single subagent lane.
- The shared
main cap is wider, not gone. Issue #16055 (five Telegram bots serialized
through main) was closed as stale in April 2026; the mechanism it described is unchanged, the lane
is just 8–16 wide instead of 4.
- One correction to older write-ups, including our own runbook. Issue #48488 (“Lane queue has no
task-level timeout”) was closed as not planned on June 16, 2026, which reads like the gap survived. It did
not: the timeout shipped in 2026.4.29 under a different title. If a page tells you a hung promise jams a lane
forever, it is describing a runtime older than April 2026.
How do you read lane state in logs and diagnostics?
The wait diagnostic fires when a task waited more than two seconds before starting:
lane wait exceeded: lane=<lane> waitedMs=<n> queueAhead=<n> activeAhead=<n> activeNow=<n> queueBehind=<n>
queueAhead and activeAhead are captured at enqueue time; activeNow and
queueBehind at dequeue. A lane whose activeAhead equals its width was saturated when you
enqueued; a lane with activeAhead=0 and a long wait was either suspended (width 0) or blocked by a
group budget or a sibling’s reservation, which the diagnostics.lanes gateway method reports as
blockedBy. The Control UI shows the same data in its System busyness overlay, with
static lanes listed individually and all dynamic session lanes collapsed into one row. The real-world line that
opened issue #43235 read lane=main waitedMs=151783 queueAhead=1 — eight Feishu bots sharing one
main lane on a 4-wide default.
A worked reading from a managed instance on OpenClaw 2026.9.3, September 9, 2026 (UTC): a user message was
appended at 20:37:55; the watchdog reclaimed the reply at 20:43:58 after 363 seconds with last marker
global_lane:wait_ended; cooperative recovery reported aborted=false drained=false
forceCleared=true released=1; the command lane hit its 30-second abort grace at 20:44:28. Everything the
lane machinery did there is what the source says it does. The important inference is the marker:
wait_ended means the run was admitted, so the six minutes were spent inside the run, not in
the queue. The cause turned out to be prompt preparation waiting on a memory-manager rebuild, a 2026.9.3 defect
unrelated to lanes. If you see wait_ended followed by silence, stop tuning maxConcurrent.
How does Lobsterland run the lane model?
Lobsterland does not replace the scheduler; it constrains the one number that decides how long a bad run can hold
a slot. Managed instances are generated with agents.defaults.timeoutSeconds: 1800, so a hung
run’s global slot is reclaimed after about 30 minutes 30 seconds instead of 48 hours 30 seconds.
maxConcurrent and subagents.maxConcurrent are left at upstream defaults.
The isolation upstream lacks within one process, Lobsterland provides between customers by
construction: each instance is its own gateway process in its own pod, so one customer’s saturated
main lane cannot queue anyone else’s turns. If your multi-agent design needs a slow agent kept
away from an interactive one today, the honest answer is upstream’s own: a separate gateway process per
group — which is what a second managed OpenClaw instance is.
Cron-heavy workloads, where the fixed cap of 8 and the cron-nested remap do the most work, are
covered on the hosted cron automation page, and the routing side
of running several agents on one host on the multi-agent OpenClaw hosting
page.
OpenClaw lane concurrency FAQ
What is a command lane in OpenClaw?
A named FIFO queue inside the gateway process with a concurrency cap. Every agent run is admitted through a per-session lane (width 1) and then a global lane such as main, cron-nested or subagent. Lanes are plain TypeScript promises; there are no worker threads.
What is the default maxConcurrent in OpenClaw?
Since OpenClaw 2026.8.1, the main lane defaults to min(16, max(8, available CPU parallelism)). Before that it was 4. The subagent lane defaults to 8, cron and cron-nested are fixed at 8, nested is fixed at 1, and background plugin work shares a budget of 3.
Does cron.maxConcurrentRuns still work?
No. In OpenClaw 2026.9.4 it is a retired configuration key that openclaw doctor --fix removes, and cron concurrency is a fixed 8.
Why does a cron agent turn run in cron-nested instead of cron?
Because the outer cron job already holds a cron slot. If the inner agent turn queued on the same lane it could wait on itself. The runner remaps cron to cron-nested, and heartbeat runs use cron-nested for the same reason.
How long does a hung run block a lane?
For the run timeout plus 30 seconds. The upstream default timeout is 48 hours; Lobsterland sets 30 minutes on managed instances. Stall recovery (session.stuck, session.stalled) can release a session lane earlier.
Can I give one OpenClaw agent its own lane?
Not in OpenClaw 2026.9.4. Per-agent lanes are tracked in issue #43235 and remain unimplemented; the maintainers' stated workaround is a separate gateway process for the slow agent.
Run the scheduler with a 30-minute leash
Managed instances ship with the run timeout that reclaims a stuck lane slot in half an hour, one gateway
process per customer, and the current stable OpenClaw image. Compare that with tuning it yourself before
deciding where to run OpenClaw.