Runtime Operations | Midfleet Docs

Runtime operations overview

Runtime operations connect workspace config, console routes, optional Relay transport, runtime credentials, and live agent processes into one routable system.

Prepare the runtime before spawning agents

bash

midfleet login \
  --hub-url https://app.midfleet.ai \
  --workspace <workspace> \
  --pull-runtime

midfleet config doctor
midfleet config redact --json

# If local runtime values are stale:
midfleet config reset --runtime
midfleet config pull-runtime

midfleet runtime up --expected-workspace <workspace>

midfleet runtime status --bundle ~/.midfleet/runtimes/<workspace>

# Keep the host up across logout/reboot (macOS LaunchAgent or Linux systemd):
# ./scripts/install-midfleet-runtime-service.sh --workspace <workspace> --role primary

Install a durable runtime service (servers and laptops)

Foreground midfleet runtime up stops when the terminal exits. Production and always-on hosts should run the supervised process under the OS service manager.

Copy scripts/install-midfleet-runtime-service.sh from the Midfleet repo onto the host (or curl it from the default branch). The script installs a macOS LaunchAgent or a Linux systemd unit with automatic restart.

bash

# After install root exists and dry-run succeeds:
midfleet runtime up --bundle ~/.midfleet/runtimes/<workspace> --dry-run --json

./scripts/install-midfleet-runtime-service.sh \
  --workspace <workspace> \
  --role primary
  # secondary compute host: --role secondary

# macOS status / logs
launchctl print gui/$(id -u)/io.midfleet.runtime.<workspace-slug>
tail -f ~/.midfleet/logs/runtime-up-*.log

# Linux status / logs
systemctl status midfleet-runtime-<workspace-slug>.service --no-pager
journalctl -u midfleet-runtime-<workspace-slug>.service -f

Runtime config fields

Field	Meaning	Check
`consoleRouter.publicBaseUrl`	Public URL Midfleet uses for chat and terminal routes.	Use this exact value as `--serve-public-base-url`.
`consoleRouter.listenPort`	Local router port.	Router process should listen on this port.
`relay.listenPort`	Local relay port.	Relay should accept runtime events.
`auth.workspace_api_key`	Workspace automation credential.	Saved locally and redacted in evidence.

Keep relay and console router separate

Component	Responsibility	Failure symptom
Console router	Routes public chat, terminal, and health paths to agent serve ports.	Control cannot open terminal/chat or public health route fails.
Relay	Accepts authenticated Hub or Control spawn requests and starts safe `midfleet start` commands. Direct CLI starts bypass it.	Remote spawn fails even though direct starts and existing agents may still work.
Cloudflare tunnel	Forwards public hostname traffic to the runtime host.	Browser sees Cloudflare/Access/HTML response instead of Midfleet route response.

bash

export MIDFLEET_RELAY_TOKEN=<relay-token>
midfleet internal relay start --host 127.0.0.1 --port 8787 --daemon
midfleet internal relay status
midfleet internal relay health

# Public relay shape:
# https://relay-<workspace>.midfleet.io/spawn
# Header: x-midfleet-relay-token: <relay-token>

Start routable agents directly on the runtime host

Two launch paths exist. Direct CLI (this section) runs on the runtime host and does not call Relay. Hub/Control spawn posts to the host relay (POST /spawn) and is documented under tunnels/relay. Networks agent counts only include agents attributed to a runtime inventory id (rti_*) that are not evicted.

Before every spawn, the host must already show spawn ready on Networks (fresh heartbeat + relay ready). Install a durable supervisor first (scripts/install-midfleet-runtime-service.sh or midfleet runtime up under launchd/systemd).

Run as the runtime account on the target host (the machine that owns the rti_* you want in Networks).
Set MIDFLEET_RUNTIME_INSTANCE_ID to the inventory id (rti_…) from Networks / runtime topology — not the bundle wrt_… value in runtime.env.
Set MIDFLEET_AGENT_WORKDIR_ROOT to the same absolute path as Networks → server config → workdir_root (Mac example: /Users/…/Dev/agents). Workspace onboard may still advertise a Linux /srv/… root; the env override makes multi-host fleets reproducible.
Use --serve-public-base-url from that server’s console_public_base_url (Networks server config), not a different host’s console hostname.
Use --project or --workdir, never both. New workers need --project and --team-id.
Use one free --serve-port per agent (or omit for auto-allocation when supported).
After start: local listen, Hub agent runtime_instance_id=rti_*, then Networks agents_running increments on the next runtime heartbeat (~30s).

bash

# 0) Preflight on the runtime host
midfleet config show
curl -fsS "http://127.0.0.1:<relay-port>/healthz"      # from runtime-config relay.listenPort
curl -fsS "http://127.0.0.1:<console-port>/readyz"    # console-router
# Networks UI: this host spawn_health=ready

# 1) Identity + workdir (required for correct Networks counts on multi-host workspaces)
export MIDFLEET_RUNTIME_INSTANCE_ID=rti_<this-host-from-networks>   # inventory id, not wrt_*
export MIDFLEET_AGENT_WORKDIR_ROOT=/absolute/path/from/server-config  # same as workdir_root

# 2) Direct CLI spawn (no Relay POST /spawn)
midfleet --profile <profile> start \
  --name <unique-agent-name> \
  --command <agent-command> \
  --model <provider/model> \
  --capabilities implementation,coding,debugging \
  --project <project-code> \
  --team-id <team-id> \
  --serve \
  --serve-port <free-port> \
  --serve-public-base-url <console_public_base_url-from-this-server> \
  --worker

# 3) Verify (same host)
midfleet list
curl -i -sS "http://127.0.0.1:<serve-port>/global/health"   # expect 401 with auth required
# Hub: agent status=online and runtime_instance_id=rti_…
# Networks: select this server → agents count >= 1 within ~30s

Current credential-isolation limit

The current runtime materializer writes one workspace-runtime canonical provider auth file and links agent auth paths to it. Do not mix provider identities or materialize agents concurrently in one runtime. Per-agent immutable auth files remain a hardening target.

Verify the public route, not only localhost

The unauthenticated console health request should return 401, proving that the public route reaches the agent and enforces auth. For a direct start, Hub audit should also show one runtime credential issuance from the intended authorization source and no Relay POST /spawn.

bash

midfleet --profile <profile> list
midfleet internal inbox --name backend-agent --json
midfleet --profile <profile> ask backend-agent "Reply exactly: SPAWN-OK" --timeout-seconds 120
curl -i https://console-<workspace>.midfleet.io/<workspace>/agents/<agent-id>/global/health

# Use the exact host printed by:
midfleet config show
# Runtime -> console / consoleRouter.publicBaseUrl

Stop and evict without erasing workflow history

Stop the local runtime. Run midfleet --profile <profile> stop <agent-name> on the runtime host. This stops the process and runtime session, removes a sandbox container when present, and clears local running state while preserving the agent ID for restart.
Evict the Hub identity in Control. Eviction marks the Hub agent as evicted. It does not delete runtime-host files or historical workflow records.
Run host cleanup separately. Use the approved retention-aware host cleanup process for stale console routes and agent-specific managed artifacts. Inspect a workdir for uncommitted work before removal, and never delete the workspace runtime's shared canonical provider auth file as if it belonged to one agent.
Verify and retain history. Confirm no agent process or public console route remains and inspect the expected workdir/auth disposition. Keep an evicted Hub row when workflow runs, handoffs, claims, or evidence reference it.