Configuration¶

Moraine reads TOML configuration. The default template is config/moraine.toml, and runtime installs normally use ~/.moraine/config.toml.

For a copyable, full-file reference with comments beside every supported key, see Fully Commented Configuration TOML. That file is intended as a documentation reference; the shipped runtime template stays more compact.

Resolution Order¶

The top-level moraine command resolves config in this order:

1. --config <path>
2. MORAINE_CONFIG
3. ~/.moraine/config.toml, when it exists
4. MORAINE_DEFAULT_CONFIG, when it points at an existing file
5. config/moraine.toml, when running from a source checkout

Inspect the small public allow-list without printing credentials or the full effective config:

moraine config get backend.start_on_up
moraine config get clickhouse.url
moraine config get clickhouse.database --output json

moraine config get --help lists the supported keys. Other keys, including authentication values, are rejected during argument parsing.

Service-specific binaries accept the same --config <path> flag and add service-specific environment variables:

Unknown top-level sections and unknown keys inside normal config tables are rejected at load time. The only intentionally loose file is a repo-level .moraine.toml, which is a name-only backend reference and ignores unknown future keys.

Top-Level Sections¶

Minimal Example¶

Most users only need to override paths, ports, or watched sources:

[backend]
bind = "127.0.0.1"
start_on_up = true

[monitor]
port = 8080

[runtime]
root_dir = "~/.moraine"

[[ingest.sources]]
name = "codex"
harness = "codex"
enabled = true
glob = "~/.codex/sessions/**/*.jsonl"
watch_root = "~/.codex/sessions"

Identity¶

[identity] configures the stable person identity stamped on newly ingested raw_events and events rows:

[identity]
author = "alice@example.com"

The value is free-form by design; email-shaped strings are conventional but not enforced. Moraine never infers this identity from git, $USER, $HOSTNAME, the machine host, or the working directory. Missing and whitespace-only values normalize to empty.

Local/default-only installs may leave author empty. If any route targets a non-default backend and author is empty, that mirror is disabled for the process and reported as disabled_missing_identity_author; default ingest continues. After setting author, restart the ingest service. Retained source files are then replayed to the mirror using the configured author; files deleted before the mirror catches up cannot be recovered by this replay path.

ClickHouse¶

[clickhouse] controls database connectivity:

[clickhouse]
url = "http://127.0.0.1:8123"
database = "moraine"
username = "default"
password = ""
timeout_seconds = 30.0
request_compression = "none"
async_insert = true
wait_for_async_insert = true
allow_newer_server = false

For a managed local install, leave these values at their defaults. For an external ClickHouse instance, point url, credentials, and database at that server. For a complete single-user walkthrough, including Docker and personal server examples, see Remote ClickHouse Tutorial.

[clickhouse] is an alias for [backends.default]. To mirror specific projects to additional servers, see Backends and Per-Project Routing.

Moraine does not impose a default elapsed execution deadline after a query is admitted. Work continues until it completes, its caller cancels or disconnects, the owning service shuts down, or an internal caller supplies an absolute deadline. Every statement is tagged with a collision-safe moraine-<workload>-<uuid>-<child> query ID. Abandoned logical work cancels all of its active child statements through a separate administrative path; cleanup is best effort and bounded to five seconds. The url query string and per-request parameters may not override Moraine's reserved ownership, deadline, or resource settings (query_id, replace_running_query, max_execution_time, max_execution_time_leaf, timeout_overflow_mode, timeout_before_checking_execution_speed, workload, max_memory_usage, max_memory_usage_for_user, max_bytes_before_external_group_by, max_bytes_before_external_sort, max_bytes_ratio_before_external_group_by, max_bytes_ratio_before_external_sort, max_temporary_data_on_disk_size_for_query, or max_temporary_data_on_disk_size_for_user).

Query resource governance¶

Moraine assigns every ClickHouse statement to one of four fixed resource profiles. These limits are internal safety policy rather than user-facing TOML:

The running and waiting bounds apply per Moraine process and per statement, not per logical multi-statement operation. Waiting work opens no ClickHouse connection and allocates no server query memory. A full client or server admission queue returns the typed busy outcome. ClickHouse memory, disk, row, and byte limits return the distinct resource_exhausted outcome. Cancellation, caller deadlines, and service shutdown also wake queued work without waiting for a slot.

Managed ClickHouse adds server-wide backstops. It permits at most 16 concurrent queries, limits query memory for the default user to the smaller of 2 GiB or one quarter of detected host/container memory, caps each mark and uncompressed cache at the smaller of 256 MiB or one sixteenth of memory, and limits total server memory to 75% of memory. The remaining server envelope is reserved for merges and other server work. Effective per-query memory is therefore the smaller of the profile cap and the remaining aggregate user allowance.

Managed startup creates four ClickHouse workload queues matching the profiles. Each queue has its own running and waiting capacity, so saturated interactive reads cannot consume background, migration, or administrative slots. New installs and existing managed installs receive the rendered memory and disk settings on moraine up; startup idempotently refreshes the workload scheduler on the already-running server before declaring ClickHouse ready. Moraine refuses to manage ClickHouse when the detected memory limit is below 2 GiB.

For named or remote ClickHouse backends, Moraine checks system.workloads once per client. It attaches a managed workload only when all required scheduler nodes exist. A missing hierarchy, denied probe, malformed response, or probe timeout leaves the workload setting unset instead of failing the caller; the per-query memory, spill, and temporary-disk settings still apply. This keeps external servers compatible without assuming Moraine owns their scheduler.

Sorts and groups spill to ~/.moraine/clickhouse/tmp after the profile's absolute byte threshold; managed profiles disable ClickHouse's competing available-memory ratio so that the lower per-query memory ceiling cannot fire before the spill threshold. A statement may use only the profile's temporary-disk allowance, while all queries by the managed user share a 4 GiB temporary-disk ceiling. These are hard limits: exhausting either returns resource_exhausted rather than risking an unbounded allocation or filling the managed disk.

Environment-backed ClickHouse values¶

The url, database, username, and password fields accept either a literal string or an explicit environment reference. Environment references keep credentials out of the TOML file and work independently for every named backend:

[backends.team-ch]
url = { env = "CLICKHOUSE_TEAM_URL" }
database = { env = "CLICKHOUSE_TEAM_DATABASE" }
username = { env = "CLICKHOUSE_TEAM_USERNAME" }
password = { env = "CLICKHOUSE_TEAM_PASSWORD" }

Moraine reads the variables when it loads the config. A referenced variable must exist and contain valid Unicode; its value is used exactly, including an empty value or surrounding whitespace. Missing or invalid variables are config errors. Literal strings remain supported and are still the default.

Environment references are accepted only from a config you selected with --config, a config selected through a Moraine config environment override, or your home config at ~/.moraine/config.toml. Moraine rejects environment references in config/moraine.toml when that file was discovered only because it exists in the current repository. To trust that repository file, opt in by passing --config ./config/moraine.toml or setting MORAINE_CONFIG=./config/moraine.toml. Literal ClickHouse values continue to work in an implicitly discovered repository config.

This works with environment injectors and secret managers. For example, a 1Password environment can launch Moraine without writing the resolved password to disk:

op run --env-file=.env.1password -- moraine up

Every independently launched Moraine process loads the config for itself. Run other commands through the same injector, including moraine status, moraine db doctor, moraine db migrate, moraine down, moraine logs, moraine setup, and moraine run mcp. Services started by the injected moraine up process inherit its environment. If a referenced variable is unavailable, moraine setup reports an environment error and does not offer to repair or replace the config.

An environment reference protects the resolved value from being stored in the TOML file. Moraine still holds the value in process memory, and a running service must be restarted to pick up a rotated value. Keep authentication in the dedicated username and password fields: do not embed credentials or secret query parameters in url, because URL, database, and username values may appear in diagnostics. For agent integrations, inject the environment only into Moraine's MCP process; see Install by Harness.

Backends and Per-Project Routing¶

A project may belong to a team with a shared ClickHouse deployment. Named backends plus routes mirror that project's sessions to the team server while keeping the complete local history:

[backends.team-ch]
url = "https://ch.team.example:8443"
database = "moraine_team"
username = "svc-moraine"
password = { env = "CLICKHOUSE_TEAM_PASSWORD" }
request_compression = "gzip"
allow_newer_server = false

[[routes]]
dir = "~/src/teamproject/**"
backend = "team-ch"
mode = "mirror"

Named backends¶

Each [backends.<name>] block takes the same fields as [clickhouse], plus allow_newer_server (see the schema version handshake). The default backend always exists after config load: [clickhouse] and [backends.default] are aliases for it, so when [backends.default] is not declared it is synthesized from [clickhouse] (or built-in defaults). Declaring both blocks is a load error.

Routes¶

Each [[routes]] entry maps session working directories to a named backend. Routes are ordered; the first matching route wins.

A session's routing directory is sticky: the first non-empty working directory observed for the session decides its route, so a mid-session cd never splits a session across backends. The working directory is extracted from session trace content during ingest (record-level where the harness stamps it, session metadata otherwise); sessions whose traces carry no discoverable working directory stay on the default backend.

Repo-level .moraine.toml¶

A repository can opt into a backend without a home-config route by carrying a .moraine.toml at its root:

backend = "team-ch"

Moraine walks up from the session's working directory, stopping at $HOME or the filesystem root; the nearest file wins and ends the walk. The file is a name reference only — never URLs or credentials — and resolves only against [backends.*] entries in your home config. This is the trust boundary: a hostile cloned repo cannot redirect traces to a server you never configured. A name with no matching backend logs a warning (once per name) and the session stays on the default backend. Unknown keys in the file are ignored, and an explicit [[routes]] match in the home config takes precedence over the repo file.

Mirror semantics¶

Routing never replaces local history. The default backend receives every session unconditionally; a routed session is additionally mirrored to its backend. Each mirror runs its own sink with its own ingest_checkpoints stored in that backend's database, scoped per host (migration 018), so team members sharing one backend never disturb each other's mirror progress — even when session files on two machines share an absolute path.

A slow or unreachable backend never stalls local ingest. Mirror forwarding uses a bounded queue; on overflow the backend is marked lagging and live mirroring to it pauses. The source session files on disk act as the write-ahead log: at startup and whenever a lagging or unreachable backend recovers, a targeted replay pass re-reads tracked files against that backend's own checkpoints and closes the gap, without touching the default sink. Files deleted before a backend catches up are lost to that backend only (the local copy already ingested them live); Moraine logs when this happens. One known limitation: rows observed before a session's working directory is known resolve to the default backend only and are not retroactively mirrored once the session pins to a route. The harness adapters keep this window effectively empty (the working directory rides the records themselves, or is recovered from the session header), so in practice it only affects traces that carry no working directory at all.

Per-backend mirror status — connecting, catching_up, ok, lagging, unreachable, disabled_skew, or disabled_missing_identity_author — is written to ingest heartbeats as a backend_sinks map and surfaced through the monitor's /api/v1/health. This uses a column added by migration 017; until moraine db migrate runs (and ingest restarts), ingest warns and omits the field rather than failing heartbeats. disabled_missing_identity_author means [identity].author is empty; set it and restart ingest before mirroring to team backends.

catching_up means the remote sink is reachable and retained source files are replaying. New live rows may drain during this phase, but their checkpoints are withheld until replay establishes contiguous coverage. Moraine switches to ok only after an ordered durability barrier confirms earlier replay batches were durably handled — inserted or recorded by the existing explicit oversized-row quarantine — with their checkpoints committed, and no routed batch was dropped across the promotion boundary.

Schema version handshake¶

Moraine migrates only the default backend (moraine db migrate, or automatically on moraine up). It never runs migrations against a non-default backend. Before mirroring starts, it compares the backend's schema_migrations ledger against the migrations bundled in the running build — a strictly read-only probe — and enforces:

For ingest mirroring, a skew failure disables that one mirror until the ingest service restarts and shows as disabled_skew in /api/v1/health; the default backend is unaffected. A backend that simply does not answer retries the handshake periodically and shows as unreachable. The same handshake also runs when moraine run mcp starts in a routed directory, where it fails the MCP process instead (see below). The handshake exists to make skew loud, not to manage it.

MCP in routed directories¶

Routing changes what agents search, not just what ingest mirrors. When moraine run mcp starts in a directory that routes to a non-default backend (home-config [[routes]] first, then the repo .moraine.toml walk-up), it skips the central proxy entirely — the central server only serves the default backend — and runs an embedded server against the routed backend. Search results then come from the team server, not from local history.

Startup in a routed directory is deliberately fail-fast: the schema handshake runs first, and if the backend is skewed or simply unreachable, moraine run mcp exits with an error naming the backend instead of falling back to the default backend. Serving local results while the agent believes it is searching the team server would be silently wrong. So if MCP dies at startup in exactly one project, check that project's route and the team backend's health/schema first. As with ingest routes, a repo .moraine.toml naming an unconfigured backend logs a warning and keeps the default behavior.

The Unix socket connection attempt remains bounded by central_connect_timeout_ms. After a socket connects, private-route negotiation has no elapsed ACK deadline because the server may need to complete an owned backend schema handshake before replying. It continues while the caller remains connected; a full disconnect abandons and cancels that work.

Secret Redaction¶

Secret redaction is enabled by default for every ingested row. Moraine scans the secret-bearing string fields in raw events, normalized events, and tool I/O, then replaces matched secrets with typed placeholders such as [REDACTED:github-token]. Derived previews and hashes are recomputed from the redacted content.

[redaction]
ruleset = "builtin"
# extra_patterns = ["acme_internal_[A-Za-z0-9]{32}"]
# dangerously_skip_secret_redaction = true

dangerously_skip_secret_redaction = true is honored only from $HOME/.moraine/config.toml. It disables redaction for the local/default backend only; mirror egress to named backends is always redacted. The same flag in a repo or checkout config is ignored with a warning so a cloned repository cannot weaken local capture settings.

The builtin ruleset covers common API keys, provider tokens, private-key blocks, JWTs, and high-entropy generic credential assignments. extra_patterns adds local Rust regular expressions that redact the full matched span with [REDACTED:extra-pattern-N]; invalid patterns are a startup error. Redaction counts are written to ingest heartbeats in redactions_total after migration 022; before that migration, redaction still runs and ingest logs that the heartbeat audit column is not available.

Ingest Sources¶

Each [[ingest.sources]] entry describes one watched source:

[[ingest.sources]]
name = "claude"
harness = "claude-code"
enabled = true
glob = "~/.claude/projects/**/*.jsonl"
watch_root = "~/.claude/projects"
format = "jsonl"

Supported harness values are codex, claude-code, cursor, hermes, kiro-cli, kimi-cli, nac, opencode, pi-coding-agent, prime-agent, and qwen-code. Each value maps to a registered ingest source adapter; see Ingest Sources for the adapter contract and Harness Author Workflow for source development steps.

glob selects files to ingest. watch_root is the directory Moraine watches for changes. format controls the file parser:

When format is omitted, Moraine infers it. Hermes sources with a .json glob are inferred as session_json, Kiro CLI sources as kiro_session, Cursor globs ending in .vscdb as cursor_sqlite, NAC globs whose filename is store.db as nac_sqlite, OpenCode globs ending in opencode.db or opencode*.db as opencode_sqlite, and otherwise sources are treated as jsonl.

Source Matrix¶

The built-in defaults and config/moraine.toml reference cover these source families:

moraine setup derives both Prime Agent paths from the absolute PRIME_AGENT_CODING_AGENT_DIR override when it is set, so ingestion and the managed runtime skill remain rooted together.

Hermes supports both live session JSON and offline trajectory JSONL because the harness is the same but the file format differs. Use a separate [[ingest.sources]] entry for each watched directory. Cursor likewise has two trace forms under one harness: Agent transcript JSONL and SQLite chat history (cursor_sqlite); both are enabled by default. Kiro CLI stores transcript records in <session-id>.jsonl and session-level cwd, title, model, and token totals in <session-id>.json; the paired kiro_session format watches both files and checkpoints sidecar changes independently of transcript growth. Running moraine setup with the Kiro target rewrites the setup-owned kiro source to use $KIRO_HOME/sessions/cli when KIRO_HOME is set. NAC stores parent sessions and managed-worker episodes in store.db (nac_sqlite); guided moraine setup materializes this source only when NAC's effective store path is stable. OpenCode stores conversation history in default or channel-specific SQLite databases (opencode_sqlite); the template enables it by default.

Source Examples¶

Codex:

[[ingest.sources]]
name = "codex"
harness = "codex"
enabled = true
glob = "~/.codex/sessions/**/*.jsonl"
watch_root = "~/.codex/sessions"

Claude Code:

[[ingest.sources]]
name = "claude"
harness = "claude-code"
enabled = true
glob = "~/.claude/projects/**/*.jsonl"
watch_root = "~/.claude/projects"

Local Claude Cowork on macOS:

[[ingest.sources]]
name = "claude-cowork"
harness = "claude-code"
enabled = true
glob = "~/Library/Application Support/Claude/local-agent-mode-sessions/**/.claude/projects/**/*.jsonl"
watch_root = "~/Library/Application Support/Claude/local-agent-mode-sessions"

macOS defaults and moraine setup add this as a separate source while retaining the ordinary claude source. Moraine groups every nested Claude CLI transcript under its containing local_* Cowork session. The live ingest gate accepts only the .claude/projects/ transcript subtree, so sibling audit.jsonl files are not ingested.

Cowork root metadata is allowlisted into one session_meta record: session and nested CLI IDs, created/last-activity times, cwd, model, title, and archive/star flags. Account/email data, prompts, selected folders, MCP/plugin/tool settings, and unknown fields are not copied. Transcript attachment, last-prompt, and ai-title records remain available as raw records but do not become searchable events. Root metadata is refreshed when a transcript is created or appended. A metadata-only sidecar edit is picked up by the next transcript activity.

This source supports the observed local macOS layout only. Claude's JSONL format is internal and may change. A stable signed-in Claude Desktop path has not been verified across Windows installer/MSIX variants, so Moraine does not ship a Windows Cowork glob. Remote Cowork is also outside this source; its documented observability path is OpenTelemetry.

Kimi CLI:

[[ingest.sources]]
name = "kimi-cli"
harness = "kimi-cli"
enabled = true
glob = "~/.kimi/sessions/**/wire.jsonl"
watch_root = "~/.kimi/sessions"
format = "jsonl"

Qwen Code:

[[ingest.sources]]
name = "qwen-code"
harness = "qwen-code"
enabled = true
glob = "~/.qwen/projects/*/chats/*.jsonl"
watch_root = "~/.qwen/projects"
format = "jsonl"

For a custom Qwen storage root, keep the normalized qwen-code harness and use a distinct source name so moraine setup leaves the custom entry untouched:

[[ingest.sources]]
name = "qwen-code-archive"
harness = "qwen-code"
enabled = true
glob = "/srv/qwen-archive/projects/*/chats/*.jsonl"
watch_root = "/srv/qwen-archive/projects"
format = "jsonl"

The built-in adapter is fixture-tested against Qwen Code 0.19.x's internal append-only ChatRecord shape; that upstream persistence format is not a stable public API.

NAC:

[[ingest.sources]]
name = "nac"
harness = "nac"
enabled = true
glob = "~/.config/nac/store.db"
watch_root = "~/.config/nac"
format = "nac_sqlite"

This source polls NAC's SQLite store read-only. It indexes parent sessions, assistant reasoning, tool request/response pairs, token usage, and managed-worker episodes. Durable remote-session and remote-worker bodies are retained but receive no local project/worktree attribution. Credential-bearing columns and host identifiers are never selected. Qualified NAC tool names such as mcp__moraine__search_sessions normalize to canonical Moraine tool names while their raw names remain in event provenance.

moraine setup integrations nac --yes resolves the NAC config directory in this order: NAC_HOME, XDG_CONFIG_HOME/nac, then ~/.config/nac, and merges only the owned [mcp_servers.moraine] table. Model, storage, sandbox, and unrelated MCP settings are preserved.

When NAC is selected as an ingest source in regular guided moraine setup, its automatic path follows an absolute storage.store_path, or the default <NAC config dir>/store.db when the config home is stable. A relative store path depends on NAC's launch directory, and a per-launch nac --store-path override is invisible to setup, so use the manual snippet it prints instead of silently following the wrong database.

Replacing a NAC database at the same path starts a new source generation and reconciles the old rows after the replacement snapshot is durable. WAL-only changes are picked up through store.db-wal and store.db-shm; unrelated SQLite files and backups are ignored.

OpenCode:

[[ingest.sources]]
name = "opencode"
harness = "opencode"
enabled = true
glob = "~/.local/share/opencode/opencode*.db"
watch_root = "~/.local/share/opencode"
format = "opencode_sqlite"

This source polls OpenCode's opencode*.db databases read-only and ingests conversation records synthesized from the event and event_sequence tables. Account, credential, and token-bearing tables are deliberately out of scope. Moraine also reacts to the -wal/-shm sidecar files so WAL-only writes trigger polls.

Cursor Agent JSONL:

[[ingest.sources]]
name = "cursor"
harness = "cursor"
enabled = true
glob = "~/.cursor/projects/*/agent-transcripts/**/*.jsonl"
watch_root = "~/.cursor/projects"
format = "jsonl"

This source watches local Agent JSONL transcripts under agent-transcripts/. Cursor's IDE chat history lives elsewhere — in state.vscdb SQLite databases — and is ingested by the separate cursor_sqlite source below.

Cursor SQLite history (macOS):

[[ingest.sources]]
name = "cursor-sqlite"
harness = "cursor"
enabled = true
glob = "~/Library/Application Support/Cursor/User/**/state.vscdb"
watch_root = "~/Library/Application Support/Cursor/User"
format = "cursor_sqlite"

Cursor SQLite history (Linux):

[[ingest.sources]]
name = "cursor-sqlite"
harness = "cursor"
enabled = true
glob = "~/.config/Cursor/User/**/state.vscdb"
watch_root = "~/.config/Cursor/User"
format = "cursor_sqlite"

This source polls Cursor's state.vscdb databases read-only and ingests composer sessions and message bubbles (session titles, chat turns, tool calls). The conversation data lives in the globalStorage database; per-workspace databases under workspaceStorage match the same glob but are mostly empty. Moraine also reacts to the -wal/-shm sidecar files so WAL-only writes trigger polls; state.vscdb.backup files are ignored.

This source ships enabled. Cursor has no stable local-database contract, so an editor update can change the schema at any time (issue #361) — tracking that drift is Moraine's job. When the schema drifts, Moraine reports rate-limited sqlite_* ingest errors and skips the database rather than ingesting bad rows, until the normalizer is updated to follow the new format. Set enabled = false if you don't want IDE chat history ingested.

Pi Coding Agent and OMP:

[[ingest.sources]]
name = "pi"
harness = "pi-coding-agent"
enabled = true
glob = "~/.pi/agent/sessions/**/*.jsonl"
watch_root = "~/.pi/agent/sessions"
format = "jsonl"

[[ingest.sources]]
name = "omp"
harness = "pi-coding-agent"
enabled = true
glob = "~/.omp/agent/sessions/**/*.jsonl"
watch_root = "~/.omp/agent/sessions"
format = "jsonl"

OMP uses the Pi session schema, so both sources share the pi-coding-agent adapter. Separate source names preserve historical ~/.pi checkpoints while allowing startup backfill and live watching of current ~/.omp sessions.

Prime Agent:

[[ingest.sources]]
name = "prime-agent"
harness = "prime-agent"
enabled = true
glob = "~/.prime/agent/sessions/*.jsonl"
watch_root = "~/.prime/agent/sessions"
format = "jsonl"

[[ingest.sources]]
name = "prime-agent-subagents"
harness = "prime-agent"
enabled = true
glob = "~/.prime/agent/session-artifacts/**/sub-*/*.jsonl"
watch_root = "~/.prime/agent/session-artifacts"
format = "jsonl"

The adapter pins normalized semantics to Prime Agent v0.7.0 session format v3. Other historical or future versions remain raw-preserved but are otherwise best-effort until covered by fixtures.

The root and RLM-child sources use distinct checkpoint namespaces. Child sessions remain independently searchable, carry substream metadata, and link to the parent session ID recorded in their header. The setup-owned defaults strictly accept UUID-named transcripts at the canonical paths, so artifact journals such as rlm-subagents.jsonl never enter the ingest queue. Custom Prime sources remain configurable under a different glob.

Both sources are enabled by default for fresh installs and backfill existing local Prime history on first startup. That makes ordinary prompts, responses, tool I/O, cwd, and allowed metadata durable and searchable under the configured ClickHouse/redaction/routing policy. Set enabled = false on both entries above before startup to opt out.

When a pre-existing explicit source list still contains any setup-owned source with its canonical name, harness, glob, and watch root, config loading adds missing Prime defaults in memory. New entries stay disabled when all matching setup-owned entries are disabled. Custom-only source lists are unchanged, and a custom source whose name collides with either Prime default wins rather than being duplicated.

Prime Agent orchestration records such as status snapshots, attributed child usage, session/git state, hidden custom records, and unknown future records are retained only in raw storage. They do not create searchable events or add to token totals; visible conversation messages, tool activity, summaries, and visible custom messages do.

Hermes live sessions:

[[ingest.sources]]
name = "hermes"
harness = "hermes"
enabled = true
glob = "~/.hermes/sessions/session_*.json"
watch_root = "~/.hermes/sessions"
format = "session_json"

Hermes trajectories:

[[ingest.sources]]
name = "hermes-trajectories"
harness = "hermes"
enabled = true
glob = "~/trajectories/**/*.jsonl"
watch_root = "~/trajectories"
format = "jsonl"

Adding Harnesses¶

The config crate validates harness names before ingest-core runs, but moraine-config cannot call the ingest adapter registry without creating a dependency cycle. Adding a harness therefore requires coordinated updates:

• Add and register the adapter under crates/moraine-ingest-core/src/sources/.
• Add the harness string to moraine_config::KNOWN_INGEST_HARNESSES.
• Update defaults in crates/moraine-config/src/lib.rs and config/moraine.toml when the source should ship enabled by default.
• Update this page and the ingest source development docs.
• Run the registry/config sync tests and ingest fixture contract tests.

The [ingest] table controls batching and watcher behavior:

[ingest]
exclude_project_dirs = []
batch_size = 4000
max_batch_bytes = 8388608
flush_interval_seconds = 0.5
state_dir = "~/.moraine/ingestor"
backfill_on_start = true
max_file_workers = 8
max_inflight_batches = 16
debounce_ms = 50
reconcile_interval_seconds = 30.0
heartbeat_interval_seconds = 5.0

Exclusions use the same directory matching semantics as [[routes]]: ~ expands when configuration loads, * stays within one path component, and a trailing /** matches both the project directory itself and everything below it. Moraine keeps the first non-empty working directory for the whole session, so a later cd cannot change the decision. For example:

[ingest]
exclude_project_dirs = [
  "~/code/project-with-large-trajectories/**",
]

For JSONL sources, Moraine reads only far enough to find that initial working directory before normalization. Codex provides it in the early session_meta record, so excluded Codex trajectories do not require a full-file read and never reach the sink.

MCP¶

[mcp] sets defaults for agent retrieval:

[mcp]
max_results = 25
preview_chars = 320
default_context_before = 3
default_context_after = 3
default_include_tool_events = false
default_exclude_codex_mcp = true
prewarm_on_initialize = false
async_log_writes = true
protocol_version = "2024-11-05"
# max_parallel_requests = 16 # optional; omitted defaults to 4
use_central_server = true
central_socket_path = "mcp.sock"
central_connect_timeout_ms = 250

Raise max_results only when clients need larger result windows. Increase context defaults when retrieval snippets are too narrow. Leave prewarm_on_initialize disabled for compatibility harnesses that launch multiple stdio MCP processes at once; enabling it trades startup CPU/database work for lower first-search latency.

The shared backend applies one parallel-request budget across every direct HTTP request and private-socket MCP connection and queues at most 16 valid retrievals in FIFO order when that budget is busy. Queued and running retrievals have no fixed admission deadline; they continue until completion, client cancellation, full disconnect, or service shutdown. A clean request-side EOF is only a half-close: already admitted work finishes and its response is still written. A full queue is rejected immediately with a structured busy tool error. An embedded fallback is a separate process, so its execution budget is process-local. Validation and control requests continue to run while retrievals wait.

Shared MCP service¶

Every moraine up starts one backend process that owns the default repository, ClickHouse client, warm caches, and MCP admission budget. It exposes two MCP transports:

• Streamable HTTP at http://<backend.bind>:<monitor.port>/mcp, used directly by Codex and Claude Code after moraine setup.
• A private Unix socket used by the retained moraine run mcp stdio compatibility client.

Direct HTTP clients require the backend to be running; they do not start a tunnel process or fall back to an embedded server. The endpoint supports single-message JSON POST requests and returns JSON responses (or 202 for notifications). It does not allocate MCP session IDs or expose batch and SSE response modes.

The following fields control only the stdio compatibility path:

Directories routed to a non-default backend and moraine run mcp -- --project-only still use embedded stdio because direct HTTP requests have no launch-directory context. See MCP in routed directories and Agent MCP Search → Install. The private socket remains mode 0o600; keep the HTTP listener on loopback.

The up-managed MCP service is the unified backend daemon; the legacy per-up stdio daemon and standalone monitor service are gone. The old backend.start_on_up, runtime.start_monitor_on_up, mcp.start_central_on_up, and runtime.start_mcp_on_up launch switches remain load-compatible for upgrades, but their values no longer gate startup. Every well-typed combination loads with an effective backend.start_on_up = true. moraine setup canonicalizes the value to true and atomically removes the three obsolete aliases.

Search Ranking¶

[bm25] tunes search behavior:

[bm25]
k1 = 1.2
b = 0.75
default_min_score = 0.0
default_min_should_match = 1
max_query_terms = 32

Most installations should keep these defaults.

Backend Daemon¶

[backend] controls the unified daemon's HTTP listener:

[backend]
bind = "127.0.0.1"
# auth_token = "<generate-a-random-guard-token>"
start_on_up = true

Every moraine up starts the unified backend; --no-ingest is the only startup-selection switch. For upgrade safety, a non-loopback backend.bind also requires an affirmative existing launch setting (backend.start_on_up = true, or a legacy true alias); otherwise config loading fails rather than unexpectedly exposing the unauthenticated monitor API. Loopback configs normalize existing false values to true automatically.

Experimental HTTP bind guard¶

backend.bind defaults to the loopback interface. When the effective bind is non-loopback, backend.auth_token must contain at least one non-whitespace character or backend startup fails before creating any listener.

This is experimental configuration groundwork and a startup prerequisite only. It does not authenticate monitor HTTP requests, and exposing the monitor API or UI to an untrusted network remains unsafe. Real monitor authentication is tracked in issue #383, Phase 3.

The private Unix-socket MCP transport remains user-scoped. The direct /mcp endpoint is mounted only when backend.bind is an explicit loopback IP; a non-loopback listener does not serve MCP even when its startup guard token is configured.

Monitor¶

[monitor] controls the monitor HTTP port:

[monitor]
port = 8080

Runtime Paths¶

[runtime] controls where Moraine keeps state and where it finds service binaries:

[runtime]
root_dir = "~/.moraine"
logs_dir = "logs"
pids_dir = "run"
service_bin_dir = "~/.local/bin"
managed_clickhouse_dir = "~/.local/lib/moraine/clickhouse/current"
clickhouse_start_timeout_seconds = 30.0
healthcheck_interval_ms = 500
clickhouse_auto_install = true
clickhouse_version = "v25.12.5.44-stable"

Relative logs_dir and pids_dir values are resolved under root_dir. The MCP service keeps mcp-open-cursor.key in root_dir with owner-only permissions so continuation cursors remain valid across service restarts. Preserve this file as runtime state; replacing or deleting it invalidates outstanding cursors. service_bin_dir must contain moraine-ingest and moraine-mcp, unless MORAINE_SERVICE_BIN_DIR is set or source-tree mode is enabled. Release packages also retain moraine-monitor as a deprecated executable alias that delegates to the unified backend; moraine up never manages it separately.

Moraine's managed ClickHouse uses concurrency control with fair round-robin scheduling and a soft aggregate query-thread limit equal to the detected CPU core count. Individual repository reads do not override max_threads, so an idle query can scale up while concurrent queries share the aggregate budget. These defaults apply only to Moraine-managed ClickHouse; external ClickHouse deployments retain their administrator-defined scheduling policy.