Chimera

Select a dataset on the left.

Stats dataset metrics, replication status, and distribution

Loading stats…

Dataset

Profile
Role
Records
Bytes
Memtable rows
Segments

Tier breakdown

Tier	Records	Bytes

Status
Direction
Lag

Schema what the dataset is and how it reads on the SQL plane

delete this dataset…removes WAL, manifest and every local tier — irreversible; refused while sharded, mirrored out, paired or hosted

Query

min ts (ms) max ts (ms) series / stream token (logs)

Documents

key

Vector search nearest neighbours over an embedding field · docs/design/020

field k metric query text (embeddings provider) or vector JSON filter JSON {"/lang":"en"}

Configure object storage tier · memory & disk budgets for this dataset

Enable object storage send cold data to

When enabled, the dataset's cold tier (☁️) flushes sealed segments to the selected object store. Register object stores under System → Object Storage. Leave disabled to keep every tier on this node (🧠 memory / 💾 disk).

memory budget disk budget

Soft caps for the in-memory (🧠) and on-disk (💾) tiers. When a tier exceeds its budget, the oldest sealed segments spill to the next colder tier (disk, then object storage if enabled). Blank = unlimited (node default).

Delete dataset permanently remove the selected dataset from this node — irreversible

To delete , type its exact name below and confirm. This removes its WAL, manifest and every local tier and cannot be undone; it is refused while the dataset is sharded, mirrored out, paired or hosted.

confirm dataset name

Ingest into

One JSON object per line. time series: {"series":"host0","ts":1700000000000,"value":1.5} · logs: {"stream":"{app=\"web\"}","ts":1700000000000,"line":"INFO ok"} · documents: {"key":"pk=user1","doc":{"pk":{"S":"user1"}}} (omit doc to delete)

dry run

SQL DataFusion over every dataset · read-only

Decision log why the system did what it did

dataset limit

Lineage OpenLineage — where a table's data comes from and where it goes · docs/design/016

dataset depth

upstream

downstream

Runs latest events touching the selected dataset

Emit forward every event to another OpenLineage backend (Marquez, …); Chimera itself accepts events on `POST /api/v1/lineage`

url bearer token → secret name enabled

Schedules recurring transform jobs with a cursor · docs/design/017

name schema target dataset every (s) key columns (comma)

source SQL ($1 = cursor, $2 = upper bound)

cursor hi SQL (one scalar) initial cursor

Each run: freeze the upper bound, transform(source SQL, schema) over rows in (cursor, bound], write the conformed rows as documents into the target (one JSON field per schema column), advance the cursor. The mapping is authored once and cached (015).

Transform schemas + the model that authors mappings · docs/design/015

provider model API key → secret name endpoint attempts

embeddings model API key → secret name embeddings endpoint (OpenAI-compatible) dims

schema name columns JSON [{"name","type","required","description"}] · types: string int double bool timestamp date json

SQL: CALL transform('SELECT …', 'schema') · REST: POST /v1/transform {sql, schema}. First call: the model writes a SQL mapping (table input), it is validated against the data and cached; later calls consult no model. Hand mappings: POST /v1/transforms {sql, schema, mapping_sql}.

Secrets sealed at rest, write-only · docs/design/010

name value

Sign-in OpenID Connect (OIDC) for this UI and the REST API · docs/design/013

provider issuer

Advanced — explicit endpoints (leave blank to auto-discover from the issuer)

auth url token url jwks url

client id client secret → secret name redirect url

allowed emails (comma-separated) allowed domains API token → secret name

Machine clients (the protocol app, PocketBase hooks) send Authorization: Bearer <api_token secret>. Peer links and wire frontends are unaffected.

API tokens named, scoped bearer tokens · read ⊂ write ⊂ ops ⊂ admin · the secret is shown once

name read write ops admin datasets allowlist (comma-separated; empty = all)

Message queues MQTT subscriptions landing in a dataset · docs/design/014

name broker (host:port · mqtts://host:8883 for TLS) topics (comma-separated) qos

username → secret password → secret target dataset mode value/key pointer

Each message is a record {topic, payload, received_ms, qos, retain}; pointers address that record. Documents default to one document per topic (LWW).

REST API wizard discover an API's endpoints, add them as datasources, grouped into a collection · docs/design/010

Give an API's base URL; Chimera probes its OpenAPI/Swagger spec (or asks the configured model under System → Transform) and lists the endpoints for you to pick.

REST datasources an external API polled into a dataset · docs/design/010

Give a URL; Chimera probes it and drafts the definition (with the configured LLM when one is set under System → Transform), you review and save.

Add datasource · step 1 of 2 the URL of the API endpoint that returns the records

url

Chimera fetches the URL once without credentials to see the response shape; a 401/403 is fine — the draft will point at a {{secret:…}} header for you to fill in.

S3 datasources objects in a bucket parsed into a dataset · anonymous (public buckets) · docs/design/018

name bucket / prefix region key contains suffix

format json path target dataset mode series / key / stream pointer value / line pointer ts pointer every (s) max objects/run access key → secret name (blank = anonymous) secret key → secret name session token → secret name (optional)

Each run lists the prefix, reads new or changed objects (by ETag), parses them (CSV header → fields, NDJSON, JSON array at a pointer) and lands the records through the same extraction as REST/MQTT. Example: s3://noaa-ghcn-pds/csv/by_station/, contains USW00094728, points series /ELEMENT value /DATA_VALUE ts /DATE.

Object Storage S3-compatible write destinations — AWS S3, MinIO, Cloudflare R2, Backblaze B2, Wasabi · credentials sealed at rest

name bucket / prefix region endpoint (blank = AWS S3)

access key → secret name (blank = anonymous) secret key → secret name session token → secret name (optional) enabled

Credential secrets must exist first (System → Secrets). Test connection does a real round-trip — a signed PUT of a tiny probe object then a best-effort DELETE — so it verifies endpoint reachability, credentials, bucket existence and write permission. Endpoint blank targets AWS S3 (bucket.s3.region.amazonaws.com); set it for MinIO/R2/B2/Wasabi.

Collections named groups of datasets · a collection is a SQL schema (`collection.dataset`) · `system` is reserved · docs/design/019

name description

collection datasets

MCP servers an external MCP server as a datasource · discover → auth → resources → create

Database pairings a live replica of a DynamoDB table or Postgres table · docs/design/012

Auto-update

Chimera checks GitHub for new releases. Turn on auto-update to apply them automatically, or leave it off and update by hand from the list below.

Running version: …

auto-update channel

Releases

Loading…

This node identity · docs/design/011

Give the fingerprint and peer URL to the other operator; both sides must register each other (pinned mutual TLS, no CA).

Peers registered, pinned

name peer url fingerprint (sha-256 hex)

Mirrors outbound (this node owns) · inbound (a peer owns; promotable here)

dataset (collection → dataset; only datasets this node owns) to peer

Outbound

Inbound

Shards one dataset across several nodes · 011 §4

dataset members (peer names, comma-separated; this node is always included) keyed fraction numeric fraction

Replication one dataset · N copies · chosen nodes · chosen tiers · 026

loading…

Each row is one copy. Copy 1 is the writer (Owner); extra copies are read-only Replicas established over the mirror transport (026 §2). Health & freshness are composed live from /v1/status, /v1/mirrors & /v1/shards.

Configure replication one writer, add/remove replica copies · 026 §3

dataset desired copies mode

participant nodes this node is always copy 1 (the writer) · pick peers to hold the rest

loading…

Dataset status pairing · mirror · sharding · 011/012

loading…

Set these up under System → Pairings and Peers → Mirrors / Shards. Shard health is a live describe per member.

/v1/status

Endpoints

REST on this port · MySQL wire on 3306 · Arrow Flight SQL on 50051 · DynamoDB API on 8000 (see docs/design/004, 006). MCP: chimera mcp serve.

Add dataset

Stats dataset metrics, replication status, and distribution

Dataset

Tier breakdown

Pairing

Mirrors

Shards

Schema what the dataset is and how it reads on the SQL plane

Query

Documents

Vector search nearest neighbours over an embedding field · docs/design/020

Configure object storage tier · memory & disk budgets for this dataset

Delete dataset permanently remove the selected dataset from this node — irreversible

Ingest into (select a dataset)

SQL DataFusion over every dataset · read-only

Decision log why the system did what it did

Lineage OpenLineage — where a table's data comes from and where it goes · docs/design/016

upstream

downstream

Runs latest events touching the selected dataset

Emit forward every event to another OpenLineage backend (Marquez, …); Chimera itself accepts events on POST /api/v1/lineage

Schedules recurring transform jobs with a cursor · docs/design/017

Transform schemas + the model that authors mappings · docs/design/015

Secrets sealed at rest, write-only · docs/design/010

Sign-in OpenID Connect (OIDC) for this UI and the REST API · docs/design/013

API tokens named, scoped bearer tokens · read ⊂ write ⊂ ops ⊂ admin · the secret is shown once

Message queues MQTT subscriptions landing in a dataset · docs/design/014

REST API wizard discover an API's endpoints, add them as datasources, grouped into a collection · docs/design/010

REST datasources an external API polled into a dataset · docs/design/010

Add datasource · step 1 of 2 the URL of the API endpoint that returns the records

Add datasource · step 2 of 2 review the draft, then save

S3 datasources objects in a bucket parsed into a dataset · anonymous (public buckets) · docs/design/018

Object Storage S3-compatible write destinations — AWS S3, MinIO, Cloudflare R2, Backblaze B2, Wasabi · credentials sealed at rest

Collections named groups of datasets · a collection is a SQL schema (collection.dataset) · system is reserved · docs/design/019

MCP servers an external MCP server as a datasource · discover → auth → resources → create

Database pairings a live replica of a DynamoDB table or Postgres table · docs/design/012

Auto-update

Releases

This node identity · docs/design/011

Peers registered, pinned

Mirrors outbound (this node owns) · inbound (a peer owns; promotable here)

Outbound

Inbound

Shards one dataset across several nodes · 011 §4

Replication one dataset · N copies · chosen nodes · chosen tiers · 026

Configure replication one writer, add/remove replica copies · 026 §3

Per-copy placement only when copies > 1 · the writer stays durable (026 §8)

Dataset status pairing · mirror · sharding · 011/012

/v1/status

Endpoints

Ingest into

Emit forward every event to another OpenLineage backend (Marquez, …); Chimera itself accepts events on `POST /api/v1/lineage`

Collections named groups of datasets · a collection is a SQL schema (`collection.dataset`) · `system` is reserved · docs/design/019