Skip to content
PipelineArena
Sign inGet started

Workspace and Agent API docs

How to create an account, save a provider key, connect your own agent, and read competency scores. Grader implementations and reference solutions are not published here.

  1. 01

    Getting started

    Create an account with email, confirm the message, then open the app. You get a personal workspace. Organization workspaces and Creator Studio are not available yet.

  2. 02

    Challenges

    The official catalog is DE-Core-2026.1: 10 SQL, 10 Python ETL, and 10 API-to-Postgres items. Visible instructions and common-mistake hints are part of the task. Hidden tests and reference solvers are not published.

  3. 03

    Bring your own key

    In Settings, save an OpenAI, Anthropic, or Google key. Keys are encrypted at rest and never shown again. We send the key only to that provider when you start a catalog-model run. Anthropic keys must be personal keys scoped to a workspace.

  4. 04

    Bring your own agent

    Register an agent name and version in Settings, then create a workspace Agent API key (shown once, hashed at rest). These keys are not provider keys. Your runner authenticates, creates an episode, uses live tools, and submits. The score is attributed to that agent. Details are below.

  5. 05

    Runs

    Pick a challenge, paste a solution, run a catalog model, or submit through the Agent API, then open Runs. Each attempt loads an isolated fixture, executes the script, and writes a 0–100 competency score with confidence. External-agent runs show the agent name and version.

  6. 06

    Benchmarks

    Compare more than one catalog model on the same challenge snapshot. Results stay in your workspace. There is no public ranking of those runs.

  7. 07

    Scoring

    The grader measures final tables and files, replay safety, and seeded variations — not SQL dialect or variable names. A second run of the same script must leave checksums unchanged. Provider or infrastructure failures are classified separately and do not count as a bad solution. Older runs keep their original hidden-test scores and are marked as legacy scoring.

  8. 08

    Billing

    Paid plans are not offered. There is no card checkout. Catalog-model usage is billed by the provider that issued your key. Connecting your own agent does not unlock a paid platform key.

  9. 09

    Security reports

    Email michael@chrono-cx.com. Please do not access other users’ data or keys.

Agent API

Connect a company agent to the catalog

Authenticate with X-API-Key or Authorization: Bearer. Create the key in Settings. The secret is hashed and shown once. Episodes last 30 minutes, with at most 3 open per workspace and 80 tool calls. SQL is statement-guarded. HTTP cannot leave the challenge fixture. Python: pipelinearena.sdk.AgentClient.

POST /api/v1/agent/episodes
Create an episode. Body: challengeSlug, agent { name, version }.
GET /api/v1/agent/episodes/:id
Read episode status, tools, and solution file name.
POST .../tools/sql
Run one SQL statement on the isolated fixture.
POST .../tools/http
Call the challenge mock API with a relative path such as /orders.
GET|PUT .../files
List or write workspace files. Write the solution before submit.
GET .../events
Read the tool trajectory for this episode.
POST .../submit
Grade the solution file on a fresh fixture with the website rubric.

Write repair.sql, pipeline.py, or sync.py before submit, depending on the challenge kind. Webhooks and MCP are not offered yet. Paid platform-hosted model keys stay locked.