Otter/Docs

Docs

Install, release, run.

You do not need this website's repository to use Otter. Install the binary and work in your own directory. The authoritative reference lives in the project docs; this page is the shortest path to something running.

  • Homebrew cask · macOS
  • .deb · Ubuntu / Debian
  • release tarball
  • go install

01 · Quickstart

Install, scaffold, release, run.

install
# macOS — a Homebrew cask
brew install tkoizumi/tap/otter

# Ubuntu / Debian — a .deb per architecture, named with the version
VERSION=0.5.3; ARCH=amd64                     # amd64 or arm64
curl -LO "https://github.com/tkoizumi/otter/releases/download/v${VERSION}/otter_${VERSION}_linux_${ARCH}.deb"
sudo apt install "./otter_${VERSION}_linux_${ARCH}.deb"

Both packages install otter (the client) and otterd (the same daemon with its own flag defaults). Python 3 is required on the machine that runs jobs, unless a job opts into python.mode: managed.

first job
otter init my-project        # a workspace and one job
cd my-project

otter validate my-project    # check the manifest without a daemon
otter release my-project     # stage and activate an immutable snapshot
otter start --detach         # free loopback port; records where it listens
otter run my-project         # queue a run and follow the outcome
otter state get my-project count
otter stop

Release before you run. A run executes the job's active release rather than its source tree, so otter run refuses until the job has been released once. After an edit, release again or the run keeps executing the previous snapshot.

02 · Anatomy of a job

A directory with a manifest and an entrypoint.

layout
my-project/
├── jobs/
│   └── counter/
│       ├── otter.yaml      # what the runtime needs to know
│       ├── main.py         # the program
│       ├── logic.py        # pure logic, captured into the release
│       └── test_logic.py   # ordinary unit test
└── otter.env           # optional; loaded by `otter start`

Every otter.yaml under the jobs root is discovered recursively, validated and registered. A broken manifest is reported and marked invalid — it never takes the daemon down.

03 · Manifest

Everything the runtime decides for you, declared once.

otter.yaml
version: 1

name: salesforce-to-netsuite
description: Sync changed accounts into NetSuite.
entrypoint: main.py

trigger:
  cron: "*/5 * * * *"
  webhook:
    enabled: true

timeout: 300
concurrency: 1

retry:
  attempts: 5                # total attempts: one initial plus up to four retries
  backoff: exponential       # none | linear | exponential
  initial_delay: 2s
  max_delay: 60s

env:
  NETSUITE_ACCOUNT: ${NETSUITE_ACCOUNT}

secrets:
  - SALESFORCE_TOKEN
  - NETSUITE_TOKEN
KeyWhat it controls
nameHow every CLI verb refers to the job. The directory name does not have to match.
entrypointThe Python file executed for each run.
trigger.cronFive-field schedule, re-registered from the manifest at every start.
trigger.webhookOpt in to a token-guarded POST /v1/hooks/<job>.
timeoutSeconds before SIGTERM, then SIGKILL about five seconds later.
concurrencySimultaneous runs of this job. Extra triggers queue instead of racing.
retryAttempts, backoff shape, first delay and the cap.
secretsVariables read from the daemon's environment and injected into the child.
python.modehost (default) or managed for a prepared interpreter with locked dependencies.
capturefull, metadata or off for this job's HTTP recording.

The manifest schema is versioned by version: and covered by the manifest reference.

04 · Python SDK

Standard library only, shipped inside the binary.

On first start the SDK is extracted to <data>/sdk/python and prepended to the child's PYTHONPATH, so import otter works with no pip install step.

@run
from otter import run

@run
def main(ctx):
    cursor = ctx.state.get("cursor")
    records = get_changed_accounts(cursor)

    for record in records:
        upsert_customer(record)

    if records:
        ctx.state.set("cursor", records[-1]["last_modified"])
Context.from_environment()
from otter import Context

ctx = Context.from_environment()

count = ctx.state.get("count") or 0
count += 1
ctx.log.info("Counter executed", count=count)
ctx.state.set("count", count)
MemberDescription
ctx.state.get(key, default=None)Read a JSON value; default when unset.
ctx.state.set(key, value)Persist any JSON-serialisable value.
ctx.state.delete(key)Remove a key; returns True if it existed.
ctx.state.all()Every key/value pair for this job.
ctx.log.debug/info/warning/error(msg, **fields)Structured log line, stored against the run.
ctx.run_id · ctx.job_idIdentifiers for the current run.
ctx.trigger.typemanual, cron or webhook.
ctx.trigger.body / headers / header(name)The webhook payload, when there is one.

The authoritative state lives in the daemon; the SDK is a thin HTTP client, so a crashed or killed process can never corrupt it.

05 · Triggers

Three ways a run starts, all recorded the same way.

Cron

trigger:
  cron: "*/5 * * * *"

Standard five-field expressions, re-registered from manifests on every start, so a restart never loses a future schedule. Occurrences missed while the daemon was offline are not replayed — that behaviour is documented rather than implied.

Manual

otter run salesforce-to-netsuite

Always available, regardless of manifest configuration. Use it for the first run, a backfill, or a repair — it goes through the same queue and policy as everything else.

Webhook

POST /v1/hooks/<job>
X-Otter-Token: …

Opt in per job. Otter generates a static token on first start and persists it. A call without the token gets a 401 — unauthenticated execution is never exposed.

inspecting the token
$ otter inspect shopify-to-erp
webhook:       enabled
webhook url:   POST http://127.0.0.1:7337/v1/hooks/shopify-to-erp
webhook token: 5cde21dc53f207cb621b98df53d4caa06d1c15ffb98a084165f2094bdee757c1

06 · Reliability

The behaviours that make a job safe to leave running.

Timeouts

timeout: 300 sends SIGTERM to the process group, waits about five seconds, then SIGKILL. The run is marked timed_out and the retry policy applies.

Retries

retry.attempts is the total number of attempts. Configuration failures — a missing secret, a missing entrypoint, an invalid manifest — are never retried, because the next attempt would fail identically.

Crash recovery

On start, any run left running is marked failed with otter daemon restarted during execution, and a fresh attempt is enqueued when policy allows. Recovery is fail-closed.

Graceful shutdown

On SIGTERM/SIGINT Otter stops claiming work, gives running jobs --shutdown-grace (default 15s), terminates what is left, records those runs, and closes SQLite cleanly.

Checkpointing

A job saves progress in ctx.state and reads it on the next run. Otter does not restore Python execution at the point of failure; your code decides where to resume.

Attempt chain

Each attempt is its own run record linked by parent_run_id, so the whole retry history is auditable from one run id.

otter run-status <run-id>
run id:  afd1b271-7e83-4bbc-a202-a7592fc0320d
job:     flaky
status:  failed
attempt: 1
error:   process exited with code 1

attempts:
  RUN ID                                ATTEMPT  STATUS     STARTED   DURATION
  afd1b271-7e83-4bbc-a202-a7592fc0320d  1        failed     13:16:22  150ms
  b69229bd-96af-4c4e-a9cd-77d5190651a4  2        failed     13:16:23  120ms
  8a6c3c9e-8cbe-4a23-a287-1f7a6cd5fa18  3        succeeded  13:16:25  103ms

07 · HTTP capture

The failure you can still read on Monday.

Runs record their outgoing HTTP by default — sanitized headers and bounded JSON bodies included — because capture observes live traffic and cannot be turned on after an unattended failure. otter trace <run-id> merges lifecycle lines, captured output and HTTP exchanges into one chronological page.

otter trace 3f2a91c4
job: orders-sync   status: failed   attempt: 1
release: 8c1d4f0a9b3e   trigger: manual   duration: 41ms
error: process exited with code 1
retry context: otter run-status 3f2a91c4-7d18-4a6e-8b21-5c0d9e4a17bb

TIME      KIND       DETAIL
03:30:00  ·  lifecycle  run queued (trigger manual)
03:30:00  ·  lifecycle  run started (attempt 1 of 1, trigger manual)
03:30:00  ✓  http       POST api.example.test/v2/cursor -> 200, 6ms  [1]
03:30:00  !  http       POST api.example.test/v2/records?access_token=REDACTED -> 400, 9ms  [2]
03:30:00  ×  log        [stderr] RuntimeError: upstream rejected the batch
03:30:00  ×  lifecycle  run failed (attempt 1, 41ms), exit code 1

└1.  main.py:12; completed; payloads: full
└2.  main.py:26; completed; payloads: full
  • Covers urllib always, plus requests and httpx (sync and async) when the run's interpreter has them installed
  • Headers, query parameters and JSON fields are redacted; extra names are configurable per deployment
  • Retention is bounded by --capture-retention, defaulting to 168h
  • A job opts down with capture: metadata or capture: off

08 · CLI

Every verb, with --json where it matters.

the surface, abridged
otter status                        # queue age and depth, run counts, freshness, storage
otter jobs [--all]                  # registered jobs; --all adds invalid and retired
otter inspect <job>                 # manifest view, triggers, recent runs
otter reload                        # re-read jobs without restarting running work
otter pause [<job>|.]                # suspend cron and webhook; manual runs still work
otter resume [<job>|.]               # re-arm what a pause suspended

otter run [<job>]                    # queue a manual run; prints the run id
otter runs [<job>] [--all]           # one job, or every job with --all
otter cancel <run-id>                # stop a queued or running run; never retried
otter run-status <run-id>            # one run plus its retry attempts
otter logs <run-id> [--follow]       # captured output
otter trace <run-id>                 # context, output and HTTP in time order

otter requests <run-id>              # captured outgoing requests
otter request <request-id>           # one exchange, run resolved for you
otter state get|set|delete <job> <key>

otter init [name]                    # scaffold a workspace and one job
otter validate <otter.yaml|dir|name> # validate without a running daemon
otter start [--detach] / otter stop  # this project's runtime
otter deploy --host <user@host>      # install or update a runtime over SSH
otter prepare [<job>]                # prepare opt-in managed Python environments
otter release [<job>|.] [--all]       # stage and activate an immutable release
otter release --list <job>           # staged releases, newest first
otter release --activate <digest> <job>   # roll back to a staged release

Global flags: --api <url>, --token <token>, --json, --version. Without --api, the daemon URL is read from the nearest .otter/serve/listen.url, walking up from the working directory.

09 · HTTP API

A loopback API your scripts can rely on.

GET /v1/version and GET /health are public, so a client can establish what shape and schema it is talking to before it holds a credential. Loopback binding needs no token; binding elsewhere requires OTTER_API_TOKEN, and the daemon refuses to start without it.

endpoints
GET    /health
GET    /v1/version
GET    /v1/jobs
GET    /v1/jobs/{id}
POST   /v1/jobs/{id}/runs            ?capture=off|metadata|full
GET    /v1/runs                      ?job_id=&status=&limit=&offset=
GET    /v1/runs/{id}
GET    /v1/runs/{id}/logs            ?after_id=&limit=
GET    /v1/runs/{id}/requests        ?after_id=&limit=
GET    /v1/runs/{id}/requests/{request_id}
POST   /v1/runs/{id}/cancel
GET    /v1/jobs/{id}/state
GET    /v1/jobs/{id}/state/{key}
PUT    /v1/jobs/{id}/state/{key}
DELETE /v1/jobs/{id}/state/{key}
POST   /v1/hooks/{job}
what am I talking to?
curl -s http://127.0.0.1:7337/v1/version
response
{
  "schema_version": 1,
  "contract_version": 3,
  "product_version": "0.5.3",
  "manifest_schema": 1,
  "sdk_version": "0.1.0",
  "supported_platforms": ["linux/amd64", "linux/arm64", "darwin/amd64", "darwin/arm64"]
}

10 · Deploying

Over SSH, or to managed compute.

Self-hosted

otter deploy --host deploy@10.0.0.4

Cross-compiles locally, pushes the binaries, creates an unprivileged service account and installs the systemd unit. Each job is staged under <remote>/jobs/<name> with every declared tree at the relative depth the manifest names, so a release digest means the same layout everywhere. otter deploy --status reports what this project last deployed.

Otter Cloud · alpha

otter login
otter deploy --cloud

Promotes an existing local release onto a pooled tenant we operate — --build restores packaging if you want it. Credentials are org-bound, and jobs can be deleted. A deploy prunes inactive snapshots to --keep (default 3) while always keeping the previous release as the rollback target.

daemon configuration
otterd \
  --jobs /opt/otter/jobs \
  --data /var/lib/otter \
  --listen 127.0.0.1:7337 \
  --workers 8
Flag / envDefaultPurpose
--jobs · OTTER_JOBS_DIR./jobsRoot scanned recursively for otter.yaml.
--data · OTTER_DATA_DIR./tmpHolds otter.db and the extracted SDK.
--listen · OTTER_LISTEN127.0.0.1:7337HTTP API address.
--workers · OTTER_WORKERSCPU cores, max 8Maximum concurrent runs across all jobs.
--api-token · OTTER_API_TOKENnoneBearer token; required for non-loopback binding.
--shutdown-grace15sTime running jobs get on shutdown.
--capture-defaultfullHTTP capture for runs that do not choose one.
--capture-retention168hHow long captured payloads are kept; 0 disables expiry.
--run-retention · --log-retentionforeverRetention for terminal run history and captured output.
--log-formatjsonStructured daemon logs; pretty during development.

11 · Limits

What Otter deliberately does not do.

  • It does not resume Python execution at the point of failure.
  • It does not replay cron occurrences missed while the daemon was offline.
  • It does not schedule across a fleet or shard work between hosts.
  • It does not inspect or transform your data — it runs your process.
  • It does not provide a visual DAG editor; control flow is code.
  • It does not guarantee delivery of a webhook the sender failed to deliver.

Each of these is stated in the published runtime contract as a promise or a non-promise. If a non-promise is load-bearing for you, that is important to know before you install rather than after.

Stuck on something specific?

The full documentation lives with the source, and issues are read.