Nineveh

Reading your data

Three ways out: a REST API for asking questions, a change feed for being told, and webhooks for being told somewhere else. All three serve the same tables.

This page is the HTTP contract. If you are reading from a React app, the hooks in Reading from React do all of it for you and generate the types for your tables — but what they do is exactly what is below.

Everything below uses your project's base URL and a key:

KEY=…                                 # Settings → API keys, shown once
BASE=https://api.nineveh.dev/projects/myproject
AUTH="Authorization: Bearer $KEY"     # the whole key, which starts with nvk_

Every read needs a key, and a key opens one project. Nothing here is public: a request without one is refused, and a key made for another project is refused too. Make one under Settings → API keys — it is shown once, so keep it then. Three places it can go, and they are tried in this order: an Authorization: Bearer header, an apikey header, or ?apikey= on the URL. Prefer a header. The query string is there for EventSource, which cannot set one, and a key in a URL ends up in logs and Referer headers.

Running it yourself, that's http://127.0.0.1:4000/projects/myproject instead, and a plane in local mode serves loopback without a key. Nothing else on this page changes.

REST

curl -H "$AUTH" $BASE/v1/status           # cursor, lag, whether a rebuild is running
curl -H "$AUTH" $BASE/v1/tables           # every table: kind, key, columns
curl -H "$AUTH" $BASE/v1/tables/balances  # rows

Rows come back with the query that produced them:

{
  "rows": [ { "user": "0xabc…", "balance": "7530", "_version": "6003184262" } ],
  "limit": 50,
  "offset": 0,
  "count": null
}

Narrowing it down

Parameter Meaning
limit how many rows, default 50, max 1000
offset skip this many
order ?order=balance.desc, up to four columns: ?order=balance.desc,user
count=exact also return the total, at the cost of a second query
<column>=<value> keep rows where the column equals that
<column>.<op>=<value> compare instead: eq, neq, gt, gte, lt, lte
curl -H "$AUTH" "$BASE/v1/tables/balances?user=0xabc…&order=balance.desc&limit=10"

# a range is two comparisons on one column, and a leaderboard breaks its ties
curl -H "$AUTH" "$BASE/v1/tables/balances?balance.gte=100&balance.lt=500"
curl -H "$AUTH" "$BASE/v1/tables/sellers?order=revenue.desc,seller&limit=10"

The operator goes on the parameter's name, not in front of its value, so a value is never reinterpreted: a string column holding gt.5 means exactly that. Several parameters are AND-ed, and naming one column twice is how you ask for a range.

That list of operators is the whole set, deliberately. Each one is a single comparison against a single column, which is what a database can answer from an index on that column. There is no like, no in, no or, and no condition spanning two columns — if you need one, it belongs in a reducer, computed once as the row is written rather than searched for on every request. A boolean column takes eq and neq only; which of two booleans is greater is not a question about your data. A JSON column cannot be filtered on at all.

Two things about every row

Wide integers are strings. u64, u128 and u256 don't fit a JavaScript number, so they're returned as decimal strings. Parse them with BigInt, never Number; the loss above 2⁵³ is silent.

_version says when the row last changed. It's the transaction that last wrote it, and it's what lets you tell a stale copy from a current one.

The change feed

Server-Sent Events, in commit order:

curl -N -H "$AUTH" "$BASE/v1/changes"
event: change
id: 11292175483.0
data: {"version":"11292175483","seq":0,"table":"balances","op":"update","key":{"user":"0xabc…"},"row":{…}}

op is insert, update or delete. The id is version.seq, and it is a position you can come back to:

?after=11292175483.0 resume just after that change
?after=beginning replay everything still kept
?tables=balances,volume only these tables

A browser resumes on its own: EventSource sends the last id it saw as Last-Event-ID when it reconnects, so a dropped connection costs nothing.

const feed = new EventSource(`${BASE}/v1/changes?apikey=${key}`);
feed.addEventListener("change", (e) => apply(JSON.parse(e.data)));

EventSource can't set headers, which is why the key goes in the query string here rather than in one.

A reset event means the tables were rebuilt and swapped in. Your cached copy is from the old build; reload it.

Webhooks

Nineveh calling your server, rather than your server asking Nineveh. Make one in Studio under Webhooks → Add an endpoint:

Studio's new endpoint dialog: a name, a URL, a grid of tables against insert, update and delete, and a choice of what each delivery carries
Four decisions, and the one that catches people is the third: a log table is append-only, so its update and delete are greyed out rather than offered and never fired.

Four things to decide:

  • Name. What its signing secret belongs to, and its place in the change order.
  • URL. https, or http to your own machine while you are building.
  • Which changes it hears. A row per table, a column per kind of change. A log table is append-only — its rows are written once and never touched again — so its update and delete are greyed out: they are subscriptions that could never fire.
  • What each delivery carries. The changed row, or only its key. The row is usually what you want: it is enough to act on without asking for it.

The endpoint is saved with the project, so it travels with it, and its signing secret is made for you.

Each POST carries up to 100 changes in order:

{
  "project": "myproject",
  "endpoint": "my_backend",
  "changes": [
    { "table": "balances", "op": "update", "version": "…", "seq": 0,
      "key": {…}, "row": {…} }
  ]
}

Check the signature

Every request carries X-Nineveh-Signature: t=<timestamp>,v1=<hex>, where v1 is HMAC-SHA256(secret, "<timestamp>.<raw body>"). Verify against the raw body, before any JSON parsing:

const [t, v1] = sig.split(",").map((p) => p.split("=")[1]);
const expected = createHmac("sha256", secret).update(`${t}.${rawBody}`).digest("hex");

if (!timingSafeEqual(Buffer.from(v1), Buffer.from(expected))) throw new Error("bad signature");
if (Math.abs(Date.now() / 1000 - Number(t)) > 300) throw new Error("too old");

The timestamp check is what stops someone replaying a request they captured earlier. Each endpoint's secret is on the Webhooks page, next to the endpoint it signs, and can be rotated whenever you like.

Delivery is at-least-once

A batch is retried until your endpoint answers 2xx, so you will occasionally see the same change twice. Two ways to be safe:

  • rows: true sends the row. Compare version and seq against what you've already applied and ignore anything older.
  • rows: false sends only the key: this row changed, come and look. Then fetch it. Duplicates stop mattering, because a fetch always returns what's current.

The second is less code and harder to get wrong. Use it unless you need the row in the delivery itself.