Version 0.3: the docs on
maintrack the next release; for the release you run, read the docs at its tag. The API may change between 0.x minor releases, before 1.0. An upgrade never re-interprets a stored definition: it keeps its meaning, or is paused with the reason; see grammar versions.
A declarative API for creating data transformations in PostgreSQL. Correctness and performance are the key design goals. All state managed in Postgres for simplified architecture and efficient, provably correct transforms. The known exceptions to that guarantee are listed in known correctness gaps.
Applications often compute the same derived facts over and over — the same aggregate, join, or enrichment recomputed on every request. That recomputation is spikey read load: the cost lands at query time, scales with how often you read, and grows as the underlying data grows, so the queries that matter most are the ones that get slow under pressure.
Trellis inverts the trade-off. Instead of recomputing a derived table on every read, it maintains that table incrementally: when a source row changes, only the affected target rows are updated. This replaces spikey read load with a steady write load — work proportional to how fast the source changes, not to how often you query — and leaves behind a plain table that is cheap and predictable to read.
The trade-off is worth it when a fact is read far more often than its inputs change, or when read latency matters more than write throughput. It is the same bargain a database index makes — pay a little on write to make reads fast.
Connections to Postgres use TLS when the database URL asks for it (sslmode, sslrootcert, sslcert, sslkey) and try it first when it does not; see TLS to Postgres.
Pick the form that matches how you run it:
- The
trelliscommand-line tool for defining transforms and running the pipeline from a shell: downloadtrellis-v<version>-<target>.tar.gzfor your platform from the GitHub release and unpack it, or build it withcargo install --path clifrom a checkout. - Rust: the engine crate is published as
trellis-pg(its library istrellis), sotrellis-pg = "0.3"inCargo.tomlanduse trellis::...in code. - Elixir: the Hex package
trellis_pg; see clients/elixir. - Ruby: the gem
trellis-pg; see clients/ruby.
Trellis supports PostgreSQL 15 to 18 (details). docs/embedding.md is the guide to running Trellis inside an application, docs/upgrading.md says how to upgrade Trellis and roll it back, and docs/releasing.md lists what each release ships.
Trellis runs against a PostgreSQL database and a role that can create objects in it (see Roles and permissions). Given a source table and, for an aggregate, an index on its GROUP BY columns:
CREATE TABLE orders (id bigint PRIMARY KEY, customer_id bigint NOT NULL, amount numeric(12,2) NOT NULL);
CREATE INDEX ON orders (customer_id);define a transform and run the pipeline:
export TRELLIS_DATABASE_URL=postgres://user@localhost/shop
trellis apply "TRANSFORM customer_totals FROM orders GROUP BY customer_id
SELECT customer_id AS customer_id, COUNT(*) AS order_count, SUM(amount) AS total"
trellis run # until interrupted: captures changes and keeps the target current
trellis status # in another shell: each definition's status, warnings and quarantined rowscustomer_totals is now an ordinary table. It holds one row per customer, backfilled from the rows already in orders, and stays exactly equal to SELECT customer_id, COUNT(*), SUM(amount) FROM orders GROUP BY customer_id as orders changes. docs/transforms.md is the reference for the grammar (1-1, aggregate, JOIN and 1-1 JOIN transforms) and for which sources and targets are supported, docs/expressions.md for what a calculated field may contain, and docs/known-correctness-gaps.md lists the ways a target can go stale or wrong without Trellis repairing it.
This repo is home to a few key components:
trellisis the rust crate that users will download and install into their application codecli(packagetrellis-cli) builds thetrellisoperator-facing binary, which wraps thetrellislibrary crate for defining transforms, running the live pipeline, checking status, and dumping definitions to restore into another database from a shell — runcargo run -p trellis-cli -- --help(ortrellis --helponce installed) to see what it can dodocsoutlines the key ideas and decisions this project has taken; docs/README.md indexes it by reader, docs/glossary.md defines its terms and docs/architecture.md maps modules to concepts; docs/recommendations.md is the operator's guide to running Trellis well (WAL and checkpoint tuning, ring sizing, roles and permissions), docs/operations.md says what to do when Trellis stops a definition and how each repair behaves, docs/status-reference.md defines each status word and status field, docs/upgrading.md how to upgrade and roll back, and docs/releasing.md says how a release is cut and publishedclientsholds the Elixir and Ruby bindings for running Trellis inside a host app; docs/embedding.md is the guide to embedding itbenchmarkis a harness for measuring throughput, latency and other key-metricsgenerativeis a generative test suite that is used to validate correctness under various scenarios and loadstestkitis a reusable, test-only harness that spins up a throwaway PostgreSQL instance and hands out isolated, migrated databases per test scenario. It is never shipped in a user's binary — other crates pull it in as a dev-dependency and exercise it throughcargo test. Itstrellis-testkitbinary provisions the same kind of cluster for test suites written in other languages; seecargo run -p testkit --bin trellis-testkit -- --help.
This project uses asdf to define the default versions used for local/development work.
From the root of this project run asdf install postgres to make sure you have the appropriate version.
For MacOS you often need to set brew install
icu4cand then setexport PKG_CONFIG_PATH="$(brew --prefix icu4c)/lib/pkgconfig:$PKG_CONFIG_PATH"before running the asdf install command