03 What gets built
Written by both. Signed off by a human.
Every layer of your platform lands as a pull request in your repo. Here is what you get in each one, and the kind of thing the engineer catches.
Ingest
Get the data in
Pipelines written as plain Python and orchestrated as assets, from SaaS tools, databases and files into your warehouse. Version-controlled, testable and readable by any Python developer.
Source inventory and load strategy
Incremental loads with sensible watermarks and retries
Credentials in your secret store, never in code
What the engineer caught
The AI wrote the watermark on a creation date, which never sees edits to old rows. The engineer moved it to the update date with a lookback and added retries.
Under the hood
DagsterPythondlt
Pipelines are Python-first. dlt, the open-source Python library for loading data, runs inside Dagster assets, so its REST API and SQL database sources, its library of verified sources and its many destinations are all within reach. A source is declared as configuration (base URL, auth reference, resources, pagination, incremental cursor), and a read-only probe proposes that configuration for you to approve.
- Declared API sources with dlt✓ Supported
- Plain Python assets for anything else✓ Supported
- Landing in Snowflake✓ Supported
- Landing in PostgreSQLPreview
- Tables landed by another toolPlanned
Transform
Make it trustworthy
Layered SQL models with tests, lineage and living documentation. Every column that matters has a definition next to the code that produces it.
Staging, core and mart layers
Tests on keys, relationships and freshness
A data dictionary generated from the code
What the engineer caught
Lifetime value was computed before refunds. The engineer netted them out so finance and product see one number.
Under the hood
dbtElementary
dbt models in layers: a generated, warehouse-neutral staging layer, then your own models, with tests, documentation and lineage. The engine is pinned and checksum-verified, and packages are pinned and locked. Elementary adds observability: run and test results, schema-change tests and a report.
- dbt (Fusion engine)✓ Supported
- dbt-core, used for PostgreSQLPreview
- Elementary observability✓ Supported
- Snowflake✓ Supported
- PostgreSQLPreview
Orchestrate
Make it run, in order
Dependency-aware runs, retries, alerting and freshness gates, so a failure is a notification and a report never refreshes from a half-built model.
An explicit run order, a then b then c
Freshness gates before reports refresh
Alerting to where your team already is
What the engineer caught
The report was wired to run after the load, skipping the model in between. The engineer fixed the order and added a freshness gate.
Under the hood
DagsterPythondlt
Dagster is the orchestrator and Python is the language. Pipelines, dbt models and checks all become assets in one graph, so run order, retries, freshness checks and alerts come from the same definitions and lineage is visible end to end. dlt runs inside the assets as a package, which opens up its whole catalogue of integrations. Where Dagster runs, self-hosted or hosted, is your choice.
- Dagster✓ Supported
- PrefectPlanned
- Apache AirflowPlanned
Govern
Own the foundation
Environments, roles and cost controls as code. Plans are reviewed by a person before anything is applied.
Dev, test and prod from one definition
Least-privilege access by design
Cost guardrails built in
What the engineer caught
The analyst role was granted everything. The engineer narrowed it to read-only on one schema.
Under the hood
TerraformOpenTofu
Terraform describes the infrastructure. Policy-as-code rules check it before anything is applied (no public storage, encryption on, key rotation on, least privilege), and a person reviews the plan. Azure is the first-class cloud; AWS is the same platform with only the cloud-specific parts swapped. Secrets stay in your store and code holds references.
- Microsoft Azure✓ Supported
- Amazon Web ServicesPreview
- Google CloudPlanned
- TerraformPreview
- OpenTofuPlanned
- GitHub and GitHub Actions✓ Supported
- Azure Repos, Azure Pipelines, BitbucketPreview
- GitLabPlanned
Visualize
Land it where people look
Reports and semantic models under version control, with definitions and descriptions that cannot drift from the models underneath.
One governed model, many reports
Descriptions that do not rot
The definition of revenue lives in code
What the engineer caught
Revenue was summed before refunds. The engineer changed the measure to net, so the report, the dictionary and any question asked of the data agree.
Under the hood
Power BI
Reports and semantic models are kept as plain files (the Power BI project format), so they diff and review like code. Descriptions from the data dictionary are written into the semantic model through a pull request, so what the report says matches what the models say. Power BI is the supported BI tool; others are on the roadmap.
- Power BI✓ Supported
- LookerPlanned
- TableauPlanned
- MetabasePlanned
Ask
Questions in plain language
Business users ask in a chat and get answers built only from governed measures. The AI chooses a measure and filters; it never writes the query. Every answer shows its definition and query.
Governed measures only
Runs as the person asking, so their permissions apply
Refuses what it cannot answer well, and says why
What the engineer caught
The AI ran the questions as a shared account, so everyone would see every region. The engineer switched it to the asking user and made the query visible.
Under the hood
Power BI measuresYour AI provider
Questions are answered from governed Power BI measures. A model of your choice picks the measure and the filters; it never writes the query. You bring your own provider, and Microsoft's own AI surfaces are available as an alternative engine for teams that want questions to stay inside that boundary.
- Anthropic, OpenAI, Azure OpenAI, a local modelPreview
- AWS Bedrock, Google Vertex AIPlanned
Apps
Small apps on governed data
Internal tools, planners and write-back apps live in the same repo, read the same models and appear in lineage like everything else.
Apps are nodes in lineage
Reads and writes mapped to columns and tables
Write-back that returns to the pipeline
What the engineer caught
A planner had a hardcoded assumption and unattributed writes. The engineer read the assumption from a governed table and recorded who submitted each scenario.
Under the hood
StreamlitPython
Apps are Python, typically Streamlit, reading your marts and writing their own tables back. The platform reads app code without running it, works out which columns each app reads and which tables it writes, and puts apps in lineage and blast radius. Hosting apps for you is not built yet; apps run where your existing ones do.
- Apps in lineage and blast radiusPreview
- Hosting apps for youPlanned