The Semantic Layer Nobody Maintains

In Everyone Is a Data Analyst Now, I described how anyone at AngelList can ask a data question in Slack and get an answer from Dana, our AI data analyst, in minutes. That post covered the pipeline and the experience. This one is for the dbt developers and analytics engineers who read it and asked the obvious follow-up: how do you make an LLM reliably write correct SQL against 1,400 mart models without a semantic layer?

The answer is that we do have something that plays the role of a semantic layer. We just didn't build it the way dbt (now part of Fivetran) or Snowflake would typically tell you to. There's no MetricFlow YAML, no CREATE SEMANTIC VIEW, no query-planning service between the question and the warehouse. Instead, there's a corpus of plain markdown files, knowledge and skills, that lives in our dbt repo. Most of it writes itself.

The problem semantic layers solve (and what they cost)

Everyone agrees on the problem. Raw tables aren't self-describing. A business concept like "net revenue" might live in a column called amt_ttl_pre_dsc, and unless the definition is centralized, every consumer reinvents the logic, and the logic diverges. At AngelList, "customer" alone is ambiguous: do you mean fund managers (GPs) or the limited partners who invest with them? "ICV" (Investment Capital Volume), "bookings", "ARR", "take rate": each has a canonical definition and a canonical way to query it, and getting them wrong produces numbers that look plausible and are quietly incorrect.

The mainstream answers to this problem share a shape:

• The dbt Semantic Layer, powered by MetricFlow, has you define semantic models and metrics in YAML on top of your dbt models. Downstream tools query metrics through dbt platform APIs (JDBC, GraphQL), and MetricFlow plans the SQL. Centralized definitions, consistent everywhere, and gated behind Starter/Enterprise tiers of the dbt platform.
• Snowflake semantic views move the same idea into the database: schema-level objects that model logical tables, relationships, dimensions, and metrics via DDL or YAML, consumed by Cortex Analyst and SQL.

Both are real solutions and both carry the same costs. You author a new artifact in a new DSL, separate from your models, and you maintain it forever. You put a query-time service between your users and your warehouse. You couple yourself to a vendor's runtime and pricing tier. And the maintenance burden lands on the data team: every new model, renamed column, or redefined metric is a change you must remember to mirror into the semantic layer, or it drifts.

That last cost is the one that kills semantic layers in practice. Documentation and metadata that require manual upkeep go stale, and stale semantics are worse than none. They're confidently wrong.

The contrarian bet: the semantic layer is documentation, not infrastructure

Here's the observation that changed our architecture: if the primary consumer of your semantic layer is an LLM agent, you don't need a query-planning service. You need legible, current, well-organized text. The agent writes the SQL itself. MetricFlow exists because BI tools can't read prose and reason about grain; an agent can.

So our "semantic layer" is two primitives, both plain markdown, both version-controlled in the dbt repo:

Knowledge is declarative: what things mean. A business glossary. An analytics knowledge file. A generated catalog of every mart model and column. A domain quick reference.

Skills are procedural: how to do tasks. Step-by-step instructions with names like query-warehouse, add-dbt-model, create-streamlit-app. Thirteen of them today.

Nothing about this is specific to one agent. The corpus is plain markdown in the repo, so any capable model can consume it. At AngelList we run both Devin and Claude against the same files, and the skills-and-knowledge convention (AGENTS.md, SKILL.md) is one both ecosystems understand. Swapping or adding an agent doesn't mean rebuilding the semantic layer; it means pointing another reader at the same text.

An AGENTS.md file at the repo root acts as the router. When a question arrives (say, via @Dana in Slack), the agent's session loads AGENTS.md, matches the task to a skill, and the skill tells it which knowledge files to consult and which integrations to use. The whole thing forms a decision tree: question → skill → knowledge → execution backend (Metabase MCP, Snowflake, dbt). We even have a skill whose job is to render that tree as ASCII art, because the corpus is now large enough to need a map. Here's the path an analytics question takes through it (a pruned version of that skill's own output):

Anatomy of the knowledge base

The knowledge side has three main components, and they map surprisingly cleanly onto what a semantic layer vendor would typically sell you.

The business glossary is our equivalent of metric definitions. It's a markdown file with canonical definitions and agreed-upon query logic for every ambiguous term, validated with the business owners and stamped with a "last validated" date. Here's a real excerpt:

Notice what this is: a metric definition, its measurement methodology, and, crucially, instructions about ambiguity. MetricFlow can encode count(distinct gp_id). It cannot encode "always clarify which segment is needed before answering." Prose can.

The dbt model catalog is our equivalent of the semantic model: the map from business concepts to physical tables. It's three tiers: a 13,000-line index covering all 1,400+ mart models across 59 domains (names, descriptions, schemas, upstream dependencies, column counts), per-domain column detail files (one for finance, one for operations, and so on; the agent loads only the domain it needs, because context windows are finite), and a separate column file for the staging-prefixed mart domains. In total it documents some 27,000 columns from dbt YAML, enriched with another ~12,000 columns' worth of type information pulled from Snowflake itself.

The analytics knowledge file encodes the tribal knowledge a senior analyst carries: Snowflake dialect rules (IFF(), LATERAL FLATTEN(), DATEADD()), which execution backend to use when, output conventions, and, most valuable of all, table-selection gotchas:

Every one of those rows is a bug that happened once and can never happen again. This is the institutional memory that normally lives in one analyst's head and leaves when they do.

The novel part: the catalog generates itself

Hand-maintaining a 13,000-line catalog would be a nightmare, so we don't. This is the piece of our setup I suspect is genuinely novel: the semantic layer is generated from the dbt manifest and from the upstream application codebases, automatically, on every merge.

The generator is a single Python script, generate_model_catalog.py, that lives next to the knowledge it produces. It parses manifest.json (produced by dbtf parse; we run dbt Fusion for CLI operations), then cross-references Snowflake's information_schema to pick up actual warehouse data types and columns that exist in the warehouse but have sparse YAML documentation. It emits the three-tier catalog plus a cached CSV of the Snowflake column inventory so CI can regenerate the catalog without a live warehouse connection.

A GitHub Actions workflow ties it to the development loop:

on:
  push:
    branches: [snowflake]
    paths:
      - 'models/**'
      - 'dbt_project.yml'
      - 'packages.yml'

Push a change to any model on our primary branch, and CI runs dbtf parse, regenerates the catalog, and commits the diff back to the repo. Add a mart, and it's discoverable by the next agent session. Rename a column, and the column file reflects it minutes later. The documentation cannot go stale because it's governed by the same trigger discipline as tests. Nobody maintains it; it maintains itself.

The second generator goes a step further, and this is the part I personally haven't seen anyone else do: it mines the application codebases that produce the data. dbt only knows what you tell it in YAML. But the meaning of a column is usually defined upstream: in the Rails app whose schema.rb declares it, in the Prisma model that shapes it, in the Go migration that created it. Our enrich_staging_columns.py script walks the upstream repos (nine of them: our venture monolith, banking, treasury, identity/KYC, workflow, and other services), extracts column types, defaults, nullability, foreign keys, and model-level comments, and writes them into the staging models' YAML schemas, where the catalog generator then picks them up. With a flag, it will even scaffold complete staging models and mart models for upstream tables that don't have one yet: sources entry, stg_ SQL, schema YAML, and a Metabase-facing mart, all generated.

The newest input is the warehouse's own usage telemetry. A dbt model aggregates Snowflake's ACCOUNT_USAGE.ACCESS_HISTORY (with service and CI accounts filtered out), and the catalog generator annotates every mart with how often it's queried, by how many humans, and when it was last touched, flagging the annotations if they go stale. When the agent faces five similarly-named tables, it can now weigh which one people actually use, and the same annotations double as a dead-table detector for us. The semantic layer doesn't just describe the warehouse; it observes it.

So semantic context flows automatically along the same path the data does: application code → staging YAML → mart catalog → agent context. When an engineer three teams away adds a column to a Rails model, its description can end up in Dana's knowledge base without any human in the data team touching a file.

Distribution is automated too. When the knowledge corpus changes, a workflow publishes it as context to Hex (where a different agent answers questions inside notebooks), and another syncs skills to Runlayer, our MCP gateway, so agents outside the repo can use them. One source of truth, many consumers: precisely the promise of a semantic layer, delivered by git push.

Skills: how self-service actually happens

Knowledge tells an agent what's true; skills tell it what to do. A skill is a markdown file with YAML frontmatter (name plus a description used for matching) and numbered steps. Devin discovers every SKILL.md under .agents/skills/ at session start and auto-invokes the one that matches the task.

The workhorse is query-warehouse. When someone asks Dana an analytics question, the skill walks the agent through: determine the execution backend (Metabase MCP by default; queries run against Snowflake under the hood with proper access control); find the table by reading the catalog index and the relevant domain column file; check the gotcha table; write Snowflake-dialect SQL with the metric definitions from the glossary; execute; and present the result with the SQL shown and a Metabase playground link so the requester can modify the query themselves. That last step matters: every answer is also an invitation to self-serve.

But querying is only the entry point. The skills corpus covers the whole surface a data team normally owns:

Dashboards are self-service too. When a question graduates from "give me a number" to "I want to watch this over time," Dana can build the dashboard. There's a skill for each platform, plus explicit guidance on choosing between them:

• The Metabase skills (metabase-manage-cards, -dashboards, -collections, metabase-query) create and update SQL-based cards and assemble them into dashboards over MCP. Best for standard charts and tables in the tool the whole company already uses.
• The create-streamlit-app skill handles requests that need real interactivity: Python logic, complex filters, custom visualizations. It walks the agent from data discovery (delegating back to query-warehouse), through scaffolding a Snowpark-connected app.py with our conventions baked in (cached query functions, fully-qualified table names, a KPIs-then-charts-then-detail layout), to deployment into Snowflake via the snow CLI, per-app access grants, and a version-history commit to a dedicated streamlit-dashboards repo. The skill even encodes data-isolation guards: finance apps deploy to a separate schema with a restricted audience role, and the skill states, in bold, which combinations must never happen.
• The create-arti-dashboard skill covers the lightweight end: a self-contained HTML page deployed to our internal artifact server, querying Snowflake through a governed MCP proxy. No backend, no API keys, shareable by URL.

Each of these skills opens with a "Step 0: Confirm platform choice" that lays out the trade-offs and redirects to the right sibling skill. The routing logic a data team lead would apply in their head is written down, so the agent applies it consistently:

Beyond dashboards, add-dbt-model walks through creating staging, intermediate, and mart models with proper naming, YAML, and tests; verify-dbt-models runs parse, lint, and build against Snowflake before every push; regenerate-catalog and enrich-staging-columns operate the knowledge machinery itself; and manage-fivetran-connectors handles ingestion operations. The agent doesn't just answer questions about the warehouse; it develops the warehouse, following the same procedures a human engineer would.

And the loop compounds. When a session surfaces a gap (an ambiguous table, a missing definition, a new gotcha), the fix is a small PR to a markdown file. The next session is smarter. Institutional knowledge accretes in version control instead of evaporating in Slack threads.

The boring-tools dividend: GitHub Actions instead of dbt Cloud

None of this requires a vendor's orchestration platform. The entire dbt operation runs on GitHub Actions and the dbt Fusion CLI: thirteen workflows in the repo.

• CI on every PR builds changed models into a per-PR dev schema in Snowflake (dbtf build with state deferral), so reviewers see real results; a cleanup workflow drops the schema when the PR closes.
• Deploys and schedules: production deploy on merge, plus daily, intraday, and monthly scheduled runs and snapshot jobs, each a cron trigger in YAML rather than a scheduler UI.
• Failure notifications post to Slack.
• Knowledge automation: the catalog regeneration workflow described above, Hex context publishing, and Runlayer skill sync.

The economics are hard to argue with. GitHub Actions runners cost a few dollars a month at our scale; dbt Cloud's team and enterprise tiers price per seat, and the Semantic Layer specifically requires the paid platform. But the deeper point is architectural: because our semantic layer is files in the repo, it needs zero incremental infrastructure. There is no semantic-layer service to deploy, monitor, or upgrade. The "runtime" is whatever agent happens to be reading the files.

Honest trade-offs

This approach is not a drop-in replacement for a governed metric API, and it's worth being clear about what we gave up.

No query-time contract. A BI tool can't ask our semantic layer for revenue and get planner-generated SQL. An agent has to read the docs and write the SQL, which means correctness rests on documentation quality and model capability rather than on a deterministic planner. Our mitigations are the gotcha tables, the glossary's explicit query logic, and the convention of always showing the SQL so a human can inspect it. But a MetricFlow query is guaranteed to use the canonical definition, and ours is only very likely to.

Context is a budget. A 13,000-line index plus 59 domain files exist in that shape precisely because an agent can't load everything at once. Tiering, per-domain splitting, and a quick-reference file are the price of making the corpus fit the consumer.

It works because our consumers are agents. Humans at AngelList mostly touch this system through Dana or Metabase. If your organization's consumption pattern is Tableau-first with humans dragging pills, MetricFlow or Snowflake semantic views may genuinely fit better. Our bet is that agent-mediated consumption is the direction everything is heading. If that's true, the natural form of a semantic layer is text, not a service: agent-agnostic by construction, with no lock-in to any one model or vendor.

The semantic layer nobody maintains is the one that stays correct

Step back and look at what the pieces add up to. A business glossary plays the role of metric definitions. A generated model catalog plays the role of semantic models. Skills play the role of the query interface. GitHub Actions plays the role of the platform. And the sum is a semantic layer with the one property the traditional versions never achieve: it regenerates itself from the systems of record (the dbt manifest, the warehouse's information schema, the upstream application code) every time any of them change.

Today this system lets one person run all of data at a company administering hundreds of investment-vehicle launches a month, with an AI agent as a full partner. The pattern is portable, and you can start remarkably small: an AGENTS.md, one query-warehouse-style skill, and a script that turns your manifest into a catalog on every merge. Grow it one gotcha at a time.

Beau Rothrock works on the data platform at AngelList. If building systems that bridge AI, data, and finance sounds interesting, check out our open roles.