Semantic Layers in Apache Superset: SIP-182 and Apache Ossie
Ask four different systems "how many monthly active users did we have in June?" and you'll get four different answers. The CRM says 41,208 (it's counting trial signups). The warehouse says 39,655 (it excludes internal accounts). The BI tool says 42,730 (someone's ad-hoc metric, defined once, forgotten why). And if you ask an AI agent? It picks one of the above pretty much at random, with total confidence, and no way for you to know which.
Surely, you've run into this. Analysts have been arguing about things like what "customer" means since long before anyone had an LLM to make the argument worse. What's new is the cost of getting it wrong. An analyst who picks the wrong number gets corrected in a meeting. An agent that picks the wrong number gets shipped into a dashboard, a Slack summary, or a decision, with nobody in the loop to say "wait, that's not how we count trials."
A semantic layer is the fix: an agreement about what a number means, defined once and served everywhere. Metrics, dimensions, and the joins between them, in one place instead of four.
Here's the thing most people don't know about Apache Superset™: it's had one of these since the beginning. We just called it the dataset editor. And until a few months ago, it was the only semantic layer you were allowed to have.
Last month, I hosted a webinar with Beto Dealmeida (Apache Superset PMC member, and the author of SIP-182, the proposal that changes this) to walk through what's shipping, what it opens up, and a new standard called Apache Ossie that's about to make the whole thing a lot less bespoke. This post is the write-up I promised at the end of that session (thanks for your patience), with the architecture, the roadmap, and the parts of the Q&A worth keeping. If you'd rather watch than read, the recording is up.
Superset already had a (thin) semantic layer
Superset's dataset editor didn't happen by accident. It happened because Superset started as a front end for Apache Druid, back when Druid had no SQL interface at all. You talked to it in JSON. Writing a raw query meant hand-building a JSON request, and Druid was bad at grouping on high-cardinality columns if you didn't warn it first.
So Superset needed somewhere to consolidate that knowledge: which columns are metrics, which are filterable, which are safe to group by. That somewhere became the dataset. Users stopped querying tables directly and started querying a curated, guarded version of the table instead. When SQL support arrived later, the underlying representation shifted to SQL, but the job of the dataset never changed: be the one place that knows what a metric means.
That job description, it turns out, is a semantic layer. Superset had a thin one, opinionated toward "compile-time" safety, running the entire time.
Metrics & Dimensions: a quick, necessary detour
Before going further, it's worth being precise about two words that get used loosely across the industry. In Superset's world:
A metric is an aggregation, almost always over a fact table. COUNT(*), COUNT(DISTINCT user_id), SUM(price), for example. Sometimes something gnarlier, because if you need a filter on a metric, you can't just add a WHERE; you jam a CASE WHEN into the aggregation expression instead.
A dimension is a related column, often reached via a join, that you'd break a metric down by. Usually low-cardinality on purpose: device type, country, age bucket. Use user_id as a dimension and you're not adding insight, you're just exploding your row count.
And underneath both of those sits a category of knowledge that lives nowhere near your database: the tribal kind. "You need to exclude user 42 from that count, we ran a test on that account." Nobody encodes that in a table schema. A semantic layer is where that knowledge finally gets to live somewhere durable, instead of in the head of whichever analyst has been around the longest.
By the way: there are two schools of thought on when a semantic layer resolves a dimension. Superset does it at compile time, when you build the dataset: rigid, and it requires planning ahead, but fast, since the join is already written and the database isn't guessing at query time. Other semantic layers resolve it at run time, joining on demand when you ask for a breakdown you didn't plan for: flexible, but a bet, since a many-to-many relationship joined at the wrong moment can quietly multiply your row counts. Neither is strictly better. It's a trade between planning cost and query-time risk.
The part that doesn't scale: pretending everything is a database
Superset's compile-time approach works great, with an asterisk: it works great for databases. Explore inspects the columns and metrics on a dataset, emits SQL, sends that SQL to a database, and gets a table back. Clean.
The trouble starts once you want to point Superset at something that isn't a database. Over the years, Beto has personally built or overseen integrations with Airbnb's in-house Minerva layer, dbt's semantic layer and later MetricFlow, a Snowflake integration, and Data Junction, another open-source semantic layer. The pattern every single time: pretend the semantic layer is a database. Invent a pseudo-database. Give it pseudo-tables. Emit SQL at it like always, then parse that SQL back into whatever the real semantic layer actually speaks, whether that's a REST call, a GraphQL request, or something else entirely.
If that sounds brittle, it is. Superset's own SQL-generation function runs to roughly 700 lines, threading the needle between dozens of database dialects that disagree on whether you can alias a column in a GROUP BY, or how to express a time grain. Now take that already-complex SQL, and parse it a second time on the way out the door, into a second query language on the other side. You end up with a semantic layer bolted onto a semantic layer, and nobody can tell you what happens if a user defines a metric on the dataset that sits on top of the pseudo-database that sits on top of the real semantic layer. Usually, it just doesn't translate.
Making "semantic layer" a first-class connection
SIP-182 is Beto's fix, and the core move is refusing to keep pretending. Semantic layers stop being disguised as databases and become siblings of databases instead: a data connection in their own right, sitting next to your database connections rather than underneath a costume.
To make that possible, Beto first had to clean up what a chart actually asks for. Explore (a.k.a. Chart Builder) has always sent a payload called the QueryObject to request a chart, and it's a complicated object: 29 fields, one of which (extras) is itself a 7-field dictionary, filters showing up in more than one place, from_dttm and inner_from_dttm sitting a few lines apart with no obvious reason why. It evolved the way most things evolve inside a project running for a decade: correctly, one PR at a time, into something nobody would design on purpose.
Asking every future semantic layer author to learn all of that felt unreasonable. So Beto wrote a mapper (map_query_object()) that translates a QueryObject into a SemanticQuery: metrics, dimensions, filters, order, limit, offset, group limit. Nothing else. If you've ever worked with any semantic layer, that vocabulary needs no introduction.
Sitting underneath both objects is Explorable, a protocol that formalizes what a thing needs to support in order to be turned into a chart. It's a generalization of ExploreMixin, originally built by Hugh Miles to let charts come directly from ad-hoc queries in SQL Lab rather than only from saved datasets. Dataset and Query already implemented it. SIP-182 adds a third implementer: SemanticView, Superset's name for what a semantic layer calls a model, or a cube, or (fittingly, since Snowflake uses this exact term for its own semantic layer product) a semantic view.
A SemanticLayer implementation is a short, deliberately small interface: configuration and schema methods, accessors to list or fetch semantic views, and on the view itself, get_dimensions(), get_metrics(), get_compatible_metrics(), get_compatible_dimensions() (a concept datasets never needed: pick a metric, and not every dimension is still valid alongside it), and the data-retrieval trio get_values(), get_table(), get_row_count(). The payload that comes back is an Arrow table, not a bag of JSON, and not SQL parsed out of a pseudo-database.
By the way: semantic layer connections aren't hand-built forms. Each one defines a schema, Superset turns it into JSON Schema, and JSON Schema builds the config UI, which is a rare case where the better user experience and the simpler implementation are the same thing: add a field to the schema and it shows up in the form, no frontend code required, no per-layer form to hand-maintain as Cube, Snowflake, dbt, and whatever comes next all ask for wildly different setup information. The schema can also react to itself. Connect to Snowflake, and only once your credentials check out does "database" turn from a text box into a dropdown of your actual databases, fetched live instead of typed from memory.
Semantic layers as Superset Extensions
The first implementation of this wasn't built as an extension. It became one once Superset's extension system matured enough to make that the obvious choice: separate repo, separate release cycle, automatic discovery, and a config form you don't strictly need to hand-write, since a future version can load a bespoke React component in its place if the auto-generated JSON Schema form isn't pixel-perfect enough for your taste. That's the part that actually changes who gets to write one: a semantic layer integration no longer has to earn its way into Superset core to exist. Anyone can build one, package it, and ship it on their own schedule.
Building one is superset-extensions bundle, which validates, installs dependencies, rebuilds the frontend, syncs backend files, and packs the whole thing into a .supx file (yes, a real file extension of our own invention). Beto's pandas-semantic-layer is the reference implementation, small enough to read start to finish in one sitting.
There's a version of this that's more than a nice-to-have. In the Q&A, someone asked how to wire up Databricks. Neither of us had a confident answer, but Beto's response is worth keeping: "If anybody wants to, you can use AI, get the Databricks API for a semantic layer, give it the interface, and AI will write the extension for you." The SemanticLayer interface is small precisely so that this claim can be true, and it's a good test of whether an interface is well-designed: can a model with no tribal context implement it correctly from the docs alone?
Five consequences opportunities worth sitting with
Getting semantic layers out from behind a pseudo-database isn't just a cleaner diagram. It changes what's possible in ways that compound.
Any semantic layer becomes a contribution, not a request
Snowflake Semantic Views and dbt MetricFlow are shipping on Preset now, with the Pandas implementation already available as a reference. Cube, Malloy, Cantrip, AtScale, and Minerva-style in-house layers are the kind of thing that's now a well-scoped extension for someone to write, rather than a line item on Superset's core backlog. The list of supported layers stops being the point. The interface is the point. Whatever ships next quarter can be someone else's extension.
Superset's own dataset editor gets to become an extension too
Beto and I both agree it's a hairball, and honestly, it's earned the right to be one. The code's been evolving with the project since the Druid era. The plan is to migrate it onto the same Explorable/SemanticLayer interface every other extension uses, which hardens that interface (a builtin consumer is the best QA an interface can get), shrinks Superset's core, and reduces the ongoing maintenance burden that comes with keeping a decade of dataset-editor logic wired directly into core. It's a work in progress, not a shipped feature. "Heal thyself" is a good instinct/ethos for a project this size, and our own semantic layer should test and harden the extension interface.
Migration between semantic layers becomes a real workflow, not a rewrite
If Superset can read a semantic layer and write one, it can move you between them: connect the source, serialize what it finds into a neutral model instead of a bespoke script, let a human reconcile whatever didn't map cleanly, then write the target. Converters for the dbt Semantic Layer and Apache Polaris already exist as reference implementations. Read/write coverage varies by layer today, and full round-tripping is roadmap, not shipped, but the direction matters on its own: expressing Superset's own datasets the same neutral way makes Superset's modeling portable too, which is a stronger no-lock-in claim than most BI tools get to make honestly.
You can run more than one at a time, on purpose
Nothing forces you down to a single semantic layer. The SIP includes a nice worked example: one Snowflake connection for power users who write semantic views directly in SQL Lab, and a second, separate semantic layer connection that exposes a curated, locked-down subset of those same views to everyone else. Multiple regions with disconnected data stacks is another legitimate case, and it's one we see a lot at Preset already, since a single Preset org can run several Superset workspaces for different departments or customers. The one thing to actually watch for: don't let two active semantic layers both claim ownership of "ARR." That's the exact metric-drift problem this whole effort exists to fix, reintroduced by having too many sources of truth active at once instead of zero.
Context compounds in layers
An agent pointed straight at a warehouse is guessing at what a column means. An agent pointed at a semantic layer is reading a definition someone actually wrote down. The semantic layer defines what a metric means and how it joins, the semantic view narrows that to what's safe to query, charts capture which questions people actually ask, and dashboards show how those questions cluster into a job. Ossie's spec (more on that in a moment) includes an ai_context block, freeform instructions and worked examples written once and read by any compliant tool instead of configured separately per vendor copilot. Superset's own MCP service exposes charts, dashboards, datasets, and SQL Lab as tools already, and extensions can add their own with @tool and @prompt. The layer Superset uniquely gets to add on top is usage signal: which metrics actually get charted, by whom, next to what. No warehouse has that. It's the difference between handing an agent the full restaurant menu and handing it the printout of what regulars actually order. Given the second one, it's going to guess a lot less.
The standard underneath all of it: Apache Ossie™
None of the above holds together without agreement on what a semantic layer's own definition file actually looks like, and that's a separate, new piece of news: Open Semantic Interchange was accepted into the Apache Incubator in June 2026, and renamed Apache Ossie in the process (OSI, it turns out, was already very much spoken for).
Ossie is a declarative spec, JSON and YAML, that any tool can read or write: semantic models made of datasets, fields, relationships, and metrics, plus an ontology layer for business concepts and rules that don't reduce cleanly to a column. It's vendor-neutral by construction, not by promise: over 100 commits already from contributors at Snowflake, Dremio, Salesforce, Databricks, and dbt Labs among others, 17 launch partners grown to more than 50 organizations, and three active working groups (Metric Language, Catalog, Ontology). It's still early. The core spec sits at v0.2.0.dev0 and is explicitly marked draft. But Superset and Ossie are now governed under the same ASF roof, which is a very different foundation to build on than "a consortium of vendors agreed on a format."
Why a standard changes Superset's math
Here's the argument that made the calculus shift for me as Beto was giving the talk. Every semantic layer integration built so far has been bespoke, and the bespoke part was never where the value was.
Cube gets integrated by emulating the Postgres wire protocol with pseudo-tables. Minerva gets integrated with one giant dataset and a pile of compatibility overrides. MetricFlow gets integrated by parsing pseudo-SQL into a GraphQL request. Snowflake's own integration rewrites generated SQL into a UDTF call. Four semantic layers, four completely different kinds of hard problem, none of which teach you anything about the third one.
Once Cube, MetricFlow, Snowflake, and Polaris can all export the same Ossie YAML, Superset only has to write one thing: a provider that parses that YAML and maps it onto Explorable. That's a file to parse, not a wire protocol to emulate. It puts a working .supx for a new semantic layer within reach of a weekend, instead of a quarter.
Why this is safe to commit to, not just plausible
The honest version of this argument has to address the obvious objection: standards proposed by vendor consortiums have a way of becoming whatever the biggest vendor in the room wants next year. That's the part that actually changed with the Incubator acceptance, not the spec itself. There's no vendor consortium left to bless anything. Ossie is governed the way Superset is governed: two Apache projects sharing licensing, IP provenance, ICLA coverage, and release policy, where committership is earned by contribution rather than granted by employer. That means Superset gets to help shape the spec from inside the room, not lobby it from outside.
Which is why the next step isn't "wait and see." It's ship a provider as an extension, since the interface already exists and no core changes are required to start. Then prove it on something real: convert a live model, chart against it, diff it against the source. Then, if that holds up, graduate it out from behind the flag and into something the PMC maintains long-term. Where that actually gets decided is the usual place for anything this size: an SIP, discussed on dev@superset.apache.org. If you have an objection to any of this, that's where it belongs, and where it'll get read.
Where things stand today
If you're wondering when you can actually touch any of this: the SEMANTIC_LAYERS feature flag exists on Superset master right now, but it's not part of 6.1. It's targeted at Superset 7.0, and at the time of the webinar we were in that release's breaking-change window, wrapping up within days. On Preset, we tend to run ahead of the open-source release cycle, so this is closer to generally available there already, with Snowflake Semantic Views and dbt MetricFlow rolling out over the next few months. If these three built-in options don't cover your stack, building a small extension on top of the same foundation gets you the rest of the way.
A couple of things worth knowing are already in flight behind this: Beto has an open PR for a semantic cache that can apply filters to a cached payload and roll up dimensions when they're additive, instead of re-running a query every time someone excludes one country from a chart they already ran. And there's a separate, larger PR rewriting Superset's own semantic layer as an extension on top of SQLGlot instead of a hand-rolled SQLAlchemy query builder, which on its own removes something like 5,000 lines of code and, as a side effect, gets predicate pushdown almost for free.
How to get involved in Semantic Layer Extensions
If you're a Superset contributor, Explorable is in master and stable enough to build against today. A semantic layer provider is about as well-scoped and high-visibility as a first contribution gets. SIP-182 is the place to start reading if you want to see exactly what the mapper simplifies away.
If you already run a semantic layer we haven't touched yet, an Ossie .supx is parse-and-map, not protocol emulation, and whoever ships the first one for a given layer sets the pattern everyone after them copies.
If you're a Superset or Preset user, turn on SEMANTIC_LAYERS, point it at something you already run, and tell us exactly where it falls over. That feedback is the roadmap, not a nice-to-have on top of it.
And if you want a hand with any of it, or want to talk about running Snowflake, dbt MetricFlow, or your own semantic layer on Preset, come find us in the Superset Slack. Thanks to Beto Dealmeida for writing SIP-182 and walking through it live, and to Elizabeth Thompson for handling all the Q&A. The design is still moving. Now's the time to disagree with it.
Further reading
- The Semantic Layer Is Back. Here's What We're Doing About It. — why the semantic layer is having a moment, and where Preset fits.
- A Case Study in Dataset-Centric Visualization Using dbt and Snowflake — datasets, dbt, and Snowflake working together in practice.
- Preset MCP: Analytics Your AI Agent Can Build, Not Just Read — giving AI agents a grounded way to query and build on your data.
- Building Better BI Chatbots: Why Context and Triggers Matter — why context, like a semantic layer, is what keeps AI answers honest.
- Preset Joins the Open Semantic Interchange (OSI) Initiative — the cross-vendor standard for sharing semantic definitions, and why Preset is in.
