I’ve been building analytics software for more than two decades. I’ve watched waves of innovation promise to democratize analytics, including self-service BI, cloud warehouses, and natural language query systems. Each wave delivered value, but also exposed new challenges. We’re now facing the next evolution: AI agents that not only respond to questions but also actively analyze data and make recommendations.
The promise is compelling. Instead of teaching business users SQL or new BI tools, AI agents handle the technical complexity while delivering insights. But there’s a critical question enterprises must address: How do you trust an AI agent’s analysis when you can’t validate its reasoning or content?
This question drove our latest conversation on AtScale’s Data-Driven Podcast. I sat down with Dianne Wood, our co-founder and data architect, and Petar Staykov, our product director, to explore why semantic layers are becoming critical infrastructure for AI, not just business intelligence. The discussion revealed something important: the architectural requirements for trustworthy AI analytics are fundamentally different from traditional BI.
Why MCP and the Semantic Layer Belong Together
Model Context Protocol is a relatively new standard introduced by Anthropic that gives large language models a structured way to interact with external systems. Similar to JDBC for databases, MCP provides a universal interface that allows LLMs to safely access tools, metadata, and governed data sources.
With MCP, LLMs can discover what data exists, understand business meaning beyond raw schemas, and run analytical queries without being embedded directly in your data platform. But MCP on its own isn’t enough.
“MCP tells the agent how to interact, but without context, it still doesn’t know what it’s looking at.”
– Dianne Wood, AtScale
This is where the semantic layer becomes essential.
When a user asks for “sales,” an LLM doesn’t inherently understand whether that means gross sales, net sales, or booked revenue. Without those definitions explicitly encoded in a semantic layer, LLMs will hallucinate when pointed directly at raw data. A semantic layer provides deterministic definitions for metrics, dimensions, hierarchies, and filters, presenting them as a single logical model. The LLM can then query what appears to be a clean, well-defined table, one that represents potentially dozens of joins, calculations, and governance rules.
Dianne described this distinction clearly:
“AtScale isn’t exposing columns in tables. We’re exposing business concepts, metrics, and attributes that are defined, governed, and usable.”
The LLM is probabilistic, generating different insights from one session to the next. The data layer can’t be. Semantics ensure the numbers, definitions, and governance are deterministic every time.
From BI Dashboard to AI Collaborator
During the podcast, I demonstrated this by asking Claude for sales by product in France. The agent didn’t just return a result set. It added context, surfaced patterns, and highlighted anomalies without being explicitly prompted to do so.
Because the semantic layer guarantees consistent definitions, the agent is free to reason on top of the data. It can explore variance, seasonality, concentration risk, or product mix without redefining what “sales” means each time.
Many early “chat with your data” experiments failed because natural language alone isn’t enough. The combination of semantic precision and probabilistic reasoning is what makes AI usable in an enterprise setting. Semantics build trust by layering AI’s analytical capabilities on top of deterministic data.
The Next Step: AI That Builds the Semantic Layer
Querying data is just the beginning. AtScale customers are already using LLMs to build the semantic layer itself.
“A customer used an LLM to generate business descriptions for all their metrics and dimensions. Something that normally takes months of meetings was done in minutes. And it was surprisingly accurate.”
– Petar Staykov, AtScale
Consider the challenge of keeping semantic layers current. As KPIs evolve and new data sources appear, keeping semantic models aligned with the business is one of the biggest challenges in analytics. We believe this will be handled by multiple specialized agents in the future, each focused on a different part of the semantic lifecycle: creation, documentation, governance, versioning, and change management.
“There isn’t one agent that does everything. Just like there isn’t one human who knows everything. You need an army of agents, each responsible for keeping the semantic model in sync with the business.”
– Petar Staykov, AtScale
MCP becomes the interface that allows those agents to operate safely, while open standards like SML and Git provide transparency, history, and control.
Trust Is Still the Path Forward
The fundamentals haven’t changed. Consistent definitions, governance, performance, and explainability—these requirements become more critical, not less, when AI agents are making recommendations instead of humans. What’s different is the speed and scale of decision-making. Analyses that once took weeks now happen in minutes, but only if your data foundation can support that velocity.
Petar captured it well:
“The semantic layer, agents, and MCP together form the brain of the enterprise. Data flows in from operational systems, and decisions flow out, but only if the brain understands what the data actually means.”
This is why semantic layers are becoming core infrastructure for enterprise AI. The organizations that build this foundation now will be the ones capable of trusting their AI agents when those systems are making critical business decisions at machine speed.
To learn more, listen to the full conversation on the Data-Driven Podcast.
SHARE
Case Study: Vodafone Portugal Modernizes Data Analytics