Building a Common Data Language: How a Semantic Layer Drives Collaboration and Trust

Dataveristy first published this post on June 17, 2024. You can read the original article here.

—

The self-service analytics revolution was intended to free business users from IT chains. The idea was to break the IT bottleneck by allowing businesses to create their own data products. After all, businesses understand their respective domains better than anyone in IT could, and decentralizing the work of creating data products should theoretically increase the throughput for developing these products.

This decentralized approach failed to deliver on the promise, though. Instead, self-service meant business users needed to become data processing experts. At the same time, embedding semantics into various analytics tools without a common language or standards created inconsistencies in business metrics, destroying trust in their data products.

In this blog post, I will explore how a common semantic modeling language can facilitate collaboration across business teams while drastically improving data accuracy and trust to deliver on the original goal of self-service analytics.

Understanding the Universal Semantic Layer

A semantic layer is a data abstraction that provides a consistent, business-focused way of representing data independent of the underlying data sources or consumption tools. It translates complex data into meaningful and understandable terms for end users. This layer is a universal translator, allowing teams to centralize their business definitions in one place, independent of consumption and data storage.

A standard semantic layer can bridge the gap between organizations and business domains to allow data teams to describe their data consistently, breaking down silos that hinder a common understanding of the business while promoting collaboration. Adopting a common semantic framework allows all stakeholders to understand and use data consistently, fostering interdisciplinary collaboration.

For example, the marketing, sales, and customer service teams might use different terms for similar customer interactions. A standard semantic layer ensures that all teams refer to these interactions consistently, reducing confusion and aligning business definitions across the board.

The Object-Oriented Semantic Model

To serve as a common data language, a universal semantic layer must include an object-oriented architecture for driving collaboration and consistency across business domain teams. Semantic models should be composable and built from simple semantic objects to express more complex business processes inside a semantic model. Like software, an effective semantic layer platform needs to satisfy the following object-oriented design principles:

1. Encapsulation: Each semantic object should be capable of hiding its implementation and complexity. For example, a modeler should be able to create a “gross margin” calculation that other modelers can include in their semantic models without needing to understand how it was calculated.
2. Modularity: Each semantic object should be able to break down more complex concepts into simple constructs. For example, a modeler should be able to compose a composite model such as “Campaign Cost” that references two smaller models, such as “Campaign” and “Cost.”
3. Inheritance: Each semantic object should be extendable to promote reuse and flexibility. For example, a modeler should be able to add additional attributes to a conformed dimension, like “product,” to enable a further use case.

The illustration below demonstrates how different business domain teams can collaborate and share semantic objects like Lego building blocks.

A use case that leverages an object-oriented semantic layer

Managing a Universal Semantic Layer

A universal semantic layer must include an object-oriented semantic modeling language, fine-grained user access controls, and versioning. Like other software development projects, semantic model development should conform to an organization’s software development life cycle (SDLC). Ideally, a robust semantic layer platform should integrate seamlessly with a Git-compatible version control system to allow domain teams to build repositories for managing their semantic objects, define read and write access controls, and share objects with other teams.

Implementing a Universal Semantic Layer

Implementing a universal semantic layer involves several key steps:

1. Stakeholder Engagement: Engage stakeholders from various business domains early to ensure that the semantic layer meets the organization’s diverse needs.
2. Standardization of Terms: Develop a standardized glossary of terms and metrics that everyone across the organization agrees to use.
3. Technology Investment: Deploy technology solutions that support creating and maintaining the semantic layer, ensuring it can integrate with existing systems.
4. Training and Adoption: Conduct training sessions to familiarize all users with the new system and encourage its adoption through ongoing support and resources.
5. Continuous Improvement: Regularly review and update the semantic layer to accommodate new data sources, changing business needs, and user feedback.

Conclusion

Establishing a universal semantic layer is more than a technical upgrade; it is a strategic investment in the future of an organization’s data infrastructure. By enhancing collaboration, building trust, and improving data accessibility, a universal semantic layer serves as the common data language for building data products.