The Metrics Store: The Connecting Language between Data and Business

Businesses rely on metrics as a foundation for self-assessment. These metrics offer a quantifiable means to evaluate a business’s performance across various facets. By diligently monitoring key performance indicators (KPIs), organizations gain a clear understanding of their progress toward attaining predefined goals and objectives. Metrics, therefore, serve as invaluable sources of data and insights, steering decision-makers in their strategic choices.

However, the world of business metrics is not without its complexities. In a large organization, each business unit may employ slightly different methods for calculating metrics, adjusting the underlying logic, or enhancing existing ones. This variance can lead to inconsistencies both within and between organizations. Furthermore, the proliferation of multiple analytics, Business Intelligence (BI), and Machine Learning (ML) tools within decentralized data and analytics teams further compounds this issue.

In this evolving landscape, a notable trend has begun to emerge: the concept of a metric store. This trend has garnered significant attention, particularly as the market progresses toward the next generation of BI and analytics capabilities, driven by advancements such as GenAI.

This year, Gartner, a prominent authority in technology research, has identified the metric store as one of the critical capabilities within the realm of BI and Analytics platforms. The metric store has also found its place in the Gartner Hype Cycle for Analytics and Business Intelligence 2023 report, and AtScale was named in the metric store category. In this blog post, I aim to succinctly outline the reasons behind the growing interest in this concept, especially in light of the market’s journey towards embracing the next level of BI and analytics capabilities driven by GenAI.

First I would like to cover the must-have capabilities of a metric store.

• Unified Metric Repository for User Flexibility: In today’s diverse landscape of Analytics, BI, and ML tools, organizations often embrace a portfolio approach due to varied user preferences. A metric store must act as a single, consolidated source for all metrics, ensuring seamless integration with any BI or ML tool of choice. This empowers users to analyze metrics comfortably in their preferred environment.
• Robust Support from an Enterprise-Scale Semantic Layer: To cater to thousands of enterprise users effectively, a metric store requires robust backing from an enterprise-scale semantic layer. Semantic modeling capabilities enable developers to effortlessly structure metrics, create essential drill-down and hierarchical analysis features, and ensure enterprise-grade security, entitlements, and scalability for stringent security and access control requirements.
• Optimizing Performance and Cost Efficiency: The metric store serves as the final gateway for delivering valuable insights to users. Therefore, a well-structured cloud workflow management system should underpin metric store consumption, guaranteeing a seamless and cost-effective analytics experience. Efficient performance management workflows automate and coordinate the creation of physical materializations (such as aggregate tables) and make real-time decisions regarding what to materialize and when. This functionality must remain dynamic and adaptable, responding to user query behavior, query execution times, and other relevant metadata. These strategic choices also have a direct impact on controlling cloud costs associated with analytics programs.
• Streamlined Metrics Engineering with Code and Non-Code Options: Organizations typically manage a multitude of metrics, demanding automation capabilities within the metric store. These features empower developers to efficiently define and manage metric definitions. The development experience should be inclusive, catering to both code-savvy and non-code users. Embracing software engineering principles ensures comprehensive version control, CI/CD integration, and workflow management for a reliable single source of trust.
• Collaboration and User Engagement with Version Control: Metric definition is an ongoing effort, often requiring collaboration among diverse business departments. A metric store should serve as a collaborative hub, enabling users to comment, share, and refine metrics collaboratively. Strong version control capabilities facilitate continuous integration and deployment (CI/CD) aligned with software engineering best practices.
• Open APIs for Seamless Integration: Metrics hold significant importance for business users in their daily tasks. Therefore, a metric store should facilitate actions on these metrics directly within business applications. Open APIs are essential for seamless integration with third-party tools and applications, ensuring that metrics are readily accessible and actionable for users.
• Composability and Business Innovation at the Edge with Governance: Imagine a metric store where its components, such as metrics, dimensions, relationships, and datasets, are akin to Lego building blocks. They are designed to be composable, allowing business analysts to assemble these essential “Lego blocks” with the guidance of an intelligent semantic layer. This composable framework empowers organizations to create an analytics mesh—a dynamic environment where business units can innovate with metrics, and swiftly gain new insights, all while maintaining rigorous governance and compliance standards

Gartner’s recent acknowledgment of AtScale’s metrics store, enhanced by the semantic layer, signifies only the inception of our exhilarating journey. At AtScale, we are embarking on a mission filled with promise—to reshape the future of metrics stores as a pivotal element within a universal semantic layer platform. Our vision transcends the conventional; we aspire to revolutionize the way metrics are conceptualized, managed, and utilized in the business world. Stay connected for an in-depth exploration of our vision for the next-generation metrics store in the forthcoming second part of this blog post.