CHALLENGE
Limitations with scaling in-house custom OLAP solution to business users
Since EverQuote was a spinoff from Cogo Labs, a technology-driven venture accelerator, the new company already had a data architecture consisting of a MySQL cluster, Python services, a direct connection to Excel, and a custom OLAP interface. But this custom legacy approach, which was over ten years old, had a number of bottlenecks that prevented many use cases and suffered from poor query performance.
The other challenge EverQuote faced was the limited technical knowledge throughout the organization. While EverQuote started as a group of engineers and analysts, as the company grew, it proved difficult to scale self-service analytics to non-technical employees. The company needed a modern data architecture that could democratize data analytics for all.
“Our custom OLAP tool has served us very well for over a decade, but it’s owned by engineering,” stated Jonathan Spencer, Data Engineering Lead at EverQuote. “Over time, the tool has required a lot of maintenance and governance by the engineering team. It also wasn’t flexible enough to serve all analytical needs.”
SOLUTION
Migrating from legacy OLAP to Snowflake’s cloud data platform using AtScale
Using AtScale’s Semantic Layer, EverQuote was able to seamlessly transition its analytics workloads to Snowflake without impacting the existing business user experience. The semantic layer now enables the business team to access data stored on Snowflake within Tableau, Excel, and many other business analytics and visualization tools.
Through AtScale’s semantic modeling capabilities, EverQuote is able to flexibly add new metrics and data definitions that provide consistency across consumption tools. Data virtualization also makes it easier to onboard new data quickly so that it can be queried from BI tools almost immediately.
“We found AtScale, and it became the right tool at the right time to revamp our custom OLAP tool,” Kwan Lee, EVP of Engineering explained. “In 2020, we adopted Tableau, and this year we started allowing business users to create data cubes in AtScale, which has been very useful for our organization.”
RESULTS
Democratizing data analytics for both business and data science teams
By modernizing its data architecture with Snowflake and AtScale, EverQuote have been able to dramatically reduce the time-to-insight for business users. In fact, the ability to perform both incremental and full cube refreshes has enabled EverQuote to provide more real-time data to all downstream lines of business.
With AtScale, EverQuote has also been able to more easily leverage its data for machine learning use cases. Data teams can now use Jupyter and other Python-based approaches to generate data science insights against Snowflake data. This has broken down the traditional silo between business intelligence and data science teams so that both teams can produce better insights for the organization.
Now that users across the organization can leverage data using the tools of their choice, EverQuote has shifted its culture to embrace data as a strategic asset. By promoting data literacy through training and offering the right business intelligence tools, EverQuote has also made self-service data analytics a reality.
“The challenge for data engineering teams should be building self-service components that can be used by other teams,” concluded Spencer. “Using technology and the principles of self-service, we can reduce the cycle time for those looking to do analytics and give them more autonomy in their own business.”