August 2, 2019
Cloud Data Warehouse Modeling: Traditional vs. NewIn our most recent webinar, Matt Hartwig, Associate Director of Data Infrastructure at Wayfair and Dave Mariani, Co-Founder and Chief Strategy Officer of AtScale shared the techniques Wayfair uses to make data a competitive advantage.
Wayfair is an e-commerce leader that specializes in home goods. According to Hartwig, “How we actually get that furniture into someone’s home, largely comes from a company that is backed by thousands of software engineers focused on building out proprietary technologies and also integrating best in breed third-party technologies like AtScale, as well as cloud technologies from our preferred partner in GCP.” The company was founded by two Cornell graduates who are still involved today.
Hartwig credits Wayfair’s success to the way that they use their data. “We have about 900 million interactions every single year, related to data. And that could besomeone in our supply chain going and running a query to figure out how many items are currently sitting at a certain warehouse. It could be, an analyst going and trying to figure out why a certain item is getting lost in transit on a regular basis. It could be someone going in and looking at a report to just see what Wayfair’s current revenue is for the given day.” He speaks to the divide of the queries between people and systems as 250 million of them are driven by people.
How does Wayfair serve these users? Matt speaks to the role of the data infrastructure team that he is part of, “We have hundreds of BI developers, we have hundreds of data scientists, and we have hundreds of analysts. And Wayfair’s approach to managing and serving all of these users has been to form a data infrastructure team,” and how they are responsible for “providing, managing, and operating,” the data.
Hartwig introduces the concept of velocity and why it’s something that he personally focuses on. According to Hartwig, velocity is “how we talk about speed and scale … it’s all about how we sustainably grow from the company we are today to a $50 billion company and to a hundred billion dollar company.” He continues, “From an infrastructure point of view and how we think about serving our user community who are often trying to analyze sales patterns and seasonal trends, everything we’re focused on is how do we make it so that we can do all of that more efficiently in the future.” What does it have to do with data? Matt states that, “When it comes to data, we think about that specifically as the speed with which Wayfair can go from data collection to driving business decisions, outcomes and insights.”
Matt shares an example as to how customers interact with Wayfair’s product pages and how the data converts them from being in the “storefront” to the desired “decision” outcome.
What problems do companies like Wayfair run into when solving for data velocity? Matt shares five of the most common challenges:
- Fragmented tool space
- Fragmented IAM
- Rapid Data Volume Growth
- Data Everywhere
- Limited Growth
“We were a historical SSAS user. We had query volumes about 300,000, every single month. We had many thousands of users every single month.” Matt shares and speaks to two charts, one showing the decline of SSAS usage and the increase of AtScale usage.
In this use case, Dave Mariani shares how to load foot-traffic data from SafeGraph. 351 millions rows of data is too much for Excel to handle and strays from Wayfair’s mission of velocity and agility. With the power of AtScale, Mariani shows us how one can leverage data virtualization with the weekly patterns data in a virtual cube in a fast and governed way.
In the second part of the use case, Dave also shows us how to create an Excel model to forecast sales, sharing why it’s important to leverage OLAP and MDX to compute those calculations server-side.
Dave later demonstrates how one can refresh the forecast by leveraging time-relative functions and direct connections to the data.
Why do Google BigQuery customers choose AtScale? Dave shares our TPC-DS 10 TB Benchmark results.
Related Content:
- Analytics Leader Spotlight, Matt Hartwig of Wayfair
- Watch the on-demand webinar here.
- Download our Cloud Data Warehouse Performance Benchmark report
- How to Optimize Sales Analytics Using 10x the Data at 1/10th the Cost (SlideShare Presentation)
Power BI/Fabric Benchmarks