Is the semantic layer suitable for you?: A Guide

What is the Semantic Layer?  Why should the people who aren’t trying to make money from it care? This blog is the first of a series to offer organizations a simple understanding of the semantic layer.

This post reviews these questions and helps you determine whether the semantic layer suits you.

What is the semantic layer?

The semantic layer is an abstraction over your organization’s data. It allows you to refer to your business metrics without having to interface with tables, rows, schemas, and columns. You ask questions about your business, and you get answers - without the 15 metrics and 38 dimensions in between.

For the more technical, there are a bunch of articles about how the semantic layer does these things; you can start by reading the excellent resource shared by Drew Banin, Co-founder of dbt Labs.

How can the semantic layer help your organization?

The semantic layer is a fit for you if your organization is facing the following problems:

1. Business teams are suffering in efficiency due to a lack of data
2. Data teams are unable to take up critical projects because they are clogged with data requests for the business
3. Non-technical users are learning SQL because data teams aren’t able to service their requests
4. Business teams need consistent definitions of the same KPIs; different functions use different meanings for the same metrics.
5. Business metrics are currently derived from a mess of SQL queries spread across local workspaces across different team members.

How does the semantic layer help solve these problems?

1. Business teams no longer have to spend time figuring out the “right” definition for metrics or raise tickets to modify SQL queries that report business-critical metrics. Instead, each metric has one consistent meaning across the entire organization.
2. Business teams get access to accurate metrics, regularly updated in their dashboards and reports.
3. Business teams no longer have to wait for days (or weeks!) to access the business metrics they needed yesterday - with the semantic layer, you get it when you want it.
4. Data teams no longer have to spend hours implementing mundane requests for new aggregations requested by business teams.
5. Data teams get much sharper visibility into which metrics are being created or modified via software development level monitoring into data pipelines.

How do I know if I need the semantic layer right now?

If you’ve felt that the Modern Data Stack is the simple answer to your organization’s data needs, but have been struggling to answer, “How?”, the semantic layer is probably the missing piece in the puzzle.

One helpful way to think about this is to look at your business’ tool stack. If you find that your business operations are spread across multiple different tools and platforms, something like:

• Google Ads and Facebook Ads for running paid ads
• Google Analytics for tracking website performance
• Segment/Amplitude for tracking product analytics
• Stripe for tracking finances
• Intercom for tracking customer experience
• Hubspot for tracking marketing and sales CRM

… then you can safely expect the people running these tools to have questions about the impact of their actions.

The answers to these questions are usually expressed via metrics. More tools ⇒ more metrics. Additional requirements over time include more control to slice and dice these metrics.

Consuming and creating metrics means creating a layer of querying your organization’s databases for aggregations over tables, schemas, rows, and columns. Setting this up can become a complicated problem at scale - as you probably know - which is where a semantic layer is just what an organization needs.

What is the setup cost (time, effort, money)?

(We’re following up on this soon in a more detailed post: Semantic Layer - Build v/s Buy)

There aren’t many implementations of the semantic layer in the market today. However, most mature organizations have some version of the semantic layer running internally - like Minerva for Airbnb or uMetric for Uber.

Each organization has its own unique set of challenges, and we believe that the semantic layer should adapt to its own needs. However, the broad requirements that we see for any semantic layer are as follows:

1. A central data warehouse to store all your organization’s data
2. This enables several things, from enabling cross-functional use cases to simplifying the queries made to fetch data from the warehouse
3. A transformation layer to model data and expose it to business teams in the form of metrics
4. While this can be complicated to implement in-house, many open sources like dbt allow data teams to setup a robust data pipeline coupled with the best practices of a development workflow
5. Hot Tip: dbt has launched its semantic layer implementation last week, do check it out!

Depending on your organization's needs, we believe that this is a data project which could range between several weeks to a few months. If you choose to go this route, do drop us a message we would love to hear about your experience (and hopefully guide you)!

There are, of course, several companies in this space that allow you to buy a semantic layer implementation for your organization.

There are solutions like (shameless plug) Houseware, Lightdash, and the list below, allowing you to get started with a semantic layer implementation with minimum effort (days to weeks).

Who are the stakeholders for this adoption?

The primary stakeholders for the semantic layer are business users who require data to make everyday business decisions and data teams who currently spend their teams writing out SQL queries for business metrics.

These are stakeholders right at the intersection of data and business - the ones who have to understand the business context to write SQL queries; or learn an organization’s data schema to get accurate numbers for their MIS dashboards.

In the ideal world, the semantic layer is as intuitive as running an app on your smartphone.

However, to run and maintain a semantic layer today, you would need a data team with the knowledge of dbt and an interface between data and business teams regarding what kind of metrics to create on top of the data warehouse. That’s it - no other changes to the existing visualization layers are required.

Bottomline: What's the ROI?

In a nutshell, the semantic layer can help your organization flip your relationship with data. Instead of working for data, you can make it work for you.

It not only helps you resolve some of the most pressing data issues that plague all organizations but also unlock new use cases to take your business efficiency to the next level. Some of these include:

• Trigger downstream actions in apps based on metric values (e.g., increase the budget for an ad campaign whenever you see high ROI for it)
• Track users across their value journey in the organization (from sales, as a prospect → to product, as a user → to finance, as revenue)

That’s a wrap on our guide! Stay tuned for our next blog on the series. Let us know in the comments below if we could sell you the semantic layer or if you have any more questions 😉.