The past year has seen adoption rates for cloud analytics skyrocket.
And while organizations are spending more on their cloud infrastructure than ever before, 61% of them plan to optimize their cloud spending in 2021. This makes cost optimization the top cloud initiative for the 5th year in a row according to a poll by Flexera.
If you’re in charge of managing your organization’s cloud infrastructure, you have to garner a deep understanding of the way your company uses cloud analytics in order to make sure cloud facilities meet the needs of business users.
If you’re a CDO, an IT System manager, a database admin, or a data project manager, you need to be able to measure your Snowflake ROI quickly in order to make better usage decisions.
Here's the catch with cloud data: they’re notoriously flexible, scaling up and down according to your needs. To those who aren't optimizing their usage of BI tools, this can be a disaster. However, careful monitoring and focus of your organization’s usage can allow you to optimize costs even more efficiently.
One such flexible cost structure is that of Snowflake: the new-generation data warehouse has innovated by decoupling storage and compute functions. By bringing this kind of flexibility to the world of big data, Snowflake has set itself apart as one of the most disruptive cloud analytics solutions on the market. This means, if you’re the one in charge of monitoring Snowflake usage, you’ll need instant access to important usage insights continuously.
Here are the four types of KPIs that will allow you to do just that. Tracking them will help you paint a complete picture of how your data warehouse is running and share clear and actionable insights with company stakeholders.
1- Monitor cost with credit KPIs
The most obvious place to start when you’re trying to optimize your Snowflake costs is the number of credits. Choose a dashboard that allows you to follow the evolution of credit usage over time for all your warehouses and that allows you to:
- Drill down for credit consumption by the warehouse
- Pinpoint particularly credit-intensive warehouses and users
- Understand why those users need so much data infrastructure investment
Are they misusing Snowflake? Query performance KPIs (below) can allow you to figure it out. Spikes in usage that are not explained by particular business needs should drive you to ask more detailed questions and uncover potential wasteful queries. You’ll also get more detail on your consumption drivers by figuring out what proportion of your credits are used on cloud services.
2- Monitor performance with query KPIs
Take a closer look at queries to understand which level of performance your data warehouse users are experiencing. Before diving into the query error rates or length of results, understanding when most of your queries are made can already get you a long way. A heatmap of queries by time and day will allow you to understand your organization’s habits when it comes to Snowflake. Spotting the slots that are particularly query-intensive will allow you to adapt your data warehouse so that it can support more compute at these specific times, or will enable you to spread out queries throughout the day.
When it comes to queries themselves, tracking the number of rows produced or the duration of queries by second can allow you to answer some important questions:
- Do you have the right warehouses to serve Snowflake users?
- Are there suboptimal queries generating bad performances or even costs?
Similarly, tracking the error rate for your query can allow you to spot red flags before they become a problem so you can troubleshoot faulty queries or extend your help to the analysts that might need it.
3- Monitor adoption with usage KPIs
When you’ve advocated for a specific solution, you need to be able to follow adoption rates and make sure the infrastructure you’ve chosen actually fits your organization. A straightforward metric that you can follow is the total number of queries.
You’ll also need to know who are the most active users of Snowflake and pinpoint the warehouses that are used the most which will allow you to get a sense of adoption rates across various teams or departments in your organization.
Being able to share important adoption KPIs across the organization can help you explain your decisions and give stakeholders the bigger picture.
4- Monitor security with login KPIs
Security is a prime concern when it comes to cloud data analytics. Ultimately, the cost of a data breach would be so high that any cost optimization effort should put security at the forefront.
This is why it is crucial to also monitor login attempts, especially those that fail. A high number of failed attempts should be a red flag to follow up on in order to make sure no one is trying to hack into your data warehouse.
Another useful KPI you might want to keep an eye on is a map of logins localization. Logins from geographies that look unusual can raise a red flag.
There are many free tools on the market allowing you to track these important KPIs, and we’ve recently opened up ours to the public. Head here to give it a try, or contact us: we’ll be happy to discuss your Snowflake monitoring needs.
