the saas CPO checklist

36 steps — free template

The SaaS CPO
checklist to build
scalable dashboard
experiences

We packaged 8+ years of R&D experience and 100s of data visualization projects in this checklist. It provides a step-by-step plan and actionable best practices for CPOs or product managers. We hope it will help you frame your project and reduce the complexity of building dashboards for your users!

download pdf version
icon-cross

Continue reading
and get the PDF

Always start with your users’
decision-making needs

Product Manager for 1 to 4 weeks

Carry-out user interviews with at least 5 customers

• Only 45% of people with access to analytics actually report using them. The most frequent reason they cite— they’re too difficult to use.
• So before getting your hands dirty with crunching the data or building the dashboards, it’s important to identify different user groups and the problems they need to solve. Which means figuring out their goals, the questions they want answered, and the steps they take to make decisions.
• The best way to do that is in-person interviews or field visits, but if that’s not possible, calls are great too!

Define end-user personas and match their needs and skills to clear product requirements

Different people make decisions differently and will require different outcomes. To make your dashboards relevant, you need to personalize the insights they show. Since personalizing them on a one-to-one basis is nearly impossible, you need to identify end user personas, to determine which information is most relevant to each user. With that context, you can align what you’re building with what they need to achieve.
This could be broken down into:
Role: C-level executives, business users, data analysts, IT users, etc.
Pain points: what prevents them from doing their job? From making the best decisions?
Outcomes and metrics: What business questions I’m trying to help them answer? What metrics do I need to collect? How are those metrics defined? What are the relevant business dimensions and time granularity?
Technical expertise and skills: to define the degree of sophistication of your application. How familiar are they with BI tools? Are they business users or BI analysts? Can they write or customize SQL? Are they proficient in Excel?
Data fluency: The analytics’ level of detail should match the users’ comfort zone and the degree that they understand nuances in the data. How sophisticated are they with using data? Do they enjoy digging into the numbers?
Mobility: field workers or traveling executives may need access to data via smartphone or tablet, others on the other hand, may mainly work from a desktop
Understanding the complex needs of each persona is crucial and will have a substantial impact on adoption and engagement.

Understand user workflows and how they influence your project

• In order to build engaging dashboards, you need to understand your users’ habits when it comes to making decision based on your product data. The format and information displayed inside your analytics need to seamlessly fit their existing workflows.
• Determine how your analytics will connect to their decisions, what information can they act-on? When do they need it? Where in your product should you make it available (full dashboard pages vs single widgets at the point of decision-making)
• For more information:
https://toucantoco.com/blog/en/successful-data-projects-checklist
https://docs.toucantoco.com/concepteur/07-method.html
https://toucantoco.com/blog/en/how-to-rock-a-kick-off-in-a-data-project

Design outcome-oriented dashboards
for your product

Product Manager for 1 to 4 weeks

Frame your project and set a solid foundation

Now that you’ve defined what your dashboards should accomplish for your users, you have to define how they will look and how they will work. You need to think about big-picture elements and building blocks for what you will create.
Conventionally, single-page dashboards are the most common and conventional way of building analytics. But there are many forms to consider: Online portal or standalone app, dashboard inside your product, visualizations and widgets built in-context inside users’ usual workflows, etc.
Before deciding how you want to deliver your analytics, think about a few factors:
• Data detail & density: will it offer your users the ability to drill down to see more elements? And how rich your views will be?
• Mobility: do your users need to access it on the go?
• Collaboration: is it important to be able to share the content of your analytics with other users?

Identify core functionality and outline specific features

Some basic interactive features you should consider:
Drill down: give your users the ability to deep dive and breakout the information you’re providing, from high level metrics to smaller details.
Filters: allow your users to define the scope of their data to reflect their needs. Filters can either be global, meaning they refine the scope of your entire dashboard. They can also be local, meaning they only impact individual charts, metrics, or views.
Comparison: ability to see two or more data subsets side by side for comparison reasons.

Define out-of-scope items

• It’s also important to decide where you draw the line on things that won’t be included in your analytics roadmap.
• Although it’s impossible to identify all out-of-bounds features in advance, at least try to create a set of guidelines. For example, you might choose to deliver dashboards that users can customize by resizing different components or moving them around, but not allowing them to build the visualizations from scratch. Or, you may decide that you’ll provide a standardized set of analytics covering a variety of use cases you’ve identified, but you won’t build tailored, customer-specific, “one-off” data models.

Leverage the art of data storytelling

Think of your analytics as more than just charts in a grid. There is an entire art behind turning data into insights in a clear pedagogical way, and it’s called Data Storytelling. It is the ability to tell a story with data and to personalize it according to the audience.
If you’re choosing to build your own analytics, it’s up to your team to find the way to tell stories, and thankfully, they’re not flying blind: at Toucan we live and breathe for data storytelling and are more than happy to share with you what we learned:
here are our top 10 design and data storytelling best practices to build easy-to-use dashboards for untrained business users.

Choose the metrics you want to display through your analytics

• Consolidate your users’ needs and the questions they’re trying to answer into a core set of indicators and metrics. This helps users focus on what really matters instead of being distracted by useless or redundant metrics or graphs.
• Find unique or specialized metrics that make the data highly valuable in your industry.

Pick the right visualizations to answer the right questions

• Questions before visualizations: A single metric can be displayed through different visualization methods. For example, the metric “Revenue” can be displayed as a single metric or as a line chart to show its evolution over time, the difference between the two is the question you’re trying to answer.
• Toucan has built extensive documentation with detailed information on when to use which chart: head over to our viz gallery for inspiration.
• For more information:
Dataviz Dos & Don’ts

Build a prototype for user feedback and design iterations

• Don’t wait until you’re finished to show your users what you built! Even a simple prototype of how the actual dashboard would look and function shared with a few beta testers can be a great way to kick off a feedback loop and constantly iterate to make your new features even more effective.
Based on their response, you can build an initial 1st version, and proactively invite feedback from end-users to constantly improve until you get it right.
This process can be extremely helpful to give you guidance for your development efforts, it also gets users intrigued and encourages them to share valuable feedback.
• Platforms like InVision, Figma or even an interactive PPT are sufficient. At Toucan, we even sometimes work with a pen and paper approach on design books.
• For more information:
Design-first approach

Define and build the right supporting
data architecture

• Data Architect / Senior Back-end Developer for 1 to 4 weeks
• 2 Back-end Developers + 1 Product Manager for 3 to 12 months
+ 1 to 2 weeks of testing

Find what data is available, and what it looks like

• Once you’ve defined what metrics you want to display and how your analytics will look, it’s time to do the dirty work: start looking for your data.
• Data scoping can be tedious, but it’s important to find all relevant data sets and most importantly assess if what you have is able to support your project.
• Deep dive into the structure and indicators of all possible data systems to zero-in on the final indicators.
• Define the aggregation / compute rules when needed
• If the data you need is not available, or not suitable to answer the questions at the top of your list, you may need to table some of your ideas until you can get the data you need to answer them.

Assess data volumes and define refresh frequency

• Quantify the data you need and anticipate the impact of your decisions on your project and your cost.
• Your chosen data pipeline and storage structure needs to fit and support the volume of data and type of analysis you’ll be offering: i.e. if you offer advanced drill downs or even further, self-service capabilities, your database should support high concurrency, meaning it should be able to handle a large number of users accessing your analytics and running simultaneous queries.
• Determine storage cost and querying & performance requirements.
• Define refresh intervals and query frequency.

Historicize your data based on time intervals that make sense for your users

Make sure that your database structure allows temporal querying. i.e. if you’re building a visualization to illustrate the evolution of a metric over the years or if you want your users to go back to data dated a few years back, your database should be able to support this.

Define granularity levels and operational views

During your research and scoping phase, you should have identified granularity levels needed for the analytics you’re building. The level of granularity and aggregation has an impact on how you structure your data (think Elasticsearch or MongoDB indexes for instance). During this step, you need to replicate those decisions inside your data structure.

Assess the impact of self-service capabilities on your data structure

• If you’ve identified during the research and scoping phase the need to offer your users with self-service capabilities, this has an impact on your data structure.
• It’s time to define exactly what you mean by self-service functionalities and draw the line to determine what you’ll be doing and what’s out of scope: if you’re giving your users complete freedom to query anything in your database, your data structure needs to be able to handle this.

Build a future proof API

• You need to anticipate future possibilities to leverage the data structure you built to support new use cases: i.e. Business Intelligence, Machine Learning.
• Anticipate multi-tenancy support: you need to consider security and user permissions.
• Multi endpoint support: if your data will be used for various formats and mediums, take that into account when working on your data structure. i.e: data exports, PDF, Excel/CSV files, etc.

Assess available market solutions

Don’t reinvent the wheel and choose a best-in-breed approach, here are a couple of solutions to consider:
• Data connection & integration: Fivetran, Mulesoft, Bearer
• Data storage: Snowflake, Mongo Atlass, Redshift, etc.
• Authentication: open source (SAML2, OpenID Connect) or managed (Amazon Cognito, Azure Active directory) For more information:
https://docs.toucantoco.com/concepteur/method/07-create-data-model.html

Build visualizations according to defined
user needs and requirements

• Solution design: Architect / Senior Front-end Developer for 1 to 4 weeks
• Solution implementation: 2 Front-end Dev + 1 Product Manager for 2 to 6 months
• Testing: 2 Front-end Developers + 1 Product Manager for 2 to 4 weeks

Take into account responsiveness requirements

• If you’ve identified during your research and scoping phase that responsiveness and mobility is key to address your target audience’s pains, then you need to take that into consideration when building your visualizations.
• If responsiveness is not a requirement now, think about whether it would be required in the long run: it’s better to anticipate future needs, since adding it as an afterthought can be extremely painful, and you might risk losing precious development time when the need manifests itself!

Assess the need for interactivity to define the requirements of your chart API

Depending on the form you chose for your analytics, they need to be much more than charts on a static page to be actionable and useful.
A couple of things to consider:
• Basic user filtering, cross filtering, master filtering
• Interactions between visualizations and the rest of your product and vice versa

Assess available solutions on the market and select one

There are many available solutions to make it easier for your developers, from code libraries (vanilla JS, d3.JS , highchart , vega ) to chart vendors and on the shelf BI products to specialized Embedded Analytics vendors (like Toucan ).
If you’re mixing and matching different solutions in your toolkit, make sure they fit with your existing stack and data architecture
Rely as much as possible on peer evaluations and user communities

Think beyond charts, deliver what your
users need to take better decisions

Resources depend on actions you decide to take

Provide explanation before information

• When building analytics features, you need to ensure all users are on the same page when it comes to the displayed data.
• Context and explanation are crucial to understanding new and unfamiliar concepts, especially for business users. Providing data without context is the difference between a Japanese chef explaining the art of sushi making and a fishmonger throwing a fish at your head. If you just present raw data, users will have a hard time uncovering the story behind the numbers on their own.
• Phone calls and reporting requests to your success & support team are often a result of lack of context, misalignment in how a KPI was calculated or how it is defined, or simply lack of cohesion.
• You can avoid this by adding a glossary to your visualizations containing a definition of keywords used on your graph to align everyone on a single definition of the data.

Enable easy sharing and collaboration

Your users will want to share data, collaborate with colleagues, and distribute their findings. Collaboration features make sure to keep your users engaged and centralizes exchanges inside your product. Some features to consider:
Data annotation: ability to highlight certain components in your visualizations and share them with colleagues by adding a commentary to a specific number or chart. It’s the virtual equivalent of writing notes in the margin of a paper.
Comments: a chat area where users can share insights with their colleagues, ask questions and interact.
Call to action: users are more likely to act when explicitly guided towards next steps.
Resources:
• 2 Front-end Developers and 1 Product Manager for 1 months)
• x 2-4 if you need end-user self-service

Add alerting & push notifications

Alerts are a great way to keep your users engaged with your analytics and help them keep an eye on critical business information in the quickest and most efficient possible way. The alert may be activated when a metric goes outside a particular threshold. For example, if stock levels of a critical item falls below or rises above a certain level, a store manager can be automatically informed to take action immediately. They can be based on:
• a set frequency: every hour / day / week
• certain event triggers: if metrics x condition, then email/ in-product notification/SMS, etc.
Resources
• 1 Front-end Developer, 1 Back-end Developer and 1 Product Manager for 1 month
• x 2-4 if you need end-user self-service alerts. For example, “as a user I can set up my own custom alerts & notifications”

Add CSV or Excel export options directly from charts

Excel or CSV files are the most basic formats to consider when it comes to export, but are also a must-have. You should allow your users to pull information out of your app to give them more detailed information of what’s behind the dashboards for further analysis.
Resources
• 1 Front-end Developer, 1 Back-end Developer and 1 Product Manager for 0,5 months per export type

Add PDF export option directly from the dashboard

PDF exports may sound simple, but they can be some of the trickiest parts of an analytics project. If you want to become a key insights provider and answer your client organizations’ reporting needs, then allowing your users to export PDF versions of your analytics is crucial. What you need to consider when building PDF export features:
Format: simple static PDFs or interactive PDFs on webpages to allow end users to drill through to get more information.
Pagination: each page needs to have a meaningful amount of data to avoid over-cluttering, it’s important to decide how much information goes into a page and when it makes sense to start a new page.
Personalization: ability to be individually customized down to specific user types (c-level executive, field manager, etc.) and data types (per country, channel, store, etc.)
Report bursting or scheduled delivery: ability to schedule the delivery of customized reports for a specific set of recipients run on a daily, weekly, monthly, or custom schedule.
Here are some tips for you:
• Ability to support small and large datasets, cross-browser compatibility, print quality, customization per user.
• Think about your design: placement of logos, labels, images, text, legends, sources, etc. Precise dimensions and placement per type of chart displayed. Maintain branding and overall look and feel of your product. Rendering in both portrait and landscape orientation.
Resources
• 1 Front-end Developer, 1 Back-end Developer and 1 Product Manager for 0,5 months per additional export type
• x 2-4 if you need end-user self-service. i.e. “as an end user I can build my own PDF report”

Link your analytics to PowerPoint directly from your dashboard

Today, when users want to present results, PowerPoint is still the preferred format for presentations both internally (team meetings, retrospective reviews, follow-ups) or externally (investment decks, board meetings, etc). Building a link from your analytics to PowerPoint to smoothen the transition for your users is a great way to increase adoption and engagement and make your dashboards even more useful and actionable.
Resources
• 1 Front-end Developer, 1 Back-end Developer and 1 Product Manager for 1 month

Add user group dynamic customization or parent-children templating

Different people consume data differently and need to take different decisions. In that context, you need to allow data and view customization per user or user group.
Resources:
• 1 Front-end Developer, 1 Back-end developer and 1 Product Manager for 1 month
• x 2-4 if you need end-user self-service. i.e. “as an end user I can add my favorite charts to my dashboard”

Apply 0 bug policy, test coverage
and CI/CD requirements

• 1 to 2 months to build your automated test pipeline on data and chart API
• 1 day per release for manual testing

Test for cross-browser and cross-device support

As you’re building your analytics features, it’s your responsibility to make sure that not only your features answer specific business needs and are useful to your users, but that they work for all of your users, no matter what browser or device they’re using. They should provide the same interactivity and user experience across all browsers and devices. You need to consider:
• Different browsers including older ones that some users might still be using which don’t support the latest advances.
• Different devices with different sizes and capabilities: smartphones, tablets, big screens, small computer screens, etc.

Carry out a mix of automated and manual testing approaches

Manual testing and automated testing cover two vast areas. There are several methods within each category. Some of them are better suited to manual testing, and some are best performed through automation, this is why it’s recommended to carry out a mix of both methods for optimal performance.
For example, Toucan’s acceptance list is currently running on 120+ items.

Organize test & support to minimize R&D perturbations

• Designate a rotating dedicated tech resource for support to keep your tech team focused
• Define beforehand what you will support and what you won’t. i.e. do you want to support legacy browsers? iPad formats? What makes or breaks your releases? What continuous improvements are acceptable as ongoing Kaizen?
• Build a framework to prioritize dashboarding linked bugs
• Keep an eye on your tech stack dependency
• Look out for security patches & depreciation
• Look out for continuous improvements offering new business opportunities. i.e. could this new release of d3.js allow you to reach one of our business goals at a reasonable dev cost? Would it power new use cases?

Bonus: If you want your non-technical
teams to set it up, use it, and maintain
it in full-autonomy, build a WYSIWYG
interface

Resources depend on actions you decide to take!

Determine your non-technical teams’ data fluency and technical expertise

It’s important to define who will do the bulk of the work when it comes to building your dashboards. You’ll need to align the tools they will use with their skills and level of understanding.
For example, if you want to keep your tech team focused on your main offer and choose to let your CSMs build a part of your dashboards or maintain them, you know that they’re not data savvy and have limited technical skills. They will need easy to use tools that don’t require advanced technical and SQL coding skills.

Build a studio for your builders

Not everyone works with or responds to technology in the same way. What looks like a piece of cake to some might be a hair-pulling nightmare to others. This is why it’s important for you to gauge how comfortable your teams are with the tools you’re providing them with. While you can’t expect CSMs or Product managers to write SQL queries, here are a couple of stories you can choose from to anticipate needs and resources:
• Let my PM or CSM build visualizations from curated datasets: 1 Front-end Developer, 1 Back-end Developer 1 Product Manager for 2 to 4 months
• Allow my PM / CSM to handle dashboarding access rights and roles (viewer, contributor, builder): 1 Front-end Developer, 1 Back-end Developer 1 Product Manager for 1 to 2 months
• Allow my PM / CSM to customize graphic elements for each customer (colors, logo, …): 1 Front-end Developer, 1 Back-end Developer 1 Product Manager for 2 weeks to 1 month
• Allow my PM / CSM to set up push notifications using emails templates: 1 Front-end Developer, 1 Back-end Developer 1 Product Manager for 1 to 2 months
• Allow my PM / CSM to build programmatic PDF/PPT reports: 1 Front-end Developer, 1 Back-end Developer, 1 Product Manager for 1 to 2 months
• Allow my PM / CSM to connect to new data sources: 1 Back-end Developer, 1 Product Manager for 1 to 4 months
• Allow my PM / CSM to do data preparation (joins, aggregations, reshaping, …) 1 front-end developer, 1 Back-end Developer, 1 Product Manager for 3 to 6 months
We open-sourced our connector module at Toucan https://github.com/ToucanToco/toucan-connectors
Assess available market solutions:
• Reuse our query builder interface http://weaverbird.toucantoco.com
• Explore what’s out there! https://vistools.net/, https://www.g2.com/categories/embedded-business-intelligence#grid

You built your dashboards… now what?
— get to future proofing!

Write the F****** manual! (RTFM!)

• We live in a world where we can no longer get by with siloed information. Best in class organizations thrive by embracing a knowledge sharing culture, and when it comes to your analytics, that is no exception.
• RTFM is a nod to geek culture. This is what developers tell each other when a question is asked and the answer is already documented somewhere. We made it part of our core values, and you should too!
• Imagine this, you built the first version of your analytics and shipped it to your customers, everything is going well. A couple of years down the road, the key people involved in the project have left your company, but you’ve decided to improve that first version.
• By making sure that every step of the process is well documented, it makes it easier for teams and new recruits to learn from past mistakes, so they can move forward with an improved strategy. It also allows them to follow in their previous coworkers’ footsteps, build upon what worked, so they can scale success through a systematic approach.

Brace for user requests

Once you’re done with the first batch of analytics functionalities, your job doesn’t end there. As your users start putting your dashboards to good use, you will receive increased requests for new features, better visualizations, or more seamless workflows. You need to build a feedback loop to stay ahead of user requirements.

Prepare for market evolution, and competitor innovations

“The Only Constant in Life Is Change.”- Heraclitus —Your users are not the only ones changing, the market too. Whether it comes to innovations in the embedded analytics space made possible with AI and ML technologies, or competitor products upping their analytics game, you’ll need to keep an eye on both and adapt accordingly.

Define what happens if you no longer have the resources to upgrade in-house

Scaling your product is extremely hard, especially if you need to work on your core product while also managing a parallel analytics roadmap. Something to consider is partnering with an embedded analytics vendor to scale and take your analytics to the next level.

Want an alternative to
build-it-yourself analytics?

Here at Toucan Toco, we’ve proven our ability to support organizations on that exciting journey of exploring the world of embedded analytics. In fact, we’re the acknowledged #1 embedded analytics partner for growing teams who want to add a dashboarding experience inside their software or SaaS product. And don’t just take our word for it: your peers have spoken on G2.
If you’re looking to future proof your organization and take your embedded analytics capabilities to the next level, get in touch with Toucan today.

Did you like this resource?

Spread the world about building great analytics features for your users!

download pdf version
icon-cross

Continue reading
and get the PDF