Strong roadmaps are a result of properly assessing a great deal of options and identifying the ones that best fit your constraints and goals. The more information you have, the more options you’ll be able to identify in your roadmap.
The challenge with information is that it (1) requires regular collection, (2) requires proper cleaning to be useful, and (3) both collecting and cleaning are time-consuming.
This article goes into how to acquire information on a regular basis and how you can attempt to do so efficiently.
“The universe is only as large as our perception of it. When we cultivate our awareness, we are expanding the universe. This expands the scope, not just of the material at our disposal to create from, but of the life we get to live.”
— Rick Rubin, The Creative Act: A Way of Being
The feedback factory
In most businesses, there is an abundance of customer contact. The reality is that, in many cases, the daily contact your business has with customers covers a very broad and very deep slice of the customer experience and perception of your product. It includes:
Product data that is created by users using the product,
Support interactions between your team and customers,
Interactions your sales team has with customers,
Reviews, NPS surveys, social media interactions,
In-person events.
In addition to this contact, there are also specific efforts you can undertake to gather additional information, including customer interviews, surveys, analysis, usability tests, experiments, and gradual feature roll-outs, among other things.
The main challenge is that many times this rich stream of feedback never makes it to your desk, and as a result, you’re left with incomplete information and blindspots about your product and business.
The question is, then, how do you sustainably set your team up to gather information at scale, process that information, and have it feed your decision-making? The answer is in building a feedback factory.
Building the feedback factory
A feedback factory is a model for systematizing the routine acquisition and processing of feedback at scale. To build the feedback factory you have to assemble the three core systems and ingrain these systems into your culture. These three systems are:
Collection
Cataloguing
Socialization
Collection
A key idea behind building a feedback factory is to try to expand the ways you can capture valuable customer input. By tapping into all sources of contact with customers you’ll be able to collect more feedback than you thought was possible.
Collection includes any acquisition of information about your customer, business, market, or products, and the storage of this information. There are 4 core mechanics that make up a great collection system:
The feedback portal
You want to create a space where both team members and customers can log any feedback or insights. The feedback collection portal setup is simple: you want one or more forms that team members or customers can fill out that links to a database where you store all logged feedback. You then want to permeate the usage of this form to everyone at your company and use it whenever there is applicable customer contact, whether it be used on calls, emails, ticket reviews, or even integrated directly into your products.
Ultimately, by expanding this approach to every customer contact across your business, you’ll end up building a small window into those interactions, and through that window will come meaningful nuggets from which you can build projects. This will become the most abundant form of feedback that you have stored. It’ll be composed of short messages about your product experience but collected at high levels of volume allowing you to effectively quantify and qualify the intensity of problems and opportunities.
Contact with customer-facing team leads
Connecting with team leads, I’ve found, is a great way to get high-level feedback, but it is also how you build a shared commitment to feedback collection. This is because team leads are the ones who can keep individuals outside of your team committed to capturing and logging feedback.
You’ll want a weekly or bi-weekly sync dedicated to their domain area where you can learn from their customer contact and understanding of the business line. I’ve found this recurring cadence strengthens the bond between teams and gives you the ability to quickly course correct if feedback collection is lacking.
Research integration into the software development lifecycle
Once you move onto a world where you’re trying to solve a specific problem you need to collect information in more targeted ways. The key here is integrating gathering feedback and information into the software development process.
Specifically, it means building the time and appetite to do validation/discovery activities at the beginning, middle, and end of projects. Not all projects need research, and when they do, they might only need it at certain stages. However, without the ability to dig in during a project, you’ll often find that you’re missing the necessary information to make correct scoping and problem-solving decisions.
Practically speaking, this means determining what kind of validation (if any) your approach to a project needs, what information you’re missing that could enable better decisions, and deploying efforts designed around capturing this information.
This concept is an article on its own, but some examples of this are interviewing customers before a project starts, doing usability sessions, and running early access programs (to name a few). These exercises can be critical to your decision making process and make up a key part of collecting feedback.
Reviewing data on a regular cadence
It’s easy to think of feedback as verbal customer communication, however, any customer reaction is feedback. I believe data often reflects customer sentiment, preference, concerns, and confusion about the product. Data is a form of non-verbal feedback and is just as important as feedback delivered verbally by customers.
You want to track key metrics from your product experience on an ongoing basis so that you can develop a more cohesive understanding of the area you manage. I look at certain datasets daily (often every few hours), certain data sets weekly, and certain data sets monthly. By building these regular checking windows you’ll end up building a sophisticated understanding of the area you manage and will be spurred to undertake investigations that can generate valuable projects.
Helping others understand what feedback is:
I’ve found that people often discount information that is valuable feedback because they don’t see it as feedback. It sounds obvious, but it’s worth remembering: how is someone supposed to collect feedback if they don’t fully know what feedback is?
I like to describe feedback in the following way:
Feedback is any information pertaining to how customers experience your business. Fundamentally, any experience that a customer expresses about your product, services, value, and so forth is feedback. Confusion about your product is feedback. Anger, love, and anything in between is feedback.
As you can surmise, almost every single customer interaction contains feedback. This can be overwhelming for people in customer-facing roles, but the more they buy into this idea and the more motivated they are about that information being used, the better off your business will be in terms of its ability to make decisions.
Cataloging
Ok, so you have tons of information now. New problem, how do you make that information accessible weeks, months, or even years after it was originally captured? Well, the answer is painful but necessary - you need to catalog information and create and maintain libraries.
There are four keys to doing this:
Put as much as you can into the feedback portal.
The more you concentrate your knowledge base in a single place the less things are likely to get lost. The feedback portal is one such place and it has the distinct advantage that the information making it into the feedback portal is getting there via form.
Make sure that your forms are proactively capturing information that can be used to catalog insights. Specifically, ask form fillers to provide user type, feedback type, and product area along with the feedback itself. Doing so will save you hours of feedback re-processing.
Create re-usable knowledge hubs.
As you build more and more feedback materials (interviews, write-ups, analysis, etc) the challenge is how to make it intuitive for people to find this information and remember to store any new information in that same place. It is essential for a sustainable feedback factory that the information you’re collecting does not get scattered. There are a few key ways you can group these materials effectively.
Group by research activity type: Here, you’re grouping information by the type of feedback-gathering activity that was used instead of the content area. Think about a hub where you can find sections containing all of your interviews, all of your surveys, all of your usability sessions, etc.
Group by specific projects or features: Here you’re creating a hubs specific to projects or features. This allows anyone who is looking for information about that specific feature or use case to easily find previous research efforts around that area.
Group by user types: Here’s you’re grouping materials and information by specific user types that you work with allowing anyone to search for information by that user type.
Ultimately the type of aggregation you do depends on your business, what information is most valuable, and how you often go about solving problems. That being said, the key is that you are grouping information in some way so that it is easier for others to find it and so that people always know where to store additional information.
Try to consolidate your knowledge to a small number of resources
The more spread out your information is the less likely you are to use it and the more likely you are to forget you have it. As much as you can, try to keep a small number of places where information lives as this will help create sustainable patterns around collecting, cataloging, and retrieving information.
Try to have 3-4 main structures where your feedback lives with key differentiation between them. In my experience, a model where you have the feedback portal (for short-form insight), knowledge hubs (for theme specific longer form insight), and metric trackers (with data on a specific domain area) has been an effective practice as I always know where to find insights based on the kind of information I’m looking for.
Routinely clean and reorganize insights
Maintenance is an essential component of a productive feedback-collection machine. You need to allocate time on a regular basis to undergo feedback cleaning, reorganization, cataloging, etc efforts. I often schedule deep cleans during slow times during the year and find that this practice yields cleaner spaces but often also yields new insights as I’m looking at feedback with a fresh perspective.
Socialization
The final piece to this system is the socialization of feedback, its use, and embedding it in your culture. This is the heart of the system, and without it, the rest of the feedback factory will fall apart.
Simply put, the feedback factory exists to help make better product decisions, if it is not clear that this is the case then the factory itself will cease production as individuals will stop being motivated to collect feedback.
Socialization is the routine broadcasting of the usage of collected feedback to power the roadmap. For me, this works in a few ways:
I routinely make projects in the backlog of the roadmap based on feedback items that were logged and tie the project to those specific items. This allows others to see feedback be used to foment actual projects and see in a very practical way me making use of their feedback.
I routinely make use of captured feedback to help drive the product development process and onboard team members onto a project. This sets up product development teams to expect and seek out feedback to make decisions and allows them to speak about their projects using the language of our customers.
I routinely highlight the specific knowledge items that helped drive the execution of the roadmap project as a project comes to a release. We often follow up with customers and team members directly about how their feedback was used as we deliver a project. Team members and customers alike find this to be valuable and feel that this shows that their opinions are listened to.
I routinely acknowledge the usefulness of feedback in large company or team-wide meetings. I specifically highlight the impact certain logged feedback had on our ability to make good decisions, avoid bad ones, and evaluate the full set of possibilities. I also deliberately call out specific individuals who have done a great job at gathering feedback as this (1) rewards them publicly and (2) creates an incentive for others to collect feedback.
By consistently making use of feedback you’ll find it clearer to define what you need in terms of future feedback as well as motivate those capturing it. The more explicit the connection of feedback to decision-making, the higher the buy-in level for participants in the feedback process.
Parting thoughts:
Any roadmap-building process is based on information. The less information you have the weaker your decision set is. The more information you have, the more options are available to you, and as a result, the more likely you are to arrive at the right option.
To ensure you have information, you need a company-wide buy-in to the importance of collecting feedback. To pull this off, you’ll need really good working relationships with other teams, and they’ll need to trust you in your ability to execute and make use of the information they’re collecting. You’ll also need to build a system where you can collect and catalog feedback at scale and keep people motivated to support that system.
In a functioning organization the whole business is bringing you the information you need to make good decisions, leaving you with a better foundation from which to make these decisions and putting the onus on your ability to evaluate and prioritize. This structure has been essential to my roadmap for many years, and hopefully, it will be to yours too.