How to track customer feature requests across multiple sources

At Microsoft, I work for a team called Adaptive Cards. We build a public SDK for Android/iOS/Windows apps, and like many teams, we want to be customer focused. But how do we track what customers are asking for?

Voting portals

The classic solution is to create a public voting portal. UserVoice is the traditional go-to solution for this.

Seems great, right? You have a single source of truth for what your customers want!

But this ignores the reality that feedback comes in from multiple sources. Your customers won’t only be giving you feedback via your portal. For example, what if you’re meeting with a customer in person, and they say “I need input validation”? If you’re using a traditional voting portal, you would have to tell the customer, “That’s great, please go to our portal at xyz.com and vote for that!”. Essentially, you’re putting the burden of aggregating feedback into a single source onto the customer. They might not go through the effort of going to the portal, and their feedback might be dropped. Maybe if you get enormous amounts of feedback, dropping some feedback wouldn’t matter, but we care about every little drip of feedback :)

Some other sources of feedback include Twitter, GitHub issues, emails, IMs, and more.

What we really need

We need to be able to take feedback from any channel it comes through, link it up to feature requests, and then prioritize based on the top requests. For our team, our feedback channels are primarily email, meetings, and GitHub issues (and we have a relatively low volume of feedback such that we can inspect each one ourselves).

Aggregating feedback to a backlog

The first task to be done is to consolidate our feedback into one location.

One key thing to note is we call this a “backlog”. We often don’t have time to “process” the feedback (link all the requests to existing requests) right when it comes in. But we need to make sure the feedback goes to a backlog that we can later process when we have time.

• Productboard: ✅
• ProdPad: ✅
• UserVoice: ❌ (no backlog concept, and no easy add-from-email option)

Processing feedback

Once we get some free time, we need to sort through the feedback and link it to existing or new feature requests. One key thing to note is that each piece of feedback might have multiple feature requests in it (a one-to-many relationship), as seen below with the first piece of feedback.

• Productboard: ✅
• ProdPad: 🚧 (while you can have one-to-many relationships, it doesn’t support the really nice highlighting features to visually show which asks inside the notes are already linked)
• UserVoice: ❌ (no one-to-many relationship means processing meeting notes takes way longer, you have to add each request separately)

Prioritizing / planning the release

Once we have all the feedback captured, we obviously need to be able to prioritize the top requests. But keeping the original verbatim text is useful too. Sometimes while we’re building a feature, we question “is this actually achieving the customer’s goals?”, and if we have the verbatim text of what the customer asked for, we can confirm whether it is (or know we need to ask them more details)!

There’s a whole bunch of other useful features here related to prioritizing and planning. For example, some requests are more important to others. Like in the first request, the basic isRequired validation is quite critical to the customer, whereas email/phone validation is nice-to-have.

Additionally, we might assign different weights to each company. If Contoso is an important partner of ours, we would want to assign the weight of the company overall to a higher weight, so that all their requests are subsequently treated with more importance.

• Productboard: ✅ (although you can’t assign weights to companies)
• ProdPad: 🚧 (no ability to assign importance nor weights)
• UserVoice: 🚧 (no ability to assign importance nor weights)

The future with productboard

After doing tons of research, productboard seems to be the only company out there that truly understands the end-to-end feedback processing cycle for teams like ours! Here’s how the future looks when using productboard.

Aggregating insights to the Inbox backlog

The first thing to do is to aggregate insights (feedback) into a backlog of insights that needs to be processed/triaged. If you receive an email where a customer asks for some features, that needs to be captured. If your meeting notes have feature requests, that needs to be captured. If a customer posts on GitHub asking for some features, that needs to be captured. Productboard makes it easy to integrate all that feedback into their Inbox.

For emails, simply forward the email to your productboard inbox email address (which they provide). You’re done! 15 seconds :)

For meeting notes that are in OneNote, the experience isn’t as seamless as one might wish. You have to select and copy the text of your notes, open productboard, and paste it into a new insight. About 1 minute total. Notice that during this, I even created a new company and it still only took a minute total!

For insights posted on the web, their browser extension is quite awesome! Just highlight some text, click the extension, add who it’s from, and submit it! It’ll automatically grab the link to the site itself so you can always go back later. 20 seconds total!

Processing insights

After you’ve accumulated a backlog of insights, the next thing to do is process those insights! That means linking each feature request inside an insight to an existing or new feature request. Each insight might have multiple feature requests (especially meeting notes, but even the simple GitHub comment seen below has multiple too). To do this, you simply go to productboard’s inbox, and work your way through each insight in the inbox, and mark them processed when you’re done! Insanely quick and easy.

Planning for the next release

When it’s time to decide which features you’re going to build next, you want to see what your customers asked for! productboard makes that easy to do. You can quickly see the top feature requests by sorting by user impact score.

Proving why we need to build a feature

Sometimes co-workers or management will ask why we’re building a feature. Is it worth the effort? Do we really need it right now? Are we building the right thing? With productboard, you can quickly see how many people/companies asked for the feature, and how important it is to them.

But that’s not all. You can also view the raw list of insights to see which companies in particular asked for it, and their verbatim text. And you can always go back to the original source if you need to see more context. Being able to see the original context and verbatim text is super critical. As we’re building features, we often start to have questions around how it specifically should work. To accurately answer those questions, we need to understand exactly what the customer needed. Maybe they needed a 6-star rating control, not just a “rating control”. Keeping that original context is critical!

Where productboard could improve

While productboard is nearly perfect, it’s not perfectly perfect :) But it still seems to be the best tool for capturing customer feedback!

Browser extension doesn’t show if insights were already captured

If my co-worker Matt goes to our GitHub, he has no way of knowing whether I’ve already captured insights from an issue or not! The extension should show already-captured insights from the same page URL when you click the extension, to avoid submitting duplicate insights.

Browser extension doesn’t allow processing insights inline

Sometimes I want to process the insights right then and there. Instead, I have to submit the insight to the inbox, click the link to view it, wait for the page to load, and then link the text to features.

Capturing insights from Microsoft Teams

If a customer asks for a feature in a Teams channel or message, you have to resort to copy/pasting into productboard’s inbox. Not a huge deal for our team since we don’t use Teams for this usually, but occasionally it happens.

The insights view could use some improvements

Viewing the list of customers that asked for a feature isn’t exactly super user-friendly. There should at least be an aggregation view at the top (similar to the really useful aggregation view that appears in the list items).

You can’t assign different weights to each company

Even though our partnership with, say, Contoso, is more important than our partnership with random GitHub user Jessica, we have no way of expressing that through productboard. If both people say that something is super important, their requests are currently equal weight. We should be able to say that Contoso’s requests all get extra weight since they’re a key partner.

Insights inbox is currently useless if you have multiple product teams using the same productboard

They currently don’t support having separate inboxes for separate products, even though they support having separate products within everything else! If our Adaptive Cards team and the Project Rome team wanted to both use the same productboard, it works well for everything except the inbox. All of our insights go to the same inbox! They’re working on fixing this though :)

Responding to feedback from non-email sources

It’d be super awesome if I could have feedback that some people posted on GitHub and some people emailed us directly, and easily respond to all of them saying that we implemented their request! And it would automatically respond on GitHub if the user originally posted on GitHub, or respond to the original email the customer sent. That would just be so epic! But that would certainly be the small cherry on top, and it’s definitely pretty complicated to implement, and it isn’t a deal breaker if we have to manually respond. An alternative could be we tell customers to monitor the public portal for their idea, but just like at the beginning of this article, that’d be putting the burden on the customer!

Other solutions

Here’s some other solutions, and reasons they fall short of ideal.

ProdPad

ProdPad seems to be the closest match to productboard. They surpass productboard by having an open API and DevOps integration, but they fall behind in the core essentials — their UI and experience for capturing and tracking feedback isn’t as good, and that’s the bread-and-butter of where we spend our time!

Aggregating feedback to unsorted bucket

ProdPad has the same “inbox” concept as productboard. All feedback comes in as “unprocessed” (they call it unsorted), so you can have a backlog of feedback you need to triage.

For emails, exactly the same as productboard: Simply forward the email to your Zapier inbox email address (you have to set this up but it took ~5 minutes). You’re done! 15 seconds.

(GIF coming soon)

For meeting notes that are in OneNote, exactly the same as productboard: The experience isn’t as seamless as one might wish. You have to select and copy the text of your notes, open ProdPad, and paste it into a new insight. About 1 minute total. Notice that during this, I even created a new contact/company and it still only took a minute total!

(GIF coming soon)

For insights posted on the web, exactly the same as productboard: Their browser extension is quite awesome! Just highlight some text, click the extension, add who it’s from, and submit it! It’ll automatically grab the link to the site itself so you can always go back later. 20 seconds total!

(GIF coming soon)

Processing feedback

After you’ve accumulated feedback, the next thing to do is process that feedback! That means linking each feature request inside a feedback to an existing or new idea. Each feedback might have multiple feature requests (especially meeting notes, but even the simple GitHub comment seen below has multiple too). ProdPad doesn’t handle meeting notes with multiple requests nearly as well, but at least it supports one-to-many.

(GIF coming soon)

Planning for the next release

When it’s time to decide which features you’re going to build next, you want to see what your customers asked for! ProdPad’s UI isn’t as great for this, but it can be functional enough.

(pic coming soon)

Proving why we need to build a feature

Sometimes co-workers or management will ask why we’re building a feature. Is it worth the effort? Do we really need it right now? Are we building the right thing? With productboard, you can quickly see how many people/companies asked for the feature, but you can’t see how important it is to them.

(pic coming soon)

But that’s not all. You can also view the raw list of insights to see which companies in particular asked for it, and their verbatim text. The only problem is seeing their verbatim text is difficult if the feedback had multiple feature requests. But you can always go back to the original source if you need to see more context. Being able to see the original context and verbatim text is super critical. As we’re building features, we often start to have questions around how it specifically should work. To accurately answer those questions, we need to understand exactly what the customer needed. Maybe they needed a 6-star rating control, not just a “rating control”. Keeping that original context is critical!

(pic coming soon)

UserVoice

UserVoice does actually support feedback from other sources, but it doesn’t seem to be their primary flow. But they do have the same browser extension that lets you highlight text and link it to a feature request.

Where UserVoice falls short is aggregating feedback from non-online sources, like meeting notes, but most importantly, it lacks capturing a backlog of feedback that needs to be triaged.

Aggregating/processing feedback

In UserVoice, there’s no concept of a “backlog” of feedback that needs to be processed, so the aggregation and processing step is one-in-the-same (which has drawbacks since we often don’t have time to process immediately, which means we might drop the feedback completely).

For email, since they don’t have a backlog, you can’t simply forward email to an inbox. And since they don’t have a one-to-many concept of feedback-to-suggestions, you have to copy each individual suggestion from the email, losing the context of the entire email. This all takes an average of about 3 minutes per email, including finding the correct suggestion to link to.

For meeting notes, this is even more painful, since meeting notes especially often have multiple feature requests, which you have to copy/paste in separately rather than pasting the entire meeting notes and then linking the requests from within it. This could take 3+ minutes.

For GitHub/Twitter/other online sources, this is a bit more efficient thanks to their browser extension, but you still have to link the feature immediately, so this will still take a minute or more. Additionally, you lose all of the original context of the overall meeting notes since they don’t support one-to-many! You only include the text you think is relevant at the time (which later you might realize you need more context).