From Gut to Plan: The Thoughtful Execution Framework

If there’s one big takeaway I’ve gotten from years of meditation training and silent retreats I’ve attended, it's that we shouldn’t blindly trust our minds. Our minds only offer a subjective representation of reality, colored by emotions and past experiences. And when you go to work, you can’t really “leave your subjective mind behind”. The opinions you have and the choices you make are always somewhat based on gut. And there’s nothing wrong with that. The beauty of diversity is that people with different points of view and experiences come together and try to figure out what makes sense, it’s just that we should be more aware of our subjectivity when making strategic business decisions. 

I started thinking more deeply about this about five years ago when I was founding a new team at Spotify with a product lead and an engineering lead. Together, we were thinking about how to best set up our group for success, which came down to three things. Firstly, we wanted to make sure that all teams in our organization had clear goals informed by insights and our company strategy. Secondly, we wanted to ensure that those goals were measurable so that we could track how our work impacts the business. Lastly, we wanted to make sure that the teams had thought-through plans on how to go about trying to reach their goals. And that’s where we saw some opportunities for improvement, as what would frequently happen was this:

After a business goal was set, teams felt tempted to quickly jump into generating ideas on how to reach it. They then would often get excited about a particular idea that they wanted to build and ship as an A/B test, to see if it had the desired effect. But what we soon learned was that if you go from a goal directly to a single solution, and the solution doesn’t work, it's really hard to backtrack why. Is it because you designed the solution in the wrong way? Or is the underlying hypothesis not correct? Or are you even solving the right problem? 

To avoid teams ending up in this state of feeling stuck, we started thinking about ways to challenge this mindset.

The Thoughtful Execution tree

To change the mindset of jumping too quickly into one solution, we realized that we should remind our teams of the necessary steps in a thoughtful product development process. And that presenting those steps in a tree structure would encourage them to follow the steps in order. We wanted to lead them to go wide in problem identification and hypothesis creation before zooming into a single solution. We named it the Thoughtful Execution tree.

It all starts with data and insights

If your friend complained that she feels lonely after moving to a new country, you’d probably feel tempted to suggest solutions to her problem. "Why don’t you join a book club since you’re so into reading?" you might say, while another person would recommend her to spend more time with her colleagues. But only through carefully examining your friend's behavior would you identify the choices she makes that might lead to her feeling lonely. Maybe she works too much and doesn't have enough time to make new friends. Or maybe the fact that she doesn’t speak the local language makes her shy away from social situations where she could meet new people. Only by understanding the problem space and addressing the root causes of why your friend is not making new friends can you give meaningful advice on how to help with her loneliness. 

Similarly in the business world, after a goal has been set it's crucial to spend enough time gathering data and insights to understand what exactly are the potential problems or opportunities that should be solved. And those can be quite different from each other—like “mountains” of their own, to use the metaphor we commonly use here at Spotify. Like mountains, opportunities can be of different heights and difficulties to conquer, which means you need unique tactics for each.

One of our teams was working on ways to keep users coming back to listen to Spotify in Brazil. Sure they could have just come up with a bunch of ideas on how to potentially make the app stickier to the Brazilian audience, but through careful gathering and analysis of data and insights, they were able to identify very different kinds of problems and opportunities that, with the right solutions, could lead to more people staying as active users of our service. One area to improve was to surface the local Brazilian music catalog more efficiently to new users, while another one was to optimize the application performance for slower networks. These very different problems and opportunities require very different solutions, so it’s crucial to leverage data and insights to point you in the right direction.

After identifying the different problems or opportunities that could be solved, the next step is to understand where the biggest impact lies. If possible, you can try to seize the opportunities with the help of your data scientist friends or to run rapid surveys or experiments to detect which opportunity “mountain” is the highest. Depending on the scope of the project, you might first decide to zoom into only one of the identified problems or opportunities, or maybe addressing several of them at once is needed in order to move the metric of your goal.

If you do decide to zoom into only one opportunity, it’s recommended to make another, more detailed, Thoughtful Execution tree just for that problem or opportunity. For example, when a team started looking into optimizing the mobile sign-up experience on Spotify, their initial goal was to increase the number of successful registrations. Funnels in any product look roughly the same: 100% of people enter the funnel, and then different amounts jump out of it at different stages, depending on their motivation to stick around. 

In most apps, you have to have an account before you can use the service. And it’s a common thing for companies to try to optimize that experience to be as frictionless as possible, to make sure even people who aren’t yet super motivated to try out the service get “through to the other side of the door.” Once our Sign Up team started looking into data and insights, they identified three key points in the funnel that could be further-optimized:

1. Some people downloaded the Spotify app, but didn’t proceed further to the sign-up form; 

2. Some people navigated to the sign-up form, but didn’t complete it; and 

3. Some people navigated to the sign-up form, completed it, but failed to sign up due to an error they made.

Even though addressing any of the three opportunities would likely improve the metric of their goal, the team decided to first address the third opportunity, since there were people who clearly showed motivation to sign up, but still failed to do so. That’s why the team created another Thoughtful Execution Tree, this time with the goal being more specific:

Through looking at instrumentation data of the sign-up page and by running qualitative tests of the experience, the team could identify more specific problems and opportunities related to the registration form that they could start generating hypotheses for. 

From opportunities to multiple hypotheses and solutions

Any problem or opportunity can be addressed in multiple different ways with varying results. That’s why it’s important to create multiple hypotheses per opportunity to see which one leads to the most desirable results. Each of the hypotheses can have several solution executions, so you can’t prove or disprove a hypothesis by just trying out one design solution. 

When we were exploring an opportunity of how to better explain why certain playlists were recommended to each user, we came up with multiple hypotheses on how to do that, and some of them you can see in the tree below. The tree also illustrates an example of how a single hypothesis can be designed in many different ways. And each of those solutions might perform differently. 

From solutions to learnings

In the design solution phase, you should think boldly and freely before scoping to a minimum viable product that will be tested. Explore both short term incremental improvements as well as longer-term directions that require bigger changes. Even though in the example above the changes to the experience are pretty incremental, Thoughtful Execution also works when instead of iterating on something which already exists, you’re exploring something new. Your tree should just be much more high level when you are first figuring out the value proposition of your product and optimization follows after you’ve gotten the basics right. The less mature a product you are working on is, the bigger leaps you might need to make to reach a measurable impact on your metric.

When you have finally designed solutions, it’s time to test the most promising ones to see if you’ll get the impact you’re after. Start analyzing the ideas in terms of desirability and feasibility. Consider anticipated impact and speed to execute. Evaluate which ideas would bring the biggest learnings versus what would be the easiest to build. 

Remember that qualitative tests can inform a lot about what makes sense to A/B test. Through running tests and experiments you start learning about different solutions and can update the tree the more you learn. The aim is to try to prove or disprove hypotheses by seeing whether your solutions manage to move the metrics of your goal. If you try out multiple solutions without seeing the effect that you’re after, it might be time to move on to another branch in the tree. If on the other hand your MVP shows promising results, it’s likely that by iterating the solution further, you will see even more positive impact, so it’s important to keep iterating before jumping onto another opportunity.

The power of Thoughtful Execution

When we introduced the framework to our teams, we could notice a clear improvement in the quality of hypotheses that they were exploring, as well as positive trends in the business metrics we were set out to improve. Spotify’s culture is all about autonomy, so we didn’t want to introduce Thoughtful Execution as a formal process, but rather as a reminder of the needed steps in a thoughtful product development process. We gave teams the freedom to decide how they wanted to tackle each step in the tree in terms of methods and tools they want to use, just as long as they are able to document their Thoughtful Execution tree. It’s been a great tool for teams to structure and align their work and communicate outwards what they are working on and why, and what the upcoming areas of focus may be. 

Your turn

We have now used Thoughtful Execution in our Growth organization for almost five years and decided that it's time to pass this framework along. Before you start using it yourself, here are our tips, gathered from our key learnings along the way:

1. Don’t get paralyzed by not having enough data

Even though it’s crucial to leverage as much data and insights as possible when framing the problems and opportunities you should tackle, it’s unrealistic to assume that you should know everything before moving forward. Different companies and teams have different levels of access to data and research, and not having enough insights available is a common situation. Think of alternative ways of gathering quick insights. Benchmark similar products or run a quick expert evaluation on your product. If you have the possibility, you can also run learning experiments on your product that provide you with additional insights. It’s also perfectly fine to use your gut when framing opportunities, but do it consciously and check if your assumptions were actually true. The important thing is to keep on moving and you can update your Thoughtful Execution tree when more learnings are being generated.

2. Be mindful of which testing method to use

Although quantitative experiments ultimately will give you hard data on how your core business metrics are affected, you can learn a lot through qualitative means. It is fast and cheap to try out bold ideas in a qualitative setting before investing time and resources on building something. For example, when we were redesigning Spotify’s core navigation, the first thing we did was a card sorting exercise with both new and existing users, presenting Spotify’s different content sets and features on individual cards and asking them to structure them in groups that made sense to them. It only took a day to run the research and we didn’t have to invest in building any prototypes, yet we gained valuable learnings on how people mentally structured our product.

3. Holistic changes tend to move metrics

Moving big business metrics like retention is hard. So don’t get disappointed if your experiments come back neutral. When we were exploring onboarding features on our free tier, we often ran into situations where a single piece of product education didn’t move a quantitative metric, even though it was well-received in a qualitative test. But once we ran a test where we introduced multiple changes to the experience at once, we could see a positive uplift in retention. 

4. MVP only gets you to basecamp

After you’ve run an experiment that manages to move the metric of your goal in a positive way, it is tempting to ship the experience at scale and move on to the next opportunity. Keep in mind that minimum viable products only get you to the “basecamp” of the opportunity mountain, and there is likely more impact that you can get out of it by further improving the experience. MVPs typically leave out delightful elements that aren’t necessary to the functionality of the experience but can have a great impact on the desirability of the product. That’s why it is important to remember to optimize the MVPs further to ensure high quality throughout your product. 

Share and test this framework with your teams: Download our PDF guide and access our Figma or MURAL templates through the links at the top of this page. We hope that this tool inspires you to build products in a more thoughtful way and ship with impact.