How does spotify drive data-driven decision making?

Spotify The infrastructure team shared how they prioritized the data Build an automated data collection platform , Thus in DevOps Data driven decision making is realized in , And improve the productivity and product value of developers .

Spotify Infrastructure teams using Gradle（Gradle Enterprise Edition） As its Android Application building system . It can generate 、 Collecting and storing the data needed to understand the software based on local development experience . It needs to focus on the visualization of data pipelines and dashboards . about iOS System data generation 、 Collect and store , There is no mature solution yet , So the team developed these tools themselves .

Spotify It's been a long time in the field of data .Spotify The technology learning team launched data University （Data University）, This is a series of training courses covering all aspects of data science and Engineering , Designed to help engineers solve product related problems .

Android Infrastructure teams apply these lessons to their build time and local development experience , But they found that they lacked the data to drive decisions .

Spotify By summoning certain specific “ tribe ” The team came to specifically provide the data infrastructure , The engineers are equipped with building modules to collect data and visualize data input , This kind of data requirement is solved . They pointed out that , There are still many challenges , For example, how to apply this data-driven approach to their architectural decisions .

The team uses this new data infrastructure to clarify where technology and product teams should invest to reduce build time . When they look at build time trends and Swift and ObjC The total number of components used in , They realized that investing in Swift Optimization makes sense .

This technology investment for data-driven decision making with Harvard Business Review Analytics Services （Harvard Business Review Analysis Services） The results of a recent study are quite different , The study shows that , Only 7% , provides their teams with the analytical tools and resources they need to drive data-based decision-making and autonomy .

essentially ,Spotify The method is very simple ： The team asks questions they can't answer , And then in the backlog of to-do （backlog） Give priority to these issues . After the data is available and the questions are answered , The team collects feedback during the evaluation phase , To see if the work has had an impact on the local development process . To prevent data quality from deteriorating , The team must conduct a quality check on the data consistency and data pipeline of each component .

In the planning stage , The team uses historical data to identify scenarios that need improvement . These data may not be able to describe the current situation , But it provides a baseline for identifying improvements . If they already know when the system will be built in a particular situation , So they want to keep the same number , Or improve these numbers , And no matter how the code base grows . This is crucial , Because as the system becomes more and more complex ,DevOps Workflow can also become complex and opaque .

Agile naturally tends to give priority to products , therefore DevOps The challenge is , How to add features to improve product efficiency and improve development efficiency or service reliability Find a compromise .

In the planning stage , The team introduced tasks to collect and display the data needed to validate the changes . The questions raised at this stage are one of the key outputs , for example ：“ Do we collect enough information to check that the developer has turned on the remote cache ？” perhaps “ In a single PR How many components did they change on average ？”

As the infrastructure team's data plans gain more internal recognition , Other teams began to prioritize platform related work . The product team began to pay attention to data visualization , To verify the drive to move DevOps Product discussion of team decision making process .

Data driven decisions by the product team help to assess the effectiveness of the solution and satisfaction with adoption . Product managers usually use user surveys to evaluate products from an early stage . by comparison , Data driven processes bring this assessment to product conceptualization .

InfoQ Of Data driven decision series This paper outlines how data-driven decision supports three major activities in software delivery —— Product management 、 Development and operation and maintenance .

Link to the original text ：

https://www.infoq.com/news/2020/10/Data-Driven-Decisions/