How Spotify Ships Its App: A Look Inside (Part 2)

Previously, all information was scattered across Jira, forcing Release Managers to switch between tabs to check status, and handling Slack inquiries on top of that made it easy to miss information. To solve this, Spotify built the Release Manager Dashboard, which brings release-related information together on a single screen.

The dashboard was designed so that not just Release Managers, but anyone who understands the basic mobile release flow, can quickly grasp the situation. There were four goals:

• Minimize context switching
• Reduce cognitive load
• Support fast, accurate decision-making
• Provide an at-a-glance view of release progress along with the ability to drill down into details

Spotify treats each combination of platform and version as a track, and manages Android, iOS, and Desktop independently. The screen displays each track's release status, blocking bugs, sign-off status, whether the RC build succeeded, whether the Build Verification Test passed, whether it was uploaded to the App Store, along with quality and usability metrics such as crashes, ANRs, CPU exceptions per song, and daily active users.

Information that doesn't belong to a track is also important. The dashboard separately shows blocking bugs with no version and bugs with no priority, and supports the work of tracking down the owning team, including internal reports as well as external alpha/beta tester reports.

The implementation was built on top of Backstage, Spotify's internal developer portal. It's a plugin built with React and TypeScript, which configures app build deployments through the Software Catalog and also connects with other Backstage plugins such as app builds and crashes.

The backend acts as an API gateway that collects and integrates data from roughly 10 systems. Initially there were too many re-fetches, making it slow and expensive, but after adding caching and pre-aggregation every 5 minutes, loading time was reduced to 8 seconds, and costs dropped significantly as well.

The layout uses color to let people quickly read status.

• Green: Ready to move to the next step
• Yellow: Items pending action exist
• Red: Errors that require immediate action

The Production section at the top shows crash metrics for Android, iOS, and Desktop versions already in production, along with the most recent 24-hour rolling daily active users. Current and Upcoming cover versions that haven't rolled out yet; in the Current example, iOS 8.9.2 is shown as not yet ready due to a release-blocking bug, incomplete regression tests, and a crash rate exceeding the threshold.

The ITGC section ensures that full production rollout is only possible after passing IT general controls testing, and the Release Status Ping is a link that generates a Slack message to share release status. Upcoming has the same structure as Current, but items that aren't yet applicable are grayed out and disabled.

Once data started being saved every 5 minutes, enabling analysis of trends over time, the two biggest bottlenecks became clear:

• Time spent testing and fixing blocking bugs
• Time spent waiting for App Store approval

Another problem was that if a manual stage transition was completed outside business hours, moving to the next stage could be delayed by up to 12 hours. To address this, Spotify built Robot, which automates the transition to the next stage.

Robot is a backend service that executes the next action once conditions are met — for example, when conditions such as manual test sign-off completed, no open bugs, and automated tests passing on the latest git commit are satisfied, it performs actions like submitting for review. The overall flow is managed as a state machine that includes release branched, final release candidate, submitted for review, roll out 1%, roll out 100%, as well as rollout paused and release cancelled.

Thanks to this automation, the average release cycle has been shortened by about 8 hours. Right now it's mainly applied to the main Spotify app, but going forward they plan to extend it to smaller apps as well, and are evolving it into a more modular structure to support that.

There are also clear reasons why Spotify maintains its own system. Backstage components can be reused, other teams can query release data based on the Software Catalog's data model, and org data can be used to automatically assign bugs to relevant teams and ping progress on critical tickets. Above all, since every developer is already using Backstage, adoption friction is low.