Toss builds its own device farm

Toss built an in-house device farm called Nebula to solve the inefficiencies that arise when each team manages devices individually. Nebula provides an environment where developers can perform automated testing by remotely controlling real smartphones with just a single API call, without having to touch the devices directly.

Nebula's architecture consists of four layers.

• Client: Various entry points such as SDK, CLI, and API
• Server: An orchestration layer that manages device discovery, occupancy, and test execution (utilizing Kafka and distributed locks)
• Agent: Runs on hosts (Mac mini, Linux) and manages local devices
• Device: The bottom layer that executes actual commands

In particular, Toss developed its own Nebula Driver to overcome the structural limitations of the existing Appium. According to benchmark results, the Nebula Driver showed overwhelming performance differences compared to Appium, as follows.

• Element click: About 13.5x faster (52ms vs 702ms)
• Text input: About 10x faster (138ms vs 1,377ms)

This performance improvement was made possible by adopting a speed-first philosophy that delivers commands to nodes immediately, instead of Appium's 'robustness-first' philosophy (which incurs wait times).