A Concurrency Bug Found via strace, Down to an 8μs Difference

While packaging video into DASH and HLS formats using shaka-packager, a bug occurred that intermittently failed only when processing multiple input files concurrently. The logs only showed Cannot open file to write, with no visible cause.

After ruling out disk/memory shortage, file descriptor exhaustion, specific video issues, and version problems in turn, a concurrency issue was suspected, and strace was used to trace file-related system calls. -ff was used to separate per-thread logs, and -tt was used to leave microsecond-level timestamps to compare success/failure cases.

In the failure case, execution never reached the stage of opening the output file with openat; right before that, mkdirat was failing with EEXIST. When two threads entered at nearly the same time to create the same output directory, one thread created the directory first by a mere 8μs margin, and the other thread received EEXIST.

The problematic code checked for existence with is_directory() before creating the directory, then called std::filesystem::create_directories() if it didn't exist.

• create_directories() returns true if newly created
• If it already exists, it returns false, but ec may be empty
• However, the code only looked at the return value and treated false as an error

In other words, when two threads entered simultaneously, one thread passed through normally, but the other thread mistook the normal situation of the directory already existing as a failure, and terminated without ever reaching openat. Ultimately, the bug arose from the combination of not checking ec from create_directories() and a race condition between is_directory() and mkdirat.

The solution was clear. Fundamentally, the error handling approach for create_directories() needs to be fixed, and as an immediate workaround, pre-creating the output directory before running the packager works. As a result, subsequent packaging succeeded reliably.

This was a problem that couldn't be seen from a single log line, but by going down to the system call level with strace, it became clear exactly where and what was failing, allowing a quick path to the actual defect point in the source code.