Reinforcement Learning-Based Bitrate Control for Neural Video Compression

Existing rate control for neural video compression (NVC) mainly approximates the relationship between bitrate and distortion using models such as R-λ or R-D-λ. However, these approaches assume window-wise temporal stationarity and rely on local optimization, failing to sufficiently reflect the global context.

The authors redefine NVC's rate control as a sequential decision-making problem. This avoids the complexity increase caused by window length, leverages global information across frames, and allows direct determination of code parameters without a separate two-stage procedure.

To achieve this, they designed an Actor-Critic RL framework. The current frame x_t and the reconstruction result of the previous frame are input together to create a spatial-temporal embedding, and the state is constructed by also embedding the target bitrate and previous code parameters. During inference, only the Actor is used to greedily select actions, making it simpler and faster than the existing two-stage approach of bitrate allocation followed by parameter derivation.

When resolution differences occur, the reference frame is resampled to match the current frame's resolution, and the reconstructed frame is restored to the original resolution via bicubic upsampling. This joint λ-m action policy is designed to maintain rate control accuracy while reducing computation by leveraging lower resolution.

Experiments were conducted on three representative NVC models: DVC, DCVC, and DCVC-DC. The comparison targets were the hyperbolic model from Chen et al. (2023) and the neural network-based method from Zhang et al. (2023), using a combination of experimental data from the original papers and some self-implemented results.

The results were generally excellent, achieving an average -15.96% BD-Rate improvement, 1.30% rate error, and 19.7% encoding time savings. Notably, even on DCVC-DC, which uses a complex inter-frame reference mechanism, an average -13.98% BD-Rate gain and 1.13% rate error were achieved.

The advantages in terms of complexity are also clear.

• Reduced network parameters and KMACs per pixel, lowering computational cost
• Encoding time reduced by 19.7%, decoding time reduced by 26.5%
• Operates with linear complexity while maintaining global reference, increasing the potential for real-time NVC applications

In conclusion, transforming NVC rate control into reinforcement learning-based sequential decision-making overcomes the limitations of existing model-based approaches while simultaneously improving performance and speed. Going forward, jointly optimizing the coding model and rate control module could pursue more flexible and stronger R-D performance.