Core ML-based Stable Diffusion Acceleration

Through new optimization techniques for Core ML announced at WWDC'23, Stable Diffusion can now run faster and with less memory on iPhone, iPad, and Mac.

The key technical changes are as follows:

• 6-bit palettization: Compresses model weights from 16-bit floating point to 6-bit, drastically reducing storage space and download time.
• Real-time decompression: Previously, the entire model was decompressed at load time, resulting in high memory usage, but the new approach converts weights layer by layer on the fly during inference, dramatically lowering memory footprint.
• coremltools support: Through the coremltools.optimize submodule, PyTorch or TensorFlow models can be converted into optimized Core ML format.

These improvements are useful for developers who want the convenience of on-device AI execution, privacy protection, and reduced API costs.