Websocket Rate Limiting in Distributed Environments

While typical HTTP API Rate Limiting restricts the number of requests, Websocket maintains a persistent connection, so a different approach is needed. Especially in distributed environments, since each server must share state, it's common to leverage Redis.

When you need to limit the number of concurrent connections, as in Replit's case, you can use a method where each server records the per-user connection count in Redis and sums these up for validation. The server generates a key in the form customer-limit-{userId}:{serverUid} based on a unique ID, incrementing it on connection and decrementing it on disconnection.

However, this approach has a critical flaw. If a server shuts down unexpectedly or a network failure occurs, the decrement operation on disconnection is not performed, leaving 'zombie' connection data in Redis.

To solve this, Redis's EXPIRE feature is leveraged. An expiration time is set on each key, and while the connection is maintained, a background refresher process (e.g., a Go goroutine) periodically renews the key, continuously managing the validity of the data.