Multi-Camera Self-Calibrating Homography

This is a multi-camera tracking pipeline that predicts camera A's target position from camera B's observations even when camera A loses sight of the target.

• When two cameras view the same person simultaneously, a 64-dimensional HSV appearance descriptor is L2-normalized and stabilized for matching using EMA (alpha=0.3).
• Bottom-center foot-point correspondences from both bboxes are accumulated, and once 4 or more pairs are gathered, a 3x3 homography H is computed using cv2.findHomography() and RANSAC to create a B→A coordinate transform.
• Retraining occurs every 5 new pairs, and if reprojection error spikes suddenly, it is treated as camera movement and H is discarded.

The prediction path has 3 stages.

• H-PROJ: Uses the most accurate projection path when a valid homography is available.
• EXTRAP: Uses pixel extrapolation when no homography is available, with tolerance adjusted via min(250px, 80 + 40*t).
• WORLD: Performs world-coordinate pinhole projection from the fused 3D Kalman state, always leaving a prediction path.

Tracking uses Hungarian assignment with 0.6 * IoU + 0.4 * cosine appearance cost, with DeepSORT (MobileNet) as the primary method, falling back to Hungarian(scipy), then centroid in order.

Each camera receives a 0.1 to 1.0 confidence score based on consistency, and high-innovation measurements are given lower weight. The Kalman's measurement noise R is also adjusted at every update based on confidence, bbox area, and sensor trust.

The implementation is at github.com/mandarwagh9/overwatch, with 57 unit tests covering Kalman, homography, and tracking, using GitHub Actions CI. Homography re-estimation is reported at under 0.1ms, and per-prediction projection at under 0.001ms.

The limitations are that ground-plane homography breaks down for elevated cameras at high angles, and HSV-based Re-ID has weak discriminative power for distinguishing people.