Gitframes: AI Coding Agent-Driven Video Rendering Tool with Native WebGPU Architecture

Native WebGPU Architecture

Gitframes is a beta-stage, code-first video rendering tool designed for AI coding agents, integrating Photoshop-grade VFX, After Effects-style motion, and Blender-like 3D capabilities into a single npm package. Unlike competitors such as Remotion (Chromium/Puppeteer) or Hyperframes (Canvas2D/SVG), Gitframes avoids headless browser overhead by using Dawn/WebGPU and @napi-rs/webcodecs for direct hardware rendering and encoding. This architecture results in:

• Performance: 60–120+ FPS processing throughput compared to 5–20 FPS for Remotion.
• Memory Efficiency: ~200–400 MB RAM per worker, significantly lower than Chromium-based solutions (1.5–4.0 GB).
• Deterministic Timing: Uses an absolute frame clock and discrete sample points to eliminate floating-point timer drift and dropped frames.

Key Technical Features

• Analytic Type (Slug Algorithm): Renders vector typography directly in WGSL fragment shaders, ensuring pixel-perfect clarity from 10px to 10,000px without texture atlas artifacts.
• Unified 3D & 2D Compositing: Supports nesting 2D flex/box trees into 3D homography planes and meshes (OBJ, FBX, glTF, etc.) with PBR glass, SSAO, and motion blur.
• On-Device Neural Vision: Integrates Apache-2.0 ONNX models (RTMDet-Ins, RTMO, MediaPipe Selfie Segmenter) for object tracking, instance segmentation, and pose estimation, enabling reactive signals without disk round-trips.
• Procedural Audio DSP: Generates deterministic sound effects (whoosh, impact) and supports multi-track mixing with audio-reactive visual signals.

AI Agent Integration

Gitframes is designed with an Agent-first DX, providing skills and plugins for major AI coding assistants:

• Claude Code: Installable via /plugin install gitframes or the official Anthropic marketplace.
• Other Agents: Supports Codex, Cursor, Hermes, Gemini CLI, and Copilot via npx skills add gatewai-dev/gitframes.
• Verification: Uses Zod for runtime configuration validation and provides deterministic, serializable snapshots for agent analysis.

Availability and License

The project is licensed under Apache-2.0 and is currently in Beta (API subject to change). It requires Node.js ≥ 22 and native GPU acceleration (Dawn/WebGPU or Vulkan). The repository is hosted on GitHub under gatewai-dev/gitframes.