RON-TAC Tactical AI

Implemented a DAgger-style imitation learning pipeline inside Ready or Not (UE5.3) that directly collects human demonstrations to train a multi-agent tactical squad policy.

Runtime instrumentation is handled via a UE4SS C++ mod. A single ~3.8 kLOC .cpp file and a ~270 KB DLL hook into the game internals: a D3D11 Present vtable hook captures 384×384 RGB frames, while pre/post hooks on SWATManager.Give*Command together with blackboard snapshots log demonstration data to dagger.jsonl. An activity-transition monitor uses custom activity-class matching to distinguish [PLAYER] from subordinate actions.

The real-time inference loop runs in Python + Torch 2.x + CUDA at 2 Hz. T3-Vis (roughly 40M params, a frozen ViT) produces visual embeddings, and T3-Tac (39.9M params, a set-transformer) takes visual tokens plus structured state tokens to output an 18-way CommandType, team assignment (SQUAD/RED/BLUE/GOLD), and confidence.

When confidence exceeds the threshold and the command isn't a duplicate, it's immediately re-injected into the game via ProcessEvent. The player retains first-person control at all times and can override at any point.

The training pipeline parses activity transitions into label tensors and trains with cross-entropy, using a real-data weight of 5.0 and an optional VLM-augmented data weight of 0.3. The policy is periodically swapped into the live loop, creating a closed-loop human-in-the-loop improvement cycle as human demonstrations continue to accumulate.

Results as of 2026-04-19 are as follows.

• Dataset: 1,173 player-issued commands
• T3-Tac v3 validation accuracy: 0.606 macro
• HOLD: 100%
• BREACH: 64%
• STACK_UP: 54%
• SEARCH_AND_SECURE: 52%