Local Tool 'Deckard' Released for Automatic AI Text Detection in Browsers, Performance Limitations vs. Pangram and Usage Guide

With Pangram being virtually the only major service in the AI text detection market, Deckard, a locally running AI detection tool aimed at privacy protection and cost reduction, has been released. Pangram boasts high accuracy with a 99.66% detection rate and a 0.004% false positive rate, but it requires sending all text to third-party servers, raising privacy concerns and incurring API usage costs.

Deckard is a Chrome extension that runs the Gradient MLX 4-bit model locally on Mac to mark AI-generated text on web pages in real time. It communicates with the model via Chrome native messaging without a separate web server, using approximately 400MB to 1.2GB of memory when active, but automatically shuts down after 5 minutes of inactivity to minimize battery consumption.

Local Model Performance and Limitations

According to the author's benchmarking results, the performance of local models is significantly lower than that of Pangram. Even the best-performing Gradient MLX 4-bit model achieved only a 2.712% false positive rate and a 52.35% AI text hit rate. This level is incomparable to Pangram's 99.66% detection rate.

• Gradient MLX 4-bit: 2.712% false positive rate / 52.35% hit rate
• EditLens RoBERTa-large: 2.484% false positive rate / 56.06% hit rate
• Vanguard: 2.267% false positive rate / 44.92% hit rate

Practical Usage and Outlook

Despite the lower performance, the author evaluates Deckard as a useful tool. The explanation is that if one recognizes the false positive rate is around 2% and does not treat a single flag as definitive proof of AI usage, it is valuable enough as a pre-filtering tool. It successfully identified AI summaries on YouTube and AI snippets on blogs.

With social networks expected to adopt automatic scanning for AI content tagging in the coming years, the author anticipates the emergence of small local models that are 2 to 10 times more performant than current local models. If local models significantly smaller than Pangram's production model emerge, Deckard's performance is expected to improve substantially.