AI Briefing#ab-testing
The latest AI and developer news about #ab-testing, with the original source and a short summary.
The latest AI and developer news about #ab-testing, with the original source and a short summary.
The single-model approach achieved +4.53% Active Users and +6.30% Total Listening Time in a 7-day A/B test on smart speakers.
Reducing the number of candidates by half increased orders per user by 10.13% while decreasing CTR by 3.71%
Toss Bank separated its asset management report into a floating card, improving click-through and conversion rates by 3x.
ElevenLabs defined quality, experience, and operational metrics for conversational AI and presented evaluation methods through A/B testing and simulation.
Meta has released a methodology that applies causal inference to quantify the incremental revenue of AI infrastructure investments, which could not be measured by A/B testing.
Spotify explains that guarantees in Bayesian A/B testing vary depending on configurations such as priors and stopping rules, and that adding a separate Bayesian mode to its current experimentation program does not yield benefits that outweigh the operational costs.
Yeogi-eottae achieved an increase in purchase conversion rate through an A/B test involving approximately 1 million users by improving home screen seller cards based on the JTBD framework.
Yeogi-eottae conducted a project to address low click-through rates on its home screen by redefining module roles rather than product card formats and validating new navigation experiences through A/B testing.
Anthropic is reportedly conducting server-side A/B tests to lower reasoning effort levels in the latest versions of Claude Code.
Using LLM predictions as a proxy metric for A/B testing requires specific statistical assumptions and calibration procedures.
Daangn's Data Value Team introduces an in-house experiment platform that supports everything from A/B test design to analysis.
LLM evals should be used as a pre-experiment validation funnel, not as a replacement for A/B testing.
Cursor improved agent quality by customizing the execution system per model and using dynamic context.
Cursor leveraged Vercel Microfrontends and Flags to unify its web services and build a data-driven growth experimentation loop.
ElevenLabs has launched an Experiments feature that lets users test and optimize ElevenAgents' performance using real-time traffic.
PostHog has joined the Vercel Marketplace, offering feature flags, experimentation, and analytics capabilities.
We redesigned training and evaluation metrics to align with the goal of directly increasing conversion rate.
Azar used PolicyEngine to cut experiment setup and review time from 2 days to 30 minutes.
A/B testing is a method for verifying the effect of a change and reducing risk.
Banksalad applied a POC and a 114-day phased experiment to the development of its Home tab, securing product completeness and achieving a stable launch.
KakaoPay developed its own AB testing service with flexible targeting and stability to overcome the limitations of open source.
Banksalad has established a company-wide A/B testing culture through its experimentation platform and expanded the involvement of data roles in product planning.
To overcome the limitations of manual analysis, Banksalad built an automated experiment analysis pipeline V2, improving data reliability and decision-making speed.
Devsisters' Data Platform Cell uses engineering to build an environment where data flows smoothly and members can make data-driven decisions.