AI Briefing#inference
The latest AI and developer news about #inference, with the original source and a short summary.
The latest AI and developer news about #inference, with the original source and a short summary.
Test-time training updates weights during inference to reduce memory and increase speed, but incurs the computational cost of per-user model replication.
Shortly after OpenAI acquired a 4.2% stake in Cerebras, it unveiled 'Ultrafast,' an ultra-fast service leveraging Cerebras hardware.
Lumabri has been released, enabling distributed execution of MoE models in P2P swarm environments using the Colibri engine.
Anthropic is discussing the acquisition of AI infrastructure startup Decart for approximately $6 billion.
Forecasts suggest that the spread of agentic AI could increase the CPU share in AI infrastructure.
SIE unifies serving of embeddings, reranking, OCR, and LLMs through a single OpenAI-compatible API.
SpaceX and Tesla are pushing forward with a large-scale factory in Texas to produce chips for Optimus and Cybercab.
Liquid AI released a 2.6B agent model capable of running on mobile phones and CPUs.
DeepSeek has released the official version of V4-Flash, which supports a 1 million token context window.
Scenema Audio has been released as a ComfyUI native node that runs on 8GB VRAM.
Thinking Machines Lab released an open-weight model one-quarter the size of Inkling.
AI Gateway can now be purchased via AWS Marketplace and integrated into AWS billing.
AFM3 20B is a MoE model that selects only about 20% of MLP layers to activate per prompt.
EdgeRazor improves preservation of teacher LLM performance even at 1.88-bit precision.
Xberg v1 has launched as a content intelligence engine covering PDF, OCR, document, and multimedia processing.
llama.cpp has fixed a tool calling issue in DeepSeek v4 Flash 0731.
LocalAI has revealed the performance and cost of its own C/C++ inference engines, which reduce Python dependencies.
Daum operates an AI search service generating about 500 million tokens a day based on Furiosa RNGD.
As AI models become more intelligent, the value they generate from the same amount of compute will grow, driving a sharp rise in compute costs.
OpenRouter has released a beta version of Classifiers, which can tag and classify inference tasks within a workspace.
AI hardware startup Etched has raised $300 million in funding to accelerate inference.
Hugging Face Diffusers has officially integrated Nunchaku W4A4 quantization based on SVDQuant, cutting VRAM usage in half and improving inference speed.
On OpenRouter, GPT-5.6 Sol accounts for only 15% of OpenAI tokens but makes up an estimated 34% of spending.
A paper reveals that LLMs' biased random number selection can be used as a fingerprint to verify whether an API endpoint is actually serving the model it claims to be.