AI Briefing#deep-learning
The latest AI and developer news about #deep-learning, with the original source and a short summary.
The latest AI and developer news about #deep-learning, with the original source and a short summary.
A technical guide that implements Yann LeCun's JEPA architecture from scratch while explaining its underlying principles.
A website that organizes 30 essential machine learning papers selected by Ilya Sutskever in a format that's easy for beginners to understand.
We propose PathMoE, which constrains the expert paths that tokens traverse to improve the efficiency and performance of MoE models.
ReFreeKV, a threshold-free KV Cache compression technique that automatically adjusts the compression budget based on input data, is proposed.
Meta has unveiled Brain2Qwerty v2, which combines non-invasive brainwave measurement (MEG) with LLMs to improve real-time sentence decoding performance.
A new training technique is proposed that interleaves the management of 3 layer states to improve GPU memory efficiency.
This analyzes scaling laws, the predictable relationship between model size, data volume, and compute on training loss.
Proves that Attention sinks and Representation collapse stem not from Attention itself, but from a norm-blind similarity measurement method.
A research paper has been published arguing that a Dynamical Systems Reconstruction (DSR) perspective should be adopted to overcome the limitations of time series modeling.
The core of AI research is to focus on the basics and repeat the process of steadily reading and building.
PaddleOCR has unveiled PP-OCRv6, a new OCR model series with significantly improved accuracy and speed.
OpenCV 5 has been released, significantly modernizing the DNN engine and greatly strengthening ONNX support.
Stanford University has released its CS336 course, covering the entire process from designing Transformer models to training and evaluation.
It proposes a new method that dramatically reduces memory usage by dividing the network into blocks and training them independently.
This is the first post in a guide series covering how to use torch.profiler to identify and optimize performance bottlenecks in PyTorch models.
Research has revealed that AI-generated CUDA kernels can cause loss functions to diverge during actual model training due to precision issues.
Delta Attention Residuals has been unveiled, boosting LLM performance by solving the routing collapse problem of existing residual connections.
It explains the relationship between compute, memory bandwidth, and overhead that determines deep learning performance, along with the underlying optimization principles.
An open-source AI engineering curriculum with 435 lessons covering everything from mathematical foundations to agents has been released.
Core-periphery block-sparse attention cuts the high-resolution cost of ViT.
Token AI has announced STAM, a new optimizer that enhances training stability and maximizes memory efficiency.
SATFormer is proposed, improving efficiency by selectively reusing early-layer representations on a per-token and per-head basis.
A genetic algorithm discovered an optimizer that outperforms Adam by 2.6%.
An Nvidia executive said current AI compute costs are higher than labor costs.