AI Briefing#transformer
The latest AI and developer news about #transformer, with the original source and a short summary.
The latest AI and developer news about #transformer, with the original source and a short summary.
Trained a 235M-parameter LLM from scratch using a single RTX 5080.
A 2-layer transformer implemented in 6502 assembly running on a Commodore 64.
MoDA improves depth scaling, boosting both performance and efficiency in a 1.5B model.
An open-source project that hypothesizes Claude Mythos and implements it as a recurrently-executed Transformer.
CWT reduced compute by **about 45%** while maintaining a **PPL difference of 1.7%** versus the baseline.
A story of regaining LLM/Python instincts and codebase understanding through hand-coding
ResBM presents 128x activation compression for pipeline parallel training.
ResBM compressed activations by 128x at the pipeline-boundary bottleneck.
Parcae scales up recurrence to reach 1.3B Transformer-level performance with 770M parameters.
A weight-sharing looped Transformer improves the efficiency of image and video generation.
A study was published that improves compositional generalization performance by recursively leveraging the model's depth instead of increasing computation time.
TurboOCR claims up to 1,200 img/s OCR using C++/CUDA and FP16 TensorRT.
FlashLM v8.3 (6.5M CORTEX) outperformed the v5.2 Transformer baseline under the same 2-hour CPU and same data conditions.
This analyzes how AI-powered voice cloning works and its use cases in the medical and content creation fields.
This piece examines the major challenges that arise when adopting generative AI for enterprise use and analyzes ways to resolve them to achieve operational efficiency.
AI21 Labs' Jamba model addresses the long-context processing limitations of Transformers through a hybrid architecture.
AI21 hosted a hackathon leveraging Jamba's 256K context window, showcasing innovative AI use cases.
It asks whether AI is truly intelligent by examining the differences between LLMs and the brain.
Based on Nova 2.0 Lite, attaching LoRA to only **o_proj** gave the best balance of efficiency and accuracy.
Airbnb learns behavioral sequences to recommend destinations to users who are still in the exploration stage.
User embeddings trained on long-term logs, combined with RCBS, significantly boosted recommendation performance.
This explains the principles and advantages of the MoE (Mixture of Experts) structure, which maximizes the computational efficiency of Transformers.
Netflix strengthened shot-level content understanding with MediaFM, which combines audio, video, and text.
Different temporal resolutions of PPG better predict different health outcomes.