AI Briefing#transformer
The latest AI and developer news about #transformer, with the original source and a short summary.
The latest AI and developer news about #transformer, with the original source and a short summary.
Transformers models from Hugging Face can now be loaded and used directly in KerasHub.
8Percent's CSS team experimented with credit scoring using CB data, XGBoost, and Transformer.
KV Cache quantization reduces LLM memory usage and enables longer context generation.
The Qwen team has released Qwen1.5-110B, a 100B-scale model with performance on par with Llama-3-70B.
HuggingFace has released JAT, a general-purpose agent capable of performing various tasks, along with its dataset.
AI21 has released Jamba, a Mamba-based model supporting a 256K context window.
This study demonstrates that stock returns can be predicted by using ChatGPT to analyze the sentiment of news headlines.
Improved noise sampling for Rectified Flow models and introduced a new Transformer architecture to enhance high-resolution image synthesis performance.
This explains the structure of PatchTST, a model that improves time-series forecasting performance, and how to use it with the Hugging Face library.
IBM Research's lightweight time series model PatchTSMixer has been integrated into the HuggingFace Transformers library.
This summarizes the development process of open source LLMs in 2023 and their core technical components.
This explains in detail how the MoE architecture works, how it is trained, and the trade-offs during inference.
Hugging Face announced optimized support for running LLMs on AMD Instinct GPUs without any separate code modifications.
Prodigy-HF, a plugin that directly connects the data annotation tool Prodigy with Hugging Face, has been released.
Hugging Face has introduced a Chat Template feature that automatically applies the chat format used during model training.
Compares the characteristics of the bitsandbytes and auto-gptq quantization methods natively supported by the 🤗 Transformers library.
Hugging Face integrates AutoGPTQ into Transformers to support 2-8 bit quantization of LLMs.
Covers how to optimize inference performance and memory efficiency of the Bark TTS model using Hugging Face's Optimum and Accelerate.
Llama-2-70B is cheap for prompt processing, but it's at a disadvantage compared to GPT-3.5 for completion-heavy tasks.
The transformer-based model Autoformer has proven superior to the simple linear model DLinear in time series forecasting performance.
Hugging Face and AMD have formed a partnership to optimize transformer model performance on AMD CPUs and GPUs.
Hugging Face and Microsoft have partnered to integrate the Hugging Face model catalog into Azure Machine Learning.
RWKV, a new architecture that combines the efficiency of Transformer with the advantages of RNN, has been released.
This guide walks through the end-to-end process of training a RoBERTa model from scratch using TensorFlow and TPU.