Indic 9.8 Million Document Corpus Released

·2026.05.19 08:09

Key point

An Indic corpus spanning 11 languages and 8.4B tokens has been released on Hugging Face.

Details

AM0908/indic-hplt-v1 has been released on Hugging Face.

Size: about 9.8 million web documents, about 8.4 billion tokens
Languages: 11 languages (hi, bn, ta, te, mr, gu, kn, ml, pa, ur, en)
License: CC0

An Indic multilingual web corpus including Hindi, Bengali, Tamil, and Telugu, it can be used right away for training multilingual models and for data cleaning and evaluation.

huggingface multilingual dataset indic-languages cc0

Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.