goobolabs/somnlp-corpus
The largest open Somali text corpus — 887M+ tokens from 6 upstream sources (HPLT · CC100 · mC4 · OPUS · MADLAD · MT560), cleaned through a six-stage pipeline.
View on Hugging FaceModality
Text
Size
887M+ tokens · 4.5 GB
License
CC-BY-4.0
Language
Somali (af)
Format
Parquet / JSONL
Version
v2.0
Tasks
LM, NER, POS, Classification
Domains
HPLT · CC100 · mC4 · OPUS · MADLAD · MT560
| Split | Rows | Size |
|---|---|---|
| train | 1,686,146 | 4.3 GB |
| validation | 44,372 | 115 MB |
| test | 44,373 | 115 MB |
from datasets import load_dataset
ds = load_dataset("goobolabs/somnlp-corpus")
sample = ds["train"][0]
print(sample)@dataset{somnlp_corpus_2026,
title = {SomNLP-Corpus: Open Somali Text Corpus},
author = {Goobo Labs},
year = {2026},
version = {2.0},
license = {CC-BY-4.0},
url = {https://huggingface.co/datasets/goobolabs/somnlp-corpus}
}