Quick-start guides
Load datasets, transcribe speech, and run the Somali tokenizer.
All datasets and models are hosted on Hugging Face under the goobolabs organisation. Install with: pip install datasets transformers.
from datasets import load_dataset
# Load the Somali text corpus
ds = load_dataset("goobolabs/somnlp-corpus")
print(ds["train"][0])
# {'text': 'Waxaan jeclahay barashada...', 'domain': 'news', 'dialect': 'standard'}import whisper
model = whisper.load_model("goobolabs/whisper-som-small")
result = model.transcribe("audio.wav", language="so")
print(result["text"])
# 'Beeraha Soomaaliya waxay u baahan yihiin...'from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("goobolabs/goobo-tokenizer")
tokens = tok.tokenize("Caafimaadka carruurtu waa mudnaanteenna koowaad.")
print(tokens)
# ['Caafimaad', '##ka', 'carruur', '##tu', 'waa', ...]Frequently asked
Are the datasets and models free to use?
Yes — everything here is hosted openly on Hugging Face under the goobolabs organisation. License terms vary by release, so check the license listed on each dataset or model card before using it in production.
Which models are available right now?
whisper-som-small for speech recognition, goobo-tokenizer for Somali-native tokenization, and som-en-mt for Somali–English translation — see the Models section for details on each.
How do I benchmark a model against SomBench?
SomBench v0.1 is our open Somali evaluation suite — see the Benchmarks section in the sidebar for the current leaderboard and how to submit results.
Stay in the loop
Dataset releases, new bootcamp cohorts, research papers, and lab updates — straight to your inbox. No spam, unsubscribe anytime.