Documentation
Quick-start guides
Load datasets, transcribe speech, and run the Somali tokenizer in minutes.
All datasets and models are hosted on Hugging Face under the goobolabs organisation. Install the Hugging Face libraries with pip install datasets transformers.
Load a dataset
from datasets import load_dataset
# Load the Somali text corpus
ds = load_dataset("goobolabs/somnlp-corpus")
print(ds["train"][0])
# {'text': 'Waxaan jeclahay barashada...', 'domain': 'news', 'dialect': 'standard'}Transcribe Somali speech (ASR)
import whisper
model = whisper.load_model("goobolabs/whisper-som-small")
result = model.transcribe("audio.wav", language="so")
print(result["text"])
# 'Beeraha Soomaaliya waxay u baahan yihiin...'Tokenize Somali text
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("goobolabs/goobo-tokenizer")
tokens = tok.tokenize("Caafimaadka carruurtu waa mudnaanteenna koowaad.")
print(tokens)
# ['Caafimaad', '##ka', 'carruur', '##tu', 'waa', ...]Newsletter
Stay in the loop
Dataset releases, new bootcamp cohorts, research papers, and lab updates — straight to your inbox. No spam, unsubscribe anytime.