Qeexid
Quantization wuxuu u beddelaa lambarrada float-ka saxnaanta sare (FP16/FP32) ilaa integer-yo ama float-yo bit-badan oo hooseeya. Farsamooyinka waxaa ka mid ah quantization ka dib-tababar iyo quantization-aware training.
Wuxuu suurtogal ka dhigayaa in LLM-yo waaweyn ku shaqeeyaan GPU-yo ama CPU-yo yar iyadoo tayadu wax yar hoos u dhacdo.
Si fudud
Waa sida aad muusig u kaydiso file cadaadin leh oo yar — waxaad badbaadisaa meel iyo bandwidth, mararka qaarkoodna dhib yar oo tayo ah ayaa dhacaya.
Halka aad ka aragto
- Moodellada quantized ee llama.cpp iyo GGUF ee laptop-ka lagu shaqeeyo.
- Server-yada inference ee INT8/FP8 ee production-ka.
- Fine-tuning QLoRA oo ku socda hal GPU macaamiil.
Sida ay u shaqeyso
1.Dooro dherer-bit-ka
Qorshayaal 8-bit, 4-bit, ama isku-dar saxnaan.
2.Hagaaji miisaannada (calibrate)
U beddel xaddidaadaha float ilaa grid integer ah.
3.Fuli inference
Kernel-yadu waxay ku xisaabiyaan miisaamayaasha quantized.
Maxay muhiim u tahay
- Quantization waa habka moodellada furan uga noqdaan mid la isticmaali karo qalab dhab ah oo miisaan ku xaddidan.
Inta badan la khaldo
Quantization kasta waa lumin la'aan (lossless).
Bit-yo hooseeya badanaa waxay kharashaan tayo — ku qiimee hawshaada gaarka ah.