Qeexid
LoRA wuxuu qariyaa (freeze) moodelka aasaasiga ah kadibna wuxuu ku duraa matrix-yo yar oo la tababari karo (low-rank) lakabyada gudahooda (badanaa attention projections). Waxaad kaydisaa oo beddelaysaa adapter-yo yaryar halkii ay ahaan lahayd checkpoint-yo buuxa.
QLoRA waxay isku dartaa LoRA iyo miisaamayaal quantized ah si loo helo fine-tune-yo weli ka fudud oo ku shaqeeya GPU-yada macaamiisha guud.
Si fudud
Halkii aad dib u qori lahayd buug-yaraha oo dhan, waxaad ku qasabtaa buug-yar oo khafiif ah oo leh cusboonaysiinta koorsadaada — mid fudud oo la daabaci karo, la wadaagi karana.
Halka aad ka aragto
- Adapter-yo qaybeed (domain) oo loo talagalay hawlaha sharciga, caafimaadka, ama luuqadda Soomaaliga.
- Dhawr LoRA oo ku shaqeeya hal moodel aasaasi oo la wadaago production.
Sida ay u shaqeyso
1.Qari miisaamayaasha aasaasiga
Sii hay moodelka la tababaray oo aan la taaban.
2.Tababar cusboonaysiinta low-rank
Hagaaji matrix-yada A iyo B ee yar.
3.Isku dar ama beddel
Isku dar miisaamayaasha ama shubkid (load) adapter-yada waqtiga socodka.
Maxay muhiim u tahay
- LoRA wuxuu fine-tuning-ka LLM-ka gaarka ah u fududeeyay iyada oo aan loo baahnayn kharash weyn oo GPU ah.
Inta badan la khaldo
LoRA had iyo jeer waxay la mid tahay tayada fine-tuning-ka buuxa.
Badiyaa way u dhowdahay, laakiin cusboonaysiinta buuxa ayaa weli ka adkaan karta la-qabsiyo qaar aad u adag.