1. Catastrophic Forgetting ne anohyeto ahorow a ɛwɔ RAG mu
Sɛ wɔde AI di dwuma wɔ adwuma ankasa mu a, nsɛnnennen akɛse no mu biako ne asɛm a ɛne sɛ nimdeɛ a wɔakyerɛ AI no dedaw betumi ayera bere a wɔde data foforo yɛ no fine-tuning, a wɔfrɛ no Catastrophic Forgetting.
Eyi yɛ adeyɛ a AI a egyina neural network so no, bere a ɛresua nsɛm foforo no, ɛhwere nimdeɛ anaa nhwɛso ahorow a na wɔakyerɛ no kan no ntɛm ara.
Sɛ nhwɛso no, sɛ wɔde data a ɛfa adwumakuw pɔtee bi ho yɛ open-source LLM fine-tuning a, nimdeɛ a ɛfa nneɛma a wɔtaa nim nyinaa ho anaa kasa mu tumi betumi atew.
Sɛ wɔbɛkwati ɔhaw yi a, wɔde RAG(Retrieval-Augmented Generation) mfiridwuma no di dwuma kɛse, nanso RAG nso wɔ anohyeto ahorow.
RAG anohyeto titiriw
- Ahyɛnsodeɛ a wɔahyehyɛ ho nsɛm a wɔntumi nni ho dwuma yiye (Structured Data QA): RAG nhyehyɛeɛ no agye ne ho sɛ ɛyɛ adwuma wɔ nsɛm a wɔankyekyɛ no mu, enti ɛntumi nte aseɛ na ɛmfa nsɛm a wɔahyehyɛ no (nhyehyɛeɛ, database, spreadsheet ne nea ɛkeka ho) nni dwuma yiye. Ɛntumi nso nte abusuabɔ a ɛda nsɛm no ntam aseɛ yiye.
- PDF a ɛyɛ den/nsɛm a wɔankyekyɛ no mu anohyetoɔ (Complex PDFs): Sɛ wɔreyɛ chunking (akyekyɛ mu) wɔ PDF a ɛyɛ den (a ɛwɔ nhyehyɛeɛ, nkratafa a wɔakyekyɛ mu, mfonini ne nea ɛkeka ho) mu a, nsɛm bi betumi ayera anaasɛ nsɛm no nkyerɛaseɛ asesa. Eyi betumi ama wɔankyerɛw nsɛm a ɛho hia no wɔ index mu, anaasɛ ama ayɛ den sɛ wɔbɛhwehwɛ no.
- Fallback (backup) model nni hɔ (Fallback Model(s)): Sɛ RAG nhyehyɛeɛ no ntumi nnya mmuaeɛ a ɛfata a, nteaseɛ a ɛbɛma wɔadan akɔ model foforɔ (backup) so no sua anaasɛ ɛnyɛ adwuma yiye. Eyi nti, sɛ mmuaeɛ no di nkogu a, ɔde ɔkwan foforɔ a ɛbɛma ɔde ne ho asi no ani nnye no mma ɔdefoɔ no.
- Ahobammɔ mu mmerɛwyɛ (LLM Security): Ahobammɔ a ɛbɔ ho ban fi LLM ankasa mu mmerɛwyɛ ho (prompt injection, nsɛm a ɛbɛda adi ne nea ɛkeka ho) no nnɔɔso, enti asiane wɔ hɔ sɛ nsɛm a ɛyɛ kokoam bɛda adi.
- Ntrɛmu tumi a ɛnnɔɔso (Data Ingestion Scalability): Sɛ wɔreyɛ index na wɔresie data pii a, ntrɛmu tumi ho haw ba. Sɛ data no dɔɔso a, chunking, nsie ne hwehwɛ ahoɔhare so tew, na nhyehyɛeɛ no nya adesoa kɛse.
- Nsɛm a ɛho hia a ayera (Missing Content): Mpɛn pii no, nsɛm a ɛho hia fi document no mu yera wɔ chunking nhyehyɛeɛ no mu, anaasɛ wɔankyerɛw no wɔ index mu. Eyi nti, ɔdefoɔ no ntumi nhwehwɛ nsɛm a ɔrehwehwɛ no.
- Nsɛm a ɛwɔ ranking a ɛkorɔn a ayera (Missed Top Ranked): Wɔ search aba mu no, chunk a ɛfa ho paa (top-ranked) no betumi ayera. Nea ɛde ba ne search algorithm no anoɔden, embedding no su a ɛrekɔ fam, ranking mfomsoɔ ne nea ɛkeka ho.
- Nsɛm no mu nsiesie a ɛne ho nhyia (Not in Context): Mpɛn pii no, chunk a wɔahu no ne asɛmmisa no ankasa mu nsɛm nhyia. Eyi fi anohyetoɔ a ɛwɔ search a egyina nsɛm a ɛyɛ sɛ ho nkabom so; nteaseɛ mu nkabom nnɔɔso.
- Format mfomsoɔ (Wrong Format): Chunk a wɔahu no bɛyɛ sɛ ne format mfata sɛ LLM bedi ho dwuma, anaasɛ ɛnte sɛ mmuaeɛ format a ɔdefoɔ no pɛ. Sɛ nhwɛsoɔ no, sɛ wɔdan nhyehyɛeɛ bi yɛ no nsɛm a wɔakyerɛw a, nsɛm no betumi asesa.
- Nsɛm a wɔpɛ sɛ wɔyi no adi no di nkogu (Not Extracted): Ɛtɔ da bi a, wɔntumi nnyi nsɛm a ɛho hia no adi yiye fi chunk no mu, anaasɛ LLM no ntumi nhu nsɛm no. Eyi taa si wɔ kasamu nhyehyɛeɛ a ɛyɛ den, nhyehyɛeɛ, mfonini ne nea ɛkeka ho mu.
- Nsɛm no kwan/emu dɔ a ɛmfata (Incorrect Specificity): Ɛtɔ da bi a, mmuaeɛ no yɛ amansan dodo ma asɛmmisa no, anaasɛ ɛyɛ pɔtee dodo, enti ɛne nea ɔdefoɔ no hia ankasa nhyia. Ɛyɛ den sɛ wɔbɛyɛ nsɛm no kwan ne emu dɔ no sɛnea ɛsɛ.
- Mmuaeɛ a enwie pɛyɛ (Incomplete): Awiei koraa no, mmuaeɛ a wɔayɛ no betumi awe pɛyɛ, anaasɛ wɔde nsɛm no fã bi nkutoo ama, enti ɛntumi nni ɔdefoɔ no ahiadeɛ ho dwuma pɛpɛɛpɛ. Nea ɛde ba no bi ne sɛ wɔantumi anka nsɛm a ɛwɔ chunk ahodoɔ mu abɔ mu, anaasɛ LLM no de nsɛm no fã bi nkutoo dii dwuma.
Esiane sɛ RAG de ne ho to vector nsɛdi hwehwɛ ne chunk gyina so index dodow so nti, ɛwɔ anohyeto a emu da hɔ wɔ ahotoso, ntrɛwmu ne pɛpɛɛpɛyɛ mu bere a wɔde di dwuma ankasa wɔ adwuma mu.
2. Clevi semantic database a adi RAG anohyeto so
Sɛnea ɛbɛyɛ a Clevi bɛdi saa anohyeto yi so no, wɔyɛɛ awoɔ foforo AI nimdeɛ nhyehyɛe a wɔayɛ ama nkyerɛaseɛ mu nkitahodi, nsusuwii a ɛyɛ adwuma pii ne mmuae a egyina adanse so, a ɛne Clevi Semantic Database.
Ɛyɛ ano aduru a ɛyɛ yiye efisɛ wɔwɔ wɔn ankasa Reasoning model, multimodal AI ne agent-type AI model nyinaa, na ɛyɛ Clevi mfiridwuma a ɛyɛ den no mu baako a ɛma AI tumi boa ankasa wɔ wiase yi mu.
Sɛ yɛde mfatoho a, sɛ yɛfa database a wɔde nsɛm sie no sɛ ɛyɛ nwomakorabea a, ɛnde wobɛtumi asusuw Clevi semantic database ho sɛ ɛwɔ nwomakorabea hwɛfo a ne nimdeɛ yɛ kɛse. (Sɛ wobisa nwomakorabea hwɛfo no nsɛm a wuhia a, wubetumi anya mmuae a ɛyɛ pɛpɛɛpɛ na ɛyɛ ntɛm.)
Ɔdefo betumi de nsɛm foforo aka nwomakorabea no ho na ɔsan afrɛ nsɛm a ohia bere biara.
Bio nso, wubetumi asiesie data no version nhyehyɛe anaa kwan a wɔde bɛkɔ mu sɛnea wopɛ.
Esiane sɛ nwomakorabea hwɛfo no nneyɛe nyinaa yɛ log (kɔkɔbɔ) nti, sɛ mfomso bi si mpo a, wubetumi asiesie no.
<Clevi Semantic Database Structure>
Nneɛma titiriw ne mfiridwuma nhyehyɛe
Nsɛm a egyina nkyerɛaseɛ so sie
- Ɛnyɛ chunk vectorDB kɛkɛ, na mmom ɛde nkitahodi a ɛda data ntam wɔ nkyerɛaseɛ mu (nsɛm tebea, nhyehyɛe, nea ɛde biribi ba ne nea ɛkɔ so, ne nea ɛkeka ho) sie wɔ graph DB mu
- Ɛyɛ mmerɛw sɛ wɔbɛtrɛw mu na wɔde nneɛma foforo aka ho wɔ nimdeɛ graph/semantic network kwan so
Hwehwɛ ne nsusuwii a wɔaka abom
- CTA+Context Query: Ɛde nsɛm tebea (Context) ne CTA a ɛwɔ asɛmmisa no mu nyinaa si no so
- Hybrid Retrieval: Ɛka vector nsɛdi ne mmara/graph gyina so hwehwɛ bom
- Intelligent Folder Hits: Ɛhwehwɛ folda/category a ɛfa ho wɔ nkyerɛaseɛ mu no ankasa
Multimodal nneɛma a wɔyɛ no bere koro mu
- TextReader Pool, VisionReader Pool ne nea ɛkeka ho tumi kyerɛ nsɛm, mfonini, table, nneɛma a wɔakyere wɔ nne mu ne data ahorow pii ase bere koro mu
- Bere a RAG a wɔde di dwuma dedaw no taa tumi yɛ nsɛm a wɔakyerɛw nkutoo no, semantic database no wɔ tumi a ɛkorɔn paa sɛ ɛbɛyɛ multimodal data
Mmuae a egyina adanse so ne ahotoso mu nhwehwɛmu
- Krataa Nkyerɛkyerɛmu ne Nsɛm a Wɔafa mu: Krataa nkyerɛkyerɛmu ne fibea ahorow a wɔyɛ no ara kwa
- Ka bom na Si no pi: Nneɛma a efi fibea ahorow mu no nkyerɛaseɛ mu ka bom ne ahotoso mu hwɛ
- Adanseɛ Ntomaban: Adanseɛ a egyina so mmuaeɛ ntomaban a wɔde ma
Nsusuwii a Ɛyɛ Den ne Nhyehyɛefoɔ
- Nhyehyɛefoɔ/DAG: Sɛ wɔrehwehwɛ nsɛm a ɛyɛ den a, wɔyɛ nhyehyɛeɛ anammɔn-anammɔn na wɔde DAG (Directed Acyclic Graph) gyina so susuw nsɛm ho
Agyentɔ Nhyehyɛeɛ a Wɔaka Abom
- cip-5-agent Supervisor: Agyentɔ a ɔwɔ soro a ɔhwɛ na ɔhwɛ so ma nhwehwɛmu, nsusuwii ne nkabom nhyehyɛeɛ no nyinaa kɔ so yiye
3. Nhyehyɛeɛ Mu Nkyerɛkyerɛmu ne Ntotoho
Sɛ yɛka no tiaa a, Semantic DB gye nsɛm ahorow (nne, nsɛm a wɔakyerɛw, mfonini, video ne nea ɛkeka ho) tom, na ɛnyɛ Chunk nko ara na ɛkyekyɛ mu sɛ nimdeɛ, na mmom ɛde nkyerɛaseɛ mu abusuabɔ a ɛda data no ntam (tebea, mpɔtam nhyehyɛeɛ, nea ɛde biribi ba ne nea ɛkeka ho) nso bom.
Esiane sɛ ɛde nkyerɛaseɛ mu abusuabɔ na ɛyɛ nhwehwɛmu ne nsusuwii, na ɛnyɛ nsɛmfua/ɔkwan a nneɛma yɛ pɛ ho gyinaeɛ te sɛ RAG nti, wubetumi anya nsɛm a wɔahwehwɛ no pɛpɛɛpɛ ne mmuaeɛ a ɛyɛ nokware kɛse afi AI hɔ.
Bio nso, esiane sɛ wɔde nimdeɛ graph/semantic network kwan so sie nti, ɛyɛ mmerɛw sɛ wɔbɛtrɛw mu na wɔde nsɛm foforɔ aka ho bere nyinaa.
Yɛn Clevi ahu RAG nhyehyɛeɛ no anoɔden wɔ AI nsakraeɛ kɛseɛ yi berɛ mu, na yɛnam semantic database ne AI model soronko a yɛayɛ so ama adetɔfoɔ atumi anya data a ɛyɛ pɛpɛɛpɛ na wotumi de wɔn ho to so no ntɛm.
Yɛn Clevi AI nhyehyɛeɛ no tumi ma adetɔfoɔ no data a ɛsom boɔ no yɛ nea ɛwɔ mfasoɔ wɔ tebea biara mu, na domain no su biara mfa ho.
Efi seesei kɔ so no, Clevi bɛyɛ nea ɛbɛtumi biara sɛ ɛbɛma adetɔfoɔ no data no boɔ akɔ soro paa na ɛde AI osuahu foforɔ ne anwanwa ama wɔn.
Yɛsrɛ mo sɛ mo ne Clevi nni nkɔsoɔ na monya AI daakye foforɔ no ho osuahu.
Nsemmisa ne Demo Bɛsrɛ: [email protected]
Copyright© 2025 Clevi Inc. Hokwan ahorow nyinaa wɔ hɔ.