From-scratch Foundation Model CLEVI AI(Ivy)

“How could a small company build a world-class large language model?”

How did a small startup build a large language model?

This is the question CLEVI has received most frequently since unveiling its 1.4 trillion-parameter Foundation model.

Unlike many companies that simply fine-tune open-source models, CLEVI independently carried out the entire process, from data collection and model design to large-scale distributed training and quality validation.

We introduce its secrets and competitive advantages in detail below.

1. A small company builds a large language model?

When CLEVI unveiled its self-developed large language model to the market in July 2025, numerous customers (users) were abuzz with surprise and scepticism. “Is it really self-developed?” “Didn’t they just slightly modify an open-source model?”

In fact, many companies in India and abroad simply perform transfer learning (fine-tuning) on open-source models and promote them as their own models.

However, CLEVI put forward a **‘From-scratch Foundation Model’**.

In other words, it directly designed and executed the entire process, from data collection and model design to large-scale distributed training and quality validation.

2. Why developing a large language model is difficult

To train a From-Scratch Foundation Model, all of the following areas must be learned.

A large-scale, high-quality dataset is required

• Diversity: Ensuring diversity in languages, domains and formats (text, images, etc.)
• Curation: Removing noise, managing quality, and filtering duplicate/harmful data
• Licensing: Clarifying copyright and data usage rights

Large-scale computing infrastructure

• High-performance GPU/TPU clusters: Technology that integrates thousands to tens of thousands of nodes to support large-scale distributed training
• Efficient distributed training frameworks: DeepSpeed, Megatron-LM, Ray, etc.
• Storage/network: High-volume data input and output, and a high-speed network environment

Model architecture design

• Reflecting the latest architecture: Transformer, MoE (Mixture of Experts), multimodal, etc.
• Scalability: Considering extensibility in terms of the number of parameters, layers, input length, etc.
• Efficiency: Optimising training/inference speed and memory usage

Training Strategies and Algorithms

• Pre-training objectives: Language models (e.g., next token prediction), multimodal models (e.g., contrastive learning), etc.
• Optimisation techniques: mixed precision, gradient accumulation, learning rate schedule, etc.
• Normalisation/stabilisation: dropout, layer norm, weight decay, etc.

Evaluation and Validation System

• Benchmark sets: Using standard datasets (Benchmarks)
• Internal evaluation: Evaluation based on actual usage scenarios
• Continuous monitoring: Checking quality, bias, and safety during and after training

Workforce and Organisational Capabilities

• AI/ML researchers: Model design and algorithm development
• Data engineers: Data collection, refinement, and management
• Infrastructure engineers: Distributed systems and cloud/on-premises management
• Project managers: Schedule, budget, and quality management

Ethical, Legal, and Security Considerations

• AI ethics: Bias, safety, transparency, and accountability
• Legal compliance: Personal information, copyright, and data sovereignty
• Security: Preventing data/model leaks and access control

There are currently approximately 2,000 AI research professionals worldwide, most of whom are concentrated in big tech companies such as META, Google, and XAI. In Korea, teams capable of independently designing everything from data collection to large-scale training infrastructure and building Foundation models are extremely rare.

3. Building a Large Language Model with a Small Team

Based on more than 10 years of experience researching deep learning and machine learning, CEO Lee Hwan-ho founded the company (CLEVI) three years ago with the goal of “creating the world's best AI”.

By independently designing every process, including the company's own data pipeline construction, large-scale distributed deep learning infrastructure design, model architecture development, and quality validation, the company acquired the capabilities required to develop a Foundation Model.

After training the model several times, the company now has the Clevi-5-x model, which can compete with the world's leading models.

초거대 언어모델을 효과적으로 학습시키기 위해서는 수백에서 수천 대에 이르는 GPU를 클러스터로 연결하여, 초당 수경 번에 달하는 연산을 동시에 처리할 수 있는 인프라가 필요합니다. 이 과정에서, 대규모 분산 환경에서의 연산 동기화와 통신 지연을 최소화하는 기술이 핵심적입니다. 지금까지 국내에서는 “정밀 제어 알고리즘”이 없어 대부분의 기업이 모델을 제대로 학습시키지 못했습니다. 클레비 이환호 대표는 이러한 독자적인 분산 연산 제어 알고리즘을 자체 개발함으로써, 국내 최초로 1.4조(1.4 Trillion) 파라미터 규모의 초거대 언어모델 학습에 성공하였습니다.

4. 소수 개발 인력으로 다양한 모델을 개발할 수 있던 이유

대부분의 빅테크 기업들에서도 대규모 인프라 제어와 데이터 정제 기술을 갖추고 있습니다.

이를 관리하기 위해서는 수백 ~ 수천 명의 연구 인력이 필요합니다.

하지만, 클레비는 적은 연구 인력으로 다양한 모델을 개발·보유하고 있습니다.

어떻게 가능했을까요?

클레비는 Teacher-Student Model(Knowledge Distilation) 기술을 통해 소수의 인력으로 다양한 모델을 개발 보유하고 있습니다.

그렇다면 Teacher-Student 기술에 대해서 살펴보겠습니다.

Teacher-Student(지식 증류)

Teacher-Student 기술은 쉽게 말해 초거대 언어 모델(Teacher Model)을 통해 자식 모델(Student Model)을 평가·피드백함으로써 소수 인력으로도 수백 명의 연구 인력을 대체하여 다양한 모델을 빠르게 개발하는 기술입니다.

• Mother Model(초거대 모델)이 다양한 분야의 모델을 평가·피드백하며, 수백 명의 연구 인력을 대체합니다.
• 이 방식으로 Clevi-x-platform을 통해 학습을 진행한다면 Reasoning, VLM, Physical AI, Coding Agent 등 고성능 모델을 10~15일 만에 학습·배포할 수 있습니다.

Clevi Coding Agent

Clevi Phsycal AI Learning Platform

VLM-Vision Language Model

보안 에이전트

5. Clevi-x-platform 기술·품질의 차별성

모델 성능·확장성

• Reasoning (CoT/Reasoning) superiority: cip-5-x incorporates step-by-step logical reasoning and evidence presentation into its design philosophy, demonstrating highly reliable computational and interpretive capabilities in solving scientific, mathematical, and engineering problems. Based on internal benchmarks, it is being designed and operated with the goal of achieving performance capabilities at or above the GPT-5 level, while reproducibility and verifiability under identical prompt conditions are managed as core quality metrics.

• Full spectrum of multimodality: By producing and combining a purpose-driven VLM (cip-5-vision), a large-scale multimodal processing model (ivy-4-mm), and a lightweight on-device model (ivy-3-text) on a single platform, it enables optimal combinations tailored to task characteristics (search/classification/summarisation vs. reasoning/explanation/execution).

Enterprise applicability

• On-premises security and availability: Operation across the entire closed network (input-training-service-backup), HA design, modular expansion, and SVCE (isolated safe execution environment) provide separate protection for data and model parameters.
• Rapid deployment and scaling: Ivy Chat (business-oriented multimodal conversations/agents) and API Platform (global-standard SaaS) support rapid implementation and operation.

Physical AI differentiation

• By combining NVIDIA Isaac-based simulation with Evolutionary RL, and using Sim2Real transfer and parallel training with synthetic data, it has improved training efficiency and adaptability in the field. Its coverage from digital and robotics learning through to deployment is rare among alternatives.

Service quality and governance

• Model/prompt/tool version management, evaluation suites (Korean and English knowledge, industry-specific tasks, hallucination and safety metrics), change management (SLO/SLA), and operational observability (latency, success rate, TCO) are managed through a single-platform process.

There are currently more than approximately 300 AI model service companies in South Korea, but

CLEVI is the only company that can provide both on-premises and cloud services through a high-performance, proprietary foundation model.

6. How to apply language models to industry

As AI adoption requires significant investment and thorough validation, it is essential to directly compare and experience solutions to determine whether they can genuinely benefit your organisation.

• CLEVI goes beyond simply building models, providing AI solutions that can be applied in real-world industrial settings.
• For example, it offers comprehensive enterprise capabilities, including the Ivy Chat Platform, Clevi API Platform, industry-specific customisation, and support for security requirements.
• It possesses technological capabilities that are rarely found domestically or internationally, including physical AI, robotics, and multimodal data processing.

CLEVI provides tangible AI solutions that contribute to real business efficiency and industrial innovation, positioning AI as a ‘means’ rather than an ‘end’. Experience firsthand the distinctiveness of a true From-scratch Foundation model.

Enquiries and demo requests: [email protected]

Copyright© 2025 Clevi Inc. All rights reserved.