From-scratch Foundation Model CLEVI AI(ivy)

“How could a small company build a world-class, ultra-large language model?”

How did a small start-up build an ultra-large language model?

This is the question CLEVI has received most frequently since unveiling its 1.4 trillion-parameter Foundation model.

Unlike many companies that simply fine-tune open-source models, CLEVI independently carried out the entire process, from data collection and model design to large-scale distributed training and quality verification.

The secrets behind this achievement and CLEVI’s competitive edge are introduced in detail below.

1. A small company builds an ultra-large language model?

When CLEVI unveiled its self-developed ultra-large language model to the market in July 2025, numerous customers (users) were abuzz with a mixture of amazement and scepticism. “Was it really developed in-house?” “Wasn’t it just a slightly modified open-source model?”

In fact, many companies in South Korea and abroad simply carry out transfer learning (fine-tuning) on open-source models and promote them as their own models.

However, CLEVI put forward a **‘From-scratch Foundation Model’**.

In other words, it directly designed and executed the entire process, from data collection and model design to large-scale distributed training and quality verification.

2. Why developing an ultra-large language model is difficult

To train a From-Scratch Foundation Model, all of the following areas must be learned.

A large-scale, high-quality dataset is required

• Diversity: securing diversity in language, domains and formats (text, images, etc.)
• Curation: removing noise, managing quality, and filtering duplicate/harmful data
• Licensing: clarifying copyright and data usage rights

Large-scale computing infrastructure

• High-performance GPU/TPU clusters: technology to integrate equipment across thousands to tens of thousands of nodes to support large-scale distributed training
• Efficient distributed training frameworks: DeepSpeed, Megatron-LM, Ray, etc.
• Storage/networking: large-volume data input/output and a high-speed network environment

Model architecture design

• Reflecting the latest architecture: Transformer, MoE (Mixture of Experts), multimodal systems, etc.
• Scalability: Considering extensibility in terms of the number of parameters, layers, input length, etc.
• Efficiency: Optimising training/inference speed and memory usage

Training strategies and algorithms

• Pre-training objectives: Language models (e.g. next token prediction), multimodal models (e.g. contrastive learning), etc.
• Optimisation techniques: mixed precision, gradient accumulation, learning rate schedules, etc.
• Normalisation/stabilisation: dropout, layer norm, weight decay, etc.

Evaluation and validation framework

• Benchmark sets: Using standard datasets (Benchmarks)
• Internal evaluation: Evaluation based on real-world usage scenarios
• Continuous monitoring: Checking quality, bias and safety during and after training

Workforce and organisational capabilities

• AI/ML researchers: Model design and algorithm development
• Data engineers: Data collection, refinement and management
• Infrastructure engineers: Distributed systems and cloud/on-premises management
• Project managers: Schedule, budget and quality management

Ethical, legal and security considerations

• AI ethics: Bias, safety, transparency and accountability
• Legal compliance: Personal data, copyright and data sovereignty
• Security: Preventing data/model leaks and access control

There are currently approximately 2,000 AI researchers worldwide, most of whom are concentrated in Big Tech companies such as META, Google and XAI. In South Korea, teams capable of directly designing everything from data collection to large-scale training infrastructure and creating a Foundation Model are extremely rare.

3. Creating a large language model with a small team.

Drawing on more than 10 years of experience researching deep learning and machine learning, CEO Hwanho Lee founded the company (CLEVI) three years ago with the goal of “creating the world’s best AI”.

By directly designing every process—including building the company’s own data pipeline, designing large-scale distributed deep learning infrastructure, developing model architectures and validating quality—the company acquired the capabilities required to develop a Foundation Model.

After training the model several times, the company now has the Clevi-5-x model, which can rival the world’s leading models.

To train large language models effectively, infrastructure is required that connects hundreds to thousands of GPUs in a cluster to process tens of quadrillions of calculations simultaneously per second. In this process, technology that minimises computational synchronisation and communication delays in large-scale distributed environments is essential. Until now, most companies in South Korea have been unable to train models properly due to the lack of a “precision control algorithm”. By independently developing this proprietary distributed computing control algorithm, Clevi CEO Lee Hwan-ho became the first in South Korea to successfully train a large language model with 1.4 trillion (1.4 Trillion) parameters.

4. Why it was possible to develop various models with a small number of developers

Most big tech companies also possess technologies for controlling large-scale infrastructure and refining data.

Hundreds to thousands of researchers are required to manage this.

However, Clevi has developed and maintains various models with a small research team.

How was this possible?

Clevi develops and maintains various models with a small team through Teacher-Student Model(Knowledge Distilation) technology.

Let us take a look at Teacher-Student technology.

Teacher-Student (Knowledge Distillation)

Teacher-Student technology is, in simple terms, a technology that uses a large language model (Teacher Model) to evaluate and provide feedback on child models (Student Models), enabling a small team to replace hundreds of researchers and rapidly develop various models.

• The Mother Model (large-scale model) evaluates and provides feedback on models in various fields, replacing hundreds of researchers.
• If training is conducted through Clevi-x-platform using this approach, high-performance models such as Reasoning, VLM, Physical AI and Coding Agent can be trained and deployed in 10–15 days.

Clevi Coding Agent

Clevi Phsycal AI Learning Platform

VLM-Vision Language Model

Security Agent

5. Differentiation in Clevi-x-platform technology and quality

Model performance and scalability

• Reasoning (CoT/Reasoning) advantage: cip-5-x incorporates step-by-step logical development and the presentation of supporting evidence into its design philosophy, demonstrating highly reliable computational and analytical capabilities in solving scientific, mathematical and engineering problems. Based on internal benchmarks, it is being designed and operated to target performance at or above the GPT-5 level, while reproducibility and verifiability under identical prompt conditions are managed as key quality indicators.

• Full-spectrum multimodality: by producing and combining the purpose-driven VLM (cip-5-vision), the large-scale multimodal processing model (ivy-4-mm) and the lightweight on-device model (ivy-3-text) on a single platform, it is possible to create an optimal combination suited to the characteristics of the task (search/classification/summarisation vs. reasoning/explanation/execution).

Enterprise applicability

• On-premises security and availability: operation across the entire closed network (input–training–service–backup), HA design, modular expansion and SVCE (isolated secure execution environment) provide separate protection for data and model parameters.
• Fast adoption and scaling: Ivy Chat (work-oriented multimodal conversations/agents) and API Platform (global-standard SaaS) support rapid implementation and operation.

Physical AI differentiation

• By combining NVIDIA Isaac-based simulation with evolutionary reinforcement learning (Evolutionary RL), and using Sim2Real transfer and parallel training with synthetic data, learning efficiency and on-site adaptability have been enhanced. Few alternatives cover the full process from digital and robotics learning through to deployment.

Service quality and governance

• Model/prompt/tool version control, evaluation suites (Korean and English knowledge, industry-specific tasks, hallucination and safety metrics), change management (SLO/SLA) and operational observability (latency, success rate and TCO) are managed through a unified platform process.

There are currently more than approximately 300 AI model service companies in South Korea, but

CLEVI is the only company able to provide both on-premises and cloud services through a high-performance proprietary foundation model.

6. How to use language models in industry

As adopting AI requires substantial investment and rigorous validation, it is essential to directly compare and trial whether a solution can genuinely benefit your organisation.

• CLEVI goes beyond simply building models to provide AI solutions that can be applied in real-world industries.
• For example, it offers a comprehensive range of enterprise channels, including the Ivy Chat Platform, Clevi API Platform, industry-specific customisation and support for security requirements.
• It possesses technical capabilities that are difficult to find in Korea and overseas, including physical AI, robotics and multimodal data processing.

CLEVI treats AI as a ‘means’ rather than an ‘end’, providing substantive AI solutions that contribute to real-world work efficiency and industrial innovation. Experience firsthand the differentiation of a genuine From-scratch Foundation model.

Enquiries and demo requests: [email protected]

Copyright© 2025 Clevi Inc. All rights reserved.