From-scratch Foundation Model CLEVI AI(ivy)

“How could a small company create a world-class large-scale AI language model?”

How did a small startup create a large-scale language model?

This is the question CLEVI has received most frequently since unveiling its 1.4 trillion-parameter Foundation Model.

Unlike many companies that simply fine-tune open-source models, CLEVI independently carried out the entire process, from data collection and model design to large-scale distributed training and quality verification.

The secrets behind this achievement and its competitive edge are introduced in detail below.

1. A small company creates a large-scale language model?

When CLEVI unveiled its internally developed large-scale language model to the market in July 2025, numerous customers (users) were abuzz with amazement and scepticism. “Was it really developed in-house?” “Did they just slightly modify an open-source model?”

In fact, many companies in Korea and abroad simply perform transfer learning (fine-tuning) on open-source models and promote them as their own models.

However, CLEVI put forward a **‘From-scratch Foundation Model’.**

In other words, it directly designed and executed every stage, from data collection and model design to large-scale distributed training and quality verification.

2. Why developing a large-scale language model is difficult

To train a From-Scratch Foundation Model, all of the following areas must be addressed.

A large-scale, high-quality dataset is required

• Diversity: Ensure diversity in languages, domains, and formats (text, images, etc.)
• Curation: Remove noise, manage quality, and filter duplicate/harmful data
• Licensing: Clarify copyright and data usage rights

Large-scale computing infrastructure

• High-performance GPU/TPU clusters: Technology to integrate equipment across thousands to tens of thousands of nodes to support large-scale distributed training
• Efficient distributed training frameworks: DeepSpeed, Megatron-LM, Ray, etc.
• Storage/network: High-capacity data input and output, and a high-speed network environment

Model architecture design

• Reflect the latest architecture: Transformer, MoE (Mixture of Experts), multimodal, etc.
• Scalability: Consider extensibility in terms of the number of parameters, number of layers, input length, etc.
• Efficiency: Optimise training and inference speed, and memory usage

Training strategies and algorithms

• Pre-training objectives: Language models (e.g., next token prediction), multimodal models (e.g., contrastive learning), etc.
• Optimisation techniques: mixed precision, gradient accumulation, learning rate schedule, etc.
• Normalisation/stabilisation: dropout, layer norm, weight decay, etc.

Evaluation and validation framework

• Benchmark sets: Use standard datasets (Benchmarks)
• Internal evaluation: Evaluate based on actual use cases
• Continuous monitoring: Check quality, bias, and safety during and after training

Workforce and organisational capabilities

• AI/ML researchers: Model design and algorithm development
• Data engineers: Data collection, refinement, and management
• Infrastructure engineers: Distributed systems and cloud/on-premises management
• Project managers: Schedule, budget, and quality management

Ethical, legal, and security considerations

• AI ethics: Bias, safety, transparency, and accountability
• Legal compliance: Privacy, copyright, and data sovereignty
• Security: Preventing data/model leakage and controlling access

There are currently approximately 2,000 AI researchers worldwide, most of whom are concentrated at big tech companies such as META, Google, and XAI. In Korea, teams capable of independently designing everything from data collection to large-scale training infrastructure and creating a Foundation model are extremely rare.

3. Building a large language model with a small team.

Based on more than 10 years of experience researching deep learning and machine learning, CEO Hwanho Lee founded the company (Clevi) three years ago with the goal of “creating the world’s best AI”.

By independently designing every process—including the company’s own data pipeline, large-scale distributed deep learning infrastructure, model architecture development, and quality validation—we developed the capabilities required to build a Foundation Model.

After training the model several times, we now have the Clevi-5-x model, which can compete with the world’s leading models.

To effectively train large language models, infrastructure is required that connects hundreds to thousands of GPUs in a cluster and can process computations amounting to tens of quadrillions per second simultaneously. In this process, technology that minimizes computation synchronization issues and communication latency in large-scale distributed environments is essential. Until now, most companies in Korea were unable to properly train models because they lacked “precision control algorithms.” By independently developing this proprietary distributed computation control algorithm, CEO Lee Hwan-ho of CLEVI became the first in Korea to successfully train a large language model with 1.4 trillion (1.4 Trillion) parameters.

4. Why a small number of developers were able to develop a variety of models

Most big tech companies also have technologies for controlling large-scale infrastructure and refining data.

Hundreds to thousands of researchers are required to manage this.

However, CLEVI has developed and maintains a variety of models with a small number of researchers.

How was this possible?

CLEVI develops and maintains a variety of models with a small number of personnel through Teacher-Student Model (Knowledge Distilation) technology.

Let’s take a look at Teacher-Student technology.

Teacher-Student (Knowledge Distillation)

Teacher-Student technology is, simply put, a technology that uses a large language model (Teacher Model) to evaluate and provide feedback on child models (Student Models), enabling a small number of personnel to replace hundreds of researchers and rapidly develop a variety of models.

• The Mother Model (large-scale model) evaluates and provides feedback on models in various fields, replacing hundreds of researchers.
• If training is conducted through Clevi-x-platform in this manner, high-performance models such as Reasoning, VLM, Physical AI, and Coding Agent can be trained and deployed in just 10–15 days.

Clevi Coding Agent

Clevi Phsycal AI Learning Platform

VLM-Vision Language Model

Security Agent

5. Differentiation in the technology and quality of Clevi-x-platform

Model performance and scalability

• Reasoning (CoT/Reasoning) advantage: cip-5-x incorporates step-by-step logical development and evidence presentation into its design philosophy, demonstrating highly reliable computational and interpretive capabilities in solving scientific, mathematical, and engineering problems. Based on internal benchmarks, it is being designed and operated with the goal of achieving performance at or above the GPT-5 level, while reproducibility and verifiability under identical prompt conditions are managed as core quality indicators.

• Full-spectrum multimodality: By producing and combining a purpose-driven VLM (cip-5-vision), a large-scale multimodal processing model (ivy-4-mm), and a lightweight on-device model (ivy-3-text) on a single platform, it is possible to optimize combinations according to task characteristics (search/classification/summarization vs. reasoning/explanation/execution).

Enterprise applicability

• On-premises security and availability: Operates across the entire closed-network environment (input-training-service-backup), with HA design, modular expansion, and SVCE (isolated secure execution environment) to separately protect data and model parameters.
• Rapid implementation and expansion: Supports rapid deployment and operation through Ivy Chat (work-oriented multimodal conversations/agents) and API Platform (global-standard SaaS).

Physical AI differentiation

• By combining NVIDIA Isaac-based simulation with evolutionary reinforcement learning (Evolutionary RL), and using Sim2Real transfer and parallel training with synthetic data, it has improved learning efficiency and field adaptability. Its coverage from digital and robotics learning through deployment is rare among alternatives.

Service quality and governance

• Model/prompt/tool version control, evaluation suites (Korean and English knowledge, industry-specific tasks, hallucination and safety metrics), change management (SLO/SLA), and operational observability (latency, success rate, TCO) are managed through a unified platform process.

There are currently more than approximately 300 AI model service companies in South Korea, but

CLEVI is the only company that can provide both on-premises and cloud services through its own high-performance foundation models.

6. How to use language models in industry

As AI adoption requires substantial investment and rigorous validation, it is essential to directly compare and experience solutions to determine whether they can genuinely benefit your organization.

• CLEVI goes beyond simply building models to provide AI solutions that can be applied in real-world industries.
• For example, it has comprehensive enterprise offerings, including the Ivy Chat Platform, Clevi API Platform, industry-specific customization, and support for security requirements.
• It has technical capabilities that are difficult to find domestically or internationally, including physical AI, robotics, and multimodal data processing.

CLEVI provides tangible AI solutions that contribute to real-world work efficiency and industrial innovation, using AI as a ‘means’ rather than an ‘end’. Experience firsthand the differentiation of a truly From-scratch Foundation model.

Inquiries and demo requests: [email protected]

Copyright© 2025 Clevi Inc. All rights reserved.