# Train a 3.8B LLM to 0.384 CORE for $998

> Published 2026-09-10 · https://www.promptzone.com/andres_lynch/train-a-38b-llm-to-0384-core-for-998-90c

Hugo Vergnes trained a **3.8B parameter LLM** to a **0.384 CORE** score for **$998** total, per a post that appeared on [Hacker News](https://hugovergnes.github.io/little-lm-3-8b/).

> **Model:** Little LM 3.8B | **Parameters:** 3.8B | **Score:** 0.384 CORE | **Cost:** $998 | **Hardware:** consumer GPUs

## What the Training Run Delivered
The model reached **0.384 CORE** after a single focused training pass. CORE measures combined reasoning and coding performance on a fixed benchmark suite. The run used standard open-weight starting checkpoints and applied targeted data filtering plus learning-rate scheduling.

No novel architecture changes were introduced. The result came from careful dataset curation and precise compute allocation rather than scale.

## Exact Cost and Hardware Breakdown
Total spend stayed at **$998**. This covered cloud GPU hours on RTX 4090-class cards plus minimal storage and data transfer fees. Training completed in under 48 hours of wall-clock time on a small rented cluster.

The budget excluded any human annotation or synthetic data generation beyond existing public corpora. All steps stayed within the reported figure.

## How to Replicate the Run
Clone the repository linked in the original post and install the provided training script. Point it at the filtered dataset and the listed hyperparameters. Run on any 4–8× RTX 4090 or A6000 setup with 24 GB VRAM per card.

Monitor CORE score every 500 steps. Stop once the target is reached to avoid overspending. The script logs exact token counts and GPU utilization for cost verification.

## Tradeoffs Observed
The 3.8B model trails 7B–8B baselines on long-context tasks by 12–18 points. It also shows higher variance on out-of-distribution prompts. On the positive side, inference fits comfortably on a single 8 GB card at 45 tokens per second.

No safety fine-tuning was applied, so downstream use requires additional guardrails. The low cost makes repeated experiments feasible, but each run still demands careful data hygiene to hit the reported score.

## Comparison with Typical Fine-Tuning Runs

| Approach              | Parameters | CORE Score | Reported Cost | Hardware          |
|-----------------------|------------|------------|---------------|-------------------|
| Vergnes 3.8B run      | 3.8B       | 0.384      | $998          | 4–8× RTX 4090     |
| Standard 7B LoRA      | 7B         | ~0.41      | $3,200–$4,500 | 8× A100 40 GB     |
| Full 8B pre-train     | 8B         | 0.45+      | $25,000+      | 64+ H100 cluster  |

The Vergnes run undercuts typical 7B fine-tuning costs by roughly 70 % while delivering 93 % of the CORE score.

## Who Should Run This Experiment
Independent researchers and small teams needing a capable coding or reasoning model on a tight budget will find the recipe useful. Organizations already paying for large-scale clusters can skip it and continue with bigger base models.

Anyone requiring production-grade safety or long-context performance should budget for additional post-training steps.

## Verdict
The $998 training run proves that targeted data work on a 3.8B model can reach competitive CORE scores without enterprise hardware. Teams that value rapid iteration over maximum scale now have a concrete, low-cost reference point.