# GPT-2 in CSS: A 0.5 Token/Second Technical Novelty

> Published 2026-11-07 · https://www.promptzone.com/rowan_moreau/gpt-2-in-css-a-05-tokensecond-technical-novelty-2nn5

Developer Shubham Patil published a technical experiment this week that implements a **124M-parameter GPT-2 model entirely in CSS**, a project that first [surfaced on Hacker News](https://shubhampatil.dev/blog/gpt2-css). The model runs directly in a browser, performing inference without JavaScript by leveraging CSS's native matrix multiplication capabilities.

> **Model:** GPT-2 (124M) | **Parameters:** 124 Million | **Speed:** ~0.5 tokens/second
> **Framework:** Pure CSS (No JS) | **License:** MIT | **Size:** ~700MB of CSS

## How CSS Matrix Multiplication Enables AI

The core trick uses CSS's `transform: matrix3d()` property. Each neural network weight matrix is encoded as a series of 3D transformation matrices. The model's forward pass becomes a cascade of these transformations applied to HTML `<div>` elements representing the input tokens.

The 12-layer transformer architecture, including attention and feed-forward networks, is unrolled into a massive, static CSS file. Input text is one-hot encoded and styled to trigger the correct chain of transformations, with the final output probabilities read from the computed `transform` values of output elements.

> **Bottom line:** A functional transformer LLM built not as executable code, but as a declarative set of style rules.


![GPT-2 in CSS: A 0.5 Token/Second Technical Novelty](https://promptzone-community.s3.amazonaws.com/uploads/articles/zmmmdu1ttaj4nwmu3qla.jpeg)

## Performance Benchmarks: The Cost of Novelty

The project is a proof-of-concept, not a practical inference engine. On a modern desktop browser, it achieves a generation speed of approximately **0.5 tokens per second**. The model requires **loading a 700MB CSS file**, which defines the hard limit of its usability.

| Metric | GPT-2 in CSS | Typical GPT-2 (124M) in PyTorch |
|---------|----------------|----------------------------------|
| **Inference Speed** | ~0.5 tokens/sec | ~1000+ tokens/sec (on CPU) |
| **Model File Size** | ~700 MB (.css) | ~500 MB (.bin/.pt) |
| **Runtime Requirement** | Modern Browser | Python, PyTorch, 2+ GB RAM |
| **Practical Use** | Educational Demo | Production & Research |

## How to Run the Experiment Yourself

You can test the model directly in your browser. Due to the 700MB file size, a stable, high-bandwidth connection is required.

1.  Navigate to the **project's GitHub Pages site**.
2.  Wait for the massive CSS file to load (indicator in the UI).
3.  Input a short seed text (1-2 words recommended due to speed).
4.  Click "Generate" and watch the token-by-token output appear over several seconds.

For developers, the [source code on GitHub](https://github.com/patilshubham/gpt2-css) provides the build script that converts PyTorch model weights into the final CSS file.

{% details "The Math Behind the CSS Trick" %}
The `transform: matrix3d(a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p)` property defines a 4x4 transformation matrix. By treating the 16 parameters as a weight matrix slice and applying it to an element's hidden spatial representation, it performs a vector-matrix multiplication. Chaining thousands of these rules replicates a neural network's layers. The blog post details the encoding of floating-point weights into the matrix parameters.
{% enddetails %}

## Pros, Cons, and Immediate Limitations

**Advantages:**
*   **Educational Value:** Provides a tangible, inspectable way to understand transformer data flow.
*   **Platform Agnostic:** Runs anywhere a modern browser does, with no backend.
*   **Security Novelty:** Model weights are inert style data, not executable code.

**Critical Limitations:**
*   **Impractical Scale:** The 700MB file size makes it a network-bound curiosity. Scaling to larger models is infeasible.
*   **No Training:** The system is a static snapshot; fine-tuning or continued training is impossible.
*   **Extreme Latency:** At 0.5 tokens/second, it's over **2000x slower** than standard CPU inference.

## Alternatives for Browser-Based LLM Inference

For developers actually aiming to run LLMs in the browser, several mature, practical alternatives exist.

| Feature | GPT-2 in CSS | TensorFlow.js | WebLLM (via MLC) | Transformers.js |
|---------|----------------|----------------|------------------|-----------------|
| **Speed** | 0.5 t/s | 10-100 t/s | 50-200 t/s | 10-80 t/s |
| **Model Size** | 700MB (CSS) | 500MB (Bin) | 500MB (Quantized) | 500MB (Bin) |
| **Key Tech** | CSS Transforms | WebGL/WebGPU | WebGPU, WASM | ONNX Runtime Web |
| **Practical Use** | None | Prototyping, Demos | Local Chat Apps | Client-side Apps |

**TensorFlow.js** and **Transformers.js** are robust libraries for deploying standard models. **WebLLM**, powered by MLC, is currently the leader for speed, leveraging WebGPU for near-native performance. As noted in its **documentation**, it can run Llama 3 8B at interactive speeds.

## Who Should (and Shouldn't) Use This

**Use this if you are:**
*   A **CSS enthusiast or educator** looking for a mind-bending demonstration of the language's theoretical capability.
*   An **AI instructor** wanting a visual, non-code analogy for how neural network layers transform data.
*   A **browser security researcher** interested in novel data-inert model deployment.

**Skip this if you need:**
*   A **practical tool** for client-side AI features. Use WebLLM or Transformers.js.
*   To **fine-tune or update** the model. The system is completely static.
*   **Production-grade performance**. The speed and size constraints are prohibitive.

The GPT-2 in CSS project is a brilliant hack that redefines the boundaries of a styling language. It serves as a potent reminder that the fundamental operations of AI—matrix multiplications and non-linear activations—are computational primitives, not tied to any specific framework. While you'll never deploy it, studying it will change how you think about both CSS and neural network inference.