Developer Shubham Patil published a technical experiment this week that implements a 124M-parameter GPT-2 model entirely in CSS, a project that first surfaced on Hacker News. The model runs directly in a browser, performing inference without JavaScript by leveraging CSS's native matrix multiplication capabilities.
Model: GPT-2 (124M) | Parameters: 124 Million | Speed: ~0.5 tokens/second
Framework: Pure CSS (No JS) | License: MIT | Size: ~700MB of CSS
How CSS Matrix Multiplication Enables AI
The core trick uses CSS's transform: matrix3d() property. Each neural network weight matrix is encoded as a series of 3D transformation matrices. The model's forward pass becomes a cascade of these transformations applied to HTML <div> elements representing the input tokens.
The 12-layer transformer architecture, including attention and feed-forward networks, is unrolled into a massive, static CSS file. Input text is one-hot encoded and styled to trigger the correct chain of transformations, with the final output probabilities read from the computed transform values of output elements.
Bottom line: A functional transformer LLM built not as executable code, but as a declarative set of style rules.
Performance Benchmarks: The Cost of Novelty
The project is a proof-of-concept, not a practical inference engine. On a modern desktop browser, it achieves a generation speed of approximately 0.5 tokens per second. The model requires loading a 700MB CSS file, which defines the hard limit of its usability.
| Metric | GPT-2 in CSS | Typical GPT-2 (124M) in PyTorch |
|---|---|---|
| Inference Speed | ~0.5 tokens/sec | ~1000+ tokens/sec (on CPU) |
| Model File Size | ~700 MB (.css) | ~500 MB (.bin/.pt) |
| Runtime Requirement | Modern Browser | Python, PyTorch, 2+ GB RAM |
| Practical Use | Educational Demo | Production & Research |
How to Run the Experiment Yourself
You can test the model directly in your browser. Due to the 700MB file size, a stable, high-bandwidth connection is required.
- Navigate to the project's GitHub Pages site.
- Wait for the massive CSS file to load (indicator in the UI).
- Input a short seed text (1-2 words recommended due to speed).
- Click "Generate" and watch the token-by-token output appear over several seconds.
For developers, the source code on GitHub provides the build script that converts PyTorch model weights into the final CSS file.
"The Math Behind the CSS Trick"
The transform: matrix3d(a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p) property defines a 4x4 transformation matrix. By treating the 16 parameters as a weight matrix slice and applying it to an element's hidden spatial representation, it performs a vector-matrix multiplication. Chaining thousands of these rules replicates a neural network's layers. The blog post details the encoding of floating-point weights into the matrix parameters.
Pros, Cons, and Immediate Limitations
Advantages:
- Educational Value: Provides a tangible, inspectable way to understand transformer data flow.
- Platform Agnostic: Runs anywhere a modern browser does, with no backend.
- Security Novelty: Model weights are inert style data, not executable code.
Critical Limitations:
- Impractical Scale: The 700MB file size makes it a network-bound curiosity. Scaling to larger models is infeasible.
- No Training: The system is a static snapshot; fine-tuning or continued training is impossible.
- Extreme Latency: At 0.5 tokens/second, it's over 2000x slower than standard CPU inference.
Alternatives for Browser-Based LLM Inference
For developers actually aiming to run LLMs in the browser, several mature, practical alternatives exist.
| Feature | GPT-2 in CSS | TensorFlow.js | WebLLM (via MLC) | Transformers.js |
|---|---|---|---|---|
| Speed | 0.5 t/s | 10-100 t/s | 50-200 t/s | 10-80 t/s |
| Model Size | 700MB (CSS) | 500MB (Bin) | 500MB (Quantized) | 500MB (Bin) |
| Key Tech | CSS Transforms | WebGL/WebGPU | WebGPU, WASM | ONNX Runtime Web |
| Practical Use | None | Prototyping, Demos | Local Chat Apps | Client-side Apps |
TensorFlow.js and Transformers.js are robust libraries for deploying standard models. WebLLM, powered by MLC, is currently the leader for speed, leveraging WebGPU for near-native performance. As noted in its documentation, it can run Llama 3 8B at interactive speeds.
Who Should (and Shouldn't) Use This
Use this if you are:
- A CSS enthusiast or educator looking for a mind-bending demonstration of the language's theoretical capability.
- An AI instructor wanting a visual, non-code analogy for how neural network layers transform data.
- A browser security researcher interested in novel data-inert model deployment.
Skip this if you need:
- A practical tool for client-side AI features. Use WebLLM or Transformers.js.
- To fine-tune or update the model. The system is completely static.
- Production-grade performance. The speed and size constraints are prohibitive.
The GPT-2 in CSS project is a brilliant hack that redefines the boundaries of a styling language. It serves as a potent reminder that the fundamental operations of AI—matrix multiplications and non-linear activations—are computational primitives, not tied to any specific framework. While you'll never deploy it, studying it will change how you think about both CSS and neural network inference.
Top comments (0)