# xarray-sql: Generating FlashAttention from 25 Lines of SQL

> Published 2026-10-31 · https://www.promptzone.com/arif_wu/xarray-sql-generating-flashattention-from-25-lines-of-sql-2ci

XQL Systems demonstrated that its **xarray-sql** compiler can automatically deduce a **FlashAttention-2** kernel from approximately 25 lines of declarative SQL, a technique [flagged on Hacker News](https://github.com/xqlsystems/xarray-sql/blob/6debe2495dcb4e8fac20ab97df5b89baae2a31a6/benchmarks/ml/01_attention.py) this week. The project treats GPU programming as a database query optimization problem, generating specialized CUDA code from high-level SQL statements.

> **Tool:** xarray-sql | **Approach:** SQL-to-GPU Compiler | **Example:** ~25 LOC for Attention
> **Performance:** Competitive with PyTorch | **License:** Apache 2.0 | **Source:** [GitHub](https://github.com/xqlsystems/xarray-sql)

## How SQL Becomes a GPU Kernel

The system parses a SQL query written to express the **mathematical operations of scaled dot-product attention**. It uses the Apache Calcite framework for query planning and optimization. The compiler's key innovation is recognizing patterns in the relational algebra representation of the SQL—like softmax and batch matrix multiplication—and mapping them directly to **fused, memory-efficient CUDA kernels**.

Instead of manually writing hundreds of lines of low-level CUDA or Triton code, a developer defines the data transformations declaratively. The compiler handles **loop tiling, shared memory allocation, and kernel fusion** automatically. The provided benchmark script `01_attention.py` shows the exact SQL input and measures the output kernel's performance against a PyTorch implementation.


![xarray-sql: Generating FlashAttention from 25 Lines of SQL](https://promptzone-community.s3.amazonaws.com/uploads/articles/tpn8kdddsqpb7ebrmhcv.jpeg)

## Benchmark Performance and Code Efficiency

The project's benchmark tests the generated attention kernel against a **PyTorch implementation using `torch.nn.functional.scaled_dot_product_attention`**. For a common workload size, the automatically generated kernel achieves competitive performance while originating from a radically simpler source.

The primary gain is **developer efficiency**, not raw speed. The SQL specification is roughly **90% fewer lines of code** than a manual CUDA implementation, dramatically reducing the barrier to creating custom, high-performance ML operations.

| Feature | xarray-sql (Generated Kernel) | PyTorch SDPA (Manual) |
|---------|-------------------------------|------------------------|
| Source Code Size | ~25 lines SQL | N/A (Library Call) |
| Manual Kernel Crafting | 0 lines | 200+ lines (for custom CUDA) |
| Execution Backend | Custom CUDA | PyTorch Compiled (CUDA/ROCm) |
| Flexibility | High (SQL defines computation) | Medium (API-bound) |

## How to Run the Attention Benchmark

Testing the FlashAttention deduction requires cloning the repository and running the provided benchmark. The environment needs Python, PyTorch with CUDA, and Java for the query planner.

```bash
git clone https://github.com/xqlsystems/xarray-sql
cd xarray-sql
# Follow setup instructions for dependencies
python benchmarks/ml/01_attention.py
```
The script executes the SQL compilation, generates the CUDA kernel, runs it, and compares results and timing with the PyTorch baseline. Output includes validation that the numerical outputs match within a tolerance.

{% details "Detailed Setup Steps" %}
1.  **Clone & Navigate:** `git clone https://github.com/xqlsystems/xarray-sql && cd xarray-sql`
2.  **Install Dependencies:** The project requires Python 3.8+, PyTorch 2.0+, and Apache Calcite. Refer to the [project README](https://github.com/xqlsystems/xarray-sql) for specific version requirements and Java setup.
3.  **Run the Benchmark:** Execute `python benchmarks/ml/01_attention.py`. The first run will trigger JIT compilation of the SQL to CUDA.
4.  **Interpret Output:** The terminal will print timing results for the generated kernel and the PyTorch reference, plus a check for numerical correctness.
{% enddetails %}

## Pros, Cons, and Immediate Trade-offs

**Advantages:**
*   **Declarative Abstraction:** Specifies *what* to compute, not *how*.
*   **Reduced Complexity:** Eliminates error-prone low-level memory and synchronization code.
*   **Optimization Leverage:** Benefits from decades of database query optimization research applied to GPU programming.

**Limitations & Open Questions:**
*   **Early-Stage Project:** The ecosystem is nascent compared to established frameworks like Triton.
*   **Performance Tuning:** Fine-grained control over every kernel optimization (e.g., specific tile sizes) may be obscured by the high-level abstraction.
*   **Coverage:** It's unclear which ML operations beyond attention can be efficiently expressed and compiled.

## Alternatives for High-Performance ML Kernels

For developers needing custom GPU operations, three main paths exist. xarray-sql introduces a fourth, declarative option.

| Tool | Language/Paradigm | Learning Curve | Control | Maturity |
|------|-------------------|----------------|---------|----------|
| **CUDA C++** | Imperative C++ Extension | Very High | Maximum | Industry Standard |
| **PyTorch Triton** | Imperative Python-like | Medium | High | Growing, Meta-backed |
| **PyTorch `torch.compile`** | Decorator (Imperative) | Low | Low | High, but limited to PyTorch ops |
| **xarray-sql** | Declarative SQL | **Low** (for SQL-literate) | Medium | Research Prototype |

**CUDA** remains the gold standard for absolute performance and control but requires expert-level knowledge. **Triton** has become the preferred choice for researchers and teams writing new, high-performance kernels in a more accessible language. **`torch.compile`** is the simplest path for optimizing existing PyTorch code but cannot create fundamentally new operations.

> **Bottom line:** xarray-sql is not yet a replacement for Triton or CUDA, but a compelling experiment that could influence future high-productivity GPU programming tools.

## Who Should Use xarray-sql Right Now

This tool is specifically valuable for **ML researchers and educators** exploring the intersection of databases and machine learning, or those demonstrating compiler techniques. It provides a tangible, working example of **declarative GPU programming**.

**Consider skipping it for now** if you are a practitioner needing stable, production-ready kernels for model training or deployment. The surrounding tooling and community are too small. Your time is better invested in **learning Triton** for flexible kernel writing or relying on **optimized library calls** from PyTorch or FlashAttention directly.

The project's greatest immediate utility is as a **conceptual prototype**. It successfully demonstrates that a database query planner can compete with human experts in designing a state-of-the-art ML kernel. This validates a research direction that could eventually make GPU programming accessible to a much broader group of developers who understand data transformations but not hardware details.

The evolution of GPU programming will increasingly favor abstractions that hide complexity without sacrificing performance. xarray-sql points squarely at that future.