MLIR

Quantization is one of those things where everyone unanimously agrees “it’s important”, but the details are often fuzzy; I mean, no one really wants to think about it in their compiler’s IR, right? In most stacks, quantization lives in an (awkward?) place: High-level frameworks (like PyTorch) expose “quantized layers” or post-training quantization APIs. Backends (like TensorRT, ONNX Runtime) and kernels really care about bit-widths, scales, zero points, and data layouts. The glue in between is often a mishmash of ad-hoc passes, custom operators, and brittle assumptions. MLIR is actually a sweet spot to make quantization less cursed: ...

If you only remember one thing from this post: rewriting in MLIR is “find a pattern, make a change, repeat until more changes can’t be made”, with two key components: Greedy pattern application (canonicalization and local clenups), and Dialect conversion (legalize/convert regions with invariants about the legal forms of ops). TL;DR Patterns live in a RewritePatternSet and are driven by either applyPatternsGreedily (for local greedy rewrites) or applyPartial/FullConversion (for dialect conversion with legality constraints). Write patterns by subclassing OpRewritePattern<YourOp> and overriding matchAndRewrite with your logic. Rewrite safely using PatternRewriter methods to create, replace, and erase ops. Canonicalization : MLIR has a single canonicalization pass which applies all registered patterns greedily until no more matches are found. Conversion: MLIR’s conversion framework allows you to define legality constraints and convert ops from one dialect to another while preserving invariants. We do this with ConversionTarget and TypeConverter. Folding: Take series of ops complements rewriting by simplifying constant expressions during pattern application. Part 1: The moving pieces RewritePatternSet and PatternRewriter RewritePatternSet is a container for your rewrite patterns. You populate it with instances of your custom patterns. MLIR runs these patterns for you; you don’t directly loop over operations. In your pattern’s matchAndRewrite, you Inspect the matched op. Optionally create the new IR (using the rewriter’s insertion point). Replace or erase the matched op. Greedy vs. Conversion Greedy (Canonicalization and Local Rewrites) Think “peephole + algebraic simplification”. Use applyPatternsGreedily to apply all patterns in a RewritePatternSet. applyPatternsGreedily(fop, std::move(patterns)); Conversion (Dialect Conversion) Define legality constraints for ops via ConversionTarget. Use TypeConverter to handle type conversions. Use applyPartialConversion or applyFullConversion to convert ops while respecting legality. Part 2: Your first greedy rewrite pattern. Let’s fold away arith.addi %x, 0: i32 into just %x. Yeah, it’s trivial, and MLIR’s canonicalization already does this, but it’s a great starting point. ...

GPUs aren’t mysterious - just picky. Most performance cliffs are not about the math; they’re about how warps step, how memory is fetched, and how often the registers spill. This post decodes the jargon; and to be candid, it is me “spilling” my notes, trying to explain myself. TL;DR Think in warps, not threads. Coalesce or pay. Tile for reuse in shared memory, but watch the register pressure. Matrix units (Tensor Cores, FMAs, etc) love the right data types and tile sizes. Occupancy is a balancing act: a tool, not a goal. Just enough to hide latency. 1. Execution Model, Decoded SIMT vs SIMD (why is it confusing?) SIMD (CPU): Single Instruction, Multiple Data. One instruction operates on a fixed-width vector (e.g., a single AVX-512 instruction processes 16 floats at once). SIMT (GPU): Single Instruction, Multiple Threads. Many threads execute the same instruction in lockstep as a warp (NVIDIA) or wavefront (AMD); each thread has its own registers/control flow. Warps/Wavefronts Smallest lockstep unit: ...

Every time a neural network runs, there’s a silent negotiation between compute and memory. It’s naive to think ML compilers optimize just the compute - the FLOPs. In reality, they optimize movement. The most expensive operation in modern compute isn’t your matrix multiply; it’s getting data from memory to the compute units. This post explores how layout, tiling, and fusion are the unsung heroes of ML compiler performance. 1. The Compiler’s Hidden Battle Deep Learning performance is a balancing act between compute and memory. You can think of layout as the grammar of that relationship - the way tensors are arranged, accessed, and aligned. ...

“If life had a compiler, I’d probably still be tuning the optimization flags.” Welcome to Tiled Thoughts — my verbose debug build. I’m Samarth, a compiler engineer at Qualcomm. My work revolves around building efficient ML systems, contributing to open-source compiler infrastructures like LLVM and MLIR, and exploring the intersection of programming languages and machine learning. This blog is where I log the things that don’t quite fit into a Git commit message — reflections, experiments, and observations tiled across compilers, ML systems, open-source, and education. ...