Lipi
A typographic dithering engine that renders photographs as grids of world scripts, Bengali, Assamese, Georgian, using Floyd-Steinberg error diffusion.
Overview
Lipi converts any photograph into a typographic field. Each cell in a character grid is assigned a world-script glyph calibrated to that region's luminance, then rendered with Floyd-Steinberg error diffusion so tonal gradients read accurately at a distance. The output is indistinguishable from a designed poster until you zoom in and find an entire alphabet.
Building it required three novel algorithms, Grain, Erosion, and Gravity Well, none of which exist in published shader or generative art literature in this form.
Tech
Next.js 14 · TypeScript · Canvas 2D. No WebGL, no rendering libraries. The dither pass, Sobel operator, Perlin noise, and all three novel algorithms are written from first principles on a 2D canvas context.
Solution
Empirical glyph density calibration
Each character is rasterised at 64×64px and dark pixels are counted via the ITU-R BT.601 luminance formula. The resulting ink ratio drives character selection, not Unicode complexity, not visual assumption. Any script, correctly ranked.
Two-axis structural glyph mapping
In Structural mode, glyph selection is driven by a 2D lookup table indexed by both normalised luminance (8 brightness bands) and Sobel edge density (4 edge bands), giving a 8×4 grid of Assamese characters. Tonal and structural information encoded simultaneously.
Compositable post-process stack
Flow field rotation, membrane script-swap, crosshatch opacity banding, chromatic aberration, and prismatic dispersion each operate as independent passes over the dithered output. None alter the underlying character assignment, they compose on top.
The Core Pipeline
Standard Floyd-Steinberg error diffusion operates on pixel values. Lipi adapts this to operate on a character grid. Each cell's luminance is quantised by selecting the glyph whose ink density best matches the target value. The quantisation error, the difference between the target luminance and the chosen glyph's actual ink density, is then propagated to neighbouring cells using the standard kernel.
| 1 | Right: error × 7/16 |
| 2 | Bottom-left: error × 3/16 |
| 3 | Bottom: error × 5/16 |
| 4 | Bottom-right: error × 1/16 |
The key adaptation: rather than rounding to 0 or 255, the quantised value is (1 − density) × 255, where density is the empirically measured ink ratio of the selected glyph. The error diffusion accounts for the actual optical weight of the character, not a binary approximation.
Glyph density calibration renders each character into an offscreen 64×64 canvas and counts every pixel whose BT.601 luminance falls below 128. Characters are sorted ascending by density, producing a lookup table from lightest to heaviest. At render time:
| 1 | density = darkPixelCount / (64 × 64) |
| 2 | t = 1 − normLum |
| 3 | idx = round(t × (tableLength − 1)) |
For Structural mode and the Membrane effect, a Sobel operator runs across the full source image before the dither pass. The gradient magnitude at each pixel is averaged across all pixels within each cell's bounds and normalised to [0, 1] to produce the edge density map.
| 1 | Gx = [−1, 0, +1 / −2, 0, +2 / −1, 0, +1] |
| 2 | Gy = [−1, −2, −1 / 0, 0, 0 / +1, +2, +1] |
| 3 | magnitude = sqrt(Gx² + Gy²) |
The Three Novel Algorithms
These three systems were discovered during development. None correspond to published techniques in generative art, shader programming, or typographic rendering literature.
Grain, Edge-angle glyph orientation
The Sobel operator returns a gradient magnitude, how much edge is at a point. Grain uses the discarded output: the gradient direction. Each glyph is rotated to align with the local edge angle at its cell. Glyphs along a vertical edge rotate to stand perpendicular to it; glyphs in flat tonal regions sit at their natural orientation. The character grid stops being a neutral field and becomes a map of the image's directional structure. This is distinct from flow field rotation, which applies spatially smooth Perlin noise regardless of image content. Grain binds each glyph's orientation directly to the geometry of the photograph.
| 1 | θ = atan2(Gy, Gx) |
Erosion, Luminance-weighted character decay
Erosion inverts the conceptual frame of standard dithering. It treats the density table as an ordered erosion sequence and uses luminance to determine how far through that sequence a region has decayed. Bright regions show early-sequence characters, sparse, structurally simple. Dark regions show late-sequence characters, dense, ink-heavy. The visual result is that the image appears to be emerging from, or dissolving into, its own script, as if the alphabet is the substrate from which the image is carved. The formula operates identically to standard luminance picking; what makes Erosion novel is the reframing.
| 1 | idx = round((1 − normLum) × (tableLength − 1)) |
Gravity Well, Luminance as typographic terrain
Image luminance is converted into a vertical displacement field. Dark regions sink away from the viewer; bright regions rise toward it. In the isometric 3D implementation, each cell's vertical position in screen space is derived from its normalised luminance scaled by a depth parameter. The photograph becomes a landscape where tonal data is encoded as elevation, a face renders as a mountain range, a sky becomes a flat plain. The typographic grid is no longer a 2D surface; it is a topographic map of the image's luminance. This algorithm is being prepared for publication in this form.
| 1 | screenY = (col + row) × cellHeight × 0.25 − (1 − normLum) × depth |
Key Product Decisions
What Almost Killed It
The first version of the flashlight effect was built as an overlay mask: a dark veil with a circular cutout that moved with the cursor. It looked like exactly what it was, a hole in a layer with no relationship to the underlying characters.
The rebuild implemented per-character illumination. Each glyph receives a brightness multiplier derived from its distance to the cursor using inverse-square falloff. Warm colour temperature is applied only to cells whose natural luminance exceeds a threshold, this prevents dark image regions, which map to ink-heavy glyphs, from being incorrectly tinted when the light passes over them.
The fix required the illumination system to understand the image's tonal structure. An effect that ignores the content it is applied to is not interactive, it is decorative. That distinction shaped every subsequent effect in the stack.