Regressive JPEGs
Have you ever found yourself looking at an image that loaded slowly, first revealing a blur, then more details, until it became sharp? That's the magic of progressive JPEG, a feature designed to improve user experience on slow connections. But what if I told you that this very functionality, when explored creatively (and a little mischievously), can hide an entire video within a single image?
How Progressive JPEGs Reveal Images (and Hide Surprises)
JPEG is an image format we've been using for decades, but not everyone knows its more sophisticated tricks. Instead of loading the image from top to bottom, pixel by pixel, progressive JPEG first displays a low-resolution version, which is gradually refined as more data arrives. This is particularly useful on slow networks, as the user gets a quick preview instead of a cropped area of the image.
This functionality is possible because the JPEG file can be divided into several "scans," each containing different frequency components. Think of it like drawing a rough sketch (low-frequency components, such as the average color of a block of pixels, or "DC") and then adding the details and contours (high-frequency components, or "AC"). The first scan usually contains the DC data for all color channels (luminance Y, and chroma Cb and Cr), providing a complete, albeit low-resolution, preview. Subsequent scans fill in the details, bringing the image to its full quality.
An engineer's curiosity, however, led to an intriguing question: what if we could manipulate these "scans" not just to refine an image, but to completely replace it? The JPEG format, in fact, allows future scans to overwrite already rendered data. The initial experiment involved concatenating multiple JPEG files of the same resolution, removing the start and end image markers. The result? When served over a slow network, the file started to "flicker" between the different images, like a rudimentary animation.
Navigating Limitations: From Curious Animations to Compact Videos
This initial approach, while fun, ran into a significant limitation: most JPEG decoders (like those found in browsers) give up after a certain number of scans (usually around 9). This is a security measure, designed to prevent issues like "zip bombs" – small files that, when processed, can consume massive resources and crash systems. This restriction prevented the creation of longer animations.
To circumvent this and create something closer to a video, the experiment's author needed to minimize the number of scans per "frame." The simplest idea would be to use "baseline" JPEGs (the standard mode, with a single scan), but they don't work for this manipulation in a progressive context. The solution was to create "progressive" JPEGs that contain only a single DC data scan (just the average color, without the AC details).
This technique works because, in progressive mode, a single scan cannot contain both AC and DC data simultaneously. The smallest possible progressive scan, therefore, is one that contains only DC data. Since DCT (Discrete Cosine Transform), the basis of JPEG compression, operates on 16x16 pixel blocks, a "DC-only" image is not a solid color, but a 1/16th resolution version of the original, like a mosaic of average color blocks.
This approach is fully compatible with the JPEG standard, and tools like jpegtran can be used to create them easily. The result? Chrome can render about 90 "frames" before giving up, and other browsers, like Firefox, have even more patience. Furthermore, this technique avoids the "ghosting" effect of the initial attempt, which occurred because AC scans are designed to refine existing data; with purely DC scans, there are no previous data to refine, just new information that replaces the old.
Where Curiosity Meets Utility (or the Lack Thereof)
The author admits that, outside of "rickrolls" and other pranks, this technique has no direct practical applications. There's no way to add timing information, for example, so playback depends entirely on network latency — which makes it unpredictable for serious use.
However, the exploration of the format yielded fun and thought-provoking results, such as a "Bad Apple!!" video rendered purely in HTML using <dialog> tags, or an interactive single-page application without CSS or JavaScript, all within the concept of partial rendering.
This experiment is a fascinating reminder that a deep mastery of a standard or file format can open doors to unexpected uses. Even if it doesn't result in a commercial product, the journey of understanding and bending the rules of a system is invaluable for any engineer or builder. It teaches us to look beyond the surface, to question assumptions, and to see "limitations" as starting points for creativity. For developers and tech leaders, it's a lesson on the importance of understanding the lower layers of infrastructure and how that understanding can be a source of innovation – or, at the very least, a lot of technical fun.
Sources
📬 Enjoyed this? Subscribe to the Vaccari's Code newsletter for the next wave of software & AI trends, straight to your inbox: Subscribe here