In early 2020 I saw a post about someone automating the act of painting a picture in Microsoft Paint. I had just found Processing, so I wrote a desktop program in Python that took a photograph and put down random brush strokes until the result looked like an expressionist painting. It took minutes to finish, and the best part was that you could sit and watch each stroke land.
painterly is that idea rebuilt properly, six years later, in a browser tab. The Python sketches are still in the repository as the origin story. What replaced them is a port of Aaron Hertzmann's Painterly Rendering with Curved Brush Strokes of Multiple Sizes (SIGGRAPH 1998): five brushes, each half the size of the last. Every brush blurs the photo to its own scale, measures where the canvas still disagrees with it, and puts a stroke down only there. The likeness converges rather than filling in at random. And each step of a stroke moves perpendicular to the image gradient, which is the whole difference between brushwork and a smear filter — the strokes follow fur, edges, and the line of a roof because the picture tells them where to go.
Everything is seeded, so a photo and a seed always produce the same painting. That one decision is what makes the rest possible: the timeline can be moved to any moment, and the tests can compare pixels.
The engineering I'm most pleased with is the part you never see. Strokes go down with alpha over one another and cannot be lifted, so there is no playing backwards — which means the transport is a DVD player's, not a scrubber: back a brush, play, on a brush. The player keeps a copy of the canvas at each handover and never spends more than a fraction of a frame painting, so a jump costs a few frames of catching up instead of freezing the page. Planning runs on a worker and hands the plan back as packed typed arrays — 15 MB transferred rather than 110 MB of objects cloned — and the photos you haven't picked are planned quietly in the background so choosing one starts immediately.
My favourite thing about it was an accident. Early on I watched the three-second frame where only the first, biggest brush had run, and preferred it to the finished painting: a few hundred ribbons of colour, unmistakably the photograph and nothing like it. That frame became a style of its own. It was meant as a stage on the way to a painting and turned out to be the picture people liked.
You can watch it paint with your own photo — it never leaves the tab — or read the source on GitHub, where the 2020 Processing sketches are still sitting in processing/.