The Mechanical Painter is an experimental composite system that utilizes modern image processing techniques to imitate a true painterly style conversion of standard bitmap images like pictures into digital paintings.

I first had the idea for this process after studying the works of Monet and Van Gogh. Standing in a museum looking at one of Monet's works, I imagined breaking the painting down into the requisite steps of layering required to produce the final product. This involves blocking out the scene, filling spaces, and edging at increasing levels of detail. The variance in style coming from how the brush strokes and palette is delivered. Suffice to say, the problem was an intriguing one to me.

I started with a basic Kuwahara algorithm. This filter was originally used for medical image analysis, but now is used in a number of artistic applications and machine learning tasks. Though for my purposes, the end result produced something that while by definition was "painterly" did not embody what I was looking for.

Soon, I discovered numerous research papers with Aaron Hertzmann as a contributor. Many of which, suggested modifications to the classic Kuwahara algorithm and presented additional observations into the process of converting digital images into something that really reads as a painting.

After I collated this research, it was clear to me that many of the papers pertaining to my end goal did not necessarily touch current edge level systems. This includes things like single image depth estimation, tunable facial recognition, single image shape segmentation, and other systems that, today, can be run on a consumer GPU but were pure fantasy when the papers were written.

My approach to painterly conversion takes these modern image analysis and uses them to produce a novel guidance matrix for placing strokes onto a canvas. From there, a Kuwahara filtered reference is created through a modified, paint stroke focused implementation. Then, strokes are placed along a weight ladder guided by the matrix as such to retain curvature, shading, and edges. The final result appearing as a true painting.

More insight into the system can be found in the other artifacts of this exhibit.

This is the output painted image resulting from a run on one of my photographs. Note the strokes and general sense of progression to the way image is built up from the canvas. This effect is what the Mechanical Painter project set out to achieve.

The image on the left is the original photograph as I took it. The image on the right is the output painting on the "medium" conversion preset.

Note how the texture of the concrete has been smoothed and the lower leaves appear to have been painted with a lower number of smaller strokes than the main flower. This is an effect of the novel composite stroke guidance matrix.

Depicted here is the step progression of painting an image using the standard "medium" preset.

The order is as follows:

1. Blank canvas is created.

2. Modified Kuwahara filter is run on the original image.

3. Segmented shapes are blocked out.

4. Corse large brush strokes are added for fill.

5. Smaller brush strokes are added for fill.

6. Detail brush strokes are added for fill.

7. Smaller brush strokes are added for edging.

8. Detail brush strokes are added for edging.

9. The output painting is produced.