Hello, and welcome to the Paper of the Day (Po’D): SOM Based Artistic Styles Visualization edition. Today’s paper comes straight from ICME2013: Y. Wang and M. Takatsuka, “SOM Based Artistic Styles Visualization”, in Proc. ICME, July 2013. My one-line description of this work is:
Extract features from digital images of paintings, arrange them by self-organized map, show that paintings with similar artistic styles cluster together.
The authors take several images of paintings by six artists (Gris, Braque, Raphael, Titian, Van Gogh, Gauguin, all available here. Looking through the data, I find at least three replicas), extract from them 37 numerical features related to color (temperature, weight, expressiveness), composition (rule of thirds, golden section) and lines (“different types of straight lines”), and then cluster them using the self-organizing map. They analyze the results and make several conclusions. For the paintings of the six artists, some results are shown below.
We can see that painting collections of the same art movements are much more similar to painting collections of different art movements.
I think this is supposed to be: “a painting in an art movement is more similar to other paintings in the same art movement, than paintings in other art movements.” This is a good sanity check, but the observation does not prove anything. What is needed is to answer why the feature vectors of the paintings have arranged themselves so. Is it due to the artistic style of the paintings, or something irrelevant?
Among three art movements, postimpressionism is closer to renaissance than cubism in feature space. This could be explained by different time periods of three art movements (renaissance being the earliest, cubism being the latest). Also cubism, different from postimpressionism and renaissance, is a form of abstract art which does not directly depict reality.
None of these conclusions (and many others in the paper) follow from the evaluation because the experiment provides no proof that artistic style is the cause of the results. Particularly troubling to me are the following. First, a centered, cropped, and right-sided image of the entire painting is necessary to compute the compositional features used in the paper. Does this mean that a detail of a painting using the impressionist style is not impressionist? Does this mean that a sideways image of a cubist painting is not cubist? Shouldn’t stylistic features be invariant to irrelevant transformations? Is style a property only of an entire painting? Second, since several features rely on color, the clustering results could be quite sensitive to changes in the illumination of the paintings, digital processing of the colors of the images, and so on. Is a black and white image of a cubist painting less identifiable as cubist? Is style something that ceases to exist then?
Third, since all the images are downloaded from museum websites, it could be that all digital images of paintings by Van Gogh and Gauguin are taken by the same photographer, or processed by the same software, etc. Hence, the clustering results in the image above could be museum/software-related, and not style-related.
In summary, I am highly suspicious that the results in this paper are not at all related to style. A reproduction of the evaluation, and a deeper analysis of the results, are necessary. For instance, how do things change when images become gray-scale? Does an artificially-generated image smack in the middle of the cubist cluster look cubist? Where does an image like this lie, and why? If we blur the image of a cubist painting, does it move closer to impressionism?
If the problem is to be addressed, much better evaluation is needed.