Paper of the Day (Po’D): SOM Based Artistic Styles Visualization

Hello, and welcome to the Paper of the Day (Po’D): SOM Based Artistic Styles Visualization edition. Today’s paper comes straight from ICME2013: Y. Wang and M. Takatsuka, “SOM Based Artistic Styles Visualization”, in Proc. ICME, July 2013. My one-line description of this work is:

Extract features from digital images of paintings, arrange them by self-organized map, show that paintings with similar artistic styles cluster together.

The authors take several images of paintings by six artists (Gris, Braque, Raphael, Titian, Van Gogh, Gauguin, all available here. Looking through the data, I find at least three replicas), extract from them 37 numerical features related to color (temperature, weight, expressiveness), composition (rule of thirds, golden section) and lines (“different types of straight lines”), and then cluster them using the self-organizing map. They analyze the results and make several conclusions. For the paintings of the six artists, some results are shown below.

SOM.jpg
One conclusion is:

We can see that painting collections of the same art movements are much more similar to painting collections of different art movements.

I think this is supposed to be: “a painting in an art movement is more similar to other paintings in the same art movement, than paintings in other art movements.” This is a good sanity check, but the observation does not prove anything. What is needed is to answer why the feature vectors of the paintings have arranged themselves so. Is it due to the artistic style of the paintings, or something irrelevant?

Among three art movements, postimpressionism is closer to renaissance than cubism in feature space. This could be explained by different time periods of three art movements (renaissance being the earliest, cubism being the latest). Also cubism, different from postimpressionism and renaissance, is a form of abstract art which does not directly depict reality.

None of these conclusions (and many others in the paper) follow from the evaluation because the experiment provides no proof that artistic style is the cause of the results. Particularly troubling to me are the following. First, a centered, cropped, and right-sided image of the entire painting is necessary to compute the compositional features used in the paper. Does this mean that a detail of a painting using the impressionist style is not impressionist? Does this mean that a sideways image of a cubist painting is not cubist? Shouldn’t stylistic features be invariant to irrelevant transformations? Is style a property only of an entire painting? Second, since several features rely on color, the clustering results could be quite sensitive to changes in the illumination of the paintings, digital processing of the colors of the images, and so on. Is a black and white image of a cubist painting less identifiable as cubist? Is style something that ceases to exist then?
Third, since all the images are downloaded from museum websites, it could be that all digital images of paintings by Van Gogh and Gauguin are taken by the same photographer, or processed by the same software, etc. Hence, the clustering results in the image above could be museum/software-related, and not style-related.

In summary, I am highly suspicious that the results in this paper are not at all related to style. A reproduction of the evaluation, and a deeper analysis of the results, are necessary. For instance, how do things change when images become gray-scale? Does an artificially-generated image smack in the middle of the cubist cluster look cubist? Where does an image like this lie, and why? If we blur the image of a cubist painting, does it move closer to impressionism?

If the problem is to be addressed, much better evaluation is needed.

Advertisements

2 thoughts on “Paper of the Day (Po’D): SOM Based Artistic Styles Visualization

  1. Hi,
    Thanks for comments. Could you please explain a bit more on “First, a centered, cropped, and right-sided image of the entire painting is necessary to compute the compositional features used in the paper.” ?
    The composition features are composed as follows:
    1. average saliency value in each of the nine sections following “rule of thirds”
    2. Aspect ratio of the painting
    3. Extract regions that have saliency value > 0.5 ( these regions could be separate and concave. I refer these regions as “golden section” but it is actually different from conventional definition of golden section). Calculate Size, symmetricity, rectangularity of these regions.
    I don’t really get what do you say “a centered, cropped, and right-sided image of the entire painting is necessary to compute the compositional features used in the paper.”

    Like

  2. Hi Florence! Hope you made it back to Australia in fine shape. Thanks for the extra details.
    What I meant by “centered, cropped, and right-sided image of the entire painting” is that a digital image being processed by the system must show the entire painting and only the painting (no frame and no detail, for instance), and that it must be oriented the correct way (not rotated 90 degrees, for instance). If we zoom in on a painting, the features will be quite different, though the style will not be.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s