Final OPF results

An SQL error has kept me locked out of my blog for the past month. Now, I can finally post the final results of my experiments with OPF. Previously, I discussed how my reproduction of the optimum path forest approach to music genre recognition does not generate results near those reported, until I train and test with the same dataset. I have now run the same experiment, but used a partitioning of GTZAN that considers the duplication of artists, and its faults. (My work on GTZAN is now available at arxiv.) I predicted before the classification accuracy to drop “from 74 to at least 55”. Let’s see how I did!

First we look at the classification of all 23 ms segments. Quite poor in all degrees.
(True classes are columns. In percentages, precision is last column on right, F-score is last row, recalls along diagonal, and accuracy is bottom right corner.)

Now, we apply a majority vote to classify excerpts,
which produces much better results than before, but 7 points worse than 55 I predicted.

Finally, when we take into consideration the mislabelings in GTZAN, the accuracy drops a few more points.

So, the performance of OPF in GTZAN has gone from the 99.8% published in ISMIR2011 (which comes from testing on the training data), to 76% without using artist and fault filtering, to 47% with artist and fault filtering, finally to 45% taking into consideration the mislabelings in GTZAN.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s