Here is my late breaking demo for ISMIR2015: We find several significant problems in the Latin Music Database (LMD):
- more than 6.8% of its tracks are replicated (exact or with minor changes to recording playback speed);
- there are tracks that have a large amount of speech (e.g., live concert setting);
- the spectral signatures of Gaucha appear distinct from those in other classes (leading to a possibility of confounding);
- the use of LMD in the MIREX Audio Latin Genre Classification train/test task (ALGC) appears ambiguous and flawed.
I stress (here but not in the short paper): the biggest problem is not that there are replicas, not that the ALGC metrics are ambiguous, and not that artist filtering is not really used. Instead, the experiment itself as designed does not address the stated motivation of the task:
[This task] requires algorithms to classify music audio according to the genre of the track.
For examples of this, see our experiments with the winning system of ALGC 2013.
- B. L. Sturm, C. Kereliuk, and A. Pikrakis, “A closer look at deep learning neural networks with low-level spectral periodicity features,” in Proc. Int. Workshop on Cognitive Info. Process., pp. 1–6, 2014.
- B. L. Sturm, C. Kereliuk, and J. Larsen, “¿El Caballo Viejo? Latin genre recognition with deep learning and spectral periodicity,” in Proc. Int. Conf. on Mathematics and Computation in Music, vol. Lecture Notes in Computer Science Volume 9110, pp. 335–346, 2015.
- B. L. Sturm, “The “horse” inside: Seeking causes of the behaviours of music content analysis systems,” ACM Computers in Entertainment (accepted).
Now it’s time for some of my favorite Salsa! :)