Faults in the Latin Music Database and With its Use (ISMIR 2015 Late Breaking Demo)

Here is my late breaking demo for ISMIR2015: We find several significant problems in the Latin Music Database (LMD):

  1. more than 6.8% of its tracks are replicated (exact or with minor changes to recording playback speed);
  2. there are tracks that have a large amount of speech (e.g., live concert setting);
  3. the spectral signatures of Gaucha appear distinct from those in other classes (leading to a possibility of confounding);
  4. the use of LMD in the MIREX Audio Latin Genre Classification train/test task (ALGC) appears ambiguous and flawed.

Here is a list of the replicated tracks I find. They are more numerous than I originally found a year and a half ago.

I stress (here but not in the short paper): the biggest problem is not that there are replicas, not that the ALGC metrics are ambiguous, and not that artist filtering is not really used. Instead, the experiment itself as designed does not address the stated motivation of the task:

[This task] requires algorithms to classify music audio according to the genre of the track.

For examples of this, see our experiments with the winning system of ALGC 2013.

  1. B. L. Sturm, C. Kereliuk, and A. Pikrakis, “A closer look at deep learning neural networks with low-level spectral periodicity features,” in Proc. Int. Workshop on Cognitive Info. Process., pp. 1–6, 2014.
  2. B. L. Sturm, C. Kereliuk, and J. Larsen, “¿El Caballo Viejo? Latin genre recognition with deep learning and spectral periodicity,” in Proc. Int. Conf. on Mathematics and Computation in Music, vol. Lecture Notes in Computer Science Volume 9110, pp. 335–346, 2015.
  3. B. L. Sturm, “The “horse” inside: Seeking causes of the behaviours of music content analysis systems,” ACM Computers in Entertainment (accepted).

Now it’s time for some of my favorite Salsa! :)

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s