Where’s the computer? Where’s not the human?

Some in the audience of our recent concert commented on the questionnaire we administered that it was hard to hear where the computer ended and the human began:

  • “I felt that the obfuscation added by introducing human development made it difficult to assess the AI.”
  • “In most pieces, it was hard to understand where the AI was used.”
  • “It was surprising that the use of AI is not blatant.”
  • “I enjoyed the difficulty in working out what was AI and what was human. The pieces with more liberal human development became very listenable.”

We wanted to focus our concert on the partnership of the computer and human in music making — in the composition, in the performance, and in the listening. Some pieces involve little human involvement, e.g., folk-rnn melodies harmonised by DeepBach and played by an organist. Other pieces involve a lot of human involvement, e.g., the iterative construction of “Bastard Tunes” by Oded Ben-Tal from seeding folk-rnn differently and sampling at different temperatures. Above all, we wanted our concert to engage listeners with the idea of computer systems as collaborators in human music making. That many mentioned that is was hard to identify the roles of the two, or that perhaps the computer played a small role, is a sign, to me, of success.

In discussing our folk-rnn system, we try to remain consistent that it has not learned to compose music. It is merely generating symbol sequences according to a conditional probability model that it has learned from a bunch of crowd-sourced sequences. These sequences, generated or not, can then be transformed into music by people who know how to interpret the symbols and their context. So, the human is absolutely essential to the music making, both in composition and performance but also listening.

In each of the folk-rnn compositions played at the concert, we can identify where and how the system contributed. Sometimes this requires a post hoc untangling of a variety of threads because the compositional process is often not direct and linear. But it can be done — and will be the subject of some upcoming presentations and papers.

So, how much has the human added to the computer-generated sequences. I constructed the following comparison to help understand this. Oded found at random among the 35,809 synthesised folk-rnn tunes at The Endless Traditional Music Session the following tune:

T: Fortootuise Pollo, The
M: 3/4
L: 1/8
K: Dmaj
EF | G2- GA BG | A4- AG | A2 G2 E2 | D3- D3|
A,2 A,E A,2 | A,2 AB- BA | G2 G2 cA | D2 G2 EC | F3 G EF | E2 F2 G2 | A4 A2 | G2 D4 |\
C2 EC EC- | C2 E3 E | F2 GF- FE | F2 G2 F2 | E3 G cE | DE FG AB | c3 d cB | A4 ||
A2 |cG EG cG | d2 DD Fd | cA GA BG | AB AG FE |
d2 ed cB | cB AG EF | G2 FE D2 | C4 :|
|: AB |cB AG EC | D2 CD EF | G2 B2 G2 | G4 AB |
c2 cd ec | B2 AG A2 | FG GF ED | D4 :|
|: c2 |B2 AG AB | AA BB AG | GF- FA BF | E2 ED ED |
B2 BA Bc | B2 Bc dB | A2 FD FA | G2 E2- EF |
GF ED EG | F/E/D AD FD | EF GA FG | A2 G2 A/B/A |
GE ED A,A, | _B,E EG GF | EF DE FE | D6 ||

This sequence was generated by our first generation of folk-rnn, which was essentially char-rnn applied to a large textfile of transcriptions from thesession.org.

Below I notate the first long phrase. Oded says it has “a nice balance of nice and strange.” It’s kind of a poor piece with a few aimless sections. It is definitely not a tune one would hear in a session. But it stays within my “Chicken”-themed style.


Let’s have a listen to The Fortootuise Pollo (the computer generated the name) played by a synthetic piano.

Here is a score of my arrangement. I use what the computer system has created as a skeleton, but take several liberties. For instance, I make a first ending that is a deceptive cadence. This freshens the second repetition with a satisfying resolution. I add fermatas, break up the rhythm a bit, move portions of the melody between the different voices, and make a percussion line that adds to the medieval flavour.

The result is quite a bit different, which is not surprising, but the melody stays more or less the same (the top line of the score shows the original melody). This is in-line with our goals of using computer systems to augment one’s creative explorations. I certainly wasn’t intending to write a medieval-sounding piece when I first encountered The Fortootuise Pollo, but write one I did.

Now, can I do the same just as easily with a tune generated by the magenta system that we have trained on the same material as folk-rnn? Our preliminary experiments with the magenta system — which only differs from folk-rnn by its music representation — suggests that it is much harder to find useful material, but I’m going to give it a try.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s