Sampler-like musical instrument using unsupervised learning

and dimensionality reduction methods

Leandro Garber
Universidad Nacional de Tres de Febrero, Buenos Aires, Argentina
CIIPME
CIIPME-CONICET, Buenos Aires, Argentina
leandrogarber@gmail.com

The connections between artificial intelligence techniques and art have been growing in the recent years, in part due to the efforts of Google's

Magenta project and the ISMIR community in the case of music. Influenced by this movement, we present a work in progress sampler-like

musical instrument that uses machine learning tools. The pipeline works as follows:

Pipeline

Audio
Files Preprocess Principal t-SNE DBScan
Component (metric: cosine
Analysis perp: 30
lr: 200
(99% variability n_iter: 1000)
explained)

All audio converted to Mono, WAV, 22050Hz, 16bits

STFT from the first second of audio (truncated or zero-padded), 44 hann windows ~23ms,

1025 frequency bands, ~11hz bandwidth

Resulting matrix is vectorized in order to get one vector for each audio

Results

Using a dataset with 2128 files containing sounds from

well-known drum machines from the 80s and 90s

Cymbals
this pipeline makes it possible to visualize the Bells, calimbas,
high pitched sounds
latent 2D structure unveiling different groups

of similar sounding files.

Pluck sounds
We are developing AudioStellar, a

software for musicians that allows to

Bassdrums
explore and play given samples in a

novel way using various innovative

music composition modes such as

random walks, defining playable paths

Snares
in latent space and using distance as

rhythm.
Bassdrum with notable attack ("click")

Future work

Create a rich audio dataset with user-annotated tags

from professional musicians and develop an experiment to

allow the comparison between different pipelines using quantitative methods with metrics like trustworthiness and continuity.

Feature learning with recurrent and convolutional deep neural networks will be tested against traditional audio features and diverse visualization

methods. Current pipeline will be used as a baseline.