CONTENTS

Home
Updates
Software
Electronics
Android / iOS
Videos
Music
Resume
Contact



HTTPS VERSION


MIDI Guitar (Frequency Analysis)

Posted: February 14, 2015

Introduction

A long time ago I played on a guitar in a music store that had a special "midi" pickup and a midi device so as I was playing guitar, the sound coming out of the speaker was piano, violin, drums, etc. I always wanted one of those but luckily didn't have the money to waste on that.

Over the past week or so I playing around with the idea of using DCT (discrete cosine transform) to convert sound samples from an uncompressed .wav file into a list of frequencies. From those frequencies, midi notes are written to a .mid file so they can be played back as a piano. Below on this page is the source code, sound samples, and explanation of what was done.

Related Projects @mikekohn.net

Related pages on www.mikekohn.net: MIDI guitar

Example

Here's an example of me playing Jan Johansson's Polska Från Medelpad on guitar followed by the generated midi output of the same song. There are obvious errors in the midi version of the song which are explained below. Also, I tune my guitar down a 1/2 step. When I play an A here it's really an A flat.

polska_fran_medelpad.mp3 (original mp3 of guitar)
polska_dct3.mid (test_wav's .mid output from the mp3 above)
polska_dct3.mp3 (test_wav's .mid output converted to mp3)
polska_dct3.txt (test_wav's text output)

The original Jan Johansson recording can be heard here: https://www.youtube.com/watch?v=B5dzy7G1yUc

Explanation

I started by working with the MATLAB clone called Octave. Using the following piece of code I generated a wave of 440 Hz and used Octave's DCT function and plotted it:

a=cos(2*pi*440*[0:0.001:1]);
b=dct(a)
plot(b)

discrete cosine transform (DCT)

After getting a visualization of how my input is probably going to look like after the DCT, I started writing code. I originally started with flat C, but decided it might end up looking a little cleaner as C++. I used the version of DCT as described on this page: http://octave.sourceforge.net/signal/function/dct.html. Unfortunately, the complexity of this algorithm is O(N^2) so it's quite slow. There is a O(N * log(N)) way to do this called FCT (fast cosine transform) but I haven't found a nice web page describing this yet, so for now I can't do the calculations real-time. The test_wav.cxx program expects a .wav file recorded at a 44.1kHz sampling rate mono. Samples are read in 8192 at a time and run through the DCT algorithm producing 8192 DCT coefficients. Based on the values of the coefficients will tell what frequencies are present and how strong. The MidiMap.cxx module is used to figure out which notes are present by filtering on a threshold and rounding the frequency to the closest midi note. I tried first outputting the lowest note in the result from MidiMap, but then used the loudest frequency instead. I might try some kind of sliding window at a later time, but for now this is how it works. Since my input sample rate is 44100 samples a second and I'm processing 8192 samples at a time, I have the possibility of only 5.3 notes per second. If I'm playing guitar too fast it means the notes are going to get garbled.

Below is a chart of a 4 seconds of sound done on several platforms. I ran a disassembly on DCT.o to kind of get an idea what's going on and why one platform beats another. I can see on x86 and ARM platforms that SIMD is being used, but not on PPC. The two columns labaled "Lookup" are the code running through a lookup table that replaces the cos() along with a multiplication. The top number represents the entire time the program ran and the bottom number is just the time it took to compute the DCTs with the look-up table.

Platform Compiler Time (float) Time (double) Time (float)
Lookup
Time (int)
Lookup
PC
AMD FX(tm)-8320
3.5 GHz
gcc 4.8.2
SIMD used
9s 22s 1.2s
0.8s
0.8s
0.38s

PC
AMD Phenom(tm) 9850
2.5 GHz
clang 3.3
SIMD used
19s 25s 1.8s
1.06s
1.58s
0.6s

Mac
PPC 7447A (G4)
1.2 GHz
gcc 4.2.1
SIMD not used
29m 2m 20s

Playstation 3
Broadband Cell
3.0 GHz
gcc 4.1.2
SIMD not used
25m 35s 2m 31s 1m 14s
7.2s
1m 15s
8.4s

Raspberry Pi
BCM2708
700 MHz
gcc 4.6.3 43m 22s 6m 27s

Raspberry Pi 2
BCM2709
900 MHz
gcc 4.6.3
SIMD used
22m 55s 3m 15s 1m 8s
8.0s
1m 11s
11.0s

Nvidia Jetson
Tegra Tegra124 PM375 (K1)
2.3 GHz
gcc 4.8.2
SIMD used
6m 2s 2m 3s 17.4s
1.2s
17.6s
1.14s

CuBox-i
Freescale i.MX6
1.0 GHz
gcc 4.8.2
SIMD used
16m 10s 3m 19s 51s
9.0s
52s
9.6s

Source code
git clone https://github.com/mikeakohn/midi_guitar.git

Copyright 1997-2018 - Michael Kohn