Python Plays Bach

I showed this Youtube video to a respected colleague and massive Python fan a few days ago. I never expected the reaction I got.

Bach Passacaglia and Fugue in C Minor

He thought my efforts in Python produced the spectrogram which I used for the visual part of the video. Whilst I would love to take credit, they are actually produced via ffmpeg.

When I explain to him that Python made the music; he did not believe me. He said that I was 'having him on' and generally taking the piss. When I further went on to explain that the sounds are not samples, but generated from pure mathematics, the only way to get him to believe my words was to show him some Python...

On github:

1. https://github.com/nerds-central/SonicFieldRepo/blob/master/SonicField/scripts/python/Bach-Large-Organ.sy
2. https://github.com/nerds-central/SonicFieldRepo/blob/master/SonicField/scripts/python/reverberate.sy

The programming model is that Python control the sounds and Java does the mathematical heavy listing. For example, here is the FFT code from Sonic Field:

1. https://github.com/nerds-central/SonicFieldRepo/blob/master/SonicField/src/com/nerdscentral/audio/pitch/algorithm/FFTbase.java
2. https://github.com/nerds-central/SonicFieldRepo/blob/master/SonicField/src/com/nerdscentral/audio/pitch/algorithm/CacheableFFT.java

See here for the entire 'Art Of Fugue'

He believed me when he saw code: It has been an amazing journey to get to such a point where Sonic Field has started to sound good enough to not be 'synthy'. It am very proud to have proven it can be done, that samples are not required to make beautiful sounds. Clearly, some people will not find the sounds my code has made pleasing, some will be shocked at how synthetic they sound. But at least now we can say at Python, at least for some people, makes music.

Reverb's So Real You Cannot Hear It

If I said this myself - then I would be boasting but this is a paraphrase of what someone else said about this double reverb' effect.

What (if I remember correctly) he said was that he was expecting the effect but was confused because he did not hear it. Then he realised it must be there because the music would not sound like that if there was no reverb'; "Yes, it sounds like is it played in a room, but not like a reverb' effect".

My contention is that we have gotten very used to the sound of reverberation effects. We are used to them to such and extent that we expect that artificial sound of early reverberation units and forget that the original idea was to mimic real spaces. Indeed, even impulse response reverberation does not sound real.

How can impulse response reverberation not sound real?

I have a theory as to how this happens. Consider that you are going to make an impulse response reverberation for cathedral. One way of doing this is to set up your equipment in the nave or choir and send out your frequency sweeps. You measure the returned sound and compute the impulse response.

What actually happens when music is created in such a space is very different. The sound is typically made near a wall and other reflective surfaces and then heard by the listener some distance away. My guess is that the upper frequencies of the early reflections are more highlighted this way. Indeed, some early reflections will be sound bright they will interfere with the original signal and make the upper frequencies dance.

What could be more this way than a pipe organ in a church or cathedral? The instrument is always near a wall; what is more that pipes themselves are reflectors of other pipes and the instrument is highly distributed through space. Somehow we must make the sound shine an vibrate (emotive words, not technical) in a way that a simple impulse response cannot.

It try and mimc the effect of listening to a large organ in a large space I have worked on this patch. As Joerg Sprave would say - "let me show you its features"

def excite(sig_,mix,power):

    def exciteInner():

        sig=sig_

        m=sf.Magnitude(+sig)

        sigh=sf.BesselHighPass(+sig,500,2)

        mh=sf.Magnitude(+sigh)

        sigh=sf.Power(sigh,power)

        sigh=sf.BesselHighPass(sigh,1000,2)

        nh=sf.Magnitude(+sigh)

        sigh=sf.NumericVolume(sigh,mh/nh)

        sig=sf.Mix(sf.NumericVolume(sigh,mix),sf.NumericVolume(sig,1.0-mix))

        n=sf.Magnitude(+sig)

        return sf.Realise(sf.NumericVolume(sig,m/n))

    return sf_do(exciteInner)

    

(convoll,convolr)=sf.ReadFile("temp/bh-l.wav")

(convorl,convorr)=sf.ReadFile("temp/bh-r.wav")

convoll=excite(convoll,0.75,2.0)

convolr=excite(convolr,0.75,2.0)

convorl=excite(convorl,0.75,2.0)

convorr=excite(convorr,0.75,2.0)

ll  = reverberate(+left ,convoll)

lr  = reverberate(+left ,convolr)

rl  = reverberate(+right,convorl)

rr  = reverberate(+right,convorr)

wleft =sf.FixSize(sf.Mix(ll,rl))

wright=sf.FixSize(sf.Mix(rr,lr))

wright = excite(wright,0.15,1.11)

wleft  = excite(wleft ,0.15,1.11)

sf.WriteFile32((sf.FixSize(+wleft),sf.FixSize(+wright)),"temp/wet.wav")

wleft =sf.FixSize(sf.Mix(sf.Pcnt15(+left),sf.Pcnt85(wleft)))

wright =sf.FixSize(sf.Mix(sf.Pcnt15(+right),sf.Pcnt85(wright)))

sf.WriteFile32((+wleft,+wright),"temp/mix.wav")

(convoll,convolr)=sf.ReadFile("temp/v-grand-l.wav")

(convorl,convorr)=sf.ReadFile("temp/v-grand-r.wav")

left  = sf.BesselLowPass(left  ,392,1)

right = sf.BesselLowPass(right,392,1)

ll  = reverberate(+left ,convoll)

lr  = reverberate( left ,convolr)

rl  = reverberate(+right,convorl)

rr  = reverberate( right,convorr)

vwleft =sf.FixSize(sf.Mix(ll,rl))

vwright=sf.FixSize(sf.Mix(rr,lr))

wleft =sf.FixSize(sf.Mix(wleft ,sf.Pcnt20(vwleft )))

wright=sf.FixSize(sf.Mix(wright,sf.Pcnt20(vwright)))

sf.WriteFile32((wleft,wright),"temp/grand.wav")

First notice the simple harmonic exciter. I have discussed this elsewhere. However, the first trick to getting a bright reverb' is to excite the initial impulse response:

(convoll,convolr)=sf.ReadFile("temp/bh-l.wav")

(convorl,convorr)=sf.ReadFile("temp/bh-r.wav")

convoll=excite(convoll,0.75,2.0)

convolr=excite(convolr,0.75,2.0)

convorl=excite(convorl,0.75,2.0)

convorr=excite(convorr,0.75,2.0)

Then the resulting 'wet' (i.e. reverberated) signal is excited again (just a little):

ll  = reverberate(+left ,convoll)

lr  = reverberate(+left ,convolr)

rl  = reverberate(+right,convorl)

rr  = reverberate(+right,convorr)

wleft =sf.FixSize(sf.Mix(ll,rl))

wright=sf.FixSize(sf.Mix(rr,lr))

wright = excite(wright,0.15,1.11)

wleft  = excite(wleft ,0.15,1.11)

This makes a bright, vibrant sound. The impulse I am using is a lot like a large concert hall. With excitation effects added it starts to sound very bright and close and looses some of its size. I guess it is a bit like an ornate church.

I wanted to add much more size. To do this, I used my imagination rather than just mathematics. Consider listening to a organ with the nave of a huge cathedral behind you and the aisles to the sides. The bright sound will come from the organ and its surround but, somewhat delayed and at lower frequencies, will come the long, slow reverberation from those huge stone enclosed spaces. To get that effect I created a completely separate reverberation with a much longer, denser reverberation which and pre-filtered the signal to remove some of the high frequencies.

(convoll,convolr)=sf.ReadFile("temp/v-grand-l.wav")

(convorl,convorr)=sf.ReadFile("temp/v-grand-r.wav")

left  = sf.BesselLowPass(left  ,392,1)

right = sf.BesselLowPass(right,392,1)

ll  = reverberate(+left ,convoll)

lr  = reverberate( left ,convolr)

rl  = reverberate(+right,convorl)

rr  = reverberate( right,convorr)

vwleft =sf.FixSize(sf.Mix(ll,rl))

vwright=sf.FixSize(sf.Mix(rr,lr))

wleft =sf.FixSize(sf.Mix(wleft ,sf.Pcnt20(vwleft )))

wright=sf.FixSize(sf.Mix(wright,sf.Pcnt20(vwright)))

sf.WriteFile32((wleft,wright),"temp/grand.wav")

It is the mixture of a little of this very long low reverberation with the bright faster reverberation which makes the final result.

Here is an example:

Ethereal Reflections

Reverberation Like Effects With Enveloped White Noise 

About half way through The Chase there is a pause where the sound goes very much more quiet and distant. This was a pure 'I wonder if that will work' moment for me. The effect was done not by a change in synthesis parameters, or adding an effect in the traditional sense - it the sound is nothing but an effect. It was done using impulse response reverberation using an white noise envelope.

convoll=sf.Multiply(sf.WhiteNoise(beat*4.0),sf.NumericShape((0,0),(beat*0.5,1),(beat,0.1),(beat*4,0)))
convolr=sf.Multiply(sf.WhiteNoise(beat*4.0),sf.NumericShape((0,0),(beat*0.5,1),(beat,0.1),(beat*4,0)))
left13 =reverberate(+left12,convoll)
right13=reverberate(+right12,convolr)

The Chase:

Normally we think of capturing an impulse response. We do this from recording in a space (an abandoned factory for instance) or from the output of a classic reverberation unit. How about designing one from scratch to produce exactly the effect we want?

I have continued with these ideas. Maybe the most extreme example is the use of 'diamond shaped' white noise signals in convolution for the stretch effect in Invasion:

rLen=20480
ns=sf.NumericShape((0,0),(rLen/2,1),(rLen,0))
convoll=sf.Multiply(+ns,sf.WhiteNoise(rLen))
convolr=sf.Multiply( ns,sf.WhiteNoise(rLen))

wleft =reverberate(left,convoll)
wright=reverberate(right,convolr)

Invasion: