Synth linked to mouse movements

Pad

Hi,

I would like to build an experiment where the movements produce sounds.The frequency of the sound would vary according to vertical movements and some other dimension (maybe the volume ? If anyone has a better idea i'm listening) would vary according to horizontal movements.

Here is my first try :

from __future__ import division
from openexp.synth import synth
my_mouse = mouse()
my_mouse.show_cursor(show=True)
my_canvas = canvas()
start = clock.time()
while clock.time()-start <=10000:
        button, position, timestamp = my_mouse.get_click(timeout=20)
        while my_mouse.get_pressed() == (1,0,0):
            (x,y), time = my_mouse.get_pos()
            # freq from 100 to 900
            freq = (800 - (y + 400)) + 100
            # volume from 0.2 to 2.2
            volume = 0.2 + (x + 640)/320
            my_synth = synth(exp, freq=freq, length=50, attack=10)
            my_synth.volume = volume
            my_synth.stop()
            my_synth.play()
            #self.sleep(50)

Not very pleasant for the ears. Does anyone has an idea to make this better ?

sebastiaan

Hi,

That's an interesting concept. I'm guessing that you'd ideally have a continuous sound that smoothly changes in pitch and volume as a function of mouse position. Is that right? If so, then the synth is not going to help you, because it will only produce fixed-length sounds that do not smoothly transition from one sound into the other.

Instead, my first thought would be to use PyAudio in combination with a callback function to (semi)continuously generate a sine wave of which the pitch and volume is based on the mouse coordinates. Does that make sense?

• https://people.csail.mit.edu/hubert/pyaudio/docs/#example-callback-mode-audio-i-o

Callback programming is tricky. But it's possible.

Cheers!
Sebastiaan