/
the BLUE GUITAR
Jing Yan
They said, "You have a blue guitar, You do not play things as they are." --- Wallace Stevens
Inspired by David Hockney, who is inspired by Wallace Stevens who is inspired by Pablo Picasso. The blue guitar is used as a metaphor for not only an instrument that you can play with but also a distorted and unrealistic acoustic world that you might encounter.
The “BLUE GUITAR” is a motion based spatial sound environment project. The intention is to create a potential audio environment that allows user to interact with and gradually reveals and evolves itself through interaction.During the project, I also attempt to balance the designed narrative of sound scene and the audience trigger interaction. While the story is telling during certain time sections, users can still participate in the performance of the sound.
The initial concept of the structure of sound and sound materials.
Interaction process:
The sound is triggered by the entrance of the audience. the amplitude and panning are related to the amount of users, distance to the Kinect device, position of the body part, and the position in space.
Sound Scene:
^ Opening
Imagine in a dark empty room, there is something whispering which allures you to come closer.
And the story begins.
^ Piano - instrument
The chords of piano is triggered by the appearance of the users. The closer the user and the longer it plays, more complicated the sound becomes. Interact with body gestures.
^ Sea - space
Now, you come closer to the sea. When you get higher, the body grows into the sky and you hear the gulls. When you get lower, you dive deep into the sea.
^ Memory - time
There are invisible doors on the plane, whichever you step into will lead you to a brand new space and time.
Visualization:
In contrast to the complexity and the direct control of the sound, the visualization is simplified as a wave like thing consisting of lines that is waving up and down according to the sound volume. Verlet integration is used to create a creature like movement of the entire body. Thus, the visual object is built like a deep sea creature, which gradually changing the shapes and colors due to the motion of the entire body.
https://vimeo.com/169680830
Technic:
Sound materials are controlled by minim library.
Motion detection are achieved through Kinect.
All parts are connected by Processing.
Future: I intended to build more real time synthesis sound in the supercollider, however due to some technical difficulty this real time transition from the processing to supercollider was not well built. Also the sound environment is too much realistic and there needs to have some sound improvement.
Reference:
Minim library example,Hilda's demo, Supercollider example.
the VerletBall&VerletStick class is based on Ira Greenberg's VerletStick example
(
https://github.com/irajgreenberg/worksh ... tStick.pde)
Main tab
Code: Select all
/* 2016-6-7 (Processing 3)
M265 Optical-Computational Processes:
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::::::::::::::::::: the BLUE guitar ::::::::::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::::::::: Jing YAN ::::::::::::
::::::::::::::::::: theuniqueeye@gmail.com :::::::::::::::::::::::
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::::::: [VERSION 9] ::::::::::::::::::::::::::::::::::::::::::::*/
/*
-------------------------------------------------------------------------------
They said, "You have a blue guitar, You do not play things as they are."
The “BLUE GUITAR” is a motion based spatial sound environment project.
The intention is to create a potential audio environment that allows user
to interact with and gradually reveals and evolves itself through interaction.
During the project, I also attempt to balance the designed narrative of sound
scene and the audience trigger interaction.
-------------------------------------------------------------------------------
Version History >>
[version8]
connect kinnect with skeleton detection
[version7] key interaction version
refined version6
piano chord refined
color and color transistion refined
strokewieght refined
sound cycle refined
[version6]
piano chord delay fixed
visual color transition built
total sound add to the posY
[version5]
scene setting
[version4]
add the whisper the walking step and the opening scene
[version3]
1st scene - Piano sound built
[version 2]
set up with minim sound
try different component of minim library
[version 1]
build verlet wave form with verlet integration
-------------------------------------------------------------------------------
Reference >>
minim library
http://code.compartmental.net/minim/ugen_class_ugen.html
https://forum.processing.org/one/topic/counter.html
the VerletBall&VerletStick class is based on Ira Greenberg's VerletStick example
https://github.com/irajgreenberg/workshopExamples/blob/master/apression/VerletStick.pde
Hilda's demo about the user detection
try supercollider's example to build synthesize sound
-------------------------------------------------------------------------------
Special thanks to Zhenyu and Donghao for rescuing me from brain stuck
Guide by Prof. George Legrady.
-------------------------------------------------------------------------------
*/
import peasy.*;
import peasy.org.apache.commons.math.*;
import peasy.org.apache.commons.math.geometry.*;
import peasy.test.*;
PeasyCam cam;
import SimpleOpenNI.*;
SimpleOpenNI kinect;
import processing.opengl.*;
import ddf.minim.*;
import ddf.minim.ugens.*;
import ddf.minim.spi.*;
import java.util.Iterator;
import supercollider.*;
import oscP5.*;
// sound
Minim minim;
AudioOutput out;
LiveInput in;
Oscil deepSea;
Noise theNoise;
Summer synth; // sum the outputs of multiple UGens
//Wavetable table;
//Midi2Hz midi;
//supercollider
Group group;
Synth synth0, synth1, synth2, synth3;
// sound materials
AudioPlayer whisper, step, tic; // Scene 1
AudioPlayer chord1_0, chord1_1, chord1_2, chord1_3, chord2_0, chord2_1, chord2_2, // Scene 2
chord2_3, chord3_0, chord3_1, chord3_2, chord3_3, chord4_0, chord4_1, chord4_2, chord4_3;
AudioPlayer sea, seaWave, gull_1, gull_2, wind, wave; // Scene 3
AudioPlayer bell, road, park_1, park_2, choir, walk; // Scene 4
// timer
int timerSet = 0;
float targetTime1 = 0, targetTime2 = 0, targetTime3 = 0, targetTime4 = 0;
float time_to_play_sound = 1e20;
int phase = 130;
int timestamp_1 = 5;
int timestamp_2 = 5+5;
int timestamp_3 = 5+5+10;
int timestamp_4 = 5+10+10;
int timestamp_5 = 5+10+10+5;
int timestamp_6 = 5+10+10+25;
int timestamp_7 = 5+10+10+30;
int timestamp_8 = 5+10+10+30+10;
int timestamp_9 = 5+10+10+30+25;
int timestamp_10 = 5+10+10+30+20;
int timestamp_11 = 5+10+10+30+25+40;
int timestamp_12 = 5+10+10+30+25+40+10;
// interaction switches
boolean playStep=false, playChord=false, playNoise=false, playSea=false, playMemo=false, playSynth=false;
boolean user1In, user1Hug, user2In, user2Hug, user3In, user3Hug, user4In, user4Hug;
boolean saveFrame=false, rotateX=false;
// visual
int particles = 100;
int layers = 30;
VerletBall[][] balls = new VerletBall[particles][layers];
int bonds = particles + particles/2;
VerletStick[][] sticks = new VerletStick[bonds][layers];
float tension;
float posY=0;
float trans=100, transP=100;
// kinect
PVector rightHand, leftHand, rightElbow, leftElbow, head;
boolean skeleton = true;
int[] user_list;
boolean[] handsUp = new boolean[10];
int userId;
//User location
PVector[] com = new PVector[10];
float[] peak = new float[10];
float[] rightHandy = new float[10];
//HashMap<Integer, UserTrackingInfo> user_tracking_info = new HashMap<Integer, UserTrackingInfo>();
// -------------------------------------------------------------------------------
HashMap<AudioPlayer, Float> next_play_times = new HashMap<AudioPlayer, Float>();
void setup() {
//size(1960, 1080, OPENGL);
size(1080, 720, OPENGL);
cam = new PeasyCam(this, 1000);
float theta = PI/4.0;
float jump;
tension = 0.1;
////////////// kinect setup ///////////////
kinect = new SimpleOpenNI(this);
kinect.enableDepth();
kinect.setMirror(true);
kinect.enableUser();
rightHand = new PVector(0, 0, 0);
leftHand = new PVector(0, 0, 0);
com[0] = new PVector(); // user location
com[1] = new PVector(); // user location
//coms = new PVector();
handsUp[0]=false;
///////////// visual setup /////////////////
//colorMode(HSB, 255);
colorMode(RGB, 255);
// balls + add interaction
for (int j = 0; j<layers; j++) {
jump = -400.0;
for (int i=0; i<particles; i++) {
PVector push = new PVector(0, 0, 0);
PVector p = new PVector(jump, 200, 25*j);
balls[i][j]= new VerletBall(p, push, 10);
theta += TWO_PI/particles;
jump += 8;
}
}
// sticks external
for (int j = 0; j<layers; j++) {
for (int i=0; i<particles; i++) {
if (i<particles-1) {
sticks[i][j] = new VerletStick(balls[i][j], balls[i+1][j], tension);
}
}
}
///////////// sound setup /////////////////
minim = new Minim(this);
out = minim.getLineOut(); // use the getLineOut method of the Minim object to get an AudioOutput object
synth = new Summer();
deepSea = new Oscil( 700, 0.05f, Waves.SINE );
//deepSea.patch( synth ); // patch to the output
synth.patch( out );
//// load sound materials ////
whisper = minim.loadFile("openning_3.mp3");
step = minim.loadFile("step.mp3");
tic = minim.loadFile("tic.mp3");
chord1_0 = minim.loadFile("chord1_0.mp3");
chord1_1 = minim.loadFile("chord1_1.mp3");
chord1_2 = minim.loadFile("chord1_2.mp3");
chord1_3 = minim.loadFile("chord1_3.mp3");
chord2_0 = minim.loadFile("chord2_0.mp3");
chord2_1 = minim.loadFile("chord2_1.mp3");
chord2_2 = minim.loadFile("chord2_2.mp3");
chord2_3 = minim.loadFile("chord2_3.mp3");
chord3_0 = minim.loadFile("chord3_0.mp3");
chord3_1 = minim.loadFile("chord3_1.mp3");
chord3_2 = minim.loadFile("chord3_2.mp3");
chord3_3 = minim.loadFile("chord3_3.mp3");
chord4_0 = minim.loadFile("chord4_0.mp3");
chord4_1 = minim.loadFile("chord4_1.mp3");
chord4_2 = minim.loadFile("chord4_2.mp3");
chord4_3 = minim.loadFile("chord4_3.mp3");
sea = minim.loadFile("sea.wav");
seaWave = minim.loadFile("seaWave.mp3");
wave = minim.loadFile("wave2.mp3");
gull_1 = minim.loadFile("gull.mp3");
gull_2 = minim.loadFile("gull_2.mp3");
wind = minim.loadFile("wind_2.wav");
bell = minim.loadFile("bell.mp3");
road = minim.loadFile("road.mp3");
park_1 = minim.loadFile("park_1.mp3");
park_2 = minim.loadFile("park_2.wav");
choir = minim.loadFile("choir.wav");
walk = minim.loadFile("walk.mp3");
/* try connect to supercollider
group = new Group();
group.create();
// uses default sc server at 127.0.0.1:57110
// does NOT create synth!
synth0 = new Synth("sine");
synth1 = new Synth("DecaySin");
synth2 = new Synth("DecayPink");
synth3 = new Synth("Reverb");
// set initial arguments
synth0.set("amp", 0.5);
synth0.set("freq", 80);
synth1.set("amp", 0.5);
synth1.set("freq", 80);
// synth2.set("amp", 0.5);
// synth2.set("freq", 80);
synth3.set("amp", 0.5);
synth3.set("freq", 80);
*/
}
// -------------------------------------------------------------------------------
void draw() {
// set the stage
background(0);
///////////// kinect detect /////////////////
kinect.update();
user_list = kinect.getUsers();
//user__________________________________________
if (user_list.length > 0) {
for (int numbOfUsers = 0; numbOfUsers<user_list.length; numbOfUsers++) {
userId = user_list[numbOfUsers];
try {
kinect.getCoM(user_list[numbOfUsers], com[numbOfUsers]);
}
catch(Exception e) {
}
// if we’re successfully calibrated
if ( kinect.isTrackingSkeleton(userId)) {
if (skeleton) { // draw user
pushMatrix();
translate(0, 0, -2500);
rotateX(radians(180));
//drawSkeleton(userId);
popMatrix();
}
rightHand = new PVector();
kinect.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_RIGHT_HAND, rightHand);
leftHand = new PVector();
kinect.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_LEFT_HAND, leftHand);
rightElbow = new PVector();
kinect.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_RIGHT_ELBOW, rightElbow);
leftElbow = new PVector();
kinect.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_LEFT_ELBOW, leftElbow);
head = new PVector();
kinect.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_HEAD, head);
if ((rightHand.y > rightElbow.y && rightHand.x > rightElbow.x) && (leftHand.y > leftElbow.y
&& leftHand.x > leftElbow.x)) handsUp[numbOfUsers]=true;
else handsUp[numbOfUsers]=false;
peak[numbOfUsers]=max(rightHand.y, leftHand.y, head.y);
rightHandy[numbOfUsers]=rightHand.y;
}
}
}
///////////// timer set /////////////////
timerSet = -11+int(millis()/1000)%(130); // total phase: 3 Min
textSize(40);
//text (timerSet, 0, -500, 0);
///////////// visual display /////////////////
for (int j = 0; j<layers; j++) {
for (int i=0; i<particles-1; i++) {
sticks[i][j].constrainLen();
}
for (int i=0; i<particles/2; i++) {
sticks[i][j].render(0.4+map(i, 0, particles/2-1, 0, 2));
}
for (int i=particles/2; i<particles-1; i++) {
sticks[i][j].render(0.4+map(i, particles/2, particles-2, 2, 0));
}
}
for (int j = 0; j<layers; j++) {
strokeWeight(4);
stroke(255, trans);
balls[particles/2][j].render();
stroke(#68EBE3, trans);
strokeWeight(3.5);
balls[particles/2-1][j].render();
stroke(#C3B4D3, trans);
balls[particles/2+1][j].render();
for (int i=0; i<particles; i++) {
if (i!=0&&i!=particles-1)
balls[i][j].verlet();
noFill();
// draw balls
if (i!=particles/2) { //&&i!=(particles/2+1)&&i!=(particles/2-1)
//transP=lerp(transP, trans, 0.1);
//stroke(#1F6B6F, transP);
//pushMatrix();
//translate(0, -100, 0 );
//balls[i][j].render();
//popMatrix();
}
}
}
for (int l = 0; l<layers; l++) {
for (int i = 0; i < layers; i++)
{
posY =whisper.mix.get(i);
posY +=step.mix.get(i);
posY +=tic.mix.get(i);
posY += chord1_0.mix.get(i);
posY += chord1_1.mix.get(i);
posY +=chord1_2.mix.get(i);
posY +=chord1_3.mix.get(i);
posY +=chord2_0.mix.get(i);
posY +=chord2_1.mix.get(i);
posY +=chord2_2.mix.get(i);
posY +=chord2_3.mix.get(i);
posY +=chord3_0.mix.get(i);
posY +=chord3_1.mix.get(i);
posY +=chord3_2.mix.get(i);
posY +=chord3_3.mix.get(i);
posY +=chord4_0.mix.get(i);
posY +=chord4_1.mix.get(i);
posY +=chord4_2.mix.get(i);
posY +=chord4_3.mix.get(i);
posY +=sea.mix.get(i);
posY +=seaWave.mix.get(i);
posY +=gull_1.mix.get(i);
posY +=gull_2.mix.get(i);
posY +=bell.mix.get(i);
posY +=road.mix.get(i);
posY +=park_1.mix.get(i);
posY +=park_2.mix.get(i);
posY +=choir.mix.get(i);
posY +=walk.mix.get(i);
//if(handsUp[0])
posY += lerp(0, -rightHandy[0]/700, -2);
trans = abs((int)map(posY*10000, -80, 80, 0, 255));
balls[particles/2][l].update(balls[particles/2][l].pos.x, posY*100+50);
}
}
///////////// interaction /////////////////
if (key == 's')
saveFrame("frame/######.png");
if (rotateX) cam.rotateY((PI/180)*.05);
translate(0, -com[0].z/100, 0);
theNoise = new Noise( 0.01f, Noise.Tint.RED );
scene();
if (playChord) pianoChord();
if (playMemo) memo();
// if (playNoise) theNoise.patch(synth);
// else theNoise.unpatch(synth);
}
// -------------------------------------------------------------------------------
void keyPressed() {
if (key == 'p')
//println("userList: "+userList);
//println("CamX "+cam.getPosition()[0]+" CamY "+cam.getPosition()[1]+" CamZ "+cam.getPosition()[2]);
if (key == 's')
saveFrame=!saveFrame;
if (key == ' ' )
rotateX=!rotateX;
}
kinect tab
Code: Select all
void drawSkeleton(int userId)
{
strokeWeight(3);
// to get the 3d joint data
drawLimb(userId, SimpleOpenNI.SKEL_HEAD, SimpleOpenNI.SKEL_NECK);
drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_LEFT_SHOULDER);
drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_LEFT_ELBOW);
drawLimb(userId, SimpleOpenNI.SKEL_LEFT_ELBOW, SimpleOpenNI.SKEL_LEFT_HAND);
drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_RIGHT_SHOULDER);
drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_RIGHT_ELBOW);
drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_ELBOW, SimpleOpenNI.SKEL_RIGHT_HAND);
drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_TORSO);
drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_TORSO);
drawLimb(userId, SimpleOpenNI.SKEL_TORSO, SimpleOpenNI.SKEL_LEFT_HIP);
drawLimb(userId, SimpleOpenNI.SKEL_LEFT_HIP, SimpleOpenNI.SKEL_LEFT_KNEE);
drawLimb(userId, SimpleOpenNI.SKEL_LEFT_KNEE, SimpleOpenNI.SKEL_LEFT_FOOT);
drawLimb(userId, SimpleOpenNI.SKEL_TORSO, SimpleOpenNI.SKEL_RIGHT_HIP);
drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_HIP, SimpleOpenNI.SKEL_RIGHT_KNEE);
drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_KNEE, SimpleOpenNI.SKEL_RIGHT_FOOT);
stroke(255, 200, 200);
strokeWeight(1);
}
void drawLimb(int userId, int jointType1, int jointType2)
{
PVector jointPos1 = new PVector();
PVector jointPos2 = new PVector();
float confidence;
// draw the joint position
confidence = kinect.getJointPositionSkeleton(userId, jointType1, jointPos1);
confidence = kinect.getJointPositionSkeleton(userId, jointType2, jointPos2);
stroke(255, 0, 0, confidence * 200 + 55);
line(jointPos1.x, jointPos1.y, jointPos1.z,
jointPos2.x, jointPos2.y, jointPos2.z);
drawJointOrientation(userId, jointType1, jointPos1, 50);
}
void drawJointOrientation(int userId, int jointType, PVector pos, float length)
{
// draw the joint orientation
PMatrix3D orientation = new PMatrix3D();
float confidence = kinect.getJointOrientationSkeleton(userId, jointType, orientation);
if (confidence < 0.001f)
// nothing to draw, orientation data is useless
return;
pushMatrix();
translate(pos.x, pos.y, pos.z);
// set the local coordsys
applyMatrix(orientation);
// coordsys lines are 100mm long
// x - r
stroke(255, 0, 0, confidence * 200 + 55);
line(0, 0, 0,
length, 0, 0);
// y - g
stroke(0, 255, 0, confidence * 200 + 55);
line(0, 0, 0,
0, length, 0);
// z - b
stroke(0, 0, 255, confidence * 200 + 55);
line(0, 0, 0,
0, 0, length);
popMatrix();
}
// SimpleOpenNI user events
void onNewUser(SimpleOpenNI curContext, int userId)
{
println("onNewUser - userId: " + userId);
println("\tstart tracking skeleton");
kinect.startTrackingSkeleton(userId);
// user_tracking_info.put(userId, new UserTrackingInfo());
}
void onLostUser(SimpleOpenNI curContext, int userId)
{
println("onLostUser - userId: " + userId);
// UserTrackingInfo info = user_tracking_info.get(userId);
// historyInfo.addUserData(info.positions, info.colour, time);
// user_tracking_info.remove(userId);
}
void onVisibleUser(SimpleOpenNI curContext, int userId)
{
println("onVisibleUser - userId: " + userId);
}
//Update user position
//void updateUserPoses() {
// if (kinect != null) {
// int[] user_list = kinect.getUsers();
// for (int i = 0; i < user_list.length; i++) {
// //println("test for distance use " + i +": "+ user_list[i]);
// if (kinect.getCoM(user_list[i], com)) {
// println("user com: " + com);
// int user_id = user_list[i];
// //println("Current :" + com);
// if (com != null && user_tracking_info.get(user_id) != null) {
// user_tracking_info.get(user_id).update(new PVector(com.x, com.y, com.z));
// }
// }
// }
// }
//}
scene tab
Code: Select all
void stopAll() {
whisper.pause();
step.pause();
tic.pause();
chord1_0.pause();
chord1_1.pause();
chord1_2.pause();
chord1_3.pause();
chord2_0.pause();
chord2_1.pause();
chord2_2.pause();
chord2_3.pause();
chord3_0.pause();
chord3_1.pause();
chord3_2.pause();
chord3_3.pause();
chord4_0.pause();
chord4_1.pause();
chord4_2.pause();
chord4_3.pause();
sea.pause();
seaWave.pause();
gull_1.pause();
gull_2.pause();
wind.pause();
wave.pause();
bell.pause();
road.pause();
park_1.pause();
park_2.pause();
choir.pause();
walk.pause();
whisper.rewind();
step.rewind();
tic.rewind();
chord1_0.rewind();
chord1_1.rewind();
chord1_2.rewind();
chord1_3.rewind();
chord2_0.rewind();
chord2_1.rewind();
chord2_2.rewind();
chord2_3.rewind();
chord3_0.rewind();
chord3_1.rewind();
chord3_2.rewind();
chord3_3.rewind();
chord4_0.rewind();
chord4_1.rewind();
chord4_2.rewind();
chord4_3.rewind();
sea.rewind();
seaWave.rewind();
wave.rewind();
gull_1.rewind();
gull_2.rewind();
wind.rewind();
bell.rewind();
road.rewind();
park_1.rewind();
park_2.rewind();
choir.rewind();
walk.rewind();
}
void scene() {
// &&&&&&&&&&&&&&&&&& start over &&&&&&&&&&&&&&&&&&&&
if (timerSet % phase == 1) {
stopAll();
playMemo = false;
}
// &&&&&&&&&&&&&&&&&& whisper &&&&&&&&&&&&&&&&&&&&
if (timerSet % phase == timestamp_1) {
whisper.play(); // 11s
whisper.setPan(0);
}
// &&&&&&&&&&&&&&&&&& step &&&&&&&&&&&&&&&&&&&&
if (timerSet % phase == timestamp_2) {
tic.play();
tic.loop();
}
if (timerSet % phase == timestamp_3) {
playNoise = true; // noisy gradually>>>
}
if (timerSet % phase <= timestamp_4 && timerSet % phase > timestamp_2) {
// multi users, control times and panning >>>> some problems !!!!
// for (int numbOfUsers = 0; numbOfUsers<user_list.length; numbOfUsers++) {
// step.play(); // 17s
// step.setPan(map(mouseX, 0, 1080, -1, 1));
// // com[i]
// }
play2(step, millis());
float dis = dist(com[0].x, com[0].y, com[0].z, 0, 0, 0);
step.setGain(5-(dis/1000)*(dis/1000));
step.setPan(map(com[0].x, 0, 2000, -1, 1));
playStep = true;
whisper.shiftGain(0, -20, 2*1000);
}
// &&&&&&&&&&&&&&&&&& piano &&&&&&&&&&&&&&&&&&&&
if (timerSet % phase == timestamp_4) {
tic.pause();
}
if (timerSet % phase == timestamp_5) {
playNoise = false;
}
if (timerSet % phase <= timestamp_7 && timerSet % phase > timestamp_4) {
step.pause();
step.shiftGain(0, -50, 2*1000);
playChord = true;
}
if (timerSet % phase == timestamp_6) {
playNoise = true;
}
// &&&&&&&&&&&&&&&&&& sea &&&&&&&&&&&&&&&&&&&&
if (timerSet % phase == timestamp_8) {
playNoise = false;
}
if (timerSet % phase <=timestamp_9 && timerSet % phase >timestamp_7 ) {
playChord = false;
playSynth=true;
//theNoise.unpatch(out);
chord1_0.pause();
chord1_1.pause();
chord1_2.pause();
chord1_3.pause();
chord2_0.pause();
chord2_1.pause();
chord2_2.pause();
chord2_3.pause();
chord3_0.pause();
chord3_1.pause();
chord3_2.pause();
chord3_3.pause();
chord4_0.pause();
chord4_1.pause();
chord4_2.pause();
chord4_3.pause();
playSea = true;
//sea.setGain(5);
play2(sea, millis());
/*
// try create synth in supercollider
if (playSynth) {
synth0.addToTail(group);
synth1.addToTail(group);
synth2.addToTail(group);
synth3.addToTail(group);
playSynth=false;
}
synth0.set("freq", 40 + (peak[0] * 2));
synth0.set("amp", min(0, -0.5*peak[0]+150));
synth1.set("freq", 40 + (peak[0] * 2));
synth1.set("amp", min(0, -0.5*peak[0]+150));
synth2.set("freq", 40 + (peak[0] * 4));
synth2.set("amp", min(0, -0.5*peak[0]+150));
synth3.set("freq", 40 + (peak[0] * 3));
synth3.set("amp", min(0, -0.5*peak[0]+150));
println("freq " + 40 + (mouseX * 4)+" "+"amp", (float)(0.05+mouseX/700) );
*/
wind.setGain(min(0, -0.5*peak[0]+150));
gull_1.setGain(min(0, 0.5*peak[0]-250));
wave.setGain(min(0, -0.5*peak[0]+150));
play2(wind, millis());
play2(wave, millis());
play2(gull_1, millis());
if (dist(com[0].x, com[0].z, com[1].x, com[1].z) < 1000)
play2(seaWave, millis()); // >>> To do list: shorter the wave file
}
// &&&&&&&&&&&&&&&&&& memo &&&&&&&&&&&&&&&&&&&&
if (timerSet % phase == timestamp_10) {
tic.rewind();
tic.play();
tic.loop();
}
if (timerSet % phase <=timestamp_11 && timerSet % phase >timestamp_9 ) {
playSea = false;
sea.pause();
seaWave.pause();
gull_1.pause();
gull_2.pause();
wind.pause();
playMemo = true;
}
// &&&&&&&&&&&&&&&&&& ending &&&&&&&&&&&&&&&&&&&&
if (timerSet % phase <=timestamp_12 && timerSet % phase >timestamp_11 ) {
playMemo = false;
}
}
// &&&&&&&&&&&&&&&&&& piano chord set &&&&&&&&&&&&&&&&&&&&
void pianoChord() {
if (user_list.length==1) { // single user
float dis = dist(com[0].x, com[0].y, com[0].z, 0, 0, 0);
float gain=-(dis/1000)*(dis/1000);
float duration = 250;
float maxDis=3000;
if ((timerSet-timestamp_4)<15/2) {
if (com[0].z<2*maxDis/4) {
play2(chord1_1, millis());
chord1_1.setGain(gain);
play2(chord1_2, millis()+duration);
chord1_1.setGain(gain);
play2(chord1_3, millis()+duration*2);
chord1_1.setGain(gain);
} else if (com[0].z<3*maxDis/4) {
play2(chord1_1, millis());
chord1_1.setGain(gain);
play2(chord1_2, millis()+duration);
chord1_1.setGain(gain);
} else {
play2(chord1_1, millis());
chord1_1.setGain(gain);
}
if (handsUp[0]) {
//println("hand0");
play2(chord1_0, millis());
chord1_0.setGain(gain);
}
} else if ((timerSet-timestamp_4)<30/2) {
if (com[0].z<2*maxDis/4) {
play2(chord2_1, millis());
chord2_1.setGain(gain);
play2(chord2_2, millis()+duration);
chord2_1.setGain(gain);
play2(chord2_3, millis()+duration*2);
chord2_1.setGain(gain);
} else if (com[0].z<3*maxDis/4) {
play2(chord2_1, millis());
chord2_1.setGain(gain);
play2(chord2_2, millis()+duration);
chord1_1.setGain(gain);
} else {
play2(chord2_1, millis());
chord2_1.setGain(gain);
}
if (handsUp[0]) {
//println("hand0");
play2(chord2_0, millis());
chord2_0.setGain(gain);
}
//
if (com[0].z<2*maxDis/4) {
play2(chord1_1, millis());
chord1_1.setGain(gain);
play2(chord1_2, millis()+duration);
chord1_1.setGain(gain);
play2(chord1_3, millis()+duration*2);
chord1_1.setGain(gain);
} else if (com[0].z<3*maxDis/4) {
play2(chord1_1, millis());
chord1_1.setGain(gain);
play2(chord1_2, millis()+duration);
chord1_1.setGain(gain);
} else {
play2(chord1_1, millis());
chord1_1.setGain(gain);
}
if (handsUp[0]) {
//println("hand0");
play2(chord1_0, millis());
chord1_0.setGain(gain);
}
} else if ((timerSet-timestamp_4)<45/2) {
if (com[0].z<2*maxDis/4) {
play2(chord3_1, millis());
chord3_1.setGain(gain);
play2(chord3_2, millis()+duration);
chord3_1.setGain(gain);
play2(chord3_3, millis()+duration*2);
chord3_1.setGain(gain);
} else if (com[0].z<3*maxDis/4) {
play2(chord3_1, millis());
chord3_1.setGain(gain);
play2(chord3_2, millis()+duration);
chord3_1.setGain(gain);
} else {
play2(chord3_1, millis());
chord3_1.setGain(gain);
}
if (handsUp[0]) {
//println("hand0");
play2(chord3_0, millis());
chord3_0.setGain(gain);
}
//
if (com[0].z<2*maxDis/4) {
play2(chord2_1, millis());
chord2_1.setGain(gain);
play2(chord2_2, millis()+duration);
chord2_1.setGain(gain);
play2(chord2_3, millis()+duration*2);
chord2_1.setGain(gain);
} else if (com[0].z<3*maxDis/4) {
play2(chord2_1, millis());
chord2_1.setGain(gain);
play2(chord2_2, millis()+duration);
chord1_1.setGain(gain);
} else {
play2(chord2_1, millis());
chord2_1.setGain(gain);
}
if (handsUp[0]) {
//println("hand0");
play2(chord2_0, millis());
chord2_0.setGain(gain);
}
//
if (com[0].z<2*maxDis/4) {
play2(chord1_1, millis());
chord1_1.setGain(gain);
play2(chord1_2, millis()+duration);
chord1_1.setGain(gain);
play2(chord1_3, millis()+duration*2);
chord1_1.setGain(gain);
} else if (com[0].z<3*maxDis/4) {
play2(chord1_1, millis());
chord1_1.setGain(gain);
play2(chord1_2, millis()+duration);
chord1_1.setGain(gain);
} else {
play2(chord1_1, millis());
chord1_1.setGain(gain);
}
if (handsUp[0]) {
//println("hand0");
play2(chord1_0, millis());
chord1_0.setGain(gain);
}
} else {
if (com[0].z<2*maxDis/4) {
play2(chord4_1, millis());
chord4_1.setGain(gain);
play2(chord4_2, millis()+duration);
chord4_1.setGain(gain);
play2(chord4_3, millis()+duration*2);
chord4_1.setGain(gain);
} else if (com[0].z<3*maxDis/4) {
play2(chord4_1, millis());
chord4_1.setGain(gain);
play2(chord4_2, millis()+duration);
chord4_1.setGain(gain);
} else {
play2(chord4_1, millis());
chord4_1.setGain(gain);
}
if (handsUp[0]) {
//println("hand0");
play2(chord4_0, millis());
}
//
if (com[0].z<2*maxDis/4) {
play2(chord3_1, millis());
chord3_1.setGain(gain);
play2(chord3_2, millis()+duration);
chord3_1.setGain(gain);
play2(chord3_3, millis()+duration*2);
chord3_1.setGain(gain);
} else if (com[0].z<3*maxDis/4) {
play2(chord3_1, millis());
chord3_1.setGain(gain);
play2(chord3_2, millis()+duration);
chord3_1.setGain(gain);
} else {
play2(chord3_1, millis());
chord3_1.setGain(gain);
}
if (handsUp[0]) {
//println("hand0");
play2(chord3_0, millis());
chord3_0.setGain(gain);
}
//
if (com[0].z<2*maxDis/4) {
play2(chord2_1, millis());
chord2_1.setGain(gain);
play2(chord2_2, millis()+duration);
chord2_1.setGain(gain);
play2(chord2_3, millis()+duration*2);
chord2_1.setGain(gain);
} else if (com[0].z<3*maxDis/4) {
play2(chord2_1, millis());
chord2_1.setGain(gain);
play2(chord2_2, millis()+duration);
chord1_1.setGain(gain);
} else {
play2(chord2_1, millis());
chord2_1.setGain(gain);
}
if (handsUp[0]) {
//println("hand0");
play2(chord2_0, millis());
chord2_0.setGain(gain);
}
//
if (com[0].z<2*maxDis/4) {
play2(chord1_1, millis());
chord1_1.setGain(gain);
play2(chord1_2, millis()+duration);
chord1_1.setGain(gain);
play2(chord1_3, millis()+duration*2);
chord1_1.setGain(gain);
} else if (com[0].z<3*maxDis/4) {
play2(chord1_1, millis());
chord1_1.setGain(gain);
play2(chord1_2, millis()+duration);
chord1_1.setGain(gain);
} else {
play2(chord1_1, millis());
chord1_1.setGain(gain);
}
if (handsUp[0]) {
//println("hand0");
play2(chord1_0, millis());
chord1_0.setGain(gain);
}
}
}
/*
if (user_list.length==2) { // two users
play2(chord2_1, millis()+500);
if (handsUp[0]) {
println("hand0");
play2(chord1_1, millis());
play2(chord1_2, millis());
play2(chord1_2, millis());
}
if (handsUp[1]) {
println("hand1");
play2(chord2_1, millis());
play2(chord2_2, millis());
play2(chord2_2, millis());
}
}
*/
}
float getGainFromDistance(float distance) {
return min(0, (100-distance/1000*200));
}
boolean bellTriger = false;
void memo() {
if (user_list.length==1) {
float radius = 500;
float dis_walk = dist(com[0].x, com[0].z, 0, 1500);
float dis_park_1 = dist(com[0].x, com[0].z, 500, 1000);
float dis_park_2 = dist(com[0].x, com[0].z, 500, 2000);
float dis_choir = dist(com[0].x, com[0].z, -500, 1000);
float dis_road = dist(com[0].x, com[0].z, -500, 2000);
if (dis_walk<500&&(dis_park_1<500||dis_park_2<500||dis_choir<500||dis_road<500)&&bellTriger==false) {
play2(bell, millis());
bellTriger=true;
} else if (!(dis_walk<500&&(dis_park_1<500||dis_park_2<500||dis_choir<500||dis_road<500))) {
bellTriger=false;
}
walk.setGain(getGainFromDistance(dis_walk));
road.setGain(getGainFromDistance(dis_road));
park_1.setGain(getGainFromDistance(dis_park_1));
park_2.setGain(getGainFromDistance(dis_park_2));
choir.setGain(getGainFromDistance(dis_choir));
play2(road, millis());
play2(park_1, millis());
play2(park_2, millis());
play2(choir, millis());
play2(walk, millis());
}
}
stuff tab
Code: Select all
void play2(AudioPlayer tempPlayer, float time2BPlayed) {
if(!next_play_times.containsKey(tempPlayer)) {
next_play_times.put(tempPlayer, 1e20);
}
boolean playingState = tempPlayer.isPlaying();
if (playingState) {
println("ture");
next_play_times.put(tempPlayer, 1e20);
} else {
if (millis()>=next_play_times.get(tempPlayer)) {
println("false");
tempPlayer.rewind();
tempPlayer.play();
} else {
if(next_play_times.get(tempPlayer) == 1e20) {
next_play_times.put(tempPlayer, time2BPlayed);
}
}
}
}
// -------------------------------------------------------------------------------
/////////////// Here is my garage /////////////////
//////////// for things might be useful /////////////
// -------------------------------------------------------------------------------
/*
// set a volume variable
float vol = 0.45;
// set the tempo for here
out.setTempo( 100.0f );
// set a percentage for the actual duration
out.setDurationFactor( 0.95f );
// use pauseNotes to add a bunch of notes at once without time moving forward
out.pauseNotes();
// specify the waveform for this group of notes
Waveform disWave = Waves.sawh( 4 );
// add these notes with disWave
out.playNote( 0.0, 1.0, new ToneInstrument( "E4 ", vol, disWave, out ) );
out.playNote( 1.0, 1.0, new ToneInstrument( "E4 ", vol, disWave, out ) );
out.playNote( 2.0, 1.0, new ToneInstrument( "E4 ", vol, disWave, out ) );
out.playNote( 3.0, 0.75, new ToneInstrument( "C4 ", vol, disWave, out ) );
out.playNote( 3.75, 0.25, new ToneInstrument( "G4 ", vol, disWave, out ) );
out.playNote( 4.0, 1.0, new ToneInstrument( "E4 ", vol, disWave, out ) );
out.playNote( 5.0, 0.75, new ToneInstrument( "C4 ", vol, disWave, out ) );
out.playNote( 5.75, 0.25, new ToneInstrument( "G4 ", vol, disWave, out ) );
out.playNote( 6.0, 2.0, new ToneInstrument( "E4 ", vol, disWave, out ) );
// specify the waveform for this group of notes
disWave = Waves.triangleh( 9 );
// add these notes with disWave
out.playNote( 8.0, 1.0, new ToneInstrument( "B4 ", vol, disWave, out ) );
out.playNote( 9.0, 1.0, new ToneInstrument( "B4 ", vol, disWave, out ) );
out.playNote(10.0, 1.0, new ToneInstrument( "B4 ", vol, disWave, out ) );
out.playNote(11.0, 0.75, new ToneInstrument( "C5 ", vol, disWave, out ) );
out.playNote(11.75, 0.25, new ToneInstrument( "G4 ", vol, disWave, out ) );
out.playNote(12.0, 1.0, new ToneInstrument( "Eb4 ", vol, disWave, out ) );
out.playNote(13.0, 0.75, new ToneInstrument( "C4 ", vol, disWave, out ) );
out.playNote(13.75, 0.25, new ToneInstrument( "G4 ", vol, disWave, out ) );
out.playNote(14.0, 2.0, new ToneInstrument( "E4 ", vol, disWave, out ) );
// specify the waveform for this group of notes
disWave = Waves.randomNOddHarms( 3 );
//add these notes with disWave
out.playNote( 0.0, 1.9, new ToneInstrument( "E3 ", vol, disWave, out ) );
out.playNote( 2.0, 1.9, new ToneInstrument( "E3 ", vol, disWave, out ) );
out.playNote( 4.0, 1.9, new ToneInstrument( "E3 ", vol, disWave, out ) );
out.playNote( 6.0, 1.9, new ToneInstrument( "E3 ", vol, disWave, out ) );
// specify the waveform for this group of notes
disWave = Waves.TRIANGLE;
// add these notes with disWave
out.playNote( 8.0, 1.9, new ToneInstrument( "E3 ", vol, disWave, out ) );
out.playNote(10.0, 1.9, new ToneInstrument( "E3 ", vol, disWave, out ) );
out.playNote(12.0, 1.9, new ToneInstrument( "C3 ", vol, disWave, out ) );
out.playNote(14.0, 1.9, new ToneInstrument( "E3 ", vol, disWave, out ) );
////////////
table = Waves.randomNOddHarms(9);
table = randomNoise();
table.smooth( 64 );
table.addNoise( 0.1f );
wave = new Oscil( 440, 0.05f, table );
wave.patch( out );
////////////
// use resumeNotes at the end of the section which needs guaranteed timing
out.resumeNotes();
*/
///////////////////
/*
Summer sum = new Summer();
Oscil wave = new Oscil( Frequency.ofPitch("A4"), 0.3f, Waves.SINE );
wave.patch( sum );
wave = new Oscil( Frequency.ofPitch("C#5"), 0.3f, Waves.SINE );
wave.patch( sum );
wave = new Oscil( Frequency.ofPitch("E5"), 0.3f, Waves.SINE );
wave.patch( sum );
// and the Summer to the output and you should hear a major chord
sum.patch( out );
*/
///////////////////
/*
// make our midi converter
midi = new Midi2Hz( 50 );
midi.patch( wave.frequency );
wave.patch( out );
*/
//////////////////////////////////////////////////
/*
void keyPressed()
{
if ( key == 'a' ) midi.setMidiNoteIn( 50 );
if ( key == 's' ) midi.setMidiNoteIn( 60 );
if ( key == 'd' ) midi.setMidiNoteIn( 70 );
if ( key == 'f' ) midi.setMidiNoteIn( 80 );
if ( key == 'g' ) midi.setMidiNoteIn( 90 );
if ( key == 'h' ) midi.setMidiNoteIn( 100 );
if ( key == 'j' ) midi.setMidiNoteIn( 110 );
if ( key == 'k' ) midi.setMidiNoteIn( 120 );
if ( key == 'l' ) midi.setMidiNoteIn( 130 );
if ( key == ';' ) midi.setMidiNoteIn( 140 );
if ( key == '\'') midi.setMidiNoteIn( 150 );
}
*/
///////////////////////////////////////////////////////
//out.playNote( 2.0, 2.9, new SineInstrument( Frequency.ofPitch( "C3" ).asHz() ) );
//out.playNote( 3.0, 1.9, new SineInstrument( Frequency.ofPitch( "E3" ).asHz() ) );
//out.playNote( 4.0, 0.9, new SineInstrument( Frequency.ofPitch( "G3" ).asHz() ) );
//////////////////////////////////////////////////
/*
//// microphone input transform ////
AudioStream inputStream = minim.getInputStream( Minim.STEREO,
out.bufferSize(),
out.sampleRate(),
out.getFormat().getSampleSizeInBits()
);
in = new LiveInput( inputStream );
Vocoder vocode = new Vocoder( 1024, 16 );
in.patch( vocode.modulator );
//synth.patch( vocode ).patch( out );
*/
verlet ball tab
Code: Select all
// Reference: the VerletBall&VerletStick class is based on Ira Greenberg's VerletStick example
// (https://github.com/irajgreenberg/workshopExamples/blob/master/apression/VerletStick.pde)
// -------------------------------------------------------------------------------
class VerletBall {
PVector pos, posOld;
PVector push;
float radius;
VerletBall(PVector pos, PVector push, float radius) {
this.pos = pos;
this.push = push;
this.radius = radius;
this.posOld = new PVector(pos.x, pos.y, pos.z);
// start motion
pos.add(push);
}
void verlet() {
PVector posTemp = new PVector(pos.x, pos.y, pos.z);
pos.x += (pos.x-posOld.x);
pos.y += (pos.y-posOld.y);
pos.z += (pos.z-posOld.z);
posOld.set(posTemp);
}
void render() {
//ellipse(pos.x, pos.y, radius*2, radius*2);
pushMatrix();
translate(pos.x, pos.y, pos.z);
point(0, 0, 0);
strokeWeight(3);
stroke(127, 110);
popMatrix();
}
void boundsCollision() {
if (pos.x>width/2-radius) {
pos.x = width/2-radius;
posOld.x = pos.x;
pos.x -= push.x;
} else if (pos.x< -width/2+radius) {
pos.x = -width/2+radius;
posOld.x = pos.x;
pos.x += push.x;
}
if (pos.y>height/2-radius) {
pos.y = height/2-radius;
posOld.y = pos.y;
pos.y -= push.y;
} else if (pos.y<-height+radius) {
pos.y = -height+radius;
posOld.y = pos.y;
pos.y += push.y;
}
}
void update(float x, float y) {
pos.x = x;
pos.y = y;
//pos.x = mouseX-width/2;
//pos.y = mouseY-height/2;
}
}
verlet stick tab
Code: Select all
// Reference: the VerletBall&VerletStick class is based on Ira Greenberg's VerletStick example
// (https://github.com/irajgreenberg/workshopExamples/blob/master/apression/VerletStick.pde)
// -------------------------------------------------------------------------------
class VerletStick {
VerletBall b1, b2;
float stiffness;
boolean isVisible=true;
PVector vecOrig;
float len;
VerletStick(VerletBall b1, VerletBall b2, float stiffness) {
this.b1 = b1;
this.b2 = b2;
this.stiffness = stiffness;
vecOrig = new PVector(b2.pos.x-b1.pos.x, b2.pos.y-b1.pos.y, b2.pos.z-b1.pos.z);
len = PVector.dist(b1.pos, b2.pos);
}
void render(float stroke) {
if (isVisible) {
line(b1.pos.x, b1.pos.y, b1.pos.z, b2.pos.x, b2.pos.y, b2.pos.z);
strokeWeight(stroke);
//stroke((224-0.01*(b2.pos.y*b2.pos.y)), (82-0.01*(b2.pos.y*b2.pos.y)), (141+b2.pos.y), 255); // pink-blue
stroke((220-b1.pos.y*b2.pos.y/100)/2-b2.pos.y/9, (220-b1.pos.y*b2.pos.y/100-b2.pos.y)-b2.pos.y/7,
(250-b1.pos.y*b2.pos.y/100-b2.pos.y)-b2.pos.y/4, transP*transP/100);
}
}
void constrainLen() {
PVector delta = new PVector(b2.pos.x-b1.pos.x, b2.pos.y-b1.pos.y, b2.pos.z-b1.pos.z);
float deltaLength = delta.mag();
float difference = ((deltaLength - len) / deltaLength);
b1.pos.x += delta.x * (0.5f * stiffness * difference);
b1.pos.y += delta.y * (0.5f * stiffness * difference);
b1.pos.z += delta.z * (0.5f * stiffness * difference);
b2.pos.x -= delta.x * (0.5f * stiffness * difference);
b2.pos.y -= delta.y * (0.5f * stiffness * difference);
b2.pos.z -= delta.z * (0.5f * stiffness * difference);
}
}
Because the sound file is larger than 50M, so I upload the whole package here.
Remember to connect a kinect before having a try.
https://drive.google.com/open?id=0B_3lC ... ExVMkhUdVE
Here is the link for the class demo video.
https://youtu.be/vRNfPeQHIPI