Page 1 of 3

Cecelia (Jiayue) Wu’s Project

Posted: Thu Nov 06, 2014 3:25 pm
by saharss2533
Cecelia (Jiayue) Wu’s Project

Re: Cecelia (Jiayue) Wu’s Project

Posted: Tue Nov 11, 2014 9:47 pm
by jcw
Project Title: "Navigating Cycles" --Peace and Healing through Sound & Motion

Ancient Buddhist culture believed in Kalachakra, or the idea that time, life, and the universe are all cyclical. This is found in the rhythm and energy of the orbits of the heavenly bodies, the changing of the seasons, and the in and out of human breath. As we live our lives, we navigate these cycles, a wheel of time between birth and death, as we already have for millions of years, without beginning or end. 

I strive to create a rich, intriguing work that translates this aspect of ancient philosophy to a day-to-day spiritual experience through interactive multimedia art and music composition. Inspired by the traditional Tibetan Prayer Wheel and Tibetan Singing bowl, I propose a novel physical motion sensing controller and digital audio signal processing software that both processes the real-time voice of the operator based on his or her body motion, as part of an interactive art installation. This system is designed for a layperson to be able to easily use their voice to improvise and assemble their own unique piece, becoming part of the installation themselves through their multi-sensory experience.

Tibetan prayer wheels are metal cylinders that contain multitudes of prayers on a long roll of paper, spun with a simple, gentle, repetitive motion. Tibetan singing bowls are musical instruments made by bronze or crystal, played by gently rubbing a leather-covered mallet in a steady motion around the outer rim, emanating a harmony of deep, resonant tones. With both spiritual objects, a circular movement is the main entry point to relaxation and a meditative state.

This project will create both a set of two hand-held electronic, musical Tibetan prayer wheels, and a large, fixed Tibetan prayer wheel multimedia art installation. The hand-held prayer wheels will contain miniaturized electronics, sensors and a wireless transmitter. The sensors will measure speed at which the performer spins the wheel, the rising or falling gesture, and several carefully hidden buttons, and allow the performer or the user to seamlessly transition between sound design layers as desired. The large, floor-mounted prayer wheel will be a glowing, humming presence, requiring about a 10'x10' area, with the wheel holding a solid presence in the middle. Small speakers within and around the wheel will create a meditative atmosphere. People will be encouraged to navigate around the wheel and spin it, where embedded sensors will respond to touch and spinning of the wheel and the lights will change color and luminosity, or to sing into a microphone at one corner and modulate their voice with the smaller prayer wheels.

For software, I plan to implement three sound design layers and one visualization layer. The main layer will modulate properties of a Faust-STK physical model of multiple Tibetan singing bowls, based on the Essl and Cook banded waveguide technique. The two supplementary layers will be voice-based layers, with one filtering the participant’s voice through a "modal synthesis" model constructed from recordings of an actual Tibetan singing bowl, and the other adding reverberation and feedback. The synthesized bowl sound will be excited by the vocalist's voice energy level and thus trigger different harmonic partials. Last but not least, I hope the visualization layer could be implemented in winter if I can take the data vis class. Hopefully the schedule of the data vis class would not be conflicted with my beloved sound design algorithm class...

Re: Cecelia (Jiayue) Wu’s Project

Posted: Sun Nov 16, 2014 7:08 pm
by intae

Gyro sensor or accelerometer is promising to give you information how much your object has been moved, but it's true that the value through the analog is not always accurate. I'm not sure that this is what you are looking for. But I wish you this would be helpful.

Re: Cecelia (Jiayue) Wu’s Project

Posted: Sun Nov 30, 2014 6:17 pm
by jcw
Two weeks ago, we have a 3-axis magnetometer/accelerometer/gyroscope inside a prayer wheel, which wirelessly transmits back to a laptop, and we want to use these sensors to measure the current height of the prayer wheel.
When we first started, we figured we could integrate the accelerometer Z axis information twice to get position estimates. However, because the prayer wheel is rotating during the entire performance, there is a lot of sinusoidal noise that changes depending on how fast the performer is spinning the wheel and whether or not there is any tilt. If the performer wobbles their hand even slightly, it will measure different information.
We were excited to discover that the magnetometer Z axis actually varied in a fairly stable way proportional to height, so switched to trying to use that data. However, the magnetometer works best when there are no computers, speakers, or magnets nearby. The tilt problem is still there as well.
For now, using an average of the magnetometer Z axis data approximates relative height in a noisy way.
Also, additional challenges are that the prayer wheel is small (a cylinder of diameter 3 inches, height 1.5 inches), and is already almost full. It also must be powered off a small battery (currently a 100mAh LiPo, could be swapped for a 150mAh), so we must conserve power usage.

Re: Cecelia (Jiayue) Wu’s Project

Posted: Sun Nov 30, 2014 10:53 pm
by jcw
[1] Essl, G., Cook, P.R., ”Banded Waveguides: Towards Physical Modeling of Bar Percussion Instruments,” In Proc. Int. Computer Music Conf. (ICMC), Beijing, 22-28 October, 321-324, 1999.
[2] Smith, J.O. III. Physical Modeling Using Digital Waveg- uides. Computer Music Journal, volume 16, number 4, pages 74-91, 1992.
[3] Wilkerson, Carr, Stefania Serafin, and Carmen Ng. "Physical model synthesis and performance mappings of bowl resonators." In Proceedings of the International Computer Music Conference (ICMC-02), pp. 219-221. 2002.
[4] Hewitt, Donna G. "EMIC-Compositional experiments and real-time mapping issues in performance." (2003): 96-104.
[5] Young, Diana, and Georg Essl. "Hyperpuja: A tibetan singing bowl controller." In Proceedings of the 2003 conference on New interfaces for musical expression, pp. 9-14. National University of Singapore, 2003.
[6] Serafin, Stefania, and Diana Young. "Toward a generalized friction controller: from the bowed string to unusual musical instruments." In Proceedings of the 2004 conference on New interfaces for musical expression, pp. 108-111. National University of Singapore, 2004.
[7] Sawada, Hideyuki, and Shuji Hashimoto. "Gesture recognition using an acceleration sensor and its application to musical performance control."Electronics and Communications in Japan (Part III: Fundamental Electronic Science) 80, no. 5 (1997): 9-17.
[8] Knapp, R. Benjamin, and Perry R. Cook. "The integral music controller: introducing a direct emotional interface to gestural control of sound synthesis." In Proceedings of the International Computer Music Conference (ICMC), pp. 4-9. 2005.
[9] Winkler, Todd. "Making motion musical: Gesture mapping strategies for interactive computer music." In ICMC Proceedings, pp. 261-264. 1995.

Re: Cecelia (Jiayue) Wu’s Project

Posted: Wed Dec 03, 2014 3:45 pm
by jcw
A draft of the hardware diagram:

Re: Cecelia (Jiayue) Wu’s Project

Posted: Wed Dec 03, 2014 3:49 pm
by jcw
The following is the demonstration of the parts of the prayer wheel, the labels are:

(A) Arduino Pro Mini 328, 3.3V, 8MHz
(B) Xbee 1mW Trace Antenna Series 1 2.4GHz RF Module
(C) Flora 9DOF Acceleormeter/Gyroscope/Magnetometer - LSM9DS0
(D) Li-Ion 3.7V 110mAh or 400mAh battery pack
(E) Charging port
(F) On-Off switch

Re: Cecelia (Jiayue) Wu’s Project

Posted: Wed Dec 03, 2014 4:18 pm
by jcw
Voice processing layer code:

% input signal
ifiles = {'', ... % input signal file name, string ('' gives impulse)
'cecilia_dry_chant.wav'}; %'ElecGtr-PC_snip.wav', 'ElecGtr-PD_snip.wav', 'KM6_snip.wav'};
ifile = ifiles{2};

% architecture
load mverbtb.mat;
% fm; % mode frequencies, Hz
% rt60m; % mode decay times, seconds
% gm; % mode amplitudes, amplitude
% nmode; % mode count, modes
rt60mscale = 5;

% controls
mix = 0.95; % wet-dry mix, fraction
cascade = 2; % damping cascade order, poles

azimuth = 0; % singing application angle, degrees

% output
plotflag = 1; % display output spectrogram, indicator

%% form input

if ~isempty(ifile),
% read signal file, first channel
[isignal, fs] = wavread(ifile); % input signal; sampling rate, Hz
isignal = isignal(:,1);

nsamp = size(isignal,1); % signal length, samples

% generate impulse response
fs = 44100; % sampling rate, Hz
nsamp = 5*fs; % signal length, samples
zeropad = 100*fs/1000; % zeropad length, samples

isignal = [zeros(zeropad,1); 1; zeros(nsamp-zeropad-1,1)];


%% design processing

% generate mode damping coefficients
a1m = 0.001.^(cascade*rt60mscale./(rt60m*fs));

% form mode amplitudes
b0m = gm.*sqrt(1-a1m);

%% process input

% loop through modes
psi = 2*pi*[1:nsamp]'/fs;
wsignal = zeros(nsamp,1);
for m = [1:nmode],
% heterodyne input
mu = exp(1j*fm(m)*psi);
hsignal = isignal.*mu;

% apply damping
dsignal = hsignal;
for i = [1:cascade],
dsignal = filter(b0m(m), [1 -a1m(m)], dsignal);

% generate mode response
msignal = dsignal.*conj(mu);

% add mode response to output
wsignal = wsignal + real(msignal);


% form wet-dry mix
osignal = sqrt(1-mix)*isignal/sqrt(mean(isignal.^2)) + sqrt(mix)*wsignal/sqrt(mean(wsignal.^2));

% display output
if isempty(ifile) & plotflag,
% plot impulse response spectrogram
figure(1); ftgram(wsignal, fs, 'rir');

elseif plotflag,
% plot system response spectrogram
figure(1); ftgram(wsignal, fs, 'music', 'waveform', true);


Re: Cecelia (Jiayue) Wu’s Project

Posted: Sun Dec 07, 2014 10:56 am
by jcw
The final block diagram:

Re: Cecelia (Jiayue) Wu’s Project

Posted: Tue Dec 09, 2014 7:31 pm
by jcw
Final Presentation:
The Methodology of Making Interactive Music Instrument and Sound Art Through Voice, Motion and Tibetan Buddhism (page1)
Please note that my ppt file is too big thus I cannot upload it as one complete file. Therefore, I'm uploading them as pictures and displaying them one page after another. Please click the "next" button and go to page 2 of my post online. There are the other 11 pages of my posts.