??Audio Modulated Procedural Animation (Part 1)

??Audio Modulated Procedural Animation (Part 1)

?? Dawn of the Free Form Programmable Signal Processing shines upon all of you!?

?? What was once for the elites in higher education institutions now is available for anyone who is willing to invest reasonable amounts of time

?? Imaginative new animations become possible thanks to audio signal modulation of the animations

??Don’t have any ideas for animated production? Despair not! Simply use the Free Form Programming Language and pull together an animated production unique style of all your own!


Click on the image to view the video

Mixed Audio Modulated Animations with motion capture animations for exciting new innovations

???? What

Audio Modulation: some attributes of an Audio signal could be used to drive changes to a 3D dynamical system e.g. animated character or 3D objects with transforms and skinned-meshes.

Procedural Animation: using software and algorithms to animate an object otherwise static.?

?? Why

?? Kids, youths and adults around the globe DJ video teasers from all kinds of existing productions, in particular the Japanese anime releases, and place their own native music and attempt to create a new production that is perceived as their own. This phenomenon has been observed in TikTok for many years and shall gain strength and continue into the future as forms of expressions and forms of arts.

Click on the image to view the video

Samples of Youths video productions by recycling teasers adding their own native songs and narratives

?? CCN Studios sees a fertile opportunity for a) educating the masses of youths with its accessible cloud technologies while b) they are empowered to produce unique and innovative media in place of using the second hand teasers of other rich people’s studios.?

?? How

???? Free For Programming Language is imbued with production technologies that are custom architected for the sole utility of such production applications the likes of Audio Modulate Animations.?


Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License

?? To enquire about the Free Form Programming Language and CCN’s certified training including Unity certification please contact [email protected]


??Digital signal processing (DSP) is the use of digital processing, such as by computers or more specialized digital signal processors, to perform a wide variety of signal processing operations. The digital signals processed in this manner are a sequence of numbers that represent samples of a continuous variable in a domain such as time, space, or frequency.

Source: https://en.wikipedia.org/wiki/Digital_signal_processing


??? Etude


Components

  • Audio Source 1: One shot 808 Bass
  • Audio Source 2: Toy piano single key
  • ?? 8 Cyan-Blue bars for 8-frequency octave obtained from the audio by means of FFT

 audioSource.GetSpectrumData(samples, 0, FFTWindow.Blackman);        

https://docs.unity3d.com/ScriptReference/AudioSource.GetSpectrumData.html

https://vru.vibrationresearch.com/lesson/windowing-process-data/

  • Left most bar is indexed 1 and the right most bar 8, similar to keyboard musical notes arrangement

Cyan-Blue bars in the video are similar to the white keys on a musical keyboard

Software

  • ?? Free Form Programmers can easily access the real-time frequency values to animate something else (modulate it by audio signal)
  • Or runtime compile classes containing C#.txt code similar to these:

csharptxt = "

...
//audioAgent contains the bars frequency values
angle = audioAgent.GetFrequencyBand(0) * Time.deltaTime;

...

//rotate the mug by simply calling the game object's Rotate () 
//scale the frequency any which way and pass as angle
gameObject.transform.Rotate(0f,0f, 20f*angle,Space.Self);

...
";        

https://en.wikipedia.org/wiki/Fast_Fourier_transform

https://en.wikipedia.org/wiki/Discrete_Fourier_transform

  • If need be the entire FFT signal processing code and the animation modulations could be transformed to String C#.txt and runtime compiled to allow far more versatility for DJ and other compositional creative innovations

Input Devices

  • Unity's the New Input System could easily be adapted to additional input gear e.g. Synth keyboard to simultaneously synthesize both the music and the animations on demand!

MIDI plugin available for Unity desktops and Android as well as WebGL!

  • Audio sources need not synchronize nor follow any restrictions


??? Frequency Maps to Animations

The purpose of this sample project is for the Free Form programmers to first understand the concept behind the signal decomposition into frequency-amplitude domain.And do so audio visually and first and foremost learn how to use the signal processing in an application. For those with more experience and commitment then work the algebra of the signals.?

Follow the videos below, read the captions, relate the Frequency bars to the transformations in the scene.

Click on the image to view the video

Audio Source1: Frequency bar 7 (second from right) modulates the scale of the mug


Time-Amplitude, mono


https://en.wikipedia.org/wiki/Periodogram

??Mid frequency range of a low pitched piano key:

Frequency-Amplitude


Click on the image to view the video

Audio Source2: Frequency bar 1 (first from right) modulates the rotation of the mug and levitation of the right doughnut and sprinkles of the left doughnut


Time-Amplitude, stereo


https://en.wikipedia.org/wiki/Periodogram

??Low frequency range of a Bass:

Frequency-Amplitude


Click on the image to view the video

Both audio sources 1 and 2 running simultaneously


Click on the image to view the video

Additional Bee character with all motion capture animations deactivated , however the scale of its head is modulated by Audio Source1


Click on the image to view the video

Bee character's motion captured animations for Hip-Hop dance enabled




要查看或添加评论,请登录

Darragh O'Sheada的更多文章