Additive Sound Synthesis

Introduction

So we understand the Fourier Transform: we can express any waveform as a sum of sinusoids that are appropriately weighted and are at discrete multiples of a chosen “fundamental frequency”.

How do we use these ideas to synthesize sound?

Inspiration

TO BE ADDED (sic!)

What is Additive Synthesis?

First we need to get used to the idea of an oscillator.

An oscillator is a source: it generates waveforms that we perceive as sound. Let us play with a few oscillator types here:

https://musiclab.chromeexperiments.com/Oscillators/

Each of these waveforms, by the Fourier series, is the sum of an ( infinite) number of sine wave outputs.

In Fourier series, we normally use just sine wave oscillators, and use many of them to add up to obtain the wave form we need. Now let us hear from Mr Shiffman again:

The Math of Waveform Addition

In general, we can write a sum of sine/cos waves as:

\[ f(\theta) = \frac{1}{a_0} + \sum_{k=0}^{\infty} a_k*sin(k\theta) + b_k*cos(k\theta) \tag{1}\]

\(f\) is the desired time-waveform, the \(\theta = 2\pi\times fundamental~frequency \times t\), and the \(a_k\) and \(b_k\) are weights for the individual components that are to be designed.

How does this look like? Mr. Shiffman again:

Now let us see how we can design something using the Additive Method.

Design Principles for Additive Synthesis

How do we do this with intent? We will follow the development in Farnell and Risset and Mathews, (Risset and Mathews 1969) and Moorer.

1. The idea is to take an original sound, analyze that using the Fourier Series, and then use those coefficients to synthesize the sound with code.
2. The coefficients, or parameters, need to be manipulated and transformed with time, in order for the synthesized sound to have a “live” feel.
3. The number of such parameters and their control over time could pose a formidable data management challenge. This leads to the idea of data reduction in order to have a manageable number of these, and generate the sound in its essentials.
4. One essential part of this is to use envelopes around the amplitudes of several sine waves, what is called the ADSR method. This could also lead to several oscillators being turned on or off based on need.
5. So one needs to break down the sound into “principal components” that are harmonically related ( as with the Fourier series) and then fill in inharmonic tones using additional oscillators.

What is ADSR?

ADSR stands for “Attack Decay Sustain Release”. These related to the way a note of music varies over time in a typical piece of music.

Additive Synthesis with Code

Using p5.js

Using R

music <-
  Music() +
  Meter(4, 4) +
  Line(c("C5", "D5", "E5", "F5"))

show(music)

Wait, But Why?

To be Written Up.

References

1. James Moorer. (Nov 1976) The Synthesis of Complex Audio Spectra by Means of Discrete Summation Formulae. Journal of the Audio Society. PDF
2. Jean-Claude Risset, Max V. Matthews. (Feb 1969). Analysis of Musical Instrument Tones. Physics Today. https://sci-hub.se/https://doi.org/10.1063/1.3035399
3. p5.Sound Tutorial.https://pdm.lsupathways.org/6_resources/7_soundandmusic/p5.sound/
4. Sound in p5.js Playlist. https://www.youtube.com/playlist?list=PLRqwX-V7Uu6aFcVjlDAkkGIixw70s7jpW
5. Sounds with Tone.js. https://pdm.lsupathways.org/3_audio/
6. Mister Bomb. p5.Sound project tutorials https://www.youtube.com/playlist?list=PLIsdHp2z9wFl7A1wWb2VmQUUojEGsKELE
7. https://www.cs.cmu.edu/~tcortina/15104-f20/lectures/24-MoreSound.pdf
8. R package gm: the grammar of Music. https://cran.r-project.org/web/packages/gm/vignettes/gm.html
9. Phil Burk,Larry Polansky, Douglas Repetto, Mary Roberts Dan Rockmore. Music and Computers: A Theoretical and Historical Approach https://musicandcomputersbook.com
10. Julius O. Smith. PHYSICAL AUDIO SIGNAL PROCESSING FOR VIRTUAL MUSICAL INSTRUMENTS AND AUDIO EFFECTS https://ccrma.stanford.edu/~jos/pasp/pasp.html

R Package Citations

Package	Version	Citation
gm	2.0.0	Mao (2024)

Mao, Renfei. 2024. gm: Create Music with Ease. https://doi.org/10.32614/CRAN.package.gm.

Risset, Jean-Claude, and Max V. Mathews. 1969. “Analysis of Musical-Instrument Tones.” Physics Today 22 (2): 23–30. https://doi.org/10.1063/1.3035399.