FM based Sound Synthesis

Introduction

So we understand the Fourier Transform: we can express any waveform as a sum of sinusoids that are appropriately weighted and are at discrete multiples of a chosen “fundamental frequency”.

In the module Additive Synthesis, we learnt about how a sum of properly tuned and weighted frequency components can be used to synthesize sounds. Here is another approach: of frequency-modulation, or changing the frequency of a base carrier wave to generate a large multiple of harmonic and in-harmonic sidetones to synthesize sound.

Inspiration

To be Found!!

What is Frequency Modulation?

In Frequency Modulation, we take a carrier oscillator and change its frequency in accordance with another signal called the modulation signal. There is no change in amplitude, just the instantaneous frequency. We write this change as:

\[ \Large\omega_i(t) = \omega_c + \Delta_{\omega}\ cos(\omega_mt)\Large \tag{1}\]

where

• \(\omega_i(t)\) is the Instantaneous Frequency of the carrier wave
• \(\omega_c\) is the center (undeviated) Frequency of the carrier wave
• \(\omega_m\) is the Frequency of the modulation wave, the rate at which FM changes the carrier's frequency
• \(\Delta_{\omega}\) is an amplitude-like parameter, which controls the amount of frequency deviation.

So what does an FM signal look like?

Figure 1: FM Waveforms

Metaphorically, this resembles the Slinky toy, where the slinky is the carrier wave, and the hand moving it up and down is the modulating wave.

FM Wave Instantaneous Frequency and Phase

As we know, we need the instantaneous phase of the carrier wave to write out the equation for the FM wave, and analyze it further. The instantaneous phase is the integral of the instantaneous frequency

\[ \begin{align} \theta_i(t) &= \int_0^t \omega_i(\tau) d\tau\\ &= \int_0^t \left[ \omega_c + \Delta_{\omega}\ cos(\omega_m \tau) \right] d\tau\\ &= \omega_c t + \frac{\Delta_{\omega}}{\color{red}{{\omega_m}}} {\color{red}{\sin}}(\omega_m t) \end{align} \]

Writing out the equation for the resulting FM-wave, we have:

\[ \begin{align} x_{fm}(t) &= A_c \cos \left(\omega_c t + \frac{\Delta_{\omega}}{\omega_m} \sin(\omega_m t) \right)\\ \\ &= A_c \cos \left(\omega_c t + I \cdot \sin(\omega_m t) \right)\\ \end{align} \tag{2}\]

where

• \(A_c\) is the Amplitude of the carrier wave
• \(I = \large{\frac{\Delta_{\omega}}{\omega_m}}\) is the Modulation Index, or Peak Phase Deviation.

This looks like the phase of the carrier (the portion inside parentheses in Equation 2) is like:

Figure 2: FM Wave Instantaneous Phase

There is a linear portion from the original unmodulated carrier (slope = \(\omega_c\)) with an added-on sine wave that tends to increase and reduce the phase periodically.

NotePhase vs Time

The net phase at any time still is increasing monotonically! Slope is always positive! Hmm…what if it was not ????

FM Spectrum

So much for time waveforms.

What we hear is more easily understood using the frequency spectrum. To understand the spectrum of the FM wave, we need to expand the Equation 2. Doing so, we would obtain a rather intimidating, but ultimately rewarding expansion:

\[ \Large{x_{fm}(t) = A_c \Bigg[ cos(\omega_ct)*\color{orange}{cos(Isin(\omega_mt))} \color{black}{-} \\ sin(\omega_ct)*{\color{red}{sin(Isin(\omega_mt))}}\Bigg]}\Large \tag{3}\]

A \(sin\) and a \(cos\) inside a \(sin\) and a \(cos\) ??!! How do we expand that?? Well, we have two identities:

\[ \Large\color{orange}{cos(Isin(\omega_mt))} = {J_0(I) + 2 \sum_{n=1}^{\infty} J_{2n}(I)~ * ~ cos(2n \omega_m t)}\Large \tag{4}\]

and

\[ \Large\color{red}{sin(Isin(\omega_mt))} = {2 \sum_{n=0}^{\infty} J_{2n+1}(I)~ * ~ sin((2n+1) \omega_m t)}\Large \tag{5}\]

where the \(\color{blue}{J_n(I)}\) are the Bessel functions of the first kind of order n. (More on these beautiful creatures shortly!)

Plugging these into the Equation 3, we get:

\[ \begin{flalign} x_{fm}(t) &= A_c \Bigg[\cos(\omega_ct) \Bigg( J_0(I) + 2 \sum_{n=1}^{\infty} J_{2n}(I) cos(2n \omega_m t) \Bigg) - sin(\omega_ct) \Bigg( 2 \sum_{n=0}^{\infty} J_{2n+1}(I) sin((2n+1) \omega_m t) \Bigg) \Bigg]\\ &= A_c \Bigg[ J_0(I) cos(\omega_ct) + 2 \sum_{n=1}^{\infty} J_{2n}(I) {\color{red}{cos(\omega_ct) cos(2n \omega_m t)}} {\color{green}{~\pmb{---}~}} 2 \sum_{n=0}^{\infty} J_{2n+1}(I) {\color{blue}{sin(\omega_ct) sin((2n+1) \omega_m t)}} \Bigg]\\ \end{flalign} \]

And using the much-loved-and-much-feared \(\color{red}{cosAcosB}\) and \(\color{blue}{sinAsinB}\) expansions:

\[ \begin{flalign} x_{fm}(t) &= A_c \Bigg[ J_0(I) cos(\omega_ct) + \sum_{n=1}^{\infty} J_{2n}(I) {\color{red}{\Big( cos((\omega_c + 2n \omega_m)t) + cos((\omega_c - 2n \omega_m)t) \Big)}} {\color{green}{~\pmb{+++}}} \sum_{n=0}^{\infty} J_{2n+1}(I) {\color{blue}{\Big( cos((\omega_c + (2n+1) \omega_m)t) - cos((\omega_c - (2n+1) \omega_m)t) \Big)}} \Bigg]\\ &= A_c \Bigg[ J_0(I) cos(\omega_ct) + \sum_{n=1}^{\infty} J_{n}(I) \Big( cos((\omega_c + n \omega_m)t) + (-1)^n*cos((\omega_c - n \omega_m)t) \Big) \Bigg] //What?? How??? Where?? When?? \end{flalign} \]

And so finally:

\[ {\boxed{\begin{multline*} x_{fm}(t) = A_c \bigg[ {J_0(I)} {\cos(\omega_c t) ~ \text{//carrier}}\\ + {J_1(I)} \bigg(cos(\omega_c + \omega_m)t - cos(\omega_c - \omega_m)t\bigg) ~ \text{//first order sidebands}\\ + {J_2(I)}\bigg(cos(\omega_c + 2\omega_m)t + cos(\omega_c - 2\omega_m)t\bigg) ~ \text{//second order sidebands}\\ + {J_3(I)}\bigg(cos(\omega_c + 3\omega_m)t - cos(\omega_c - 3\omega_m)t\bigg) ~ \text{//third order sidebands}\\ ...\\ + {J_n(I)}\bigg(cos(\omega_c + n\omega_m)t + cos(\omega_c - n\omega_m)t\bigg)\bigg] ~ \text{//nth order sidebands} \end{multline*}}} \tag{6}\]

We can plot a graph of FM carrier + sideband amplitudes vs frequency. What does this look like?

Figure 3: FM Spectrum

In FM, the modulation process takes power from the carrier and distributes it into the sidebands. Since the instantaneous frequency is proportional to the modulating signal, we get a mathematical structure that looks like \(cos(\omega_c t + I*cos(\omega_m t))\). Expanding this using trigonometric identities and Taylor series leads to the Bessel functions as weights for the sidebands.

The spectrum is symmetric of course, with sidebands going off in both directions in pairs. The carrier also not necessarily the highest component, as expected. To recap, we see that with FM:

• We have a carrier component (weighted by \(\color{red}{J_0(I)}\)). The carrier disappear for certain values of \(I\) when \(\color{red}{J_0(I)} = 0\);
• Pairs of sidebands around the carrier at a separation of n-multiples of \(\omega_m\), weighted by \(\color{red}{J_n(I)}\). In like fashion these pairs can also disappear at the zeroes of \(\color{red}{J_n(I)}\).
• In general, the larger the order \(n\), the larger the \(I\) needed to produce sidebands of that order, since \(\color{red}{J_n(I)}\) are slow starters when \(n\) is large.

So…there are sidebands that go into negative frequency territory…what could that mean?? That is where Chowning’s insight must have come from!

What is Chowning’s Method for FM-based Synthesis?

John Chowning, in 1973, proposed a method of sound synthesis using Frequency Modulation. The method is based on the following ideas:

1. The carrier frequency \(\omega_c\) is chosen to be the fundamental frequency of the desired sound.
2. The carrier frequency and the modulation frequency are both in the audio band ! As opposed to conventional FM transmission where carriers are in the MHz range, with audio-range modulation signals.
3. The modulation index \(I\) is chosen to control the timbre of the sound, i.e., the relative amplitudes of the sidebands.
4. The amplitude \(A_c\) of the carrier wave is chosen to control the loudness of the sound.

Consider what happens when we choose different values of \(k\), with these ideas in mind: We will first choose \(\omega_c\) and \(\omega_m\) that have an integer ratio, and then later an irrational ratio (e.g. \(\sqrt(2)\)) to see the effect on the harmonicity of the resulting sound spectrum.

Figure 4: FM with varying index - harmonic sidebands

Figure 5: FM with varying index - harmonic sidebands

Figure 6: FM with varying index - harmonic sidebands

Figure 7: FM with varying index - harmonic sidebands

Figure 8: FM with varying index - harmonic sidebands

Figure 9: FM with varying index - harmonic sidebands

Dealing with negative frequency sidebands

Note that several sidebands go past \(0\) Hz into negative territory!! How do we understand these??

NoteNegative Frequencies

What could a negative frequency sideband mean?? In reality, negative frequencies are a mathematical construct that arise from the Fourier Transform and complex exponentials. A negative frequency component can be interpreted as a positive frequency component with a phase inversion. In the context of FM synthesis, negative frequency sidebands can be thought of as contributing to the overall waveform in a way that affects its shape and timbre, but they do not correspond to physically realizable sound waves. Instead, they influence the constructive and destructive interference patterns that shape the audible spectrum.

We can imagine these negative-frequency sidebands as phasors with opposite rotation direction, which when combined with positive-frequency components, yield the final time-domain signal.

So we reflect them at \(0\) with a phase inversion. Here they might combine with spectral components at positive frequencies, either constructively or destructively, depending on their relative phases and amplitudes. This generates the final sound with its final timbre.

Let us take one example from the graphs above, say \(I=4\) and see what the final spectrum looks like. Then we will code this in p5.js and R and make some noise!!

(a) Harmonic Sidebands (wm = wc /n)

(b) Inharmonic Sidebands (wm = wc / sqrt(n))

Figure 10: Reflecting negative frequency sidebands

FIGURES TO BE CHECKED!!!!

Design Principles for FM (Chowning) synthesis

FM (Chowning) synthesis is built from a small set of parameters—carrier frequency, modulator frequency, modulation index, and envelopes—that together determine where spectral components fall and how strong they are over time.

• By choosing a rational carrier‑to‑modulator ratio, spectra are harmonic;
• By choosing an irrational or odd ratio, spectra become inharmonic.[1][2][3]

Core FM steps

• Choose target pitch (fundamental) in the audible range, and set the carrier frequency \(c\) to that or a related harmonic. For example, for the note A3 choose \(c = 220~Hz\).[2]
• Choose a modulator frequency \(m\). Its value and its ratio \(c/m\) determine the spacing and harmonic/inharmonic character of the sidebands at \(c \pm k * m\).[1][2]
• Choose a peak frequency deviation \(d\) and modulation index \(I = d/m\); this controls how far the instantaneous frequency swings and how many sidebands have significant amplitude.[2][1]
• Compute or intuit the approximate bandwidth using Carson’s rule \(B \approx 2(m*(I+1))\) to ensure audible components stay within the desired spectrum and avoid aliasing for given sample rate.[1]
• Design amplitude and index envelopes so that loudness and brightness evolve together (e.g., high index at attack, decaying over time), yielding dynamic spectra that resemble many natural sounds.[4][2]
• Implement the FM signal as \(x(t) = A \sin(2\pi c*t + I \sin(2\pi m*t))\), optionally making \(A\) and \(I\) time‑varying through envelopes, or based on human input!! [2][1]

Design parameters and how to set them

• Carrier frequency \(c\)
  • Meaning: Average frequency and perceived pitch if the spectrum is harmonic.[2]
  • How to set:
    • For harmonic tones, set \(c\) equal to the desired fundamental (e.g., 440 Hz) or an integer multiple if the ear will still infer the missing fundamental.[2]
    • For inharmonic “pitched noise” or bells, \(c\) need not match a musical pitch exactly; it sets a central region of energy.[3]
• Modulator frequency \(m\)
  • Meaning: Spacing between sidebands \(c \pm k*m\) and, through \(c/m\), the harmonic or inharmonic pattern.[1][2]
  • How to set:
    • Harmonic: choose a rational ratio \(c/m = p/q\) (often integers, like \(1:1, ~ 2:1, ~ 3:2\)), so sidebands fall on a harmonic series of some fundamental.[1][2]
    • Inharmonic: choose irrational or “odd” ratios such as \(c/m = \sqrt{2}\) or similar non‑integer relations so sidebands fall between harmonic positions.[3][1]
• Modulation index \(I = d/m\)
  • Meaning: Ratio of peak deviation \(d\) to modulator frequency; determines number and relative amplitude of significant sidebands.[1][2]
  • How to set:
    • Small \(I\) (near 0) yields almost pure sine tones with very few sidebands (soft timbre).[2]
    • Moderate \(I\) gives a few strong sidebands (clarinet‑like or brass‑like spectra). Practical musical values often lie roughly between 1 and 10 for simple single‑operator examples.[1][2]
    • Larger \(I\) produces many sidebands and brighter, more complex or noisy timbres until aliasing or excessive roughness appears.[1]
• Amplitude \(A\) and envelopes
  • Meaning: Overall loudness and time evolution of loudness and brightness, since envelopes can modulate both \(A(t)\) and \(I(t)\).[4][2]
  • How to set:
    • Use an amplitude envelope with attack, decay, sustain, and release appropriate to the instrument (e.g., fast attack/decay for percussive sounds, slower for pads).[4]
    • Tie the index envelope to the amplitude envelope so that when amplitude decays, the index decreases, causing the spectrum to lose high‑frequency content over time, as in many acoustic sounds.[4][2]
• Carrier–modulator ratio \(c/m\)
  • Meaning: Positions of spectral components; with reflection from negative frequencies, this ratio still controls whether the pattern is harmonic or inharmonic.[3][2]
  • How to set:
    • Choose low integer ratios (e.g., 1:1, 2:1, 3:1) for clear, consonant harmonic spectra.[2][1]
    • Use more complex ratios (e.g., 5:2, 7:3) for richer but still roughly harmonic tones, or irrational ratios (e.g., 1:\(\sqrt{2}\)) for bell‑like inharmonic spectra.[3]

Example: harmonic FM tone

Design a harmonic spectrum centered on A4 with clearly related overtones, all within audible range.

• Choose fundamental and carrier: Target pitch A4, fundamental 440 Hz; use \(c = 440\) Hz so the carrier itself is the fundamental.[2]
• Choose modulator with harmonic ratio: Take \(c/m = 2/1\), so set \(m = 220\) Hz. This yields sidebands at \(440 \pm k\cdot 220\) Hz: 220, 440, 660, 880, 1100 Hz, etc., which lie on the harmonic series of 220 Hz (harmonics 1, 2, 3, 4, 5…).[1][2]
• Choose index: Let \(I = 4\); this provides several strong sidebands before amplitudes taper, giving a bright harmonic timbre without extreme spectral width.[2][1]
• Check bandwidth: With \(m = 220~Hz\) and \(I = 4\), Carson’s rule gives \(B \approx 2(m(I+1)) = 2 \cdot 220 \cdot 5 = 2200~Hz\), so significant components lie roughly between about 0 and 2600 Hz, safely inside the audible band.[1]
• Define signal and envelopes:
  • Audio formula: \(x(t) = A(t)\,\sin(2\pi\cdot 440\, t + I(t)\,\sin(2\pi\cdot 220\, t))\).[2][1]
  • Set \(A(t)\) with fast attack (5–10 ms), moderate decay, small sustain, and short release for a plucked‑string‑like sound; let \(I(t)\) start near 4 at attack and decay toward 1 or 0.5 to mimic natural bright‑then‑dull behavior.[4][2]

Result: A clearly pitched harmonic tone at 440 Hz with controllable brightness, where timbral evolution is governed mainly by the index envelope while pitch remains set by the carrier.[1][2]

Example: inharmonic FM tone

Design a bell‑like inharmonic sound centered in the midrange, with components still clearly audible.

• Choose central carrier: Use \(c = 600~ Hz\) to keep energy in the middle of the spectrum.[3]
• Choose modulator with inharmonic ratio: Let \(c/m = \sqrt{2}\); solve \(m = c/\sqrt{2} \approx 600/1.414 \approx 424.3~Hz\).[3]
  • Sidebands appear at \(600 \pm k\cdot 424.3~Hz\), giving components such as about 176, 1024, 1448 Hz, etc., which do not align with any simple harmonic series, creating an inharmonic spectrum.[3][1]
• Choose index: Set \(I = 6\) for a dense, bright spectrum typical of struck metal, with many lateral components before amplitudes fade.[2][1]
• Check bandwidth: With \((m \approx 424 ~ Hz\) and \(I = 6\), Carson’s estimate gives \(B \approx 2\cdot 424\cdot(6+1) \approx 5936~Hz\), so important components range roughly from near 0 up to around 6500 Hz, entirely in the audible band.[1]
• Define signal and envelopes:
  • Audio formula: \(x(t) = A(t)\,\sin(2\pi\cdot 600\, t + I(t)\,\sin(2\pi\cdot 424.3\, t))\).[2][1]
  • Use a percussive envelope: very fast attack, no real sustain, and a long exponential decay on both amplitude and index so the sound starts bright and metallic then gradually loses high‑frequency content like a bell.[3][2]

Result: An inharmonic, bell‑like tone whose energy is centered around 600 Hz but whose partials form a non‑integer pattern determined by the irrational carrier–modulator ratio and relatively high index.[3][1]

FM Synthesis with Code

Using p5.js

Using R

Wait, But Why?

• With Additive Synthesis, we have a large number of oscillators, one for each component derived from Fourier Analysis
• These give rise to a lot of controllable/settable parameters which may be hard to manage, and to tweak if interactivity is desired
• It m-a-y be computationally intensive…
• FM Synthesis has relatively fewer parameters and is very flexible
• Tuning the parameters for on-the-fly composition is easier
• Adding ADSR and timbre-related envelopes give rise to very authentic sounding musical instruments

References

1. John Chowning.(September 1973). The Synthesis of Complex Audio Spectra by Means of Frequency Modulation. Journal of the Audio Society. PDF
2. Chapter 4: The Synthesis of Sound by Computer; Section 4.7: FM Synthesis. http://sites.music.columbia.edu/cmc/MusicAndComputers/chapter4/04_07.php
3. https://archive.p5js.org/examples/sound-frequency-modulation.html
4. https://multimed.org/student/eim/en/04-FM.pdf
5. http://black.winny.free.fr/MAO/Books/John%20Chowning%20-%201973%20-%20The%20Synthesis%20of%20Complex%20Audio%20Spectra%20by%20Means%20of%20Frequency%20Modulation.pdf
6. https://yamahasynth.com/wp-content/uploads/images/fm_synthesispaper-2.pdf
7. https://www.perfectcircuit.com/signal/what-is-fm-synthesis
8. https://web.uvic.ca/~aschloss/course_mat/MU307/MUS307_MATERIALS/Chowning_FM.pdf
9. http://netmedia.zju.edu.cn/multimedia2013/books/Sound_Synthesis_Sampling.pdf
10. https://articles.roland.com/ultimate-guide-fm-synthesis/
11. https://cdsassets.apple.com/live/6GJYWVAV/user/ma1926_logic-pro-10-6-instruments-user-guide.pdf
12. https://thewolfsound.com/fm-synthesis-explained-for-audio-programmers-art-and-science/
13. https://www.scribd.com/document/512116578/Synthesis

Other Tools

1. Strudel
2. Pure Data
3. Envion https://www.peamarte.it/env/envion_v3.6.html

R Package Citations

Package	Version	Citation
ambient	1.0.3	Pedersen and Peck (2025)
gm	2.0.0	Mao (2024)
mosaicCalc	0.6.4	Kaplan, Pruim, and Horton (2024)
plot3D	1.4.2	Soetaert (2025)

Kaplan, Daniel T., Randall Pruim, and Nicholas J. Horton. 2024. mosaicCalc: R-Language Based Calculus Operations for Teaching. https://doi.org/10.32614/CRAN.package.mosaicCalc.

Mao, Renfei. 2024. gm: Create Music with Ease. https://doi.org/10.32614/CRAN.package.gm.

Pedersen, Thomas Lin, and Jordan Peck. 2025. ambient: A Generator of Multidimensional Noise. https://doi.org/10.32614/CRAN.package.ambient.

Soetaert, Karline. 2025. plot3D: Plotting Multi-Dimensional Data. https://doi.org/10.32614/CRAN.package.plot3D.

Appendices

Why is Modulation Index inversely proportional to modulation frequency?

From equation {eq-fm-freq}, we see that the instantaneous frequency deviation is given by \(\delta_f \sin(2 \pi f_m t)\). The maximum deviation is \(\delta_f\).

Consider two cars running along parallel race tracks. And that they are at the same speed and running neck-and-neck. Now, one car starts to speed up and slow down repeatedly. If it speeds up and slows down very quickly, then the distance gained or lost by this car in front is small. If it speeds up and slows down slowly, then the distance gained or lost by the car in front is large.

So, for a given frequency deviation \(\delta_f\) , if the modulation frequency \(f_m\) is high, then the distance gained or lost (i.e the effect of modulation) is small. If the modulation frequency \(f_m\) is low, then the distance gained or lost (i.e the effect of modulation) is large.

Hence the FM modulation index is inversely proportional to the modulation frequency.

\[ % Left alignment of multiline equations \begin{flalign} f(u) & =\sum_{j=1}^{n} x_jf(u_j)&\\ & =\sum_{j=1}^{n} x_j \sum_{i=1}^{m} a_{ij}v_i&\\ & =\sum_{j=1}^{n} \sum_{i=1}^{m} a_{ij}x_jv_i \end{flalign} \]

ImportantWhat are Bessel Functions?

Let us plot them: Bessel functions ( the ones we want anyway ) are written as \(J_{order}(parameter)\). So for the carrier we need \(J_0(I)\) and for any n-order sidebands at spacing \(n*\omega_m\) we need \(J_n(I)\). In the Figure 11 below, we plot Bessel function amplitude vs modulation index, for orders 0 to 6. The vertical black line shows the chosen modulation index \(I = 2\) in Figure 3. We can read off the values of the different Bessel functions at this value of \(I\) to obtain the sideband amplitudes. Note here that these are all positive; if \(I\) were larger, then the amplitudes can go to \(0\) or even \(negative\). What would that mean??

Figure 11: Bessel Functions