The Science of Harmony
A Psychophysical Basis for Harmony
This article is an attempt to explain my paper (by the same title) published to Research, a Science Partner Journal in common-folk English. I realize it does explode to geek-speak towards the end for the benefit of fellow geeks who make it that far, but I hope at least the main bits are comprehensible to most. doi: 10.34133/2019/2369041
​
https://spj.sciencemag.org/research/2019/2369041/
A quantifiable & verifiable basis for:
1. Chord Tension: Why do some chords sound consonant and others sound dissonant? <- EASY...
​ 2. Chord Resolutions: Why do some chords resolve better to other chords? <- HARD!
Tension
Resolution
Quick Links
Click these links to zoom to the section where you have left off
The Harmonic Series - each note is made up of a series of sinusoids.
Modulations - when two or more notes come together, groups of sinusoids are perceived together as a single dancing waveform.
Interharmonic Modulations - modulations across adjacent sinusoids (their frequencies are neighbouring).
Problems with Interharmonic Modulations - electrophysiological and psychoacoustic tests show that chords formed by notes without harmonics harmonics still sound harmonious.
Subharmonic Modulations - modulations across distant sinusoids (their frequencies are far apart).
Stationary Subharmonic Modulations - explain tension, or chord sonorities.
Pachelbel’s Cannon in D - subharmonic tension analysis
Transitional Subharmonic Modulations - explain resolution, or chord progressions.
Beethoven’s Moonlight Sonata - subharmonic resolution analysis
Algorithm, Experiments & Results - with perception and chord-use statistics
Paper - link to paper published to Research, a Science Partner Journal
Other interesting things about subharmonic modulations
The Harmonic Series
Every periodic (i.e. repeating) waveform is made up of a series of sinusoids. (See Fourier Series)
For example, the following periodic waveform
is composed of the following sinusoids summed together:
Notice the first sinusoid repeats at the same period as the composite waveform.
We say it has the same fundamental frequency which means it repeats at the same rate.
The fundamental frequency is determined by the reciprocal of its period. f=1/t.
Notice the remaining sinusoids have frequencies at integer multiples ( 2x, 3x and 4x… ) of the fundamental.
We say they repeat at harmonic frequencies and refer to them as harmonics.
Naturally occurring periodic waveforms usually have more than 4 harmonics (with the fundamental counting as the first harmonic), but, 4 are sufficient as an example since higher harmonics tend to be increasingly smaller in magnitude.
Harmonics are very important to our ears towards timbre recognition, formant formation and sound localisation. We will soon see how they also are important towards the perception of stationary harmony.
If we ignore the phase at which contributing sinusoids line up, we can represent a periodic waveform by the magnitude of its harmonics as such:
We call this plot the spectrum of the waveform, and the process of obtaining this is called spectral analysis.
Here are more examples of periodic waveforms, their contributing sinusoids, and their spectra:
Modulations
When two or more notes come together, groups of sinusoids are perceived together as a single 'dancing' waveform.
​
For example, when this sinusoid from one note:
is perceived together with this sinusoid of proximate frequency from another note:
They are perceived as a single sinusoid of their average frequency, dancing at a rate equivalent to half their difference in frequencies:
This apparent dancing is an example of what we call modulation..
Interharmonic Modulations
We can categorise modulations according to the difference in frequencies of their contributing sinusoids.
In the previous example, both frequencies were small ( with 10% difference across them ).
We call these interharmonic modulations.
Interharmonic modulations have been commonly studied over the years and were first introduced an attribute of harmony by Hermann von Helmholtz as beating frequencies.
In general, the larger the difference within a certain threshold, the greater the dissonance of the chord:
Whereas, the smaller the difference, the greater its consonance.
However, if the difference were small to the brink of zero, the amplification and attenuation effect of modulation would be too slow to notice with respect to the note’s onset and release, hence, in such cases, the pair of sinusoids in question make no contribution to harmony.
An example of interharmonic frequencies with notes c5 and e5 is seen below
Where the harmonics of the note c5 is in blue and those for the note e5 are in red.
Sinusoid pairs within the threshold that could possibly contribute to consonances (low-frequency modulations) and dissonances (beat frequencies) are marked out with their frequency differences (∆f) as labeled in Hertz.
Problems with Interharmonic Modulations
Although it has been well established as an attribution of harmony, there are several discrepancies with interharmonic modulations that remain unresolved.
For example, consider pure-tone harmony - i.e. a chord played with pure sinusoids (on instruments such as the sine-wave theremin, glass harmonium or musical saw…) without harmonics have no interharmonic modulations within threshold. c5 and e5, again, with pure sinusoids might look like this:
If consonances and dissonances were attributed to interharmonic modulations alone, the pleasant effect of harmony would not be experienced here.
However, psychoacoustic and electrophysiological experiments show that the pleasant effect of harmony is experienced even with chords played in this manner.
Here is such an example of pure-tone harmony on a musical saw duet:
Thus, interharmonic modulations alone cannot explain our experience of harmony.
By the end of this article, we will show that subharmonic modulations is the missing piece of the puzzle.
Subharmonic Modulations
Since it is established that modulations across the sum of adjacent (i.e. frequency difference within threshold) sinusoids cannot explain pure-tone harmony, it’s is easy to deduce that modulations across the sum of distant sinusoids must also play a part.
(In fact, we will later see that not only can they explain pure-tone harmony. They can even explain something which, to the best of our knowledge, has never attempted before in acoustic science: chord progression! )
-
Interharmonic Modulations: modulations resulting in the interplay across adjacent sinusoids.
-
Subharmonic Modulations: modulations resulting in the interplay across distant sinusoi
​
Common Subharmonic Periods and Subharmonic Tension
If we consider the waveform composed of the sum of the fundamental sinusoids of each note of a C Major chord composed of c3, e3, g3 and c4, (only the fundamentals will be considered (1) since we are ultimately interested in the length of their overall periods and (2) since lengths of each fundamental are integer multiples of lengths of their harmonics), we can describe it as periodic with distortions to the waveform with each successive cycle.
Common Subharmonic Periods and Subharmonic Tension
If we consider the waveform composed of the sum of the fundamental sinusoids of each note of a C Major chord composed of c3, e3, g3 and c4, (only the fundamentals will be considered (1) since we are ultimately interested in the length of their overall periods and (2) since lengths of each fundamental are integer multiples of lengths of their harmonics), we can describe it as periodic with distortions to the waveform with each successive cycle.
We can describe this repetition with 2 parameters:
-
the time it takes to repeat, marked Tsub. We call this the common subharmonic period, or simply the common subharmonic.
-
the amount of distortion to the waveform with each successive cycle - this may be determined by the time difference between the first and last negative-to-positive zero-crossing amongst each contributing sinusoid at the end of the first cycle marked ∆t in the figure. We call ∆t the subharmonic tension at Tsub.
In chords perceived to be higher in tension in traditional music, we see a correspondingly higher subharmonic tension, ∆t.
It is easier to imagine what is really happening with this animation of subharmonic tension across the two chords.
Stationary Subharmonic Modulations
Since, with subharmonic modulations, we are interested in the period more than magnitude.
We can rotate the plot of C Major above clockwise to have subharmonic periods in the vertical axis.
This frees up the horizontal axis for the representation of time spanned in bars.
Pachelbel’s Cannon in D - subharmonic tension analysis
Subharmonic plot of Pachelbel’s Cannon in D for subharmonic tension analysis::
-
Horizontal axis: Time spanned in bars
Subharmonics are colored to match their corresponding notes on the music score. (For example, in the first bar, all subharmonics of f#5 are marked out in red, followed by d5 in orange, a4 in yellow, d4 in green, a3 in blue and d3 in purple.)
-
Vertical axis: Subharmonic wave period in milliseconds
Vertical axis runs top-down to put shorter wave periods at the top because they correspond to higher notes & frequencies.Tails of each marker represent time spanned.
-
The musical score runs in parallel at the bottom of the plot as reference.
-
Equal temperament is used except where specified.
*Considering the fact that this example is not used for transitional harmony, all chords are presented in its root position at the expense of introducing parallel 5ths in the interest of normalization across chords. for fairer comparison
Interpretation
Subharmonics: Each dash mark an integral multiple of the fundamental period of a note in milliseconds
Notes and Melody Line: First dash of each colour (below the horizontal axis) marks each note. Topmost dashes (below the horizontal axis) of every bar (in this case, red) together form the melody line: f#5-e5-d5-c#5-b4-a4-b4-c#5.
Chords and Coincidence: Common subharmonics of each chord are visualised in regions with the approximate coincidence of dashes of every colour.
Key: Every note of the diatonic share a common subharmonic. Hence, it is possible to identify the key of a songs by its common subharmonic. This part is visualized and so-plot in rational tuning.
Contributing subharmonics from different notes are seldom precisely coincident. Major chords have better coincidence than minor chords, and triads coincide better than sevenths and extended chords.
Accounting for Ambiguity
Up till now, we have considered only one coincidence for every chord.
One might notice at this point, that more than one coincidences are often present.
This is why it is difficult to reduce the experience of a chord to a degree of consonance or dissonance in general.
In general, our ears identify most with the smallest subharmonic tensions at the shortest common subharmonic periods (smallest ∆t at shortest Tsub).
Even as ∆t is a good way to identify individual regions of tension, for comparing across chords we need to
consider all small ∆t across multiple Tsub within range. Apart from this, we also need to take interharmonic modulations into consideration.
A programmatic algorithm to take all this in consideration this will be presented towards the end of this article and results of which are nothing short of astounding. - But first, let us take a look at how subharmonic modulations can explain something even more astounding: resolution in chord progressions.
Transitional Subharmonic Modulations
The most interesting thing about subharmonic modulations (its biggest contribution to our understanding of harmony acoustics) is its means of explaining chord resolution.
In a chord transition, our ears identify with the shortest common subharmonics of the succeeding chord, and the nearest subharmonics of each note of the previous chord are perceived to transit to it. We write
​
∆∆t = ∆tpreceding - ∆tsucceeding
where
-
∆∆t is subharmonic resolution, or tension released.
-
∆tsucceeding is the subharmonic tension at shortest common subharmonics of the succeeding chord
-
∆tpreceding is the subharmonic tension computed with the nearest (nearest to ∆tsucceeding) subharmonics of each note of the previous chord.
Types of Subharmonic Trajectories:
Convergence & Resolutions (∆∆t>0): Because ∆tsucceeding is taken from a common subharmonic while ∆tpreceding isn’t, subharmonic trajectories almost always converge, resulting in a resolution. We can describe this as a release of transitional subharmonic tension. We will later see that the size of subharmonic resolutions correspond to the aesthetics of the resolution.
Divergence & Complications (∆∆t<0): Occasionally, it may diverge - and we call this negative resolution, or complication and describe this as a build up of transitional subharmonic tension. Negative resolutions normally occur in a transition to a more complex chord of the same root as the previous one.
Excursions: Sometimes, subharmonic trajectories of all notes in the chord rise or fall simultaneously. We still analyse them according to the polarity, but make a special note that in such cases, the cumulative effect of simultaneously rising or falling trajectories also contributes to the valence of the chord transition apart from ∆∆t.
With this, we can perform an analysis on the tensional development of complications and resolutions in a stanza of Beethoven’s Moonlight Sonata, Adagio Sostenuto.
Beethoven's Moonlight Sonata - subharmonic resolution analysis
Subharmonic plot of Beethoven's Moonlight Sonata for subharmonic resolution analysis:
The song starts off with a C#m chord, whence, the common subharmonic is observed around a wave period of c#. Our ears adhere especially to the shortest one, which is at c#2. Moderate subharmonic tension is attributed to the complex tensions within a minor chord.
-
At region 1, this transits to a C#m/B chord. The tension built up with the divergence of subharmonic trajectories are marked by arrows in the figure. The subharmonic plot provides a physical (acoustic) verification of perceptual build up of tension in music seen in a further complication to the existing minor tension.
-
At region 2, there is a convergence to a momentary (half-bar) low-tension A chord. The uplifting effect of a large tension release, is counterbalanced by the detrimental effect of a falling melodic sequence, adding to the complexity of the song.
-
At region 3, A transits to a D/F#, which is a Neapolitan chord. The low F# bass extends over 2 octaves below the treble notes, putting a strong common subharmonic at a non-root period of f#1 and creating an amount of stationary tension that is unusual for a major chord.
-
At region 4, the Neapolitan chord resolves to the Dominant 7th, marked G#7 in the figure, with a large perceptual resolution that is signature to bII6-V7 transitions in music. This large tension release is visualized as a large convergence in the subharmonic plot as indicated by the arrows.
-
Musically, the Dominant 7th typically plays the role of building an anticipation for the upcoming return to the Tonic. Beethoven enhanced its function particularly well with a double suspension with staggered resolutions in regions 5a through 5c. The subharmonic plot gives tangibility to the perceptual details with suspension-resolution that have been long theorized about in music that can now be affirmed with visualization.
-
At region 5a, the transition from the G#7 progresses to what is labeled C#m. Functionally, this C#m is however still a G# with a double suspension of the 3rd (b#) to a 4th (c#) and the 5th (d#) to a 6th (e) respectively. The perceptual complication that arises with this transition can be visualized in the subharmonic plot as indicated by the divergence of the green and cyan arrows respectively. The deviation of the suspended notes from the primary triad is visualized as a deviation of their subharmonic trajectories away from the common subharmonic.
-
At region 5b, the tension resolution with the 6th being resolved back down to the 5th can be visualized in the subharmonic plot by its subharmonic trajectory resolving back to the common subharmonic as indicated by the convergent cyan arrow. The continuation of the suspended 4th is visualized in the dotted green arrow.
-
At region 5c, the tension resolution with the 4th being resolved back down to the 3rd can be visualized in the subharmonic plot by its subharmonic trajectory resolving back to the common subharmonic as indicated by the solid green arrow. In preparation for a major resolution back to the upcoming tonic, Beethoven’s touch of genius combines this with a simultaneous complication with the introduction of the 7th at this point. This is visualized in the deviation of its subharmonic trajectory away from the common subharmonic as indicated by the divergent solid yellow arrow.
-
-
At region 6, the Dominant 7th is resolved back to the Tonic with a tension release unique to V7-tonic cadences that is so immense that it is has been long established as the de facto cadence for the end of musical passages. This immense perceptual release of tension, too, is identifiable in the subharmonic plot. From the figure, it may be seen that the common subharmonic of C#m (located at the period of c#1 this time, because of Beethoven’s use of the g#2, which is in purple) lies right in the middle of two common subharmonics of G#7 (located at the periods g#1 and g#0). This unique subharmonic behaviour allows our ears to quite possibly identify with both subharmonics for the preceding ∆t - making ∆tpreceding significantly larger than its ∆tsucceeding. Its staggering convergence produces an immense sense of tension resolution with this transition.
-
At region 7, a final landmark that is interesting to note is, where the triad in the treble flips from the 1st inversion to the 2nd inversion while the chord remains unchanged. Notice that this brings about no change to both the common subharmonic, Tsub, and the subharmonic tension, ∆t, while ∆∆t=0. This, again, shows how subharmonic analysis agrees with music theory where, despite the change of notes, harmony remains the same at this point.
Algorithm, Experiments & Results
As mentioned earlier, individual ∆t and ∆∆t are a good means of identifying regions of tension and resolution in chord sonorities and progression.
However, while our ears identify most strongly with
smallest subharmonic tensions (∆t) at shortest subharmonic periods (Tsub),
-
it is actually affected by the smallest few ∆t
-
it can be ambiguous which is the tallest ∆t at shortest Tsub can be ambiguous
Furthermore, while, individual ∆t and ∆∆t are a good means of attributing perceptual tension to regions of tension and resolution in chord sonorities and progression,
-
we need to consider ∆t at all possible Tsub when comparing across chords
-
a more programmatic algorithm is needed for large databases
In the paper published to Research, a Science Partner Journal, we use
for Subharmonic Chord Tension, &
for Subharmonic Chord Resolution.
Please refer to the paper for details regarding computation.
Results show a high correlation with traditional music theory and perception statistics
for both chord tensions (stationary harmony) and chord resolutions (transitional harmony):
Please refer to the paper for detailed results.
Conclusion
Subharmonic Modulations and the quantification of chord resoutions open the door to countless new perspectives, understanding and possibilities harmony acoustics. and music analysis.
In time I will be featuring several of them here.
​
If you have benefited from this work and would like to venture further, you may wish to read the paper here.
To quote/reference this work, please cite the paper.
​
​
Bibliography & References
Dr Paul Yaozhu Chan
PhD, The Psychoacuostics & Synthesis of Singing Harmony
Institute for Infocomm Research, A*STAR
1 Fusionopolis Way, #21-01, S(138632). SINGAPORE
Tel: +65 9727 8750