DOKK Library

Kinesynth: Patching, Modulating, and Mixing a Hybrid Kinesthetic Synthesizer

Authors Don Derek Haddad Joseph A. Paradiso

License CC-BY-4.0

     Kinesynth: Patching, Modulating, and Mixing a Hybrid
                   Kinesthetic Synthesizer

                                         Don Derek Haddad, Joseph A. Paradiso
                                         Responsive Environments Group, MIT Media Lab
                                                   [ddh, joep]

This paper introduces the Kinesynth, a hybrid kinesthetic
synthesizer that uses the human body as both an analog
mixer and as a modulator using a combination of capaci-
tive sensing in “transmit” mode and skin conductance. This
is achieved when the body, through the skin, relays signals
from control & audio sources to the inputs of the instru-
ment. These signals can be harnessed from the environ-
ment, from within the Kinesynth’s internal synthesizer, or
from external instrument, making the Kinesynth a mediator
between the body and the environment.                                    Figure 1: Crowd holding hands and connected
                                                                         across patch cords by Paolo Tofani, Area’s guitar
Author Keywords                                                          player and synth wizard who worked at EMS[1].

Synthesizer, Kinesthetic, Electrical Properties of the Body
                                                                         about the connection between humans and sound. Digital
CCS Concepts                                                             synthesis enables far more sonic possibilities than analog
                                                                         synthesis, yet modulars are more popular than ever[9]. The
•Hardware → Electromagnetic interference and compati-
                                                                         Kinesynth in its turn is a controller as well as a synthesizer,
bility; Analog and mixed-signal circuits; Digital signal pro-
                                                                         but fits well with modular systems, given their open array of
cessing; •General and reference → Design;
                                                                         control and audio sources. Although there are too many to
                                                                         enumerate here, the next section describes a few related in-
1.    INTRODUCTION                                                       struments or controllers that emphasize through-body and
The pool of signals that can be generated from the elec-                 related dynamic mixing & control.
trical properties of the human body is endless. In music,
it started with the Theremin, an electronic music instru-                2.   RELATED WORK
ment invented by Léon Theremin in 1920, the predecessor
                                                                         Electronic instrument inventor Ivan Eremeeff developed a
of all capacitive instruments[10]. It works through the ca-
                                                                         device called the Gnome circa 1932, that coupled audio from
pacitive technique known as “Loading Mode” by measuring
                                                                         keys touched by a performer through their body and into
the load formed between an electrode and a capacitively-
                                                                         a receive electrode mounted atop their chair[11]. The left-
coupled body. Another mode of capacitive sensing relies on
                                                                         hand controller of Hugh LeCaine’s mid-1940’s Electronic
putting the body in very close contact with an electrode,
                                                                         Sackbut[12] included a capacitive mixer, where the user’s
and is known as “Transmit Mode”, where electrical signals
                                                                         finger rested atop a capacitive input pad that itself rests
can flow into the circuit passing through the body connected
                                                                         atop an insulated conductive disk that contains separate
to a receiving electrode. For instance, touching conductive
                                                                         sectors with electrodes transmiting different timbres. By
objects with 110/220 VAC line power nearby can channel
                                                                         moving the disk around, you effectively mix the timbres in
a 50/60Hz sine-wave ’hum’ through the receiving electrode.
                                                                         real time, each one weighted by the corresponding surface
The Kinesynth relies on that same idea, but instead of just
                                                                         area overlapped by the pickup pad. In the late 1960s, Michel
using the 50/60Hz line tone, it also includes a miniature-
                                                                         Waisvisz, Dutch inventor of experimental electronic musical
environment with an array of oscillators each tuned to differ-
                                                                         instruments, designed and built the very first Cracklebox,
ent frequencies, various control voltages, and a “patchbay”
                                                                         also known as kraakdoos, together with Geert Hamelberg,
allowing external analog synthesizers to be mixed through
                                                                         and later in the 1970s the Crackle synthesizer. These cir-
the human skin.
                                                                         cuits do not rely on capacitive coupling, but rather use the
 The renaissance of the modular synthesizer in today’s age
                                                                         body’s resistance to control a single oscillator based of the
of embedded digital hardware raises even deeper questions
                                                                         LM709 op-amp[6]. Sal Martirano’s early-70s SalMar Con-
                                                                         struction was a hybrid digital synthesizer controlled by a
                                                                         large matrix of touch buttons[7]. In the modern era of mod-
                                                                         ular synthesizers, many Eurorack[2] manufacturers adopt
                                                                         touch plates in their modules for control (a technique dat-
                      Licensed under a Creative Commons Attribution      ing at least to Buchla[5]), like the make noise René, Pres-
                      4.0 International License (CC BY 4.0). Copyright   sure Points, and Teleplexer Modules, shown in figure 2 to
                      remains with the author(s).                        name a few[3]. Closer to our approach, the Hyve-synth is
NIME’18, June 3-6, 2018, Blacksburg, Virginia, USA.                      a 60-voice polyphonic analog synthesizer that presents cir-
cuit points from each oscillator at touch pads to enable pitch   The 8 step sequencers are implemented via two dividers us-
bend and mixing based on the resistance of the skin[4]. HCI      ing the CD4022 chip. The patchbay connects electrodes in
researchers have also made interactive surfaces that work        our matrix to both inputs and outputs of external synthe-
through skin resistance[8].                                      sizers via 3.5mm mono jacks. The output impedance across
                                                                 these circuits is matched with 10K resistors.

Figure 2: Make Noise’s touch modules, from left to
right, the Teleplexer, Pressure Points and René.

The current version of the Kinesynth includes some inputs
and several outputs. The instrument is played through
bridging the inputs to the various outputs through the fin-      Figure 4: The Kinesynth wired to a Moog Mother-
gers. The body is coupled to the circuit through an elec-        32 semi-modular synthesier via patchbays connec-
trode that connects to the inverting pin of an op-amp con-       tions and patch cables.
figured as a relaxation oscillator, shown in figure 3. The

                                                                 4.   CONCLUSION
                                                                 In this paper, we presented the Kinesynth instrument, a
                                                                 kinesthetic synthesizer, that uses the capacitive and resis-
                                                                 tive properties of the human body to mix and modulate
                                                                 analog signals coming from the environment, from on-board
                                                                 audio sources, or from external eurorack synthesizers.

                                                                 5.   REFERENCES
                                                                  [1] Area at Milan’s Parco Lambro Festival 1976.
                                                                  [2] Doepfer a-100 eurorack spec:.
                                                             man/a100m e.htm.
                                                                  [3] Make noise modular synthesizer manufacturer.
                                                                  [4] The Hyve touch synthesizer.
                                                                      URL:, 2018.
                                                                  [5] D. Buchla. A history of Buchla’s musical instruments.
                                                                      In In Proc. of NIME, 2005.
                                                                  [6] E. Dykstra-Erickson and J. Arnowitz. Michel
Figure 3: Schematic of the Kinesynth’s input, show-                   Waisvisz: the man and the hands. interactions,
ing a relaxation oscillator compared against an LFO.                  12(5):63–67, 2005.
                                                                  [7] S. Martirano. “Progress Report #1 — An electronic
simplest way to generate a sound is by making contact with
                                                                      music instrument which combines the composing
conductive appliances sitting near 110/220 VAC power lines
                                                                      process with Performance in real-time”. UIUC
that will modulate the oscillator at a rate of 50/60Hz. In-
                                                                      Internal Report, 1971.
creasing the amount of skin in contact with the modulat-
ing surface can lead to an increase on the op-amp’s input         [8] N.W Gong, et al. Printsense: a versatile sensing
impedance, resulting in a faster drop of the oscillator’s fre-        technique to support multimodal flexible surface
quency. At the second stage, both the relaxation oscillator           interaction. In In Proc. of CHI 2014, pages
and another low frequency oscillator, generating a square-            1407–1410.
wave at the rate of 1Hz-1kHz, connect to the input pins of a      [9] J. A. Paradiso. “The modular explosion-deja vu or
comparator, adding a tremolo effect to the final output. In           something new?,” Presented at the Voltage Connect
addition, the instrument provides touch pads connected to             Conference, Berklee College of Music. 2017.
four oscillators, two low frequency oscillators (LFO), three     [10] J. A. Paradiso and N. Gershenfeld. Musical
cascading oscillators (FM), two 8-step sequencers, and a              applications of electric field sensing. Computer music
patchbay with 18 outlets that can be used as either inputs            journal, 21(2):69–89, 1997.
or outputs to be wired to external sources like function gen-    [11] T. Rhea. ’Electro-Mechanical Instruments’, in Darter,
erators, and/or other synthesizers. These oscillators are             T. (ed), The Art of Electronic Music. GPI
made from a 74HC14 Hex inverting Schmitt trigger, and                 Publications, NY, 1984.
each generate a pulse wave. The 3-stage FM synthesizer is        [12] G. Young. The Sackbut Blues: Hugh Le Caine,
designed through three cascading oscillators, built using the         Pioneer in Electronic Music. National Museum of
CD4093, a quad dual-input NAND Schmitt Trigger chip.                  Science &, 1989.