Authors Don Derek Haddad Joseph A. Paradiso
License CC-BY-4.0
Kinesynth: Patching, Modulating, and Mixing a Hybrid
Kinesthetic Synthesizer
Don Derek Haddad, Joseph A. Paradiso
Responsive Environments Group, MIT Media Lab
[ddh, joep]@media.mit.edu
ABSTRACT
This paper introduces the Kinesynth, a hybrid kinesthetic
synthesizer that uses the human body as both an analog
mixer and as a modulator using a combination of capaci-
tive sensing in “transmit” mode and skin conductance. This
is achieved when the body, through the skin, relays signals
from control & audio sources to the inputs of the instru-
ment. These signals can be harnessed from the environ-
ment, from within the Kinesynth’s internal synthesizer, or
from external instrument, making the Kinesynth a mediator
between the body and the environment. Figure 1: Crowd holding hands and connected
across patch cords by Paolo Tofani, Area’s guitar
Author Keywords player and synth wizard who worked at EMS[1].
Synthesizer, Kinesthetic, Electrical Properties of the Body
about the connection between humans and sound. Digital
CCS Concepts synthesis enables far more sonic possibilities than analog
synthesis, yet modulars are more popular than ever[9]. The
•Hardware → Electromagnetic interference and compati-
Kinesynth in its turn is a controller as well as a synthesizer,
bility; Analog and mixed-signal circuits; Digital signal pro-
but fits well with modular systems, given their open array of
cessing; •General and reference → Design;
control and audio sources. Although there are too many to
enumerate here, the next section describes a few related in-
1. INTRODUCTION struments or controllers that emphasize through-body and
The pool of signals that can be generated from the elec- related dynamic mixing & control.
trical properties of the human body is endless. In music,
it started with the Theremin, an electronic music instru- 2. RELATED WORK
ment invented by Léon Theremin in 1920, the predecessor
Electronic instrument inventor Ivan Eremeeff developed a
of all capacitive instruments[10]. It works through the ca-
device called the Gnome circa 1932, that coupled audio from
pacitive technique known as “Loading Mode” by measuring
keys touched by a performer through their body and into
the load formed between an electrode and a capacitively-
a receive electrode mounted atop their chair[11]. The left-
coupled body. Another mode of capacitive sensing relies on
hand controller of Hugh LeCaine’s mid-1940’s Electronic
putting the body in very close contact with an electrode,
Sackbut[12] included a capacitive mixer, where the user’s
and is known as “Transmit Mode”, where electrical signals
finger rested atop a capacitive input pad that itself rests
can flow into the circuit passing through the body connected
atop an insulated conductive disk that contains separate
to a receiving electrode. For instance, touching conductive
sectors with electrodes transmiting different timbres. By
objects with 110/220 VAC line power nearby can channel
moving the disk around, you effectively mix the timbres in
a 50/60Hz sine-wave ’hum’ through the receiving electrode.
real time, each one weighted by the corresponding surface
The Kinesynth relies on that same idea, but instead of just
area overlapped by the pickup pad. In the late 1960s, Michel
using the 50/60Hz line tone, it also includes a miniature-
Waisvisz, Dutch inventor of experimental electronic musical
environment with an array of oscillators each tuned to differ-
instruments, designed and built the very first Cracklebox,
ent frequencies, various control voltages, and a “patchbay”
also known as kraakdoos, together with Geert Hamelberg,
allowing external analog synthesizers to be mixed through
and later in the 1970s the Crackle synthesizer. These cir-
the human skin.
cuits do not rely on capacitive coupling, but rather use the
The renaissance of the modular synthesizer in today’s age
body’s resistance to control a single oscillator based of the
of embedded digital hardware raises even deeper questions
LM709 op-amp[6]. Sal Martirano’s early-70s SalMar Con-
struction was a hybrid digital synthesizer controlled by a
large matrix of touch buttons[7]. In the modern era of mod-
ular synthesizers, many Eurorack[2] manufacturers adopt
touch plates in their modules for control (a technique dat-
Licensed under a Creative Commons Attribution ing at least to Buchla[5]), like the make noise René, Pres-
4.0 International License (CC BY 4.0). Copyright sure Points, and Teleplexer Modules, shown in figure 2 to
remains with the author(s). name a few[3]. Closer to our approach, the Hyve-synth is
NIME’18, June 3-6, 2018, Blacksburg, Virginia, USA. a 60-voice polyphonic analog synthesizer that presents cir-
cuit points from each oscillator at touch pads to enable pitch The 8 step sequencers are implemented via two dividers us-
bend and mixing based on the resistance of the skin[4]. HCI ing the CD4022 chip. The patchbay connects electrodes in
researchers have also made interactive surfaces that work our matrix to both inputs and outputs of external synthe-
through skin resistance[8]. sizers via 3.5mm mono jacks. The output impedance across
these circuits is matched with 10K resistors.
Figure 2: Make Noise’s touch modules, from left to
right, the Teleplexer, Pressure Points and René.
3. DESIGN AND IMPLEMENTATION
The current version of the Kinesynth includes some inputs
and several outputs. The instrument is played through
bridging the inputs to the various outputs through the fin- Figure 4: The Kinesynth wired to a Moog Mother-
gers. The body is coupled to the circuit through an elec- 32 semi-modular synthesier via patchbays connec-
trode that connects to the inverting pin of an op-amp con- tions and patch cables.
figured as a relaxation oscillator, shown in figure 3. The
4. CONCLUSION
In this paper, we presented the Kinesynth instrument, a
kinesthetic synthesizer, that uses the capacitive and resis-
tive properties of the human body to mix and modulate
analog signals coming from the environment, from on-board
audio sources, or from external eurorack synthesizers.
5. REFERENCES
[1] Area at Milan’s Parco Lambro Festival 1976.
https://www.youtube.com/watch?v=lOk7zWoDHrY.
[2] Doepfer a-100 eurorack spec:.
http://www.doepfer.de/a100 man/a100m e.htm.
[3] Make noise modular synthesizer manufacturer.
URL: http://www.makenoisemusic.com/.
[4] The Hyve touch synthesizer.
URL: https://www.hyvesynth.com/, 2018.
[5] D. Buchla. A history of Buchla’s musical instruments.
In In Proc. of NIME, 2005.
[6] E. Dykstra-Erickson and J. Arnowitz. Michel
Figure 3: Schematic of the Kinesynth’s input, show- Waisvisz: the man and the hands. interactions,
ing a relaxation oscillator compared against an LFO. 12(5):63–67, 2005.
[7] S. Martirano. “Progress Report #1 — An electronic
simplest way to generate a sound is by making contact with
music instrument which combines the composing
conductive appliances sitting near 110/220 VAC power lines
process with Performance in real-time”. UIUC
that will modulate the oscillator at a rate of 50/60Hz. In-
Internal Report, 1971.
creasing the amount of skin in contact with the modulat-
ing surface can lead to an increase on the op-amp’s input [8] N.W Gong, et al. Printsense: a versatile sensing
impedance, resulting in a faster drop of the oscillator’s fre- technique to support multimodal flexible surface
quency. At the second stage, both the relaxation oscillator interaction. In In Proc. of CHI 2014, pages
and another low frequency oscillator, generating a square- 1407–1410.
wave at the rate of 1Hz-1kHz, connect to the input pins of a [9] J. A. Paradiso. “The modular explosion-deja vu or
comparator, adding a tremolo effect to the final output. In something new?,” Presented at the Voltage Connect
addition, the instrument provides touch pads connected to Conference, Berklee College of Music. 2017.
four oscillators, two low frequency oscillators (LFO), three [10] J. A. Paradiso and N. Gershenfeld. Musical
cascading oscillators (FM), two 8-step sequencers, and a applications of electric field sensing. Computer music
patchbay with 18 outlets that can be used as either inputs journal, 21(2):69–89, 1997.
or outputs to be wired to external sources like function gen- [11] T. Rhea. ’Electro-Mechanical Instruments’, in Darter,
erators, and/or other synthesizers. These oscillators are T. (ed), The Art of Electronic Music. GPI
made from a 74HC14 Hex inverting Schmitt trigger, and Publications, NY, 1984.
each generate a pulse wave. The 3-stage FM synthesizer is [12] G. Young. The Sackbut Blues: Hugh Le Caine,
designed through three cascading oscillators, built using the Pioneer in Electronic Music. National Museum of
CD4093, a quad dual-input NAND Schmitt Trigger chip. Science &, 1989.