der hess

After the analysis of “Visualizing techniques with plants”, we will explore interactions for human computer interaction (HCI) through plants. The objectives of this research are revealing technical approaches and which kind of interaction can be performed.

Interactions between humans and computer is usually implemented by sensors. In 2009 Dan Saffer defined a sensor for human computer interaction like this:

 

A sensor is typically an electrical or electronic component whose job is to detect changes in the environment [Saffer2009, page 13].

 

We are surrounded by plants and plants are able to sense changes in our environment. The increasing problem with our environment forces us to reconsider our usage of technology. Hybrid solutions between natural resources and our current technology is able to decrease our current environmental issues [ICC2007, pages 15-21]. For this reason, it is obvious to explore the plant abilities of sensing for HCI. The sensor belongs to the basic components of any gestural system, which is used in HCI applications [Saffer2009, page 13]. In our context we replace the common electronic sensor with a plant. An additional electronic circuit translates these bio signals into electronic computer-readable signals. Accordingly to this approach, the gestural system is completed again.

 

The most used sensors in interactive systems detect pressure, light, proximity, acoustic, tilt, motion, and orientation [Saffer2009, page 13]. In this research we will figure out which of these detections a plant performs and how it is technically implemented. The introduced technology ranges from simple electronic circuits to complex systems. The presented solutions are ordered by its simplicity to complexity.

 

The resistor approach

A plant as a resistor between human and computer is an often used technique. The most art projects measure how conductive a plant is and react on this data. In many cases this approach is used for experimental sound performances, like “Plantas Parlantes” (2010) by Ricardo O’Nascimento, Gilberto Esparza, Javier Busturia, Jigni Wang, “Baumarktmusik” (2011) by MSHR, “Kraft Test Drummie and Robert Plant” (2012) by Cristian Martínez as well as the “Mosszillator”  (2012) [LandKuni2013, pages 34, 142-159]. It can be implemented easily through an Arduino and 1mOhm Resistor. This technique can detect touches and delivers information about pressure. Furthermore, the resistor approach enables interaction between the plant and other conductive objects. For instance, it can be used in projects with plants directly like the work “Genesis of Biosynthia II” (2011) by Benjamin Kolaitis or with fruits and vegetables (i.e. “Noisy Cauliflower”, 2013, by Cara Stewart or “Teardrop”, 2013, by J. Viewz). The resistor method is very often used for musical interfaces, which is related to its characteristics of a simple electronic signal and an easy implementation.

 

 

 The piezo approach

Another simple method of biosensing is the use of a contact microphone (piezoelectronic sensor). The contact microphone can detect vibrations on solid objects. This is useful for interactions on the plant branches (e.g. "Mogees", 2012) or thorns (e.g. “From the hills” and “Frishasin pijamas” by Cristian Martínez during 2013). The interactions are mostly connected with motion detection on the plant’s surface. This can be a tap, knock, or swipe gesture. The researchers of the project “Mogees” implemented an extensive gesture recognition system, which can be also adapted for HCI applications. Moreover, the same procedure can be used for environmental sensors as well. Especially, conditions like wind, growth or other vibrating interactions are easily detected, like the the installation “Tree Listening” (2012) by Alex Metcalf [AnHuMi2012, page 135] does. Environmental sensing is a key part of ubiquitous computing applications and therefore the piezo sensors are useful tools in this scope.

 

The capacitive sensing approach

Capacitive sensing is one of the classical technologies for human interfaces (e.g. multitouch screens). It avails from the human body capacitance, which enables us to measure the touch, proximity, humidity, and acceleration. The simplest approach of capacitive sensing through plants can measure the amount of touch and proximity. Many spatial sound installation, like “Akousmaflore” (2007) by Scenocosme [ISEA2011], “Baumarktmusik” (2011) by MSHR and “The Plant Ochestra” (2013) by Alexandra Duvekot, utilize this capacitive sensing method for creating an interaction between humans and plants. Beyond that simple touch detection with plants even more complex interactions are feasible. The new developed advanced “Swept Frequency Capacitive Sensing” technology of the project “Botanicus Interacticus” (2012) is able to detect the location of touches on a plant. In addition to that, their gesture recognition algorithm detects a comprehensive collection of gestures [PoSchoLoSa2012]. These gestures can be performed on any part of the plant. The technology behind “Botanicus Interacticus” is called “Touché” [SaPoHa2012] and was developed by Ivan Poupyrev and Chris Harrison. They measure a predefined range of frequencies from a plant, and then their gesture recognition software determines, if the performed gesture is correct or not. Fortunately, some DIY Hackers rebuild the system with Arduino (tutorial). Though the Arduino results are not as accurate as the “Touché” ones - it does not have such wide frequency range - it provides a technical access  for a broader audience.

 

The electric field measurement approach

The touch interaction between plants and humans with an electric field measurement is very often mixed up with capacitive sensing technology. From an interaction design point of view the differences are not so obvious, but the electric field analysis provides more accurate sensor values for a proximity measurement. Christa Sommerer and Laurent Mignonneau were the first persons, who used a plant as a human computer interface [SoMig1993]. Their installation “Interactive Plant Growing” (1992) measures the electrical potential difference (voltage) between the user’s body and the plant. Their interface is able to detect five different distance levels, which results in a lower or higher voltage value of the sensor [ICC2007, pages 86-87]. Depending on the user interaction the growth process of the artificial plants can be influenced. Another advantage of their approach is the location of the sensor. The sensor is hidden from the users view and is connected to the plant’s roots. The branches and leaves take over  the role of the antenna.

 

The biopotential measurement with electrodes

Electrodes are very well-researched circuits in electronic engineering. They used for measuring bioelectronic changes and are an important tool for medical treatments (e.g. EEG, ECG). In that relation, electrodes are utilized for Human Plant Interfaces. In an interactive art context, first explorations of electrical changes within a plant were explored by Yuji Dogane in his artwork “Plantron” (1992) [ICC2007, page 18] and by Christa Sommerer and Laurent Mignonneau in their installation “Anthroposcope” (1993) [GerWeib1993, pages 398-400]. Yuji Dogane utilized the generated signal from the plant and converted it to digital data (MIDI). His soundscape addresses the ecological understanding how humans influence and change their environment. In contrast to an audio feedback, the installation “Anthroposcope” combines the signal from the plant with the human heart rate and produces a computer generated animation [StSoMi2009].

At this time interactive artworks dealt more frequently with the plant signals as additional parameter for their artpiece then as an interface between humans and plants. For instance, the “Yucca Invest Trading Plant” (1999) installation by Ola Pehrson applied the method of Electroencephalography (Brain Computer Interface) to gather plants signals, which were translated to trading activities for stock exchange simulator. Later, the musician and artist Miya Masaoka used the same technique for direct interaction with a plant. Her interactive sound installation performance “Pieces for Plants” (2002) detects touches and proximity of persons. The sound changes according to the human interaction. The proximity interaction is not very accurate but good enough for detecting the presence of humans. Very similar to her approach the research project “I/O Plant” (2007) by Satoshi Kuribayashi, Yusuke Sakamoto, and Hiroya Tanaka is also able to detect biopotential signals [KuSaTa2007]. Their research results revealed that the measured signal depends strongly on the present environmental conditions. In that relation, the biopotential signal for touch or other signals are very different. A calibration method has to be implemented.

In spite of that, the artist Ivan Henriques explored the plant signals during his research for his artwork “Jurema Action Plant” (2011) in more detail. His research focused on action potential mechanism of touch sensitive plants like the venus flytrap or the mimosa pudica. The touch signals are recognized by a special action potential device that measures the action potential changes inside a plant cell. Unfortunately, the plant signal is very weak and it needs a strong amplifier to translate the touch signal to a computer. Additionally, only a few plants can send such a strong touch signal. How and when the signal is dispatched, depends on the species. The venus flytrap and the mimosa pudica are appropriate for the action potential approach, because they react immediately on a touch stimuli and dispatch a very powerful signal that is strong enough to be distinguished from other signals. For HCI application both plants are very useful, because they provide a visual user feedback after the touch. One circumstance has to be considered before touch sensitive plants can be used for a Human Plant Interface. The plant’s touch reaction is an exhausting task for the plant, and should not be frequently touched. Touch interaction can harm the plant and increases the possibility that the plant dies.

The last remarkable activities in the field of biopotential signals with plants in an art context were done by Leslie Garcia. Her plant-based biosensing kit “Pulsu(m) Plantae” (2012-2013) is able to detect environmental changes light and sound as well as human touch interactions. Fortunately, she documented the electronic circuits and the software development very well. For this reason, her biosensing approach with electrodes is very transparent and accessible for a wide range of artists and tinkerers.

 

 

The results

Biosensing with plants covers the common sensors touch, sensitive touch, touch location, pressure, motion (for gestural interactions), proximity, sound and light for HCI. Regrettably, the signal quality differs from plant to plant and has to be calibrated for each plant. Furthermore, the data of the biosensors will not be as accurate as the measurements of their electronic counterpart. Too many factors (plant growth, plant health, moisture, etc.) have a negative influence on an exact sensor value [KuSaTa2007]. In consequence of the diversity in signal quality and the lack of an exact evaluation, plants are suited perfectly as a tool for artistical interpretations. Media artists have already integrated this uncertain aspect in their experimental music performances. In spite of inaccurate data, plants are still reliable bio sensors for simple touch, tap, knock, and swipe interactions. Biosensing with plants reaches its full potential in context of spatial interaction. People are able to interact with their usual environment, and the technology behind these interactions stay invisible for the them. In this relation, biosensing with plants is a very suitable tool for ambient intelligence and ubiquitous computing applications. Especially, the research results of Alexander G. Volkov and his team look very promising for a better understanding of plant signals in an environmental sensor context [VolRufRa2006, VolAdJo2007].

Beyond the application of biosensing in interaction design, in our culture always exists ethical concerns related to biosensing. Especially, biosensing with plants prompt frequently the question about plant consciousness and plant soul (e.g. artworks “Pieces for Plants”, “Jurema Action Plant”). Unfortunately, the movie and the book “The Secret Life of Plants” put this scientific topic in a pseudo-scientific and esoteric context [Koechlin2008, page 137]. For this reason, current scientists are very careful in making statements about plants perception. The consequences in ethics and treatment of plants can be huge. This topic gets more interesting if we connect plant perception with locomotion. However, during my research I figured out that most of the above listed approaches are not real plant signals. For example, the technology behind “Botanicus Interacticus” and “Mogees” works on living and dead objects, too. The plant takes over the role as transducer. Except, the biopotential approach with electrodes comes close a to real plant signal. If these biopotential signals are equivalent to a perception, like human and animals do. It is not clarified yet [Koechlin2008, page 46]. Biologists [Koechlin2008, pp. 49, pp. 55] and philosophers [Ingensiep2001] are discussing this complex topic controversially and even emotionall.

 

References:

[SoMig1993] Sommerer, Christa; Mignonneau, Laurent (1993). Interactive Plant Growing. In Visual Proceedings of the Siggraph ’93 Conference, ACM Siggraph, 1993; pp.164-165.

[GerWeib1993] Gerbel, Karl; Weibel,Peter (1993). Genetische Kunst - künstliches Leben (Ars Electronica Festival 93). PVS-Verleger, 1993.

[VolRufRa2006] Volkov, Alexander G.; Rufus , Don; A. Ranatunga (2006). Plants as Environmental Biosensors. In Plant Signal Behav. 2006 May-Jun; 1(3): 105–115.

[VolAdJo2007] Volkov, Alexander G.; Adesina, Tejumade; Jovanov, Emil (2007). Closing of Venus Flytrap by Electrical Stimulation of Motor Cells. In Plant Signal Behav. 2007 May-Jun; 2(3): 139–145.

[ICC2007] NTT InterCommunication Center (2007). Silent Dialogue. ICC, 2007.

[KuSaTa2007] Satoshi Kuribayashi, Yusuke Sakamoto, and Hiroya Tanaka (2007). I/O plant: a tool kit for designing augmented human-plant interactions. In CHI ‘07 Extended Abstracts on Human Factors in Computing Systems (CHI EA ‘07). ACM, New York, NY, USA, 2537-2542.

[Koechlin2008] Koechlin, Florianne (2008). PflanzenPalaver - Belauschte Geheimnisse der botanischen Welt, Lenos Verlag, 2008.

[Ingensiep2001] Ingensiep, Hans Werner (2001). Geschichte der Pflanzenseele - Philosophische und biologische Entwürfe von der Antike bis zur Gegenwart. Kröner, 2001.

[Saffer2009] Saffer, Dan (2009). Designing Gestural Interfaces. O’Reilly, 2009.

[StSoMi2009] Stocker, Sommerer, Mignonneau. Christa Sommerer & Laurent Mignonneau (2009.) Interactive Art Research. Springer Verlag, 2009.

[AnHuMi2012] Antonelli, Paola; Hunt, Jamer; Midal, Alexandra (2012). Talk to Me: Design and the Communication Between People and Objects. Museum of Modern Art, 2012.

[SaPoHa2012] Sato, M., Poupyrev, I, and Harrison, C. (2012). Touché: Enhancing Touch Interaction on Humans, Screens, Liquids, and Everyday Objects. In Proceedings of CHI’12. 2012. ACM.

[PoSchoLoSa2012] Poupyrev, I., Schoessler, O., Loh, J., Sato, M. (2012). “Botanicus Interacticus: Interactive Plant Technology” SIGGRAPH 2012 Emerging Technology Exhibition.

[LandKuni2013] Kuni, Verena; Landwehr, Dominik (2013). Home Made Bio Electronics Arts. Christoph Merian - Migros Kulturprozent, 2013