Heat and Noise (Renao): Media Ecologies and Urban Futures

In this paper we report on an ongoing collaborative research-creation project focused around urban socioaesthetic ambiences. This custom media system sonifies group movement. We propose a speculative application of real-time gestural computation of group activity and poetically composed media which scopes movement and computing to a realm of concerns endemic to public life. What's at stake is the role of media, sensing, and computation in the composition of urban futures, social experience and public life (the commons). The computation intends to catalyze and intensify renao poiesis, while attuning to possibilities for improvised public life to create novel and ethical public futures.


INTRODUCTION
In this paper we report on an ongoing collaborative research-creation project focused around urban socioaesthetic ambiences. This custom media system sonifies group movement. We propose a speculative application of real-time gestural computation of group activity and poetically composed media which scopes movement and computing to realm of concerns endemic to public life such as privacy, production and place. These concerns dominate contemporary technological discourses and yet they remain unexamined by interdisciplinary communities such as MOCO although they revolve strongly around both movement and computing. Indeed, what's at stake is the role of media, sensing, and computation in the composition of urban futures, social experience and public life (the commons).
We summarize our approach to experiential media system 1 which intends to catalyze and intensify renao poiesis, while attuning to possibilities for improvised public life to create novel and ethical public futures.

Apparatus and Technical Approaches
We prototype responsive media environments using the Ozone system, a responsive media environment [1] developed by Synthesis Center @ ASU in which collaborators use state-of-the-art computational sensing and analysis of real-time (non-)human activity to compose immersive and continuously responsive media behaviors for spatialized multi-channel sound, floor projection, haptics, expressive robotics, and lighting. Notions of 'material computation' have strongly informed the design and implementation of the Ozone system. Such notions generalize computation to include not only the digital operations of what are conventionally called computers but also a broader range of material transformations effecting state transitions: "Computation, in the sense of a determined and designed set of well-defined transitions as a function of a given state to another state of an event, is always material" [1]. Sensors such as camera and floors allow the system to continuously observe and transcode values such as luminosity or pressure [2]. This observational capacity extends computational power out to fleshy bodies setting off modulations of matter through gesture or touch.

Renao
Our collaborative media project articulates with the socioaesthetic quality of renao [热闹] or 'heat and noise' as an element of urban public life. Pan and Tong define renao as having three aspects: "unceasing sound, a thronging group of people, and continuous activity [那聽不完的聲音，那人潮擁擠的活動 ，還有那走馬燈般的活動]" [3]. While any social situation can be renao, the term often denotes a quality of busy, crowded public places (streets, markets, temples, and so on) or public events (festivals, holidays, concerts, and so on). The term renao appears in Chinese-language texts going back more than a thousand years, indicating its continuing valence over various time periods. This responsive media project comprises a continuation of the first author's broader project on renao as an element of urban public life in the city of Tianjin, China. Tianjin is a city of twelve million located near Beijing in northeast China. The broader project further engages with renao through archival research, walking activities with Tianjin residents, sense mapping, audio recordings, and photography, while exploring its value in domains such as urban social life and urban design. The media environment uses field audio recordings from public places in Tianjin such as markets and parks.
Analogizing public social life as heat and noise foregrounds a logic of aesthetic creation and experience that does not reduce to realms of discrete performing bodies or art objects. Such analogization instead posits immersion in and modulation of ambient material fields through continuous movement. Following this logic, a creative subject is a "nexus of coordinated gestures," gestures which prolong rather than originate movement [4]. Gestural movement convects or propagates a continuous field, a field which returns to sensing bodies as a visual and acoustic surrounds affording open-ended exploration and play [2] [5]. To sheng or 'create' renao [生热闹] is to prolong a dense and incessantly experienced texture of activity, ambulation, speech, music, and other elements, without relying on pre-established plans or aiming toward pre-given endpoints.

Urban Placemaking
Ongoing processes of urbanization are remaking our societies and environments in profound ways. Urban planning, design, and management is often guided by logics of development, efficiency, and accumulation, logics which ignore potential everyday experiences in cities. Movement is rendered as displacement between points, often via mechanized transport, rather than a mode of ongoing sensation. Urbanization has also been accompanied by anxiety with regard to social transformations effected by the rationality and consumerism characteristic of urban life, provoking desires to diminish social life through design [6]. At the same time, cities catalyze modes of being together from which new sources of value emerge. Renao marks a mode of shared socioaesthetic poiesis and experience through which urban public places are enlivened and enjoyed as a commons open to all. Responsive media environments not only intensify these aesthetic processes, but also inspire new imaginaries about the role of heat and noise in remaking our cities. How might socioaesthetic creation contribute to new modes of living together and to more livable and sociable urban places?

Communication, Economics, and Exchange
Global social and economic transformations increasingly enable a rigid logic of mediated exchange between pre-given agents. Digital technology transmutes bodies into start and end points for transmitted exchanges of coded signals [7]. Political-economic regimes index value to quantity, equating wealth creation with private acquisition of money and goods. In particular, a growing aesthetic economy impels production of, and draws perception toward, bounded works or spectacles accessible through the marketplace [8][9]. Can we not only imagine but also create and prolong social and economic fields in which 'exchange' does not require isolation or competition? What modes of mediation might catalyze such creation, affording us convection and propagation rather than transmission, ownership, or consumption?

Surveillance, Data, and Public Life
Constant surveillance in the public spaces of much of the technologically colonized world has at best spawned productive movements of protest and at worst become accepted as part of our social contract. Techniques such as facial recognition have been deployed by state apparati to monitor flows of people. While many media artists have developed making practices which are critical of this omnisurveillance, there has been less interest in proposing alternative modes of engaging public space using camera-based tracking which does not rely on discretization (the recognition of pre-given categories such as a 'face' or a specific legal entity, e.g. 'Jane Doe'). Responsive media sensing could be seen as a proposition for the reclaiming of public security infrastructure. How can we use these systems to produce what Nora Bateson calls "warm data" -signals which communicate the interrelationality of different complex and dynamical systems [10]?

MEDIA SYSTEM: APPROACHES TO SENSING AND COMPUTATION
In this project, we are particularly interested in employing computational sensing, analysis, and media transduction strategies which do not reify pre-given subjects, discrete bodies, or anthropocentric agencies. As such we employ semantically shallow measures of movement such as optical flow, media composition techniques such as concatenative synthesis, and ambisonic spatializations to create robust immersion in rich fields of media palpable by ensembles of bodies, rather than idealized listening subjects [4].

Laminar and Turbulent Sensing and Sound Poetics
As an approach to visual sensing, frame differencing and optical flow (HS Flow) index movements of (visible) material stuff in the world. 'Stuff' denotes generically 'inanimate' matter such as water, fluid, and air. Drawing on notions from fluid dynamics, we understand this stuff as being animated as flows. Flow also recalls various Western philosophies such as Heraclitus' statement 'panta rhei' (everything flows), or the process philosophies of Gilles Deleuze and Alfred North Whitehead. Our use of the word flow knowingly invokes the popular psychology around 'flow' by Mihály Csíkszentmihályi to poetically suggest resonances and entrainments between this embodied mental state and the world as fluidic stuff [11]. We use non-discretizing computer-vision metrics such as HS flow and frame differencing which operate on images streamed from an IR camera mounted overhead. Multichannel audio recordings of renao public places collected in Tianjin, China are respatialized in the sonic space using real-time ambisonic algorithms. These recordings are layered and modulated by frame-differencing and HS-flow metrics. The amplitude of the sounds is modulated by the average movement in the space, whereas the relative laminar/turbulence of the HS flow vectors deviates (crossfades) between buffers of various recordings.

Sonification, Exploration, and Play
This sonification schema aims to catalyze experimental-experiential events composed by flows of bodies through a given space. System sensors and computation allow visitors to the space to continuously modulate ambient sounds with movement and gesture, simultaneously creating and sensing the media and environment through continual sensorimotor activity and feedback.
What are the ethical and practical implications of this approach to sensing, analyzing, and sonifying the movement of groups of people (>=3)? How are these distinguished from traditional, or object-based approaches? The system reifies neither pre-given creating agents nor created objects, instead continuously computing the total sensed movement of the space. Bodily gesture inserts itself into the ongoing modulation of the total media environment, constituting a vector or node rather than an originating or controlling subject. The system affords gestural activity that does not reduce participants' media 'use' to game completion or goal achievement, instead inviting ongoing transformation of the ambient environment through exploration and play. Instead of the restrictions of rules or goals, the Ozone responsive media system invites open-ended poiesis.
If renao is spontaneous aesthetic experience, why a media system? Responsive media, we have argued, conditions both attunement and poiesis. On one hand, responsive media conditions both creation and experience of renao in ways that attune to inspire imaginaries for its potential for social transformation. Such potential lies in domains such as communal placemaking through socioaesthetic activity, social exchange that is convivial and mutually enriching rather than isolating and competitive, and the use of sensing technology for aesthetic creation rather than surveillance or control. In a more general sense, engagement with responsive media constitutes a mode of collective meditation or self-surveillance, heightening awareness of ethical capabilities and potentialities [12]. On the other hand, responsive media affords aesthetic creation and experience adequate to the heated and noisy activity characteristic of urban public places. As a generator of renao, responsive media provides settings for playful, sociable aesthetic activity.