From Self-Portraits to Medieval Simulations: Computational Artificial Ecosystems as Multi-Contextual Authoring Tools

In this paper, we examine virtual ecosystems as general-purpose authoring tools for heterogeneity and spontaneity, and discuss the use of such systems as animation drivers in a variety of contexts: from artistic abstract work, to dance choreographies, or even in the simulation of historical spaces. First, we present the background, motivation and summary for the project. Then, we discuss the use of these systems in the artworks Senhora da Graça, xTNZ, Where is Lourenço Marques?, in the dance choreography, Vishnu's Dance of Life and Death, and in the simulation of the population of the medieval village of Mértola. This emphasizes how an artistic practice informs a movement of transition, taking us from the use of artefacts that mostly rely on their traditionally associated eco-narratives to functional tool generators of diversity and heterogeneity. Finally, we close the article to contend that these systems function as utilitarian authoring tools for multi-contextual purposes.


INTRODUCTION
Artificial Life (ALife) is a domain of knowledge characterized by the study of the processes of systems of natural life using simulations involving computer models, robotics, and biochemistry. The development of computer based systems with communities of individuals forming virtual ecosystems in silico, computational artificial ecosystems (CAE) is a well-established practice within this community [10,14]. Systems of this kind already have an established history in the field of ALife going back to at least the early 1990s [7]. CAEs can be defined as systems supporting dynamic computerized environments that emulate the processes and evolution of biological life in nature. In the traditional approach, they simulate behaviors of carbon-based individuals in their natural habitat, and the boost in community dynamics is consequent upon elements being structured in a foodchain and exchanging energy and biomass units. Within these software systems, entities emulate processes seen in basic biological life forms. They typically perform metabolic functions and actively search for resources and reproductive partners in their environment. They may engage in predatory and reproductive acts as well. These agents, -in the computational sense as they have sensors and actuators -, operate in a logic of autonomy, and part of the interest being devoted to such systems is due to phenomena of self-organization and emergence where micro scale behaviors lead to macro changes of global patterns. These features demonstrate Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. ARTECH2017, September 06-08, 2017, Macau, China © 2017 Copyright is held by the owner/author(s). ACM ISBN 978-1-4503-5273-4/17/09. http://dx.doi.org /10.1145/3106548.3106604 be helpful in the understanding of complex adaptive natural biological systems [28]. Similarly, they reveal to be useful resources as a technique for exploring dynamism and variation in artificial/digital agent-based environments [8]. The main motivation for this paper is to promote a discussion on the rich variety of usages for these systems. By looking at them as not restricted to ecological narratives but as multi-contextual and multifunctional systems, we can argue for CAEs as general purpose generative engines for heterogeneity, spontaneity, and sometimes even novelty consequently, for CAEs as useful authoring tools for a wide range of diversified contexts, be it artist or not.
As a methodological strategy, we will analyze the inherent potential of these systems mainly through the lenses of our artistic uses of CAEs. A growing number of artists have adopted this practice and use these systems to produce novel, innovative, and artistic works, and as they explore new aesthetic results, they simultaneously push the boundaries of CAEs. It is mainly through this exploratory practice that we will draw our line of discussion.

STATE OF THE ART
Assembling concepts from science, technology and aesthetics, Evolutionary Art is now a well-established art discipline. Artistic expressions take shape and form in a broad range of formats that span from still imagery to large installations with interactive and dynamic immersive environments. Creative Evolutionary Systems [8], Metacreations [30], The Art of Artificial Evolution [23] and Virtual Worlds [11], are some of the key texts providing comprehensive views of this body of work. At the roots of this artistic discipline lays a fundamental methodology informed by Darwinian evolution and Neo-Mendelian Genetics. Blueprints defining the elements evolve in a generational time scale by means of processes of reproduction and genetic inheritance. With Evolutionary Art, the representations of these blueprints usually take iconic or audible forms, but there is no master plan defining a method for this conversion to occur. Consequently, the contrast among approaches and outcomes can be unrestrained. The simplicity of this is demonstrated, for example, in the work William Latham, one of the pioneers of this discipline. Three-dimensional morphologies are generated based on a process of shape evolution when the blueprints of two parent-shapes are combined to generate a third and new one. Latham describes his work as being equivalent to a gardener playing with the genetics of plants, selecting the samples to keep alive and to mutate [27,30].
With CAEs, the complexity of this first paradigm gains a new dimension. The same simple method applies now to autonomous and interacting individuals within a population (the CAE). The speed of the progress of entities increases, occurring in real-time and in parallel evolutionary processes. Novelty and heterogeneity become fundamental properties qualifying these systems. Additionally, given their autonomy, mechanisms of selforganization and emergence act as generators of cyclic changes of density and flow. These properties are deemed fertile territory for exploratory artists. CAEs play essential roles acting as generative engines in audiovisual installations [17,22], abstract music [18], or even the choreography of avatars [4]. This is an active stream of artistic practice (e.g. [2,18,22]) that had its heyday in the early 2000s (e.g [24,22]). Rooted in aesthetic traditions such as Systemart, Generative-art, Kinetic Art, and Minimalism, the community dynamics and the individual variations offered by these systems are tempting mechanisms for artistic exploration.
Reminiscent of the methodology from the original methodology method, the blueprint (or Gtype) structures the fundamental properties of the individual. The information contained in the blueprint is translated into an equivalent phenotypic representation be it audible or visual. Wakefield and Ji's Time of Doubles provides a sophisticated example of this method. Not only are the entities rendered visible from structural information contained in the blueprint, but the entities also sing their genomes, transcribing the encoded sequence into sounds [29]. Similarly, other features are inherited by the progeny when individuals replicate in a process of gene recombination that emulates sexual reproduction. For instance, in Alan Dorin's Constellation, mosses and flowers evolve their properties, but their visual features allow their easy identification as members of a particular class [15].
With CAEs, in a step beyond the original paradigm, individuals belonging to a population mimic a simplified life-cycle from generic carbon-based life forms. Contrary to the first paradigm where the selective pressure was exerted by a human, with CAEs, this evolutionary pressure is established by the agents' struggle to keep themselves 'alive' and to perpetuate their genetic heritage. Moving, or simply breathing, are examples of activities performed by agents that might require an energetic cost. The metabolic function is one of the key instruments to exert selective pressure, and the energetic resource functions as a motivational factor for action since, when emptied of energy, agents typically 'die,' thus being removed from a simulation. Hierarchical food-chains structure global interactions, and agents exchange tokens of energy with other population members according to this framework. Typically, the energetic transference takes place within the context of predatory activity as the community competes for energy and space. The implementation of the metabolic function is not linear and allows for unfettered variations and creativity. We can find examples of systems designed around the swap of units of energy, as in Eden by Jon McCormack [22], or complex simulations of chemical processes, including biomass and waste, such as in Habitat and Constellation by Alan Dorin [15,16].
The momentary arrangements of elements resulting from the dynamics generated by the autonomous movements and actions of the agents generate interesting processes of self-organization and emergence deemed interesting enough to be visualized or sonified [29]. Many of these works support their authors' aesthetic discourses with ecological undertones [17,19]. Others, however, are constructed to exploit their inherent dynamics as exploring the abstraction of the system's internal states. The sound explorations of Eldridge and Dorin are one such example [18]. The soundscape results from the status and location of the agents in the virtual space, which acts almost as a virtual partiture. Their location in space is translated into sound [18]. Unfinished Symphonies -songs of 3 ½ worlds also offers a paradigmatic example of this exploration as each individual acts as an instrument, altering its tone according to its life stage whilst the rhythm of that sound changes according to the actions performed [9].
The versatility of the system is again illustrated by the outcomes of these explorations, which are presented to the public in a wide range formats, from personal computers [2] to large installations with interactive and immersive evolving environments [29,22]. The role of the audience also differs substantially and can be called upon to intervene with the course of virtual life using haptic devices [28,1], or more discreet processes such as having the audience's presence in the gallery tracked [14] or mapped out into the virtual world [29].

OPENING THE SCOPE: A METHOD NOT RESTRICTED TO ECOLOGICAL NARRATIVES NOR ARTISTIC WORKS
We will now look at a series of works from the author, progressively moving away from the ecological narrative traditionally associated with these systems, and moving towards pragmatically exploring the system dynamics and ephemeral states per se. This movement will demonstrate the versality provided by these systems in an almost ontological change of nature, which allows us to look at the CAE as a utilitarian technique.

Duchamp's series
The first work we bring into discussion, Duchamp's series #3, is part of a set of semi-abstract works making use of the simplest form of these systems, with a community formed by foragers and resources. An initial image provides a stage where, depending on its initial light and color, each pixel is determined to be a living entity or simply a nutrient. These living pixels will move through the image trying to feed themselves and as a result producing new imagery. This series departs from Jorge Luis Borges's Library of Babel -a vast library containing all possible 410-page books of a certain format -, and this work extends the idea to imagine all the combinatory compositions we can produce using the pixels of an initial image. In this case, using the colors and dimensions of the digital representations of Duchamp's works ( Figure 2) [25]. This work is literally eating the old order and closely aligns its sub-text with the utopian exploration of new digital technologies to reestablish art that marked the end of the 1990s. The two works we invoke next, xTNZ [1], and Senhora da Graça [2], offer sharp discursive contrast with the Duchamp's series. Both offer dystopian visions of technology, and also illustrate the classic approach to these systems, strongly drawing on the more traditional idea of the virtual habitat made of in silico creatures with biomorphic reminiscences.

xTNZ
The work xTNZ -self-portrait as virtual world questions the role of the body in the digital age in a discourse closely aligned with the ongoing debate surrounding this practice in the late 1990s and early 2000s, when AI, genetic manipulation, and prosthetics became central topics of debate in mainstream media. In xTNZ, entities are organized around classes of individuals arranged in three trophic levels with predator-prey dynamics [1]. The virtual world is in permanent change and flow, with creatures evolving shapes and forms on a generational scale (Figure 3). Time occupies a central place in the narrative as the system evolves the macro images and sounds from diverse body parts of the author that skin (texture) the virtual entities and provide a soundscape to the environment. A video of the system is available at https://youtu.be/DFTdnbGNGVw.

Senhora da Graça
Senhora da Graça offers a similar dystopian narrative of ecological loss [2]. The destruction of the natural environment for the sake of progress is the motivation for a work where a digital ecosystem composed of digital creatures evolves nostalgic images from a youth in a valley flooded for the construction of a newly built dam ( Figure 4). With irony, the virtual ecosystem resides in a computer, which feeds on the same energy production that the work criticizes. As time progresses and the creatures evolve autonomously, their shapes bend and twist per their genetic based operators. As the images become less and less recognizable, they preserve all their pictorial interest but lose the original iconic resemblance to the landscapes they depict. A video of the system is available at https://youtu.be/pKIyq2OByNE. In these two instances, the polarity of software processes and their visualization and interpretation are stressed and emphasized. If in both situations, the ecological interactions of the system are explored to convey subtexts and artistic messages, in the next generation of works, this linearity becomes less evident.

Vishnu's Dance of Life and Death
In Vishnu's Dance of Life and Death, the core dynamic generated by CAEs is used to choreograph animated dancers [4]. According to Hindu mythology, Vishnu is known as the 'Preserver of the Universe', the essence necessary for all beings, and the force maintaining the universe. In this work, the computational algorithm occupies the deity's role as maintainer of all existence. In an ironic comment on the rhetoric of life surrounding these systems, the evolutionary course of life takes place and is celebrated on a stage where 3D humanoid avatars perform a dance. The algorithmic master plan maps links between actions in the ecosystem and movements performed in the dance. A birth event within the population is represented on stage by an avatar emerging from outside and moving towards a more central position, repeatedly executing the same movement until he joins the collective performance and begins interacting with the other dancers. Each event during the lifetime of an individual avatar is translated into a predefined movement of the avatar on stage. A predator chasing prey in the ecosystem might correspond to a 'chassé' movement, and the predator feeding from the victim might correspond to a 'plié' on stage. An unfettered performance emerges from these simple rules ( Figure 5).
As in the previous works, the dance functions as an iconic map of the ecosystem. What is offered to be seen on stage is an explicit visualization of the fundamental multi-agent state-space, with each state of the ecosystem revealed in the current configuration of the dancers. Contrasting with previous works, the visualizations materialize in the ecosystem but feel discrepant with the underlying (eco)-narrative. This incongruence permits a contrasting perception of the workings of the system and its visualization or representation. On the one hand, we have a symbolic system, in permanent flux, and on the other hand, we have the action of decoding its meaning. This understanding provides a fundamental shift in the way we engage with CAEs, as it paves the way to less obvious uses away from the eco-narrative. CAEs as generators of heterogeneity and spontaneity is what the next generation of works aims to explore.
A video of the work is available at https://youtu.be/YDZxkRz6pL4.

Where is Lourenço Marques?
In Where is Lourenço Marques?, the system was experimentally employed to animate groups of conversational humanoids [3,5]. This work is an ethnographic project where a population of virtual humans inhabit the virtual world and interact autonomously among themselves as well as with the audience. The CAE is used to generate the movements and behaviors of the population of 'storytellers'.
Lourenço Marques was the capital of the province of Mozambique in the period of Portuguese colonial domination. During the process of decolonization, large communities were forced to suddenly abandon the country. A group of former residents, now living in Portugal, was interviewed for this work. Elements shared during the interviews, such as old photographs, sketches, and music were used to model the landscape and some of the iconic buildings, forming a sparse and dreamlike virtual city. These interviews form a mosaic of voices helping to describe the human geography of the city. It is this first-person account of the experiences that we hear when the user clicks on any of the characters populating the city ( Figure 6). The character approaches the camera and we can hear the voice of one of the interviewees mediated by the character looking at the camera and gesticulating as if speaking. When not interacting with human visitors, the characters are autonomous and animated by the CAE. The population in Where is Lourenço Marques?, is organized into social layers surrounding the production and trade of energy. One group works on farms to produce energy while the others wander the city, exchanging the energy produced by the former group.
Echoes reminiscent of the eco-narrative are still present in this work. As an artistic statement, the ecosystem was used as a hyperbolic metaphor to problematize the social interaction in the former city by emphatically equating the behavior of species in an ecosystem with social groups in a human society. In the next work, however, this aspect dissipates as the CAE becomes clearly dissociated from the narrative and is instrumentalized in a purely functional way.
A video of the work is available at https://youtu.be/Tp8lvdNWucw.

Simulation of life in medieval Mértola
To help us with this discussion about the functionalization of CAEs, we invoke a final work, the historical simulation of a cultural heritage site. The CAE was successfully implemented as an AI engine in the simulation of the population of the medieval city of Mértola [12]. Using manual modeling combined with procedural generation, a virtual space was generated containing some of the military structures, such as the defensive walls and the watchtower of the river, as well as some of the civilian housing within the protection of the walls. Inhabiting the space and dressed with historical rigor, a virtual population is composed of autonomous individuals. These inhabitants of the virtual city are animated by means of a CAE, and are equipped with limited intelligence, which allows them to navigate the space, self-organize, interact, and trade at the local market and with each other. Additionally, each of these individuals has its psychological features, including personality, mood, and emotions. They appear to communicate with their fellow citizens in the narrow streets of the village using expressive gestures and postures that convey their inner emotional states (Figures 1 and 7). A video of the work is available at https://youtu.be/8s9mVzNVH7A. Contrasting with the previous work, Where is Lourenço Marques?, the CAE is used in the simulation of Mértola in a more high level and abstract manner. At the time of the writing of this paper, the concepts of death and birth have been removed from the simulation and are nonexistent. Avoiding the problematization of having humans in society acting as animals in the wild, the underlying ecosystem is not revealed to the audience. Even in technical descriptions, ecological terms, such as predation or energy, are abstracted or replaced by terms that are closer to the economic model, such as trading and socialization. This offers a new mindset that takes advantage of the usefulness of the generated dynamics but attempts to move away from the quintessential Eros and Thanatos narrative. The CAE is instrumentalized as a mere generator of flux and dynamics, of behavioral spontaneity and heterogeneity.

DISCUSSION
At the base of the underlying architecture forming a CAE lies a symbolic structure of software and hardware. This provides a network of relationships between data structures, determining their behaviors and establishing the overall dynamics. In the traditional CAE, this architecture is designed considering ecological relationships. However, this system is incomplete without a complementary system of codes providing meaning to what is rendered audible and visible.
Entering the domain of codes and translation of symbols, we get into the realm of semiotics. In light of the material semiotics of Bruno Latour's Actor-Network theory, we can understand how different elements surrounding the artefacts participate in forming their meaning [21]. Per this theory, not only is the software architecture critical, but the narratives being told surrounding the works are fundamental actors as well in our understanding of the artefact. A constellation of actors as diverse as our previous knowledge about Darwinian evolution and Neo-Mendelian genetics, the website accompanying the artefact where the ecological narrative is told, or even the way the entities are rendered, all form a network of relationships from which meaning emerges. Under the appropriate network formed by the right actors, a group of rapidly changing color pixels might be read as a dramatic struggle for survival in the wild of the cyber-jungle, or, on the contrary, a different configuration might turn them into flickering pixels in a psychedelic experience. As we saw earlier, a datastructure entity might be tailored to be a changing-form predator waiting for its prey, as in Senhora da Graça, while a similar datastructure entity might be a dancer clumsily transitioning its movements in Vishnu's Dance of Life and Death. In a process of authorship, it is up to the practitioner to conveniently tailor the constellation to be read in accordance with his intentions.
Of course, the process will only be completed when the artefact is experienced by the audience, with individual audience members having their own previous knowledge of the world, and together forming what Giles Deleuze calls the mental automaton [13]. This can be described as a circuit formed between the constellation emerging from the artefact and the preexisting network in the audience.
Removed from the actors contributing to the eco-narrative, only the mechanic aspect of the CAE prevails. Works, such as the simulation of the population of ancient Mértola, demonstrate that when a) the system does not make references to life forms in its architecture and b) the audience is unaware of the workings of the system, then the CAE can operate as an abstract engine for heterogeneity and spontaneity.

CONCLUSIONS
It seems opportune to use the context of this conference to discuss how artistic exploitations of CAEs have introduced new ways of looking at these systems, thus opening new avenues of exploitation and functionalization. We have looked at the exploratory practice of artists using CAEs in their practice to see how this experience can inform and be extended to the broader community. We have seen how these systems typically appear within a context of econarratives. Visuals typically represent the underlying dynamics and represent classes of individuals in predator-prey relationships. Audio works, however, provide useful insights as artists provide full-fledged abstract explorations of the systems' internal states. We have further examined a sequence of works showing a movement transitioning from eco-narratives to the exploration of the CAE as an automated device to generate differentiation, flow, and novelty in the systems. The majority of the works invoked in this document have been analyzed in detail elsewhere, and this paper continues a line of argument initiated earlier [6], reinforcing it with more evidence, namely the work on the population of the medieval simulation of Mértola. This paper is opportunity to attempt to demonstrate, diffuse, and promote such ideas amongst the community of artists interested in this technique, expanding and enriching the language and palette available to artists using such systems. We contend that apart from the traditionally associated discourses of life, fundamentally, the CAE is a design tool for authoring a generation of dynamic heterogeneity and noticeable spontaneity in multi-agent environments. The animation of nonplayer characters is one of such contexts, as the simulation of historical Mértola indicates. As a decentralized autonomous model of agency, it is left to the practitioners to elect how to instrumentalize a useful and novel technique to broaden spheres of use.

ACKNOWLEDGMENTS
This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 655226.