Hello, I.O. So, I'm really, really happy to be back here and in this room and at the
walker. I'm just going to jump in. So, I have four major projects that I've been working
on for ten years. First, I work at my studio, creating software for exhibitions, exploring
different ideas. Second, I teach studio classes at UCLA in the Department of Design Media Arts.
Third, the ongoing development of Processing.org. And four, writing books to support my teaching
and processing. So, to say it differently, I basically weave together making, teaching,
writing, and tool building. And this presentation will attempt to loop those things together.
So, the purpose of this talk is to declare that software is a dense medium for the visual
arts with as much potential as prior media, including film, photography, and painting.
This is painfully obvious to many of us, but it's confusing and elusive to most people.
The talk has five parts. Sort of the studio talking about what an artist's studio is
and what happens there and why that is sort of a place that sort of art is created from.
We're going to do a studio visit and look at some of my work. A very brief history of
the software in the arts. I'm going to talk about my teaching at UCLA and some of the
work that the students are doing there. And then I'm going to do something that I haven't
done before, which is to basically go through one project from start to finish to compress
about six months of development into just about ten minutes and show you how something
forms over time. I mean, to be honest, I'm a little bit nervous about that, but I decided
it was a good idea to try and we'll see how that goes.
So we're going to start at the studio. Here we have 1666. This is Vermeer, a painting
by Vermeer. 1855. 1870. And then now, after the cameras being used, we have more interesting
paintings in my opinion. This is James Ensor, 1896. 1911. This is a Matisse. And then we
have the area of the photographic documentation. So photographing the artist in their studio,
which has been sort of an iconic form in the 20th century. This is Lee Krasner in 1938
in the Hoffman studio. This is Alexander Kolder, 1963. This is how I imagine the inside of
my computer to be. Or wants it to be. This is Lee Bonteq, 1963. Takis, 1975, around that
time. And Helen Frankenthaler, 1975. So I've shown these images for two reasons. First,
to show how the idea of the studio has changed over the last few centuries. The studio has
been around for centuries and how has it evolved and developed and how will it continue to
do so. Second, to show how incredibly different it is to work with physical materials as it
is to work with software. So I guess the presentation has two simple themes. One is how is working
with software different and how is it the same? So I think the last few images clearly
have a different quality than the first romantic paintings. It's my hope that one day the photographs
of the heroic 20th century painters and sculptors will feel as quaint and as antiquated as the
earlier paintings in comparison to what we will do, how we will work and how we will
change the future. I think maybe for you, these paintings, like the one on screen now,
they possibly feel old fashioned and strange. For me, though, they still hold tremendous
power and sort of the weight of that history is still very present. So more to the present,
I'm going to show you some snapshots just taken really informally on my phone when
I was over at my friend Voss's studio. So Voss is 80 years old. He's been painting,
making sculpture his entire life. He has a studio in Los Angeles. He was a colleague
of mine at UCLA for 10 years. He's just recently retired. He works in a way that's very similar
to the way that I work. Here you can see a close up. You can get some of the sense of
the texture. But Voss has never written a line of code and I've never made a painting.
But there's still a lot of similarities in the way that we think and we do. These are
Voss's paints already premixed. These are his tapes for the meticulous masking that
he does. And these are his brushes. So I think the question is, what is a software studio?
And I don't actually need to show you examples of what it is. Like I needed to show you examples
of the painters or the sculptor's studio. Because you all know what it is. It's you
hunched over a laptop, possibly in the lobby at I.O. It's a lab full of people clicking
away at keys. It's such a different place in space. A lot of it is intellectual. A lot
of it is immaterial. So in essence, the studio, my studio is this computer. I mean, headphones
are actually very essential to my studio. And the computer that I'm presenting on right
now is the most essential element of my studio. I brought it here to you. And there's nothing
special about this hardware. In a short amount of time, everything can be transferred to
a similar machine. I think the important point to make is I consider the heart of my studio,
the most important thing, even more important than the results. And the things we'll be
looking at is the thousands of files of source code and the digital images that I've created
over the years. And unlike a physical studio, the work is not material and it's not ready
to be seen. It has to be pulled from the intangible storage on the machine so that it can be experienced.
The core of my studio is the processing software. And sort of in the spirit of this talk, we're
going to be running a lot of code directly out of processing, rather than showing you
pre-rendered works or images of works. So I'm going to start with one piece of software
to see, which is called Process 18 Software 6. And I'm going to show these images, these
live-running software pieces for between one and two minutes. Kind of imagine you're
in a theater. I hope you can be in the right state of mind to look. But I think to enjoy
them, you need to look closely and think about what's happening. I'll also narrate them
a little bit, too, as if I were David Attenborough and we're looking at creatures. All the work
I'm showing you today is new work. It's made in the year 2013. And some of it was finished
just a few weeks ago. This piece of all the pieces I'm going to show is the most representative
of the work that I've been doing in the prior decade.
Pieces like this that I've been working on, they really require a long period of time
to get acclimated to the work. This piece begins, but it really has no end. It continues.
Over time, the screen reaches a certain saturation, and then at that time, it sort of maintains
that balance, but the forms are always shifting and moving over time. At one time, it's sometimes
one color dominates, and that shifts to another over time. Their slow piece is meant to be
experienced over days, weeks, years, et cetera. Can we have some of the lights down, actually,
on the stage a little bit so the image becomes more clear? Okay, next one. So, next, we're
going to look at a piece called Yes-No, which I've shown in different variations here at
IO, but just finished recently. One thing about working with software is that you can
always change it. This piece was about trying to find what I consider to be the simplest
possible algorithm, and then to explore that, to spend time iterating on it, making different
versions of it. And there's a long story behind how we see what we're seeing now, and a lot
of different versions behind it. But essentially, the algorithm is just to make a grid and then
to flip a coin. And if it's heads, you draw a line going to the left. If it's tails, you
draw a line going to the right. And these different forms that we're seeing all arrive
out of that really simple process, which can be done simply on paper. The next one to look
at is called Signal to Noise, and this is version 5 of that. So, this piece and this
family of work came out of a show that I worked on all last year. I started it actually right
after the last IO. It was like late June through the rest of the year. And the show was called
20th Century. And the idea of the show was to look really closely at works of art in
the 20th Century that, to me, we're very striking. And to take ideas from those works
and to iterate on them and to make new works that meant something to me that maybe had
a similar idea to those works at that time in their context and place, but for me translating
the idea into my context in place. The Signals to Noise works all about television. And they
actually take terrestrial television signals, and then they make collages out of those signals
directly. So, let me show you another version of a Signal to Noise piece that is formed
in a little bit different way. This piece is called 100% Gray Coverage. Okay, one more
piece to show. And this piece is called Network D.
So, this piece is called 100% Gray Coverage. Okay, one more piece to show.
Okay.
This is the piece that we're going to come back to at the end and look at the steps along
the way. The cycle continues. Okay. So, very briefly, I wanted to just touch on a few ideas
from the history of software and art. This gentleman, Marshall McLuhan, one of his many
very quotable statements is, we shape our tools and thereafter our tools shape us. And
he's not talking about software directly, but he might as well be. Next thing. Of course,
this moment, 2013, isn't the first time that people have started to ask questions about
software and art. In my opinion, many of the ideas that we're talking about here today
have been hashed out for decades. Here is one particular exhibition that I think is very
strong. It's one of the first strong exhibitions. It's 1970, Software Information Technology,
this new meeting for art. What's interesting about this exhibition is that it focuses on
information. It's about data and process rather than the art object. And it features conceptual
artists like John Baldassari, Hans Haka, Alan Capro, along with designers and technologists
such as the MIT architecture machine group. And this image is from that. There's a library
of books on this topic from that period. Here we have Herbert Franke's, Computer Graphics,
Software Art, Yazzie O'Reikart's, The Computer in Art, Artist in Computer. This one is free
online at AtariArchives.org. It's really an extraordinary kind of document at the time.
But again, there's libraries of books like this. I consider it to be the birth of software
as art, to be conceptual art. This book has one of the most, I think, the most extraordinary
title of a book that I've ever read. The full thing is the title. And it's less of like
an edited essay or like a long form point of view. It's actually a collection of materials
from the time, sort of taken at the time. And so you read through its almost encyclopedia
of conceptual art during this time period. I just want to read two very short quotes
from it, or one. So Lippard says, conceptual art, for me, means work in which the idea
is paramount to the material form. The material form is secondary, lightweight, ephemeral,
cheap, unpretentious, and or dematerialized. So Saul DeWitt distinguishes between conceptual
art with a small C, for example, his own work in which the material forms were often very
conventional, although generated by a paramount idea, and conceptual art with a capital C,
which I have more or less described above. This is an example of conceptual art, one
of my favorites. I want to say something really briefly about this extraordinary book
by Malcolm McCullough, Abstracting Craft. And there's a chapter, actually Chapter 7 in
the book about medium. And it says a lot about the medium of software, which I think it's
important to be clear about terms when we're having conversations that go deep. So in defining
medium in general, he says, to give work substance, we require a medium. The action of our hands,
eyes, and tools must be mediated. Our personal knowledge and skills must be given a habitual
setting for practice. So according to his definition, I think a largely accepted definition,
a medium is two things. It's a set of tools plus a material. Here he continues. Quite
often the word medium signifies a class of tools in raw materials. For example, metal
working is a medium, which includes welders, torches, hammers, and clamps, as well as aluminum,
iron, and bronze. Because a type of material distinguishes a particular class of tools,
the ensemble may be referred to on the whole as a medium. So I take that a little bit further
into software. And for me, the definition is with software, we have programming languages,
we have software tools, we have hardware platforms, and we have collaborative versioning systems.
And those things come together to form the medium of software, the medium that we work
within. So the next section is about teaching. I often also don't talk about my teaching.
I really want to thank Jair for organizing the Code Ed summit yesterday. That was an
extraordinary event. I hope a lot of you were there. And I look forward to continuing on
with that and things that come out of that meeting. So I started teaching art and design
students about a decade ago. One of the first things I learned is that they have to be excited
about programming before they actually have the motivation to do it. And so a few years
ago, a colleague of mine, Chandler McWilliams and I, we compiled our lecture notes and our
examples from class, and we wrote this book along with the Dutch design studio Lust, Forming
Code and Design Art and Architecture. And in order to communicate clearly to this group
of students, which was primarily our undergraduate students at UCLA, it's a foundation class
that we've been teaching for many years, we broke the kinds of things that software can
do in relation to form into these five categories. Repeat, transform, parameterize, visualize,
and simulate. The truth is that almost all the projects in the book cover more than one
of these categories, but we found it was a really good infrastructure to begin the conversation.
And so we have these three questions for the book. How has software affected the visual
arts? What is the potential for software? As a designer or artist, why would I want
or need to write software? And that's a question for each individual, of course. So first category
repetition, repeat. I'm just going to show a few examples from each category to give
like a wide overview of what software can do and be. And a lot of these examples come
from folks from I.O. Here we have Jennifer Steincamp. This piece is Daisy Bell. It's
from 2008. This is a large video projection. All the flowers are animated in kinetic. This
is a browser piece from 1997 by Jody. Another Jody. These are parameterized pieces that
were laser cut. This is how they were assembled. This is a piece called a periodic vertebrae
by the very many from 2008. Here you have Toyo Ito. This is the serpentine gallery pavilion
from 2002. It is a demonstration of recursion where you have one form then continues to
repeat on itself until that operation is stopped. And the form of the ceiling of this pavilion
was formed through a really simple recursion with squares. The next category is transform.
We have Zaha Hadid's Cirrus. This is a bench from 2008. Transform is where you have sort
of a geometric model and you manipulate it, distort it, warp it in order to create something
new. This is a piece by Alex Dragolescu called malware. This is an example of transcoding
where you took the source code of a computer virus and then had each of the characters
of the source code represent something different and this geometry was generated from there.
This is a piece called The Last Clock by Ross Cooper and this is an example of slit scanning.
It works like a clock watching a television stream. The next category is parameterize.
The idea of parameterizing is that instead of making one, you make many through adding
in different parameters. This is a piece by Morphosis. It's the fair tower, 2008. And
the central diagram is the one I want to talk about. Each color represents a unique pane.
So the idea in building a building like this is that you have as few different panels as
possible to decrease fabrication cost. So parameterization could also be used for great
utility as well as for aesthetics. This is Manfred Moore P1011-H1 and another from the
same series from 2004. This is Shop Architects, Rector Street Bridge number one from 2002.
There's a detail which shows the parameterization using existing industrial materials. Next category
visualize. Here we have Ben Fry, 2003, Isometric Blocks. This is a visualization of human genome
data. Aaron Koblin, Flight Patterns, 2005 through 2009. And Soso Limited's Reconstitution
from 2008. Next category is simulate. This is a piece by Moss Architects called Transformation
of a Necklace Dome. It was created for an exhibition of Buckmaster Fuller's work at
the Whitney Museum. So this is the simulation they created. And this is the actual thing.
This is a piece by Andy Lomas called Aggregation 4 from 2005. It's a simulation of coral growth.
And this is a piece by Gramatcio and Kohler called the Resolution Wall. And the idea here
was to use a genetic algorithm to design different fabrications that balance speed of
construction with detail. There's the actual thing as built by a robot from their models.
So it's my belief that all artists and designers in the 21st century should have an experience
programming their own software. Nearly everyone uses software in the production of work, even
in the most traditional fields. Software is a powerful tool, but when a person doesn't
understand how software works at a deeper level, other than using a program to retouch
a photo or build a 3D model by pointing and clicking, that person is engaging with software
in a shallow way. So if we remember our McLuhan aphorism, we shape our tools, and thereafter
our tools shape us, we can say that if a person can't shape a new tool, that person is shaped
and molded by another person's idea about what software is and can be. And therefore
their own work is molded by what that person's work thinks their work can be. I got tripped
up there at the end. But I think the point came through. The other book that I teach
with is Getting Started with Processing, which I co-wrote with Ben Fry. And what I want to
show you about this book is the table of contents. We spent many years trying to learn how to
teach programming within the context of art and design programs, and this is the current
iteration of it. So when we start, we talk about drawing, we talk about interaction,
we talk about media, we do a lot of work with photography and typography, we talk about
motion, we teach variables towards the beginning, but we really get into things that are the
core of programming after the students are acclimated, after they feel comfortable, after
they're already making things they're excited about, and then we get into functions, objects,
and arrays then towards the end of the class when they really have a strong desire to be
pushing further. All right. I want to show you now a set of examples from the Form and
Code book that relate to these examples that these core ideas we want the students to have
an understanding of. So I'm going to start with repetition. It's just a very minimal
example of how you can use a force structure to create repetition, how the computer can
be a tool for automated drawing and diagramming. Next example. This is an example of recursion.
Where basically you have an operation where you have a dot that moves up a little bit
and goes a little bit left to right, and that operation continues, and then at some kind
of frequency you have branches come off of that. These are all things we're familiar
with, but I'm just trying to put them in the larger context of the presentation. Next
example, parameterization. So here we have a simple structure, and we have parameters
for things like height, length, and width of a chair. So when does a chair become a
table? When does a chair become some other kind of form or object? Through parameterization.
Next example. Transformation. So here's a basic example of transcoding where you load
in an image, in this case it's satellite data from NASA, and then you just use the gray
value of the image to extrude things in space. Last example to show is from simulate. Here's
using one of Dan Schiffman's particle classes to produce this kind of motion. So these are
the kinds of things that students can begin doing towards the end of the course. They're
doing simpler things like the first example I showed at the beginning. Clean that up.
All right. So I want to talk a little bit about the program at UCLA and talk about a
specific initiative there. In our grad program, this is some of the cast of characters. This
is Noah's work. Here's Mark Essin's work. He makes some really fantastic games with
unique interfaces. This is Phoebe's graphite piano. It's an electronic piano that uses
the conductance of graphite in the pencils to work. This is Nick's machine for making
extremely large bubbles. And what I want to show are some examples from the UCLA software
studio. So the software studio is a really loose network of students across the school
of arts and architecture at UCLA. Basically, students from art design, media arts, architecture
and music. And the idea is to have these specific conversations about software as a medium and
to look at different kinds of work. So I want to show you some examples. This is David Wicks's
MFA show. And he worked with water. So this is a comparison of two data sets. It compares
where water is falling to where water is being consumed. This is one season of the year and
another season of the year. And then the other works I'm going to show from the software
studio I'm going to show with video. So we're going to look at four different projects.
This is the work of Godfried Hader. And what's not seen in this video is a really important
part of the project, is this online system that Godfried made for drawing circuits freehand.
So basically, if you look at every printed circuit board in the world, the ones inside
your computer and your phone, your Arduino boards, etc., they all look the same. They're
all industrialized in the same way. And what Godfried wanted to do was to explore an alternative
for that. So he made this tool that allows you to freehand circuits. In this case, he's
using a vinyl cutter to print out the copper and then to put these circuits into a different
context. The next works by Lauren McCarthy. And this is a new project from Lauren, but
it's based on the work that she created while she was at UCLA.
Thank you so much.
This is the work of Rafiq Anadol and this is a proposal, a proposal that's now happening.
So this is a mock-up and this is Walt Disney Concert Hall in Los Angeles which is a landmark
building in the city designed by Frank Gehry.
And the proposal is to actually respond to the music that's happening inside the hall,
to do dynamic images that are responding to that sound, and then also to be looking at
the gestures of the conductor, Gustavo Dudamel as well, and to build the visualization based
on looking at those two pieces together.
So here he's exploring.
He's essentially trying different things and seeing how it feels.
And this becomes then a presentation tool that he took to Disney Hall in order to win
the commission.
The goal of these examples from the software studio is to show the diversity of things
that are happening and I think different ways that software can be utilized within an MFA
degree.
And this last piece is by Chris Riley.
And this piece is also about relationships, in this case it's about a distance relationship.
Well, Chris was in the MFA program, he was living in Los Angeles and his partner was
living in Chicago.
And so this project was a few months of this ritual where they would take their sheets
that they had used and then they'd send them to the other partner and then they would open
up the box and then through Skype they would have this ritual of opening the boxes and
then making the bed together.
And so in this piece Chris didn't actually write software, he writes software very well,
but for this piece he was using sort of existing infrastructure and examining how that could
affect his relationship.
Okay, and then actually Lauren and Godfried and David Wicks are all here so I hope you
can meet them also while you're here.
The last thing I want to show is the process of the network D piece which was the last
software we looked at during that part of the presentation.
Before showing the process I want to show a little bit of the final exhibition.
So this was in Berlin last fall at the Dam Gallery.
This is signal to noise projected.
Here's network D. This print is 160 centimeters by 100 centimeters.
Here's a detail of that, not the actual print, but a digital version.
This is another print based on network D in a detail.
These pieces are called yes, no, and these are the four different kinds of interpretations
of that simple algorithm together as a detail of that.
And then here's a little more context and then these are pieces from, prints from a
piece called network C, and detail of that into another detail.
So let's look at the process and have a lot of windows open.
So this was the very first piece.
It's very impressive.
Basically what I was trying to do here was to just get the nodes together, how to build
that out in a way where they're distributed in a way that would be appropriate for the
project.
And this is a piece exploring the ideas of networks, but in a more visual way rather
than working with a data set.
Next version of that was this.
Now we have the node set and now we're basically moving between them.
And here's how things have been laid out, a little bit of more infrastructure, but still
very minimal.
Okay, next.
Now at that point I tried something completely different.
I was looking at TCP IP headers and looking at ways of basically taking the signatures
of those and then turning those into these different patterns.
And the next one is also based on that as well.
So it was a left turn in the process.
So after that, skipping any iterations, this is where it started to go.
I think now we have some different controls but on for different speeds and we can start
looking at things in a different way.
And then this was the end of this iteration of the piece.
It now has a set of parameters so I can explore it and look at different things that are possible
with the system that I had built.
So for example, I can speed it up, I can slow it down, I can look at it in a more regular
fashion or I can add a little bit more distortion to it.
And then I can look at a very large grid or a small tighter grid.
So that was kind of the end of this first sketching period, the first sketching stage
of the project.
And then I started working on it into more of a refinement mode.
So let's look at that.
It'd be great if we could have the lights down again on the stage, thank you.
Okay, need to keep going.
So here's a further iteration of that.
And the difference here is that the lines as they're moving from one node to the next
are actually changing colors midway and it creates a different perception of the piece.
So this became a piece that made it to the exhibition.
It's called Network C. And I'm going to let this run one run a little bit so you can see
what happens with it over time.
Okay.
So then that led to another set of work, very similar work, which you can see here.
And this piece was a printed piece rather than a real-time piece running as software.
So that's one reason why it's much slower.
But I'm going to let this one run also for a few seconds because something happens.
So it's identical to Network C, but then after a certain amount of time, the nodes move and
this is the result of that.
And so these are from the Century show and remember the idea of that show was to look
really closely at different works of art from the 20th century that I was extremely interested
in.
And so these network works came out of work from the distilled movement, like Theo van
Doesburg and Modrian.
And so that's why they used that color palette that they had been using.
And so after a while, I realized that if I really wanted to make these works sort of
for my time and my moment and to make them more sort of personal to me, I needed to take
more steps in order to do that.
And so one was modifying the palette.
I think this is a far more appropriate palette for now with these machines than the one I
was using before.
I think it was also influenced by playing with prisms with my five-year-old daughter,
too, and looking at the light that comes out of that.
But for me, that was far from the end of the project, and the next thing that happened
was the piece that we were looking at earlier.
And so the steps to get to there involved this.
So the next part of the process, after doing all this sketching with code and moving from
open possibilities to refinement for me, is when I do the print work, is to look at things
in a different way.
So here, these are some of the final images or images that came out of the software we've
been looking at.
And so I start looking at them together in groups, and I start making more decisions
about them.
So while I don't really, the program for this work doesn't take a tremendous amount of time,
what actually takes a lot of time is thinking about it and looking.
And so in that way, it's very similar, although I'm writing software, to how a traditional
studio process works.
I mean, at one point in time I started doing this, this idea being that they're modern
networks.
I felt this just felt more appropriate.
And so looking at those in different ratios and different proportions, and then that was
one step of refinement, and then the next step of refinement for doing that, that print that
we looked at an image of was this.
So at this point, it goes from creation to editing.
And so these are images that were all the result of the software.
So if I just run through them very quickly, you kind of see a sense of the range and the
options at that point of the process.
And then for my work, the last step then for doing the printed work is to render the piece
out.
To work at that scale, I need to render them out at this size, basically at actual dimension,
which you see here, and we can flip through there to see those.
And then at that point, they get aggregated sort of automatically in software into these
small units, which are pieces of the final print.
And then I run that through the software once more to generate the full 160 by one meter
print at the end.
So that's the tour through that one piece, and of course, the same thing happens for
all the other pieces that I showed in the exhibition and the prior software work.
So I think in talking about the studio, I wanted to set out the talk by showing what
the traditional art studio sort of feels like and how that working happens, and then
to juxtapose that with the software studio and working with the immaterial.
While I think the materials are radically different, and we do things in software that
are sort of unimaginable in prior media, I think the way of working in the process is
actually nearly identical.
So that's the end of the presentation.
Thanks very much.
