Tonight we have David Merrill from SIFTO.
I saw David's talk at TED as many of you probably did a few years ago, and it's actually
one of the things that got me into wanting to do stuff like designers and geeks.
I was just really inspired by the technology, what they were up to by his talk, and I wanted
to sort of see more of it, more of this mind-expanding stuff.
And so that's kind of why I'm here tonight, and very excited to bring him up.
So let's welcome David.
Thanks.
I'll take the clicker.
All right.
Hi, designers and geeks.
This is really exciting to be here, because I'm both a designer and a geek myself.
So just a little preview of what I'm going to be talking about before I dig into some
of the background.
Those are SIFTO cubes.
I'll tell you a lot more about them.
But first, some context about why we're working on this product, why we're building this.
So this is a popular vision of the future of human-computer interaction.
This image is from a Microsoft concept video that was called Productivity Future Vision,
and it's what I will call the glass slab.
It's this idea of the platonic ideal of the cell phone, the perfect glass internet-connected
pixel-packed item in your pocket, and the notion is really powerful that this is one
of the most important devices that we've ever created and that we've all gotten our pockets.
And the implication of the glass slab is that the software on the device starts to take
the place of all these other physical objects that you used to use.
So I used to have one of these, a guitar tuner, but just this morning I tuned my electric
guitar with an app on my iPhone, so I don't use that anymore.
When was the last time you used one of these?
Or one of those?
Or one of those?
Or those?
Or those?
Or those?
Or answering machines or levels or maps or even cowbells.
There's an app for that one too.
So it's really happening.
It's really a thing, but I'm here to persuade you that that perfect glass slab in every
pocket, which is the direction we're heading, doesn't actually eliminate the need for other
devices, but what it does is it opens the door for a different kind of device, and I
think a more interesting kind of device.
So think about this.
Woodworkers have this huge range of tools.
This is a very mature craft, and all these tools have the part that meets the human on
one end, your hand, and then the business end on the other end, which is for sculpting
wood in various ways, and I think there's an important lesson here.
One, the tools have to fit us, but the tools also have to be really well suited to the
task at hand.
And our hands actually have lots of different ways of interacting with physical stuff.
I stole this picture from an essay online, which I recommend you check out if you don't
know it, it's called A Brief Rant on the Future of Interaction Design, it's by Brett Victor.
He is, I would call him a user interface philosopher and expert creator, and he points out that
all the ways that our hands have evolved to interact with physical stuff are really different,
and almost none of them are like fingertips on a touchscreen.
So that is a very narrow slice of what our hands are capable of.
So Mark Andreessen put it really well in 2011 when he said software is eating the world,
and basically he meant traditional products in industries are being obsolete by their
faster, better, cheaper, software enabled cousins.
So this is happening, right, the apps on your Glass Labs are part of this, and the upshot
for me as a designer of new technology is that I think it's really important to actively
design the kind of physical objects that you want to have, lest they all get eaten by software.
So what kind of devices should we make if you're somebody who thinks about making hardware
like I am?
There's two ways to answer this, one, none, Apple, Samsung, those guys are giving us everything
we need, that's it, just make software.
But I think, I don't believe that, I think that the things to make now are the uniquely
physical interactive objects, the ones that really need to be physical, the ones that
are not eaten by the Glass Lab in your pocket.
So I have a categorization that I made up to think about the categories of devices that
fall into this.
Number one, devices that interface legacy infrastructure in the world, and there's a lot of legacy infrastructure
like the credit card system, or all the televisions that are out there, so the Roku box, interfaces
the internet to televisions, or dead bolts on doors in houses, there's a lot of those,
or heating and cooling systems for people's houses, there's a ton of opportunities to
expose that legacy infrastructure to us with a better interface on top of it, mostly using
our phone as the interface in these cases.
Another category of devices that I think make a lot of sense today is ones that manipulate
matter, ones that push atoms around in the world.
So like MakerBot, that is extruding plastic to form any three dimensional shape you can
imagine, or Romo, a cute little robot that you stick your phone into and it becomes the
face of this robot on tractor treads, or the Blossom coffee maker, which brews the ultimate
cup of coffee by pulling internet, pulling recipes from the internet based on other people's
best cup of coffee they ever made.
So the point here is that manipulating real world materials, moving them around is something
that our phones don't do.
And then my third category is devices that interface to us, our bodies, right?
Like I said, your fingertips are a pretty thin sliver of what your body can do.
And so, you know, the interface to us humans hasn't changed in a really long time.
Sometimes as somebody who does electronics, I think about the data sheet to the human,
to humans.
And if you know electronics, a data sheet is basically this PDF that you get that describes
a chip and how you're supposed to hook the chip up to the rest of the circuit and what
kind of signals and messages you can send it and what you can expect to get back.
And so, our bodies have a very static data sheet, right?
The kind of frequencies our ears can hear, the kind of spatial resolution our fingertips
can feel, the kind of visual patterns and frequencies our eyes can see, like that's
not changing at the rate of technology.
So devices that interface us in different unique ways are really important.
So like CIFTO cubes, my company's product, fitness monitors like the Jawbone Up or Fit
Bit, gesture sensors like the Leap motion, this is Cell Scope, a microphone attachment
for iPhones that turns your phone into a Otoscope so you can look for an ear infection, right?
So all these different ways of interfacing our bodies.
And I think there's not a one-size-fits-all device and there's a huge opportunity for
well-defined, well-designed tools that have these unique physical capabilities.
So does anybody know who this guy is?
Mark Weiser.
One, yeah.
Okay.
Well, for those of you who don't, I'll tell you.
So Mark Weiser was a computer science pioneer.
He was the chief scientist at Xerox Park down the peninsula and is considered the father
of ubiquitous computing, which is this predicted future that we're living in now, in which
we'd be surrounded by a lot of little computers, all programmed to do different things that
would be helpful to us in our everyday lives, just everywhere, like seamlessly woven into
the fabric of our lives.
And one of Weiser's key ideas was that in order for this future to work and to not just
piss us off all the time, these computers around us were going to need to be designed
in ways that we could interface with naturally.
They needed to relate to us on our own terms with things like gesture and voice and just
interactions that we're already good at as human beings.
So one of the specific things that he predicted in this famous paper that he wrote for Scientific
American in 1991 was that in the future, we would be interacting with computers at different
scales.
And he put labels on three of these scales that were boards, which were basically he predicted
they would be wall-sized computers that you'd interact with using gesture, which today would
be systems like the Kinect or that CNN weather wall or any big touchscreen, pads, which he
described as about yay big tablet-like computers that you'd interact with by touch.
Remember, this is 1991.
We have those today, of course.
And then tabs, which he described as inch-scale computers.
And his idea for tabs was that they'd be basically containers for information.
He paints a scenario where your physical desk has a bunch of these little computers on it
and you might still have a desktop computer in this future, but you'd shrink down one
of your application windows onto a tab, carry it over to your co-worker's computer where
you'd rehydrate it on their machine and work on something together.
So they were kind of like glorified thumb drives.
But the important part was that they were a physical representation of a digital thing.
And so what is SIFTO?
Well, we're making tabs in a way.
We're making little inch-scale computers made for manipulation with your hands.
I got inspired by that vision that Weiser put out there, but realized that I think there's
a lot more interesting things that you can do with little computers that are the size
of that you can manipulate easily by hand when you're actually using them as tools for
manipulating media and tools for play rather than just thinking of them as containers.
So at the highest level, SIFTO is enchanting everyday objects, in this case little things
like the blocks from your childhood with the magic of interactivity.
So here's a video that shows some of the interactions that we're working on.
There's an insane amount of technical wizardry hacked inside to make them work, but on the
outside, they're radically simple.
You can download new games for your cubes digitally.
Let's take a look at what you can do.
SIFTO cubes can be containers for other objects, slide the letter tiles to make words, or they
can hold particles that flow from cube to cube.
They can be windows into another world.
They understand what your hands can do, like pouring a liquid or twisting a key in a lock
to open a chest, or flips, nudges and collisions, or even making lunch.
SIFTO cubes live in our world, so you can spin them like a top and use all kinds of real
world stuff in creative ways.
Games are smart.
You can play with other people and full body gestures.
So here's some real players.
These little girls' dad sent us this video to show how they were enjoying their SIFTO
cubes.
My daughters have just started playing Chroma Shuffle.
What do you guys think?
Do you have any message for the people who made this game?
What?
Do you have any message for the people who made this game?
So, for new designers and engineers, that's the reason to do anything, right?
I just want to make that happen in every living room around the country.
So it's a system for play.
That's what we've created from this interface idea of little hand-manipulatable things, and
a little background on how we think about play, basically, this is the way we play,
we human beings when we're kids.
We play with stuff, right?
Real physical things like Legos and Rocks and Jacks, Dominos, and some of the earliest
games that people ever played were games like Moncala, games around physical objects.
And it's also how we learn, right?
You think about Montessori and other hands-on learning methods, tap into this really deep
part of our psychology for interacting with physical things.
So our minds and bodies are just evolved for this kind of interaction with real physical
stuff.
And if you fast forward to most of modern video games, whether they're TV-based or screen-based
on tablets, they're totally amazing, right?
They blur reality and they're so engaging.
But we started to think, as my co-founder and I, we were cooking this idea up as grad
students when we were at the MIT Media Lab, we started to think, well, what if you could
bring these two kinds of experiences together?
What if you could blur the line between video games and Legos just a little bit?
What would that even look like?
So that's how we think of SIFTO cubes.
They're basically classic play experiences like board games or Dominos or Legos, but
those game pieces made interactive.
So kind of like Nintendo Wii for your coffee table.
So why do we do something like this?
Well, I really love making tools to enable other people's creativity.
So there's a new design space that we can enable by building this into a platform.
And so I want to share a little bit about how we design for this kind of system.
First, SIFTO cubes have really small screens.
They're 128 by 128 pixels.
They're full color.
They can do some animation, tile and sprite-based, but you don't have very much real estate
compared to a PC or a tablet game or even a mobile game.
So with less area, you have to just strip down the inessential elements.
We don't have room to put scores on the screen at all time or any other UI elements on the
screen.
It's really just the game interface itself.
But with less area, you also get a system that is more personal because you're holding
these things in your hand, literally.
And so the design implication of that is we tend to use very large, simple graphics.
One way to think about these little tiles is that they are, they can hold a single symbol
at a time.
Although you can do much more complex stuff, like in the earlier video, you saw them acting
as kind of a window into a world, kind of Legend of Zelda style.
But some of the best games are just very simple, big graphics.
And then motion, too.
So the Nintendo Wii ushered in motion-based gaming.
And this is at a time when Microsoft and Sony were pushing high voxel count and the speed,
the horsepower, the amount of RAM that their consoles had, and the Wii came out with this
totally orthogonal, different point of view, which was, that stuff is not as important as
just the way we play the game.
So motion-based gaming has been around since then, and of course phones do the same thing
now.
So CIFTO cubes have the same kind of three-axis accelerometer for detecting shaking and tilting
and being stood up on edge or flipped upside down that a Wiimote does.
And then we've added this one feature, which is one of the, one of the real characteristic
unique ones, which we call neighboring or neighbor detection, which is basically that
the cubes can tell when they get next to each other, even if they don't touch, if they
get within a couple millimeters, cube A can tell that it's next to cube B, and it knows
which side of cube B is over there, and cube B knows which side of cube A is over there.
And so that all gets bubbled up to the running game application with a message like, cube
A is now next to B.
And so we developed this little gesture vocabulary for CIFTO cubes, which we're working to expand
into more free-form gestures, but this is the core, the core set.
So tilting, flipping, standing it up, shaking it, pressing, yeah, there's a touch screen
on top too, and a neighboring.
And then finally, just designing for a group of devices is a different beast also.
Like, your game runs in one place, this is the base station here, so this is like the
brain that has a speaker in it that controls the cubes wirelessly, but it's got access
to this fragmented little group of screens.
So as a startup company, we operate kind of like a little Nintendo, basically.
We built this platform, but we know that we've got to also put good games on it, at least
to bootstrap or to kickstart a community of developers to show, here's some examples of
great stuff, and then open it up for everybody to work with it.
So basically we know that platforms are powerful when they enable other people's creativity
and that we don't have the market cornered on that creativity.
So we did, in January, we did a game jam.
We worked with experimentalgameplay.com, which is run by Kyle Gabler, the guy who made World
of Goo, one of my favorite iPad games, it's on other platforms too, but we did a contest
and we got more than 40 entries, and then the way that contest works is after a month,
we get these entries, we pick the finalists, and then they polish for another month.
So I want to show you some of the entries that came in at the end of the first month.
So this is a game called Squaresville by a guy named Frank Force, it's a little bit
Super Mario Brothers-like, but the mind-warping thing is that once you open up the next area,
you can then pull the cube away and rotate it and put it back against the cube where
the character is, and then walk into that area, like now walking on the wall, or now
walking on what was the ceiling, when you first put those cubes together to uncover
that next area.
So it's playing with space in a really interesting way.
This is one from a French studio called One Life Remains, their game is called Paro Plouie,
it means umbrella.
So it's a racing game where you're trying to find the end of a maze, get the umbrella,
and then get back to the beginning, and it's a multiplayer competitive game.
This is a storytelling game, which I think is really delightful.
There's no winning or losing, it just shows you a sequence of pictures and you get to
make up a story for them.
So as you pull the cube away, the picture goes away, and then you put it next to another
one and you get a different picture, so they took this game just in different places and
tried it with different people.
So here's another game, you put the lettuce on there, and then you tap, until it chops.
It's kind of like Diner Dash, and you just grab it and put it on there.
Let's do the tomato.
It's basically, I think the technical term is a spinning plates game, because you're
trying to keep a bunch of things going at the same time.
So you can use the tomatoes and the lettuce to make gourmet burgers, which have lettuce
and or tomatoes on them, and also during the vegetarian attack level, you can also make
vegetarian sandwiches that just have one or the other.
It's good, but a very long explanatory video.
And then see.
Here's one called Shifted by a guy named Nick Van Sachtelen, which I think is like the translation
of a Rubik's Cube into the modern age, basically.
You're trying to get all the dots around the edge of a cube to be the same color, and you
can shift them off to other cubes by neighboring them and then touching it to rotate all the
way around the perimeter.
It's really hard, but I love puzzle games, so this is one of my favorites.
This is from the game developer conference.
We had a bunch of games that we had published that we were showing for the first time.
Nice.
So people really get into the physicality of these games, like sliding, having them bump
into each other.
You can basically reinvent lunch table games, but add an interactive element.
So like I said, I really love being able to enable other people's creativity by putting
a good platform into the world.
And in the game community, we've managed to now work with a bunch of the indie game developers
that have been our heroes for a long time, so Kokuromi is the studio that Phil Fish is
part of.
He's the guy who made Fez, if you know indie games.
Daiguta Fabric is a game studio in Copenhagen that made the game where you're stacking
cubes up and flicking them into each other.
It's called Tower of No Tumble.
If you're a fan of Magic the Gathering, Richard Garfield is working on a game on the system.
He's the creator of Magic.
Chris Osborn made the BitTrip, he was the developer on the BitTrip series.
That's a great game for Wii and other NPC.
It's like a music and rhythm game.
And then Larry Kaplan, who was one of the Activision founders, is a friend of SIFTO
and said, SIFTO cubes are fantastic.
So I want to point out also having a platform out there is really fun because not all the
applications are games that people are making on it.
So this is a studio in Portland, Oregon called Second Story.
And we didn't even know they were doing this, but we found this video show up on the internet
of a music interface they made out of SIFTO cubes and Microsoft Surface.
With a multi-touch Microsoft Surface table.
In satellite, each cube becomes an interactive game controller.
Users make music by pressing, tilting, moving their object in space, and placing their cubes
on the table to select new sounds.
I think what they did is like 2D barcode style markers on the bottom of the cube so
they could tell where the cube was put down on the surface.
And then finally, there's a company called CogCube that's building tools to help diagnose
ADHD with SIFTO cubes with a game called Groundskeeper.
So with a few minutes left, I'll tell you a little bit about the system itself in case
you're interested, if you may be interested to develop on it, a few of the technical details.
Like I said, full color, 16-bit graphics with layers of tiles and sprites.
There's a callback-based event system, so you get notified.
The application running on the base gets notified when touch or neighboring or shaking those
kind of events happen.
It'll go anywhere from 3 to 12 cubes, our standard system ships with 3, but then it
expands gracefully if you have all the way up to 12, and you just bump a cube against
the end of the base to pair it.
And then the hardest part, but the most interesting part, is just that you have multiple screens,
four orientations, there's really no up on a cube screen, and lots of permutations.
So I could talk for a long time about the hardware, but I won't.
It's pretty cutting edge.
The way these guys sense each other, this is the circuit board inside a cube, is our
own little proprietary inductor circuit so they can sense each other through the walls
of the device without having to have any holes cut or anything.
And then if anybody knows Google Native Client, which lets apps run on your computer's hardware
but in a way that's sandboxed, that's for your browser.
We kind of had to do a similar thing for embedded systems.
So we have a pretty low power, modest processor, but we needed it to run apps in a way that
were sandboxed but really high performance.
So that's some of the cool technology, software technology that came out of building this
play system.
And then finally, the SDK, like I said, it's out there, it's free, it ships with a really
good emulator that's cycle accurate.
So if you write something and it works on the emulator, it'll work on the hardware.
And then we have pretty good documentation too, getting better every day.
So we have a blog, tech.sifto.com, where we have getting started documentation that
talks about how to use the emulator.
There's a bunch of applications that are in there that you can start from so you don't
start from scratch, things like what does your main loop look like, when you're coding,
how do you use sprites to put graphics on the screen, stuff like that.
So one final note, I think something very interesting is happening in hardware and the
next talk, next time that Joe pointed out, is a perfect example of that, which is that
there's all kinds of community happening now.
So there are incubators, like Lemnos Labs here in San Francisco, or Bolt out in Boston,
or Accelerator in Shenzhen in China, bringing in startups that want to build hardware and
incubating them for a few months.
There's meetups, like there's a San Francisco hardware startup meetup.
There's solid state startups down the peninsula.
There's the Internet of Things meetup happening now.
There's a lot of informal helping happening, which I think the software, like the web people
had this figured out for a long time, but hardware people are just starting to do this.
Probably because they're now building things that also connect to the Internet, it's bleeding
over.
And then, of course, alternative funding models like Kickstarter and Indiegogo are funding
a lot of these projects and products even before venture capitalists are feeling comfortable
to get in there and fund companies.
That venture capitalists are getting in there to fund companies, too, in a way that they
definitely were not five years ago or 10 years ago, and some of these companies are
just really taking off.
So to me, it feels like we're at the beginning of the PC revolution again, but now for this
range of much more interesting and crazy and creative devices.
So I think we're going to have a lot of alternatives to that glass slab and a lot of augmentations
and things that your phone is not doing that are going to have these unique physical capabilities.
And I think that's great.
So what can you do to get involved?
Well, if you code or design, probably many of you today who are here, check out our SDK.
It's a free download.
It gives you a lot of good stuff to get started.
If you make hardware, make sure you make a platform that matters that is not going to
just turn into a casualty of apps on people's cell phones.
And then otherwise, seek out and spread the word about systems and experiences that people
have made that you think are awesome, because that's how these communities get the support
they need to start and turn into real parts of the industry.
So thanks very much.
I guess I have some time to take questions, so I'll close there.
