Interview with Pixar Cofounder Alvy Ray Smith

Our conversation has been edited for length and clarity.

?

What compelled you to tell the story of the pixel (and The Great Digital Convergence) now?

You know, it took me ten years to write that book, and that was because I kept being surprised.? It's what made it a fun process.? I spent a year on Joseph Fourier only to find out there was so much more, including his relationship with Napoleon, that I didn't know.? And that introduced me to the notion of the tyrant in technology, which turned out to be repeated chapter after chapter, becoming a major theme. You’ve got the idea, and then some crisis that drives it forward, and finally, it seems there's always some tyrant that, for all the wrong reasons, helps push it along.

In order to explain what a pixel is, I had to explain sampling theory. And you can't explain sampling theory without Fourier.? So, okay, we'll start with the basics.? It's music, everybody understands music. You add up sine waves at different frequencies - you have music, you have sound.? Same thing works for pictures. That was all I had to get across in the Fourier chapter, but then I also started thinking about the book as a celebration of the Great Digital Convergence, the moment when everything became bits.? It only started in 2000. ?ChatGPT and everything happening now is just the latest version of what is possible when all information is bits.

The computer opened up all sorts of things we weren’t even thinking about.? What else is out there? We don't know.? In fact, Alan Turing told us that from the very beginning.? Not only was computation a hell of a great idea, but Turing proved that we don't know where it's going, and we can't know.? But it's going to be awesome, because look what's happened so far.

?

In addition to the foundational scientific principles, A Biography of the Pixel gets into the foundational personalities.

I think of my book as describing this thing called Digital Light, the real theories behind it and the real heroes.?? Much to my surprise, many of those that I thought were heroes, weren't.? Fourier passes the test, but Claude Shannon doesn't. It's actually this Russian guy, Vladimir Kotelnikov.? It's an amazing story, and I'd never even heard of the guy before. ?I want to get the real stories out there.? They're not that hard to find once you get away from the ‘great man’ idea.? People want the simple narrative of one great hero from which everything flows. ?It just doesn't work.? It's rarely that way, although there are people with outsized influence. Turing, I think, was a screaming genius from which all computation flows.? There's no question about that. But he failed when it came to creating the hardware to implement his idea. ?He tried, but he didn't get there first and he should have. He had the idea.

?

What originally got you into graphics?

Well, it's because of my two loves. ?First, I love to paint, to make pictures.? I started with oils and then I got into acrylics - I even had a show once. ?And second, when these new devices came along called computers, it turned out I was really good at programming.? I had to decide which way to go, and I went with computers at first, going to Stanford in 1965 to get my PhD.? I wanted to study this brand-new subject called artificial intelligence in the newly created Computer Science department.? I later went on to teach artificial intelligence at NYU.

But then I broke my leg skiing and was in a full body cast for three months, unable to move. ?I was just sitting there being taken care of by friends and I ended up rethinking the universe.? I thought, ‘you're not doing anything about your art, this is wrong.’? I resolved that when I came out of that cast, I would drop out of academia and go to California where something good would happen. ?And it was that ill-thought, content-free idea that I executed on.? I arrived in the Bay Area and my friend Dick Shoup asked me to come over and see what he’d been working on; he had just built SuperPaint at Xerox PARC.? That's why I came to California, painting and computers.? And basically, Pixar came out of that.

?

Pixar has always had a kind of mystique about it – a magical quality, and it seems as if it was there almost from the beginning.? How did that company culture develop?

When we put a group together to make the first movie, we didn't know it was going to be Pixar. ?We didn't know what the culture was going to be. We just put out the word that we were going to make the first movie, and people just found their way to us.? And they turned out to be geniuses of all kinds. ?I didn't design the culture.? Ed [Catmull] didn't design the culture.? We were academics, so maybe, in a sense, that set the tone.? We respected academia, where the height of accomplishment was to get a paper published. ?

It's been several months now, but we had a gathering over at Pixar, kind of the original 40 who started the place. ?We were getting a bronze plaque installed, it was really a celebration of RenderMan, this standard way of rendering pictures. ?You know you’re an institution when the bronze plaques start going up!?? Since I've written the book, I've kind of become the go-to historian. They asked if I would get everybody up on stage and have a conversation about what happened and who did what, when and so forth. ?I had more fun sitting up on the stage with these geniuses, Rob Cook and Loren Carpenter and Tom Porter.

I knew they were solving unsolved problems at the time, but something about it didn't hit me until I was sitting on the stage - the density of genius and creativity that was happening all around me.? They would look at the problem and say, oh, I'll do that part, and I'll do this part and I'll do that part.? When everyone is doing great things, it’s like, try not to go to work!? We were just ourselves, and it turned out to be this really benign culture.

?

How do you maintain that culture over time?

One thing I might have been particularly responsible for was my outlook.? My background as an artist and my experience with computers allowed me to make people feel at home, whether they came from an artistic or technical background.? I would not let this “technoid” vs “creatives” distinction arise. ?I've been in graphic arts houses where there were the creatives, and then there were the “technoids”, who were treated as a lower class because they took care of the machines.? And I've been in the opposite place at Microsoft, where if you can program, you're the gods on the earth. ?And if you make pictures, they said that’s just the stuff these marketing people do, a lower form of contribution.? We didn’t allow that to happen. ?There was a mutual admiration society, because our business required different people, different personalities all contributing equally. ?Disrespect was not allowed.

One thing I'm puzzled about is how we avoided nasty politics.? We were able to keep out the kind of people who come in and try to go up the ladder quickly by playing politics - getting a kingdom going.? We just didn't allow it.?

?

Publishing in a field that was growing leaps and bounds was clearly important to you – do you think it’s the same today?

I was from academia, so I knew the importance of an opportunity to be known amongst your peers for your ideas, that's the glory of the field.?

I'm enough out of touch that I can’t really say if it’s still true today, but I hope so. ?I'm looking at some of today’s problems and it would fill five years’ worth of SIGGRAPHs! ?What I’m thinking about is mixed reality, where I’m looking into your office, and I deduce the 3D structure of your office from what comes into my glasses, and I have a computer model that I've built, and we mix those together and you can see through one of them to the other. One of them cast shadows on the other – there are all kinds of interesting, tough problems to solve when you mix a derived set with a simulated set. ?But that's the kind of thing that SIGGRAPH people have been solving for years. ?They'll just keep at it until they've got it.

?

You were operating in the early stages of a rapidly growing industry at Pixar and at Altamira Software.? What was unique to that experience because of the timing?

Well, my telling of the story says that we rode Moore's Law. ?We surfed that wave from nothing to this awesome supernova of computing power that’s still going. ?And you can interpret that as every so often it felt like somebody knew what was going to happen, but usually Moore's Law just delivered another order of magnitude of computing power. ?And that’s still happening today. ?We certainly knew we were part of that, but I didn't understand it as fully as I do now. ?But we always counted on Moore's Law getting us where we wanted to be, because we certainly weren't there when we started.

?

Were you watching the rest of the industry, your competitors, as you were evolving? ?

We didn't really know what we were ultimately going to be doing. ?We knew we were going to make a movie and that was the underlying idea, but it didn't have to be a 3D modeling, Pixar-like movie.? I think we would have been perfectly happy, in the original days, if it was a 2D “in-betweening” animation. ?In fact, that's what we first offered to Disney, but then that just didn't work. ?We found that 3D was easier, frankly, and went for 3D and everybody loved the look.

A lot of my life I feel like somebody just grabbed me by the nape of the neck and said, okay, Alvy, you're going here next.

?

I think that does happen to successful people, but there is often a lack of agency that creeps into the storytelling, as if somehow, you were magically lucky.

You're chastising me correctly.? I don't really believe in magic, but I do wonder about luck, whatever that is. ?It seems like sometimes we're chugging along on our path as carefully as we can, and then something unexpected just happens.? And if you're ready to make the leap, you're going to be assisted by it.? My experience in life says don't be afraid to make a leap.?

?

In the book, you discuss scientific rivalry and collaboration throughout, and it reminded me of Open Source.? What is your take on Open Source?

I've always been a little surprised by it, frankly. ?Even though I'm wide open for people broadcasting their algorithms, there was always a place where we'd say, “that idea, let's just keep that one quiet, because we're going to make a major business decision based on that, and we don't want it out there.”? The basic idea behind RenderMan was distributed ray tracing, and there was a certain pattern of randomness that Rob Cook came up with that was a piece of his genius.? And I said, just keep that part quiet. ?We're going to go for a patent which basically says, here's how the general system works, but don't explain that particular black box. ?It's like chess, don't tell the other side what your moves are going to be - ride the secret for a while.

?

Perhaps there’s a relationship to the idea of Moore's Law.? Business is growing, and the space is continuing to expand and scale, and it would be hard for any one individual, company or unit to solve all the problems.

That's well put. ?We all have to attack it together because it's huge, and it continues to get bigger.? I mean, I look at ChatGPT, the image diffusion model stuff, it's just exploding.? Nobody really has a handle on how to control it, how to measure truth, and how to keep track of truth - all that kind of stuff. ?It's just a wide-open mess right now, and if we don't talk to each other, I don't know how we're going to solve it.

You know, my wife compares it to a cultural technology, like printing. ?When printing came along it suffered from the same thing.? Anybody could print anything, and boy, did they.? Horrible things were printed, and it probably caused the French Revolution, maybe the American Revolution too.? Once it was printed, it became myth and there you go. ?Now we embrace the idea that reporters check on each other, and an editor checks the reporting, and the competitors check on each other. ?It's not foolproof, but at least there's a system for establishing probability of truth. ?Science has a mechanism like that with peer review, but we just haven't got there with this latest turn of the screw.

?

Do you think there is someone working now in the AI space that will have an impact like Fourier? ?And if so, how do you spot them?

I don't. ?I think it's a lot larger than that.? It's all intuition at this point, but I think it's a very large idea that a lot of humans are going be struggling to get hold of.? I'm following some of the people who are trying to just think about it; Jaron Lanier is one of my favorites.? He and I grew up in the same town in New Mexico.? He was “Mr. Virtual Reality” when that was just becoming a thing decades ago.

I'm trying to find these people who are thinking philosophically about what it all means and where it might go, as well as what it is and what it probably is not.? I'm disturbed by some of the fear, the ‘it’s going to take my job away.’? Well, it will take jobs away, I'm sure of that, but I think what it allows is going to be awesome. ?It will be an order of magnitude improvement of some sort.?

My artist friends, whom I thought would be repelled by it, are in love.? They're saying, no, no, Alvy, you don't get it, art's never been about the process of making the drawing or the painting, it's your editorial instincts as an artist to say, ‘you should spend time as a human being looking at this, rather than that.’ ?In other words, their job is really acting as an editor. ?Stable Diffusion may spew out 100 pictures, but most of them will be crap.? An artist comes along and says, oh, yeah, but look at that one over there. ?Let's have the program work on that.

One of the artists I worked with early in my career is David Em. ?He came and worked on the paint program we were using at Xerox PARC back in the early seventies, and he generated some of the very first interesting digital art. ?And he's one of the guys I'm telling you about right now. ?He’s just thrilled out of his mind - he's been watching this develop for decades. ?He thinks it's a good way to sort through the trite stuff, and I think that’s what the job of the artist is now, not just the actual drawing of an individual picture.

?

Your life has been filled with these big entrepreneurial challenges, how have you managed to maintain a balanced personal life through all of it?

Well, I lost one marriage from it. ?There's no question about it, it can be hard. ?It basically failed because I was completely focused on making the first movie.? I knew we were going to make it happen.? We had all the talent, but it took hard work and dedication. ?I made the mistake of telling my wife, “for the next six months, I'm not going to be here. ?My body will be in the house, but I won't be here.” ?Well, it turned out to be two or maybe three years.??

I married again and I’ve been with a brilliant woman, Alison Gopnik, for the last 17 years. ?She and I have a wonderful time, she's a professor at UC Berkeley.? Her specialty is children's brains, the development of cognition between zero and five years old.? The AI guys are all over her because they think maybe two-year-olds can tell us how to make these AI things smarter.? Her answer is no - you're nowhere close to a two-year-old yet!

?

Having two people that are really involved in their career, that's a hard thing to make work.

Well, again, I think I didn't make it work. ?And the reason it works so well with Alison is because we had both raised our families by the time we met each other.? She and I were both at Morehouse College in Atlanta at a TED-like conference and she got up and talked, and I was blown away.? Alison used to say, you're always talking about jumping over cliffs, I don't see the evidence. ?I said, “what do you think I'm doing right now? I'm jumping over a major cliff because I'm pretty sure it's going to work. ?My intuition is screaming yes.”?

?

Do you think part of the make-up of successful people is knowing when to trust their gut instinct and commit?

My uncle George was an artist who taught me how to oil paint when I was a kid.? He also taught me how to sail. He took me out on the harbor in San Diego and let go of the tiller. ?He just said, “okay, take over.”? And I'm sitting there, you know, computing - okay, there's the lift, the wind's going over the sail... ?He said, “would you stop that? ?Just grab the tiller and you'll figure out what to do.” ??And I just committed - it comes to you really fast.? You don't have to have a formula or a model or any of that.

But people ask me all the time, Alvy, what should I do to be successful in my career? ?I hate that question because I don't know how to answer, all I can tell you is what I did.? The first thing I did was learn what I'm really good at. ?That's hard to do - not finding what you want to be good at, but what you’re really good at.? I thought I was going to be a great animator but once I met one, I realized I was nowhere close to being a great animator.

The second thing is to become the best at what you're good at.

The third thing is to be willing to go where the action is. ?And that's the hard one, because I didn't marry until I was 40, so I was able to change sides of the country three or four times just because that's where the action was.? Sometimes that’s not an option.

?

In terms of your current advisory work, I know you're working with Baobab Studios, are you part of the audience for immersive entertainment??

No.? I blame it on old age. ?I'm 80 this year, and, boy, it's different than I thought it would be. ?One thing that goes away is balance.? If my eyes are closed, which is essentially what happens when you put on the glasses, I lose my balance.? But I enjoy it when my grandkids come over. ?They head for the goggles, they set up their boundaries and they’re off - they're in hog heaven.

I delve into entrepreneurial projects just to keep my fingers in the pie.? You know, it's just that ‘all your guts out on the table’ spirit of entrepreneurs.? I've been there several times.? I think it's the most creative you can be because you've got to have all your chops together, business, finance, personnel, everything - and be willing to put it all the line and say, “I'm going to do this!”

?

What are your story-consumption habits now? ?Do you read, do you watch movies? Podcasts??

Yes, to all. ?I’m still a sucker for old fashioned narrative. ?It can be a series or a podcast, or a movie, but it's not VR and it's not games. ?I'm as seduced by a good narrative as anybody.? I'm also seduced by acting.? Animation and acting are the same thing.? The art of animation, the skill that great animators have is exactly the same skill that great actors have.? A good actor convinces you that his or her body is somebody completely different and a good animator convinces you that a stack of polygons is conscious and can make you feel happy or sad.? It's just a different medium, a body versus polygons.

?

You’re headed back to New Zealand shortly, what are some of your favorite things about coming to Aotearoa??

Oh, God. What's not to like?? I've been there several times.

I love the Kauri trees up north, Queenstown/Blanket Bay, Milford Sound.? I like Bluff Oysters, they're the best in the world.?

I have relatives there, too, who bought land long ago. ?They were these two kids that came to Auckland in 1842 with the minister who started the church there.

And the people, they're too nice. ?It's just a wonderful place.

Alvy Ray Smith