Imaging Is Everything
John Knoll picks good hobbies. His interest in building miniatures got him his first Hollywood gig as a model maker. Patching together a rudimentary motion-control camera in his basement landed him a job as a camera operator. And tinkering with image processing led to two great advances -- the knockout special-effects work he's done for George Lucas's Industrial Light & Magic and the development, with his brother, Thomas, of the image-editing application Photoshop. "I have this history of turning hobbies into professions," Knoll says.
Knoll recently spoke with Macworld about his early dabbling with special effects and what role that played in Photoshop's development.
How did you start working with special effects?
It was a hobby when I was a kid. I had a great interest in movies, specifically in technical aspects. When I saw something on screen that obviously could not have been shot as you saw it -- it had to have been faked somehow -- I was fascinated by how they did that.
When did you figure out that was what you wanted to do for a living?
It was actually seeing Star Wars that made me decide. Up until that point, special effects were something I thought was very interesting, but I didn't think of it seriously as a potential career. Then when Star Wars came out, there was so much happening in the field, and it seemed like a viable career path. I decided in late high school that this is where I wanted to go. My choice of colleges was based on wanting to be in the film business.
What was your first experiment with special effects?
I did a lot of the same things a lot of the kids who are now in the industry did. I did little claymation films in the basement of my house. I built miniatures. I had a dark room down in the basement where we did still photography.
When did you first use a Mac?
In 1984. I was coming close to my final year at USC and my dad had gotten one of the original 128K Macs. I was at home visiting, and I played around with it for a little while. I thought it was the coolest computer ever.
When I saw the Mac, it was very obvious to me that this is the future, this is how all computers are going to be. It makes so much sense. It was logically laid out, whether it picked from menus, whether it had memorized commands that you typed. It seemed so clear to me that this is where it's all going to go.
How did you start working with image processing?
I have this long history of turning hobbies into professions. When I was a kid, I built miniatures, and that was actually the first thing I did professionally in the film industry. It was a demonstrable skill that I had, so I worked as a model maker. But that kind of killed it as a hobby. I was often around motion control cameras that we used to photograph miniatures with, and so I got interested in that. I tinkered together a home-brew motion-control system while I was at school. It was my advance animation project. I had an old Apple II computer and a cereal milling machine controller, and I started doing work with motion control. So I eventually moved my career over to that, working as a motion-control camera assistant and operator. That's how I got hired at ILM in early 1986. So that kind of killed motion control as a hobby. I've always lived by the principle of find what you really enjoy doing and make it your career. So now that I was doing motion control, I needed a new hobby.
What came next?
I'd just gotten a Mac 512K, so I started writing little computer graphics programs. My older brother, Thomas, was still in college in graduate school, at the University of Michigan, working on a doctoral thesis in vision systems. The first part of any kind of vision algorithm starts with image processing. He was doing a lot of work in image processing. ILM was the first company that I had worked at that had a computer-graphics division. I had worked at a bunch of places that had special effects equipment, but none of them has a computer-graphics division. When I visited the computer-graphics department just to see what they had there, I saw a demo of the Pixar image computer. The LucasFilm computer division built it. It had a smart image buffer that could load up an image and, by issuing commands from a terminal next to it, you could do various operations on the image. You could blur it, resize it, and rotate it. I thought it was super cool.
It had a laser scanner and film recorder where we could take a piece of film and scan it in. Then that image could be manipulated on the Pixar computer and then put back out on the film with a laser film recorder. We do more or less the same thing today with much better and faster equipment.
When I saw this demo, I thought the implications were profound. Once the image was a bunch of numbers, there wasn't really any limit as to what you could do. When it's in the digital realm, anything about the image is up for grabs. You could do whatever you like to it.
How did the idea for Photoshop come up?
I was tinkering around with this new hobby of writing computer graphics software. I had been exposed to color-image processing over in the computer-graphics department (at Industrial Light & Magic), and my older brother, Tom, was working on his doctoral thesis, which included image processing. So I was back in Ann Arbor, [Michigan] visiting Tom, and he showed me his thesis. I remarked how similar it was to some of the stuff I'd seen on this Pixar image computer. At the same time, when I was writing my software, I was interested mostly in the code for how bright a pixel should be and not how to display it on the screen. Some of Tom's tools could take a raw image back on disk and display it and save it in different formats. So Tom gave me some of the tools he'd been writing, so I could do manipulations to the images.
What's the main advantage of using Photoshop in moviemaking?
It was the first commercially available popular product that could do that sort of thing. It came about right around the same time the whole film industry was beginning to move rapidly toward digital compositing and computer graphics. It was a tool that filled a niche that was very important to the industry. A lot of image compositing -- taking a lot of images from different sources and putting them together into one image -- was actually done in Photoshop for a couple of different shows, such as The Rocketeer and Memoirs of an Invisible Man, where we had Photoshop running macros to composite all the frames of the shots.
What other role does Photoshop play?
Photoshop is used for concept art. Concept art is fully developed color artwork (as opposed to storyboarding, which is black-and-white drawings). We actually start with a sketch on art paper and that gets scanned in and they start coloring over the top of it.
How do you use your Mac for day-to-day work?
I use it for planning out shots before they're executed. Sometimes for 3-D previsualization. Sometimes just for spreadsheet programs to do some number crunching to figure out what the right frame rate is for a given model. I do a lot of writing and e-mail, too.
How has digital manipulation changed moviemaking?
It used to be in the not-so-good-old-days that you were very limited in the kinds of effects that were technically possible with the limits of the equipment. A lot of times a huge amount of effort went into a shot, making sure there wasn't a black line around objects. Now you can kind of take those sorts of issues for granted. You don't have to worry about a black line or that the colors won't match when two elements are combined. Most of the effort that goes into a shot now goes into whether it looks good or not, not the mechanical issues of putting the pieces together. A lot more energy goes into the aesthetics, and as a result, you end up with better-looking shots.
Where is the future of image processing going?
I want to see the tools get better, easier to use, so that it's more about making the image than it is about fighting the technology, waiting for the thing to render. But I'd also say there's plenty of room for improvement in the quality of things we're looking at. Also, in terms of what we can do. There are areas that are still extremely difficult, and computer graphics will move into those areas. There are people who are flirting with computer-generated human beings for the purpose of stunts -- stunts that would be very difficult to shoot with performers.
I think there are some opportunities to use computer graphics to realize some things that aren't done right now. It'll become more and more practical to do big epic pictures that are just too expensive to do right now. If you need to do a movie where you have an army of 10,000 soldiers, that's a very difficult thing to shoot for real. It's very expensive, but as computer graphics techniques make that cheaper, it'll be more possible to make pictures on an epic scale, which we haven't really seen since the '50s and '60s.
What project are you most proud of?
I was fairly pleased with how the work on Star Wars: Episode I turned out. Almost every show that we do, there's at least one or two aspects that we haven't done before, that we have to push the envelope on. In Episode I, it was 20 things, and it was in a vast quantity.... It was the largest visual effects production ever attempted. It was difficult work that we didn't quite know how to do. It was a huge challenge. We had really no idea whether or not it was going to be successful, but we dived into it.