OpenEXR, a project at the ASWF, is an HDR image file format for high-quality image processing and storage, originally developed by Industrial Light & Magic (ILM) in 1999 to fill a demand for higher color fidelity in VFX. Released to the public in 2003, it is one of the earliest open source software projects specific to VFX and set the industry on a new course. We caught up with some of OpenEXR’s key contributors from throughout the years to compile an oral history of the development, evolution, and ongoing impact of this groundbreaking file format.
Special thanks to Florian Kainz, Rod Bogart, Cary Phillips, Peter Hillman, and Andrew Pearce for taking the time to share their stories with the ASWF.
Part 1: The Creation of OpenEXR
In the late 1990s, ILM was working on the long-awaited, highly-anticipated follow up to George Lucas’ original ‘Star Wars’ trilogy: ‘Star Wars: Episode 1 – The Phantom Menace.’ The size and scope of the project led to the internal realization that the way the studio handled image processing could be improved. Florian Kainz (who was at ILM from 1995-2014) and Rod Bogart (at ILM from 1995-2005) began to tackle this issue after the film was released in 1999.
Florian Kainz: Working on ‘Star Wars: Episode 1’ and other shows around that time, we realized that our image file format and the image processing software we were using were insufficient in terms of the dynamic range they could represent. The difference between the darkest blacks and the brightest whites that could occur in the same image was too small for many real-world scenes. You would often get crushed shadows or blown-out highlights. Our old proprietary image file format stored 8 bits per channel of RGB, and not even in a very efficient manner. This made us look for methods to store and process images in such a way that we would not inadvertently discard super bright highlights or really dark shadows. Rod at the time was working on our in-house compositing software, which internally stored images with 16-bit integer samples but left several bits per sample unused. It occurred to us that we could use those 16 bits in a different way – instead of integers, we’d store 16-bit floating-point numbers, which could simultaneously have a wide range and fine resolution. This would significantly improve image quality without increasing the memory footprint of the compositing software.
Rod Bogart: That was kind of a big deal, because these programs were already taking up a lot of space and there were only so many GB of RAM available back then, and we would fill those up rather rapidly. To consider going to floating point, which was 32-bits, would have doubled the memory footprint, so we knew we didn’t want to do that.
Florian Kainz: There was no commonly used floating-point format that would have fit into 16 bits. After some experimentation we came up with a format that we thought would be useful. We then made sure we could rapidly convert between 16-bit and 32-bit floating point numbers so that we’d be able to do arithmetic without a huge processing speed penalty. Computers at the time did not have 16-bit floating-point support built into the hardware, and pure software implementations of operations like floating-point addition or division would have been prohibitively slow. The fastest way to add or divide 16-bit numbers was to convert them to 32 bits, let the processor’s 32-bit floating-point unit do the arithmetic, and convert the result back to 16 bits. I worked on the floating-point numbers themselves, and Rod integrated them into our compositing software. That turned out to work really well. Of course we also needed a file format that could store our new floating-point pictures without losing image quality, and that was the beginning of OpenEXR.
Rod Bogart: What we defined OpenEXR to represent is the light values in a scene. We would say that images are for two things: they’re for looking at and they’re for making more images. And so image file formats like JPG and PNG are good for looking at things, you look at the picture and then you’re finished. But to make more images, you need a lot of quality in the data, and there wasn’t really a good choice back then – OpenEXR filled that need. There was another thing in the air at that time which was image based lighting, and image based lighting started to require these types of files that were high dynamic range.
Florian Kainz: Aside from storing floating point values, while we were at it we were also thinking about what else do we want? Our old file format could store color pixels – a red and a green and a blue value per pixel – and we had separate file formats for all sorts of auxiliary data, for example the distance (how far is this pixel from our camera), or motion vectors (how quickly does this pixel move across the image in a moving image sequence). We thought about how to store arbitrary auxiliary pixel data in a file. One thing that I feel was quite unique about our new file format, at least at the time, was that instead of defining the format by verbally describing sequence of bits on disk, we defined an application programming interface (API), a set of C++ functions that you could call to produce a file or read an existing file. We developed the API thinking about how you would read and write a file in a piece of application software. By defining an API, I think we made it a lot easier for existing or new software to actually adopt this file format. The API borrowed from an important design principle from OpenGL: you shouldn’t have to know about features you don’t use. If all you want is red-green-blue pixels, that should be really easy, you shouldn’t have to know about the possibility of having all kinds of other data in there. I think we succeeded in that.
ILM debuted the finished product internally in 1999, but it required a leap of faith to actually get it into production.
Florian Kainz: It’s interesting about actually getting this into production. There was almost a superstitious fear that somehow floating point images would become very large on disk. We developed image compression to keep the files small, but somehow we couldn’t get it through to people that these files were not much larger than the much worse quality 8-bit files that we had previously used. So at one point we did a presentation in ILM’s theater for the digital artists who were going to use this file format, where we went down into all the details of how the image compression worked, going through an example showing “here are the pixels, here’s how it’s converted to a sequence of bits that ends up on disk,” and somehow that did help. Actually, several artists told us afterwards that they found the presentation really interesting. I thought for sure everyone had fallen asleep, but that wasn’t the case.
Rod Bogart: There was an aspect of people knowing that this was something they wanted, that the other format left something to be desired – they were certainly aware of quantization, and kind of crummy looking dark ends, and no ability to mess with the file after the fact. I think once people got their hands on OpenEXR, they didn’t ever want to go back – but it was a matter of using it that first time. The show that really took the biggest leap was ‘Harry Potter and the Sorcerer’s Stone’ in 2001. Roger Guyett was our VFX Supervisor and Denise Ream was our VFX Producer. They allowed us to do this, they were the biggest show that was really willing to take the risk, because it was a risk. Who knew if it was going to solve their problems or if it would just be a mess – not every tool supported it yet. But it worked out great. You have to talk somebody into it, otherwise the software doesn’t get used. So I’ve always felt it was important to acknowledge them in the history of this. That was the show that probably had the biggest commitment. It was kind of groundbreaking for us because we really did move the needle there.
After adopting OpenEXR internally at ILM, the developers began to talk seriously about releasing it as an open source project – something very unusual at the time. ILM released it to the public as an open source library in 2003.
Florian Kainz: Once we had support for OpenEXR inside the software that we had control over, like ILM’s compositing system and our custom build of Renderman, of course we wanted support in third party software. That’s where the idea of open sourcing it came from. What if Photoshop can deal with this out of the box, or when the next version of Renderman is released we don’t have to mess with it to implement OpenEXR support, or other applications come with OpenEXR already built in?
Rod Bogart: We were still wrestling with the idea of what it even meant to do open source. I think it’s important to highlight that back then, open source was a weird thing to do.
Florian Kainz: Open source was popular for certain categories of software – Linux existed and there were some open source programs like the GIMP, which is a free Photoshop lookalike. I think the view then was mostly that open source application programs or libraries were built by enthusiasts for other people who don’t want to pay money for the software. The idea that you could use open source also to pursue goals for your business – say achieve data exchange standards or other forms of interchange or avoid duplicating development work – I think that wasn’t quite as popular then.
Rod Bogart: I would agree, I think that was a rare thing in the early days. It’s not that open source didn’t exist, but it didn’t seem like an enterprise activity.
Florian Kainz: As far as open sourcing OpenEXR, I think we were in a pretty good position to make that happen because ILM has clout. When we suggested that OpenEXR be released as open source, if I remember correctly, that decision went all the way up to George Lucas whether the company could do this, release some of its secrets into the world. George was for it. When OpenEXR came out, ILM’s PR department did a great job publicizing the release, and it was reported a lot in the press – I remember seeing articles in Japanese, in Czech, in Thai, it was all over the world.
Rod Bogart: It was amazing how fast people picked it up and used it. That speaks to the desire for people to have this and that the library worked really well, and that’s credit to Florian and the design of the API itself. It was good and people wanted to use it. They didn’t want to rewrite it and make their own version of it, and there wasn’t a need to do that. I think that was a big part of the success of OpenEXR, that it was not a sequence of statements about how a file should be laid out and now go write the code that does that – it was code that did that, that worked well, and would actually be something that you could add to your own program really simply. I think also, given the landscape at the time, things were moving toward floating point anyway, not just at ILM. And so when we finally got the file format out there, it filled the need for a lot of people all at the same time – there were a lot of folks that knew that was what they wanted.
Florian Kainz: One thing that surprised me was that very soon after we put this out, Apple invited us to do a presentation on what this thing was and they very soon built support for that file format into MacOS. With Preview or Quick Look, if you have an OpenEXR file you can just click on it, and the picture is displayed on your screen.
Rod Bogart: Another important presentation from that time was the one we gave to the ASC at the ASC Clubhouse. We talked to cinematographers about what it would mean for that level of data to be stored compared to what previously existed. It was easy for us to show that this thing had a great many more features at not a lot more cost. There is more data in those files, there is more information. That really took off within the ASC and ended up turning into what eventually became the ACES (Academy Color Encoding System) project.
