David: will speak with emily to see if we can highlight 20th anniversary!
OpenEXR Project MIssion
The goal of the OpenER project is to keep the EXR format reliable and modern and to maintain its place as the preferred image format for entertainment content creation.
Major revisions are infrequent, and new features will be carefully weighed against increased complexity. The principal priorities of the project are:
…
TSC:
Cary Phillips, ILM
Wasn’t involved early on, became involved when original authors left. There’s a lot that goes on to running a software project which involved administrative aspects, became technical project manager.
Cristina Tempelaar-Lietz, ILM
CI infrastructure
Joseph Goldstone, ARRI
Brings connection to ARRI / SMPTE / ACES
Kimbal Thurston, WETA
“Heavy lifter” in terms of software features, most significant contributions to library in recent history
Larry Gritz, Imageworks
Involved in many projects, including OIIO, one of the main consumers of OpenEXR
Nick Porcino, Pixar
Present at creation at ILM. One of the principals behind OTIO, everyone on TSC has other projects that are their principal gig.
Peter Hillman, WETA
Deep implementation for OpenEXR, primarily dealing with security related fixes from OSS-Fuzz
Rod Bogard, Epic Games
One of the original authors, great to have him connected to the project for historical perspective, as well as imaging perspective
TSC Meeting attendance over last 3.5 years
Remarkably consistent in getting people to show up. A couple of episodes where we dropped a couple of meeting, used to meet every week, now every 2 weeks.
Getting good turnout, averaged 6.5 members at each meeting, and half the meeting have had a guest.
83 total meetings, 539 hours, 42 meetings with guests
First battle in having a sustainable project is getting people to show up, which we’ve done
Larry: had interesting presentations by guests, for instance presentation by OpenColorIO discussing implications of interactions between OpenEXR and color management
Cary: we haven’t lost any members, but we haven’t added anybody.
OpenEXR / Imath releases:
1 in 2019
Bunch in 2021
More quiet lately
Improvements / new functionality
Bug / security fixes
Build fixes
Documentation improvements
New OpenEXRCore: multi-threaded C language I/O, stateless
Commits: TSC members sv others:
pre-ASWF: 97% others
2018: 12%
2019: 8%
2020: 33%
2021: 17%
2022: 34% from 15 different people
Significant contributions from people outside the TSC
A commit is not a reliable unit of work, sometimes commits can just be meeting notes, and some contributions from other people may be small build fixes / one liners
John: fantastic stats on TSC meetings attendance, commits, merges. It’s interesting to discern that perhaps there are too few non-TSC contributors - would that be a fair assessment?
Commits: TSC members:
A lot of commits from Cary are small bugs fixes / administrative
Kimball and Peter have significant contributions
PR Reviews:
Nick and Larry contribute significant number of PR reviews, important for the health of a project. Even the small stuff is important to be reviewed. You need people to show up, and you need people to “look, watch and act”
In pre ASWF days, vast majority of PRs didn’t get any reviews
CII Badge
Silver: 76%
Gold: 65%
Outstanding issues:
Test coverage: 70%, work in progress, have intention to increase
Security
Training / documentation
Signed releases
Internationalization
This could be a can of worms, could be significant undertaking
Security: Policy Statement
Unique in ASWF projects of having vulns / CVEs filed against us
Security page lists the statement of how to report security problems and list of known CVS. Links take to NIST page that lists description of vulnerability as well as information we provided. When a CVE is made public, go to this page to submit additional info, link to the source of the bug, link to the release notes that describe the fix, and place to download new version. Dealing with this is straightforward
We have a chunk of code that reads a file in its entirety, the google servers run this on a random set of bits. Reproducing issues has been pretty easy.
OpenEXR puts the entire library in a namespace based on the minor release, making it possible to link 2 versions of the library in the same process. But complicates the release process, since it puts a fine point on a minor vs patch release.
Maintain ABI compatibility between releases. Major shuffling in the works is to retrofit the existing C++ ABI to use the new multi-core infrastructure. Kimball made early progress, but then reflecting some more. 2 fold goal: every distribution has OpenEXR as part of it (Mac, Linux, Windows, BSD), need to make it so that if we release a minor version with new functionality, we don’t make all the software which uses OpenEXR non functional. So one more change that will change the ABI yet again, but will be stable going forward. C layer is already stable, could release a version 4, people who had linked against version 3 would still be fine. Recognize that we are meant to be a stable component. A shift from the VFX Reference platform in a way, will need to coordinate with them. Previously had to lock versions because of the namespace issue, every VFX platform release had different namespaces. But by default it will now be stable.
Core/Multithreading in the C++ API
Will this incorporate TBB? Issues with VFX Platform moving to oneTBB? Kimball: switched WETA to OneTBB 2 years ago, no issues, had more issues with OpenMP. OpenEXR has its own thread pool, pre-existing TBB. In 3.x can swap out with your own threadpool, and someone recently asked questions about using TBB worker pool. Seems like it would be possible. Main issue is that different threads can be reading different files or the same file handle, so need to handle re-entrancy. Task Group changes in OneTBB may have been holding up some projects
Website overhaul
Website is in need of work
Taking cues from OCIO website, using that as a basis / reusing
Will be happening soon
Roadmap: Call for Community Contribution
GPU-side enablement
Sending images directly from disk to GPU and decode there
New compression schemes
Could be contributed by someone external
PyBind11 for Imath (retire boost dependency)
Would be a major improvement
Daniel: there seems to be a strange mapping coverage for accessing EXR files from Python, no elegant way (pip install). Seems like a strange thing given how significant EXR is for machine vision applications. There’s a very old version of the python bindings, is this a category where looking for broader contribution?
Kimball: had included this in Open Source Days update. There are a number of Python wrappings around OpenEXR out there, none of which are affiliated, or well supported. Not sure what to do about it, except we need to do something about it. We think our position should be that if you want Python bindings, use OIIO. Larry: but lacks a “pip install”. Larry: the leading bindings for OpenEXR were un-maintained for a while, now they are maintained, but put a new wrapper around old API, so can’t read deep files, it’s not maintained, not coupled to the version numbers in the project. Need a decent implementation of PYthon bindings. It’s not rocket science, someone casually familiar with C++ bindings / pybind11 should be able to do this.
Cary: who’s going to do it and when is the question and where it gets bogged down. Everyone on the TSC has this as a side gig. There used to be someone at ILM who was full time staff on EXR, but that’s not the case anymore, we’re all squeezing this “on the side”. It would be great to have a contribution from someone who is going to use it as well as develop it. How do we identify a contributor like that? Larry: it’s not a FTE, there may be periods of time where more work is needed (new features in general). If a ASWF member was looking at how to use a portion of their FTE… A day a week contribution would make a big difference.
JT: Conda and Anaconda manages OpenEXR better than pip. Those are major platforms for AI and ML. I haven’t used OpenEXR in this setting but I have used the platforms.
Need to modernize these reference images, they are pretty old, early years of the format. Want to update them with representative images in terms of pixel and metadata content.
Recommended changing the default compression ratio as result of the research. Very nice to have a suggestion for a change backed up by data. An example of someone outside the standard community making a valuable contribution.
Larry: he writes amazing blog posts, does “drive bys” on various projects and always improves things. Recommendation was just to change the default level of Zip compression to compress by 3% more disk space but double the EXR writing speed, which is very helpful.
Project weaknesses / needs
Windows support
We struggle with this. Nick does lion’s share of Windows support
Hardware-we-don’t-have support
“Test fails on some new or old hardware we don’t have access to”. We use the GitHub CI infrastructure, but it would be great to have easier access to hardware.
Kimball: if we don’t have this hardware, we can make suggestions. ARM came up recently. Wave: encouraging someone on my team to join meetings, they do a lot of EXR work here. Cary: was going to come asking! Would also be very helpful to have someone from NVIDIA join meetings.
Widening the contributor community
More substantial contributions from outside contributors
Transition planning
Future of TSC, not everyone will be there forever, figuring out who to pass the stewardship to and when
David: Unreal Engine 5.1 supports EXR 3.1.5, there in a new way.
In Unreal Engine 5.1, the new Media Plate Actor enables OpenEXR support, enabling users to simply drag and drop footage from the Content Browser. In addition, users can now play back mipmapped and tiled uncompressed EXRs both in engine and with nDisplay with the appropriate SSD RAID, and now have the ability to convert EXRs to the correct format for optimum playback
Marketing Plan / Non contributor survey
Scott: security WG proposal will probably have something in the new year