Academy Software Foundation - Technical Advisory Council (TAC) Meeting - March 6, 2024

Join the meeting at https://zoom-lfx.platform.linuxfoundation.org/meeting/97880950229?password=81d2940e-c055-43b9-9b5a-6cd7d7090feb

Voting Representative Attendees

Premier Member Representatives

Project Representatives

Carol Payne - OpenColorIO Representative
Cary Phillips - OpenEXR Representative
Chris Kulla - Open Shading Language (OSL) Representative
Jonathan Stone - MaterialX Representative
Ken Museth - OpenVDB Representative

Industry Representatives

Jean-Francois Panisset - Visual Effects Society

Non-Voting Attendees

Non-Voting Project and Working Group Representatives

LF Staff

David Morin - Academy Software Foundation
Emily Olin - Academy Software Foundation
John Mertic - The Linux Foundation
Yarille Ortiz - The Linux Foundation

Other Attendees

Mitch Prater, LAIKA
Ben Giles, Calligra
James Uren, Mo-Sys
Ben Schofield, CDSA
Doug Walker, Autodesk/OCIO
Daniel Heckenberg, Netflix
James Helman, MovieLabs
Jim Geduldick
Lee Kerley, Sony Imageworks
Spencer Stephens, MovieLabs
JT Nelson, Pasadena Open Source consortium / SoCal Blender group
Daniel Flehner, Storm Studios

Antitrust Policy Notice

Linux Foundation meetings involve participation by industry competitors, and it is the intention of the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws.

Examples of types of actions that are prohibited at Linux Foundation meetings and in connection with Linux Foundation activities are described in the Linux Foundation Antitrust Policy available at linuxfoundation.org/antitrust-policy. If you have questions about these matters, please contact your company counsel, or if you are a member of the Linux Foundation, feel free to contact Andrew Updegrove of the firm of Gesmer Updegrove LLP, which provides legal counsel to the Linux Foundation.

Agenda

General Updates
- CLOTributor for ASWF projects #589
- FMX 2024 Open Source Track #595
- New Working Group Proposal - Swift #578
- Project Proposal - OpenQMC #434
- nanoColor Working Group Charter Proposal #612
- Security Threat model analysis for ASWF projects #615
- Starting a Zero Trust Security Working Group #620
OpenLensIO #603
Close out adjustments to OpenSSF Best Practices badge in lifecycle stage requirements. #502

Meeting Notes

CLOTributor
- Rez has had a couple of “drive by” contributions
- 74 ASWF issues tagged on CLOtributor
- Anything with help-wanted tag without someone assigned to it, can add additional tags
- Already got value out of it
FMX 2024 Open Source Track
- Emily: starting to fill out speaker list, if you are planning to go to FMX and want to present, let me know. Doesn’t have to be related to a ASWF project, but has to be open source
- David: if a TAC member isn’t going to FMX but someone in your company who has been working on a project / familiar with the work and could speak, please let us know. We want to gather a group of open source people who are active in the field for a panel, not necessarily having to prepare a presentation
Security Threat Model
- Trying to get some security support for our project
- Working with the Open Source Security Improvement Fund
- Can help with making “threat models” for a project, not as in depth as a full security audit, but can help figure out what to concentrate on.
- Example for the Eclipse Mosquito project
- If you are a TSC lead, please take a look at this, could something like this be useful to your project? Could it help with your security posture? Would also help with security audits. But please take a look at what got done.
- JF: is there a ballpark as to how much this would cost? John: doing a security audit with another project, a deeper audit, around $50K. A threat model analysis should be less than that. Also opportunities to group several projects in a single engagement. Don’t worry too much about the cost, will try to be as efficient as possible. Would an analysis like this be useful? Many of the TSC Chairs are unclear of how to approach security, could be a tool to give a pragmatic place to start. Cary: would love to read a project like this on a project I’m more familiar with, would be very valuable. John: for other foundation, did an audit on a single project, which then allowed the other projects to better understand the result. So we could pick one ASWF project to start, would take a month or two.
- Cary: do they just “do it themselves”, or is the project highly involved? John: for an audit, there is interview / discovery and tooling, so they don’t just go off and do in a vacuum, they want to understand the context. JC: when doing a threat model in a company, there will be someone who leads and has to be very familiar with the project, otherwise the threat model won’t be that valuable.
- Alex: how would the report come back to the project? Would it be a public document with a lot of security issues in the open that might not be fixed? John: if you look at Mosquito project document, there will be a list of vulnerabilities the project needs to deal with. Starting with that in our projects may not be the right approach. In a threat model, they will produce a report that can be used in a public forum, helping to understand the threat model the project might face. So help you understand how security relates to the specific project. You get a series of actionable points to take away. Alex: improving security is important, and understanding how it works for our projects. For companies, it’s a double edged sword, when these projects become public, and companies need to act on these reports before the report is public. Cannot leave known issues unfixed. John: there would be responsible disclosure.
- JC: (in chat thread) a good explanation of what a threat model is
- Scott: I assume the way to put out the report is probably do up CVEs as well. Also, not sure if the Linux Foundation does this, but apparently there’s something called CNAs that can manage creating and updating CVEs (I could be completely wrong about this
- JC: I wouldn’t necessarily expect CVEs to come out of threat model exercise. It can (if you happen to find some vulnerabilities, but you don’t need to create CVEs if you just fix the issues
- Scott: From what I understand, groups such as Debian depend on CVEs to exist to prioritize updating a package or not.
- Spencer: CVEs are typically generated by software vendors when they discover vulnerabilities in released products, security companies/experts, and - probably less so - end users.
- Scott: Sounds good. I don’t know how the ASWF projects handles generating CVEs.
- Larry: On the contrary, what we have seen on some of our projects is that there are a ton of people we’ve never met before who seem to be in the business/hobby of fuzzing packages and approaching them for CVEs to be filed, in order to… I dunno, claim credit for finding them? Rarely have they seemed like legit security issues and could generally have been handled with less panic by just filing an issue.
- JC: CVEs are often (sadly) used as a tool, not in the positive way. It’s often used as a tool against project maintainers.
- Scott: Yeah, I’ve been following the creator of cURL on Mastodon and they’ve been banging their head against people generating CVEs for non-issues. Including some reports that are generated by LLMs.
- Larry: Many projects (both gcc and llvm/clang, for example) have recently produced documents that detail all the things they will no longer consider to be security issues because they’re overwhelmed by this stuff. It’s such a black mark to have an unfixed CVE, you feel like you have to fix it right away and that can overwhelm the dev capacities of many projects.
- Spencer: Welcome to the world of bug bounties. A very good way to reward people for researching and finding security issues but it’s generated a whole industry of people that take it well beyond anything useful. HackerOne
nanoColor WG proposal (Carol)
- project proposal
- working with Doug Walked, and Material X leadership
- FYI at the TAC level, call for involvement if this interests you
- overlap with the reasons we are starting the Color Interop forum
- OCIO, OpenUSD and MaterialX working on this document, but we want collaboration with other projects, but trying to solve specific issues in the context of rendering, materials and textures, want to keep it focussed on a specific context, also want a working prototype at SIGGRAPH.
- Want to avoid fragmentation as projects need to solve these problems.
- Looking for input on the work more than anything else.
- Eric E: what does this do that OCIO doesn’t, or vice versa? What would the core functions be? Carol: looking at small / light version of OCIO, only core concepts. Minimized dependencies, so you can run it on a piece of hardware. No support for external files like 3D LUT files for instance. Limited scope of color operations, only analytic instead of table driven. This charter is high level since we haven’t decided yet as to what the technical details would look like. But want first to make sure people are OK with this plan.
- Larry: imagine where there is a zero dependency, single header implementation that would give you interop with the 12 most important color spaces supported by OCIO, that’s what I’m hoping for. Eric E: need to read the document.
- Gary: OpenFX is looking for a common set of baseline color spaces right now, we’d love this.
- Carol: we’ll create a WG proposal right now if that’s required, want to start right away, don’t want to put more overhead. Larry: not sure why would need to make it an official WG, we’re already collaborating with OCIO, and you already have the structure to support this. Carol: we use the name Working Groups inside of OCIO, this would be an OCIO WG with cross-project collaboration. John: that sounds fine.
Zero Trust Working Group Proposal (Jim Helman)
- At MovieLabs we have 2030 vision, with security as we move to the cloud
- The “perimeter” is now in the cloud, Spencer developed a security architecture based on Zero Trust. Zero Trust controls being added to TPN
- At last year’s SIGGRAPH we made a proof of concept, but had issues with how to pass security tokens around.
- Want to document best practices for ASWF projects so they can have the required hooks to integrate in an environment where authentication / security checks are required.
- Have been socializing this, Google and AWS are very interested. Autodesk, Foundry and Adobe also interested. We have a critical mass of interest.
- Looking for feedback on the proposal
  - Focussed on Zero Trust, not looking at CVE, CIs, dependencies… Wanted narrowly scoped / focussed on Zero Trust
  - As opposed to talking about a Security Working Group
  - Also looking for involvement
- Have some logistical questions. MovieLabs can help organizing. How is leadership of a WG established and maintained?
- When objectives of a WG are established, does it shutdown? John: can work with you to answer logistical questions. If anyone has interest, GitHub issue is listed in agenda. Also if you have interest in participating.
- Larry: can you append to the issue / proposal if you have identified which projects and people within these projects you identified could be good to participate? And make sure you have buy-in from those people?
- Jim: one item would be to identify which projects would most likely need to operate in a Zero Trust environment. Purely local library projects may not apply for instance. Will go through the list of projects and reach out.
OpenQMC: no update?
- Eric E: correct
Vote on OpenQMC project
- Has it stalled at the vote? Should we discuss this further in an upcoming TAC meeting?
- Carol: was a bit confused by the email, but yes, seems we need to have more time to discuss, we didn’t have time to discuss it after the presentation.
- John: TAC meetings are trying to get filled up, but will try to find some time.
OpenLensIO Proposal (James)
- Having computer issues, can you bring up slides?
- Presentation Link
- This is my first engagement with ASWF or any open source projects. I’m at Mo-Sys, we make camera tracking software for virtual production. Have a software engineering background. Involved with SMPTE RIS group, see a lot of positive work, and goodwill between competitors in the VP environment to try to standardize and improve the process for VP.
- Virtual Production Context
  - VP is an umbrella term for real-time VFX
  - VP requires live accurate camera tracking and les modelling
  - Lens modelling is more than just Focus, Iris and Zoom
  - Camera and lens data can be recorded for traditional VFX (especially as the quality of the data can improve): want to deliver data that can be used in VFX
- Current State of the Art
  - Render engines and VFX tools must correctly interpret many different lens models
  - Lens calibration is painful!
  - Manufacturers like Mo-Sys have built proprietary systems
  - In the absence of a standard, lens models are diverging
  - Models are complicated: multiple lens elements, anamorphic lenses, camera mounts / shims
- Why OpenLensIO?
  - Much is already published / open but with different models
  - Standardizing to improve interoperability
  - The mathematical model itself is not commercially sensitive
  - Encourage manufactures to publish lens files to make VP easier and more accessible and publish their distortion models. Every lens and camera needs to be calibrated on set to generate data that’s usable in AR or in post. It can be done as part of on-set prep for high end productions, but can become a real pain for productions. In cinematic environment could have 18 lenses and 3 cameras, need to calibrate every combination, could take a whole day. If more of that work can happen at the manufacturer where they know the characteristics of the lenses based on a standard, would encourage manufacturers to publish in a known format.
  - Seems a good place where an open source community can help, we need to bring back things to a standard. We can agree on the math, and then an OpenLensIO community can state that this is the best possible mathematical model we can agree on, the reference implementation can derive from that.
- Goals
  - Generic, practical, end-to-end mathematical model
    - Nail down the math, at first for a spherical lens
    - End-to-end model, not just a computer vision models, but incorporate known requirements for media production (overscan for instance)
  - Training resources and documentation
    - A big knowledge gap, most people who don’t work with VP day to day don’t understand the importance of lens modeling
  - Reference implementations
    - Could include shader code, for instance an ST map
    - Reference implementations for transport of the data: data usually sent from tracking system to render engine, but how that’s done is undefined. Make it easier for producers and consumers of the data to use it.
  - Framework for an open lens library
    - As lookup tables. A lens file describes describes distortion characteristics of the lens for every zoom and (?) value
    - Could be open sourced, would be game changing for Virtual Production
    - If lenses are supported by main software vendors and packages, and supported by tracking providing, we then create a resource like OCIO, encouraging manufacturers to contribute to it
  - Is it a WG on its own? Or part of OpenUSD?
- Feedback? Next Steps?
  - Carol: thank you for presenting, we recognize this is necessary work, needs to happen. Trying to figure out how to get everybody to agree. Question is whether ASWF is the right place to help achieve these goals, what we can bring to this effort. So this needs to happen, but why specifically the ASWF? We have some examples that are example libraries in DPEL, but most of our projects are software. Are you imagining a software library?
  - James: not just a library of lens definitions, it’s also about the software that makes it usable. Happy to be redirected somewhere else if you don’t think it fits here. Our lens library is open to our customers, I want to do this work anyway and will continue to pursue it. Is the foundation the right place to do this? How much is the software critical? I think the core is not the raw data, it’s about the software that makes the raw data usable. Transport could be one reference implementation, shaders could be another. Could also be a node in VFX compositing software, could be a plugin to Unreal. There are lots of software use cases.
  - Carol: how is the software helping become something that can be used by any application. Different than what other groups are trying to do.
  - Michael: USD is not an ASWF projects, but there are WGs, in particular the USD-Camera WG, have been looking at both virtual and practical cameras. We have people from ARRI, Cooke, VES, ASC… There have been some deliverables, so you should present to this group, there are a bunch of people not on TAC call. Also OTIO are involved in USD Camera, data that could be coming off set can be interesting in both VFX and editorial. This seems like a very important effort. James: yes, will check those out.
  - Nick: on behalf of USD Camera group, we would love to have you present, lenses have come up recently for discussion, great opportunity for collaboration. #wg-usd-camera
  - JT: USD Camera WG Wiki. We have a channel on the ASWF slack server also, #wg-usd-camera
  - Eric E: do you look at characterizing other aspects of the camera, colorimetry, depth of field? James: those are outside of the scope of this specific effort, but in scope of SMPTE RIS Camera Tracking group. Hopefully some of the work emerging from that will start forming standards and guidelines. But this is a specific pain point about lenses that’s separate from the cameras.
  - John: this is proposed as a WG, does the TAC want to move forward? Michael: makes sense to first talk to the folks who are already working on this? James: yes, this makes sense, we will figure out where this can fit, and come back to TAC if it is worth being its own entity / separate enough.
  - Jim Geldudick: working on whether any of the work we do in RIS could flow into ASWF groups / projects, we have similar groups between TAC and WGs, we have a camera test coming up considering outside of lens mapping. Hoping to share more going forward. Why we starting RIS was to create standards, have been working towards this in the last 3 years. Similar to what was presented for nanoOCIO earlier.
  - Larry: neat project, this is important work. Not sure where it should live, there are overlapping efforts happening in different places, good to shop around, not just where it should live, but also how to deal with overlapping efforts. Even if it ends up incubating somewhere else, once you have a software infrastructure, that could be hosted in ASWF. You need to figure out what’s the best place for this to have its infancy, you may have more shared interested in groups that are already looking at some of these issues. But it is important work and needs to work somewhere.
  - Nick: highly complementary to USD Camera work, would love to further discuss this with you.
  - John: will keep this on the agenda and reach back
  - James: you can reach me via email at james@mo-sys.com
  - Jim Geduldick (chat): 3 camera shoot with up to 12 lenses per body takes us quite a while to calibrate across spherical and anamorphic. That could just be for 1st unit. And getting X number of samples per sense with (OpenCV) model is very time consuming.
  - Scott (chat): For some reason, this sounds like a project and not a WG. Especially if the goal is creating some API to handle the camera data.
  - Nick (chat): We treat a somewhat end-to-end camera model focussing on various aspects such as lens metadata, characterizing various components of a camera, and mappings to interactive cameras.
  - Jim Helman (chat): Links to some of the RIS OSVP camera stuff: RIS OSVP Metadata RIS-OSVP Metadata Toolkit
  - JT (chat): In high end live event media production(my main industry) we also use this at the camera/lens end, calibration and projection management software middleware to the projector lens output end, which is closely related to in, it’s just the other direction. This tech is very narrow and specific in live event high tech, and very hardware specific.
  - Jim Geduldick (chat): Agreed it’s the same tools across VP for film and tv that flows into live XR and AR / Simulcam.
  - Carol (chat): I wonder where the ASC sits on this too
  - Michael (chat): Some folks from ASC show up at the USD Camera WG
  - Jim Geduldick (chat): You can also reach out to me Jim@spaceboylabs.com for more questions on SMPTE RIS