ASWF TAC Meeting - November 3, 2021

Video Conference Link

Voting member attendance

Other Attendees

John Mertic, Linux Foundation
Andrew Grimberg, LF Release Engineering
Shannon Deoul, RazPR
Carol Payne, Netflix / D&I WG
JT Nelson, Pasadena Open Source consortium / SoCal Blender group
Scott Wilson, Rust WG
Mitch Prater, Laika
Matthew Low, DreamWorks Animation
Ashley Whetter, Python 3 WG
Deke Kincaid, Digital Domain
Morgan Prygrocki

Apologies

Agenda

Celebrate VFX Platform ‘22
- OpenVDB 9.0.0 released over the weekend, all software release commitments for 2022 have been met
- Congratulations!
- Ken: a bit of a marathon but it worked. One issue was getting CI to pass, Nick dived in, and looks like we only have 2 threads. Is there any way this could be bumped up to 4? Current CI takes 2-3 hours, can be an issue when lots of people are submitting PRs. Had to go in and manually kill jobs. Kimball: that’s the option that GitHub gives you. Andrew: may or may not be possible, but breaking down the job to be more parallel. Larry: no, need more threads in a single matrix entry. GitHub runners limit you to 2 cores. Andrew: we are limited on the size of the instances. GitHub has said that selectable instance sizes should come up before the end of the year (would need to be paid for -> cost passed through via LF). Larry: you don’t need every PR to run every entry in test matrix, so pick a smaller number of representative matrix entries to run on every PR, and then run the whole matrix nightly to get full coverage, while still getting faster turnaround for PRs during the day. Ken: looking into this approach, also looking at caching not to have to build everything. Larry: yes, caching helps a lot. Ken: the firm deadline meant everyone was submitting at the same time, and CI builds weren’t finishing. Larry: used to use Travis and Appveyor, those had the feature to be able to kill individual matrix entries, so could let a specific entry finish by killing others, whereas for now GHA only allows killing an entire run.
- Would it be a good idea to invite Nick and Francois again to talk about a strategy for 2023? Ken: would be a great idea, one request is that the VFX Platform highlights specific versions of packages, for OpenVDB can define the API, so for instance can build OpenVDB 9 with API 8, so would be interesting if the platform could also specify the API version. Kimball: other issue is how to deal with sub dependencies such as PNG, Zlib… Would be good to find a way to define the ABIs of these sub dependencies. Scott: also would be good if it said something about build environment: what’s the namespace for dependencies, trying to build for all the apps (Maya, Nuke, Houdini) can be problematic. Kimball: we should schedule a time to meet with VFX Platform again some time in the new year.
- Overall, congratulations, much better state than last year.
TAC chair
- Kimball will be joining Governing Board of ASWF, but can’t vote twice as TAC Chair and as Governing Board member for WETA. Have to refresh TAC chair in December, will still be TAC member, but can’t be the chair. Larry: if we keep you as TAC chair, could you just have 1 vote on the board instead of two? Wave: Governing Board votes typically not close. Kimball: yes, but this is also decorum. Would like to give someone else the opportunity to be TAC chair, if I remain the TAC Chair, should nominate an alternate who can sit in on Governing Board meetings. Don’t have to settle this today, if anyone is interested, please let me know.
- We should address this during first meeting of December, so have a think about it.
Updates
- Rust WG (Scott): some new faces at WG meetings since the PR release.
- OSL (Chris): OSL doing well, nothing major to report. Will do master->main rename soon. Kimball: has someone experimented? Michael: OCIO did this, posted writeup in TAC channel, it doesn’t update git clone commands so broke the CI, but overall the process was simple. Larry: can change outstanding PRs that refer to branches and redirects web requests, but it does not do the same for git commands, so scripts will break. Easy to fix, but requires human intervention. Kimball: should we set goal to have this done before End of Year? Chris: yes, that’s reasonable, will report back at next meeting.
- MaterialX (Jonathan): interesting work is happening in other teams, SideFX has released Houdini 19 with deep MaterialX support, getting feedback on this. SideFX has implemented some code changes, expecting more as it moves into production. And working with Adobe on Adobe Standard Material, public version of the specification, need to reach out to Adobe to get access. Giving feedback on that, and how it could be implemented as a MaterialX graph. Larry: how does it differ from Autodesk? Jonathan: uses a different Fresnel conductor, uses a sheen model different from Imageworks sheen, have given feedback on it, need to make a strong case for a different sheen model, and a few other areas where they have done reparametrization, so different UI than what’s found in Autodesk standard shader, even though model is the same under the hood.
- OpenVDB (Ken): talked about volumetric assets from Intel at last meeting, large library of VDB clouds of different resolutions and types, also need some level set examples. Will go out to respective studios and grab challenging assets such as water surfaces, both point clouds and ground truth water surfaces. Also post mortem on 9.0 release. Next item to work with Autodesk team, thinking of contributing adaptive data structure, looking exciting. Could be optional model alongside nanoVDB. Version 9.0 is nanoVDB, faster build times, OpenEXR 3 support (thank you Cary)
- OpenEXR (Cary): pretty quiet, pushed out a patch release last week, mostly fixes from Kimball to issues uncovered by Google OSS Fuzz running against new core library, seems to be going well. Experience with OSS Fuzz testing, at first got a flurry of dozens of issues all across the map, from legitimate issues to minor annoyances, then it went and stayed quiet until new core library was exposed to Fuzz tester, which created a bunch of new issues, but mostly legitimate / worth fixing, mostly around garbage / invalid input. Back to a trickle of issues. Had a couple of exchanges with package managers from Fedora in particular reporting various issues, good to have a more engaged relationship. One person reported test failures on an IBM mainframe architecture, provided an account to try it out. A bit of a science project, big endian machine. Also offered help in giving access to more obscure systems than what we have access in the normal CI builds.
- OCIO (Michael): other than renaming the branch, also updated CII badge since it was out of date, thanks to Apple for contributing Metal support, merged into main branch, still additional work happening for testing. Kimball: USD hadn’t converted to OCIO 2, did internal libraries at WETA, is anyone else working on that? Michael: not that I’m aware of / have nor heard about it. Kimball: will contribute back the work then.
- OTIO (Josh): released version 14, over a year since last major release. Went smoothly, a lot more of the process is automated now. So far no real complaints. Approved TSC charter (multi year process), should be able to deal with licensing / CLA re-signing / moving repo to ASWF organization. Did the branch rename, went pretty smoothly, a couple of pointers in the documentation had to be update, and updated CI which was restricted to running on specific branches. Will look at whether can add anything to the document shared by OCIO. Shannon: worth putting on the blog? Josh: yes, can write a short blurb and will send to you. Larry: is OpenVDB on the radar? Shannon, no, it is not, will contact Ken via email to get a blurb.
- Assets (Eric): engagement with OpenVDB as to what would be useful as assets, Intel proposing volumetric assets. Other groups should add to the wishlist. Wave: didn’t have anyone at the meeting to assess complexity of Intel VDB assets, so hard to judge if they are “production quality”, would be good to get engagement from VDB. Also would animated VDBs be desirable? Issue of space. Ken: not only animated, but also velocity fields, which would help with testing motion blur in rendering. For complexity, those assets looked good, the largest one would be larger / more complex than the Moana cloud. Wave: would be great if they could supply a Houdini hip file so people could experiment. Came up with ALab context, people want to know if they are ingesting the asset correctly, being able to easily load the asset in Maya / Blender / Houdini would help. Eric: Houdini file would be the best way to compress it, especially for animation. Ken: does WETA has large volumetric assets that could be shareD? Kimball: probably, have a meeting next week with someone who deals with public facing “things”, can bring it up.
- USD (Cory): good meeting last week, discussion around guidelines to structure USD assets, document from SideFX has been discussed last couple of sessions, could be interesting for Asset WG. Presentation on debugging in context of Visual Studio, will post video soon. WebViz and Camera WGs continue to meet. Michael Min: ramping back up the Camera WG, having discussions about new samples that can be generated from cameras. Jonathan: there’s a dedicated Slack for USD MaterialX, lots of good discussion going on there for USD / MaterialX.
- D&I (Carol): monthly meeting next week at noon, working on getting ambassadors for every project, have volunteers for half the projects so far. Also making progress on board succession planning. Kimball: what about in the context of the TAC chair discussion? Will be a member of the Governing Board, so can’t have 2 votes. Would be good to have some guidance there.
How could we bootstrap the project review process, improve their status? Any guidance from other LF foundations?
- Kimball: starting at the beginning, what was the first project that joined and start there. Cary: we give project updates at Open Source Days outwards to the community, how would this process be different from this? Kimball: we want to know where projects are at, but want to know if project is getting new people to show up at meetings (ambassador program), overall health of the project. Doesn’t necessarily have to be “public facing” (although of course we are open). If a project is struggling, could benefit from brainpower of TAC. Cary: can use CII badge as a checklist / discussion starting point, focussing on the Project Maintainer role. Kimball: could take a portion of a meeting, open up CII badge list, how could we help. Cary: when filling out CII, or more recently looking at the requirements for Silver / Gold status, can answer a lot of questions in the form, but who is reading this? Do answers really matter? From organizational point of view, would be worth talking through interpretation of what the requirements mean, how are we meeting them / struggling with them, share experience between projects. Kimball: our role as the foundation is to foster projects, not just “collect” them. How do we help ensure the health and growth of the projects. Ken: are there skillsets shared between projects that are stressing, OpenVBD switched to CMake a couple of years ago, have only one person who knows CMake, and even if he struggles, so anyone interested from other projects to help would be appreciated. Also struggle with the discussion around security, what does it mean to deliver secure software, we follow the guidelines, but no one in OpenVDB claims to be an expert on the topic. Would be good to invest in education / expertise. Wave: identify what the projects would like advice / guidance, and then get someone to present on the topic, for instance Apple has a lot of security experts, people in VFX facility may not have a lot of exposure to. Scott: if a team can write up how they solved a specific CMake problem (for instance), other teams may then know where to look. Eric: connections between projects are important, are we getting the cross pollination we need. Kimball: this sounds like a good set of bullet points to talk through this process. Ken: setting up CI is also something that every project struggles with. Eric: also related to how high are the barriers to entry to someone joining a new project, can you just download a build a project easily. Cary: if any project presented CI workflow, that could spark discussion. Could be a “show and tell”. Ken: leading to documentation. Wave: can work in both directions: projects asking for help on a topic, or demonstrating something they do well. Kimball: Andrew / John, do we know someone who can present on the topic of CMake? Andrew: don’t know anyone, but can find out. Don’t know Craig Scott, but can try to find out. (Kimball: he works for Kitware, wrote the Professional CMake book). Kimball: have read the Professional Cmake book. Larry: bought the ebook, it’s really good, anyone who touches CMake should read it. Kimball: we should be able to invite people like that to present. Cary: don’t use Cmake in day job, so only exposure is in maintaining OpenEXR. It’s one thing to get the project to build and install, but where we run into problems is where consumers of the project encounter something we didn’t anticipate, and our CI doesn’t exercise that, it doesn’t mimic what the downstream users experience. Could get someone who works on repackaging for distributions to present as well on their challenges. Ken: OpenVDB spends a lot of time on Windows support, since we don’t have any Windows developers, a problem for the build system and the code itself. Had an interesting bug that only showed up in Windows Debug CI builds for nanoVDB, turned out to be an actual bug, the Windows compiler in debug release mode performed extra checks. Scott: also on macOS, building on Linux is easy (especially with containers), but not sure that there are many in Rust WG on Windows / macOS. Kimball: had discussion about using MacPorts vs Homebrew, how do you service those variants. Wave: could be useful topic of Windows / macOS support. Would be good to share best practices on these platforms.
Python 3 Working Group Closure request (https://docs.google.com/document/d/1RmvZPwwhLu0KmCXbn4XgS4uzVuFCT2K21eQdQfnWYLE/edit?usp=sharing)
TAC notes: repo vs wiki