ASWF TAC Meeting - 08 February, 2023

Video Conference Link

Voting member attendance

Observer member attendance

Other Attendees

David Morin, ASWF
John Mertic, LF
Yarille Kilborn, LF
JT Nelson, Pasadena Open Source consortium / SoCal Blender group
Scott Wilson, Rust WG
Stephen Mackenzie, NVIDIA
Doug Walker, Autodesk/OCIO
Sergio Rojas, Different Dimensions
Daniel Heckenberg, Netflix
Brett Russell, Fell VFX
Eric Reinecke, Netflix
Jean-Christophe Morin
Robert Bogs, WrangleBot
Axel Rother, WrangleBot
Bill Ballew, DreamWorks
Robin Rowe, CinePaint

Apologies

Agenda

David: Jonathan Stone has done a great presentation at the Metaverse
Reminder on 3-4 points and graphic for OSF opening remarks. David: we are preparing Open Source Forum next week, out of feedback from last year, the update on projects was a bit complicated, everyone showing up for 5 minute update. So only doing projects that submitted to the forum, but want to give a project on every project in the State of the Foundation presentation, so from TSC chairs, need a 1 slide topic with 4 topics in it (will provide template in Slack channel):
- Goal:
- Latest development(s)
- List of integrations (applications / vendors)
- Breaking News
- One good image
Reminder on GSoD - applications close 3/24
- Doesn’t look like GSoC had interest and the application window has closed
- John: GSoD still open, so a bit different of a structure. Happy to help coordinate if interest.
Digipro Submission Deadline: Friday May 12, 2023, 11:59pm PDT (UTC-7:00).
- Call for papers just went up
- Details: DigiPro Conference
SIGGRAPH open till Feb
Rust Working Group Project Review (Scott)
- About:
  - Goal is to create a framework for building Rust bindings for libraries
  - …
- Tracked Bindings
  - Alembic
  - Imath
  - MaterialX
  - OSL
  - OCIO
  - OpenEXR - In progress
  - OpenImageIO - In progress
  - …
- CPPMM and Babble
  - Design
    - Create C and “unsafe” Rust bindings from a C++ project
  - Goals
    - Make selling the bindings easier - bound projects get a free automatically generated C API
    - Make generating the “unsafe” bindings easier and less tedious
  - Status
    - CPPMM has been superseded by Babble due to design issues with CPPMM
    - Babble is in a working state, but very incomplete - missing core pieces, and support for complex project requirements
- Issues
  - Progress on binding projects has slowed down drastically
  - Progress on Babble has slowed down
  - Projects have a key dependency, Babble, which is incomplete
  - Babble is mostly developed by one developer, who is understandably busy with life outside of the Rust Working Group
- Possible Solutions
  - Get more developers working on Babble
    - Identify how Babble could be useful outside of creating Rust bindings
    - Make sure the onboarding process is as easy as possible
    - Make sure there are several developers who know Babble very well to manage the project
  - Investigate other binding generators
    - Drop the requirement that we also produce C bindings
    - Investigate other Rust binding generators such as cpp and cxx
  - Try to pull interested parties from the Rust community and VFX community together
    - Advertise on popular forums
    - Present small tasks that may spark curiosity
    - Make the projects as easy to hack on as possible
- We’re in a state where it could be better, but this is the direction we’re hoping to be in for 2023, hoping to bring bindings to all the packages. Overview of where we are at.
- Larry: Kimball and I were in a OpenEXR TSC meeting where we were wondering about the Rust bindings, wondering what is going on. Same for OpenImageIO. Wondering if the original effort of putting Rust binding on the most used projects has been sidelined by effort to find a general solution? Should there be a parallel track of identifying projects that need Rust bindings, willing to add a C API themselves later on. Don’t stop working on general solution, but maybe also look at project specific solutions. Could ask projects to generate C bindings themselves. Scott: there is an automatic system from C to Rust, but there’s also an automatic C++ to Rust, but the C option would be the most complete solution. If projects all have C bindings, at that point it makes things a bit easier, but also harder in other ways. Still need to make sure these C bindings are safe within the Rust environment. Larry: you want to make Safe Rust bindings around the unsafe ones. Rust group would have expertise for that. But individual projects can help. Scott: either go with cpp/cxx crate to build the bindings, but if the projects are interested in creating C bindings, we are also interested in those. Larry: projects could be motivated in creating C API to get access from Rust. If the issue is that there are constructs in C++ that are problematic, would consider an alternate different C++ interface that would be easier to wrap. Kimball: created the core library in EXR to make bindings easier, not just from Rust, but also Python and others. Eric R: do we have users that are enthusiastically waiting to adopt these? Scott: have spoken to people interested in consuming these Rust bindings. Larry: probably a lot of people interested in exploring Rust, but lack of access to important libraries makes that difficult. Scott: the WG needs to determine what’s the direction that makes most sense. We bet on CPPMM / Babble to do this work, but it hasn’t hit the mark, so need to re-evaluate.
- Kimball: My perspective tends to be language agnostic, was hoping to see from Rust WG a good framework for ownership of bindings, in EXR project we didn’t do a good job of managing Python bindings for EXR, we don’t have clear path to have those updated reliably. So would want a clear path for ownership of bindings. If there’s a path to determine that a library is safe because we’ve tested it through Rust, that can add value to our projects. Don’t know if CPPMM / Babble / cpp / cxx, do any of these provide some kind of static analysis of your unsafe bindings? What you guys have found on this front. Scott: there would be static analysis of the interface, but not much on the internal. The interface to OpenEXR is a black box to Rust. so can’t guarantee that the C++ code is 100% safe. The Safe layer won’t access a pointer that’s invalid for instance. Kimball: was trying to see if it’s something we can integrate with memory sanitizers? But you need a full fuzzer as well. Larry: you may be on the right track, what I was hoping from Rust binding was to see which parts of the API were easier or harder to make safer, and using that as feedback to the project of how to evolve the API, import some of the Rust “sensibilities”, there’s a philosophy from Rust that is worth learning from. Scott: it would be an avenue, we need to refocus our efforts, possibly in the direction of OpenEXR bound by end of 2023, here are simple things to do, here are gnarly ones, what can we do to make the code less gnarly. JF: since Rust code would be net new, don’t need to support the entire API. Chris: have you looked at rewrites from scratch for some libraries like EXR, is that another path forward? Scott: we agreed that was a non goal. If EXR wants to rewrite in Rust, we will support them, but that’s not our goal. Chris: my experience has been to stay away from Rust packages that also require C++ building since it breaks the Rust tooling. Scott: have seen some successful Rust bindings to C++ packages, having to compile / link a C++ library such a USD is a huge thing. We don’t want to force people to recompile USD. Would be better to link existing USD C++ library, and will bind against that.
- Larry: in OpenEXR, the fact that there were independently re-implemented Python wrappers that weren’t maintained by the OpenEXR project has been an ongoing nightmare / liability, intentions are good on day one, but there’s drift between official library and re-implementation / bindings. Would like to see Rust bindings owned by the project. Scott: a goal is to hand over the bindings to the project, but without just making extra work for the project. Chris: can do a fair amount with Rust even if you don’t have library projects. Are there studio projects interested in using Rust, and what’s the minimal set? Larry: we don’t have much that doesn’t need to touch ASWF projects, as well as USD, almost everything needs to connect to these projects.
- Eric R: on OTIO, for the bindings that be successful, there has to be a consumer who needs them and wants them to be maintained. Can we identify one or a small set of projects that would consume Rust bindings, and focus on getting those things going. Take feedback from this experience, so we can make this repeatable. And identify what projects can do in their C / C++ API to make things easier in the future. In OTIO, we don’t have people using Rust right now, but would love to get advice on what would be easier to make Rust (or other language) bindings.
- Kimball: do we want to keep the WG going, even with challenge to participation. Do you have enough direction to keep going? Scott: yes, we would like to keep going for 2023.
- Larry: propose motion to keep the WG going for another year. Carol: seconded. All ayes, no opposed, one abstain. The ayes have it, we reconvene WG for 2023. Hopefully you have enough talking points to keep going. Scott: thank you very much!
- Robin: bindings are nice, but do we have something more direct planned for Rust, like a tool, or just linking to other tools? Scott: for the Rust WG, the main goal is the bindings, but don’t have plans for an actual application. Personally I have plenty of plans, but currently not within ASWF. Robin: bindings aren’t the most exciting project, so having something more exciting as a project could help drive interest. Scott: I like the idea, but maybe that’s something at the TAC level. Kimball: start with some small command line pipeline tools.
- Larry (Chat): Additional possible solution: ask project to routinely produce a C binding themselves, so the Rust group can focus more on the C->Rust layer. Maybe pybind11 is a good example - it’s not an automatic binding of C++ to Python, but it’s a library that makes it relatively straightforward to wrap a C++ API in Python. Maybe there’s an equivalent to Rust?
WrangleBot project presentation TENTATIVE (download Proposal here: https://media.prsnc.io/f/f0c80ff9a4da474b98b1/?dl=1)
- Robert Bogs and Axel from Germany
- WrangleBot was released recently, multiple components, ingesting of media files, we come from the DIT side, produce TBs of data that needs to be ingested, metadata extracted. There are different tools like SilverStack, Shotput, Hedge…
- Developed WrangleBot to have something that works remotely (incentive during COVID). Started as a CLI to admin computers on set via ssh, evolved from there, but wasn’t getting the feedback needed. So developed a GUI. WrangleBot Project
- Core engine of WrangleBot allows you to copy and ingest footage, ingest metadata, puts is in a table, has API to work with metadata (kept in a database). Can go back and forth in time to track changes. Can also sync between devices. We have a network of tools that can access the assets over peer to peer connections. Can transfer large amounts of data directly between devices instead of having to go through cloud storage.
- Already have 122 people registered / using it on set.
- GUI demo: Ingest page, list of libraries that exist in a database structure. Data sets exist within libraries. Can launch jobs to copy media. There’s also a media browser, can go through all your media, can edit the metadata. Trying to get access to all the vendors. One of the issues is that we don’t have a common metadata format. Also pull thumbnails and save them to the database.
- Sometimes we need to do LTO backups, it’s very hard to look at an LTO tape and figure out what’s on there, and not everyone has access to a robot. So this makes it possible to browse a library without having to access the LTO tapes.
- A complex toolkit developed over the last 2 years. Gotten to a point that future developments shouldn’t be driven by a single person or company, hence why we should open source it.
- JF: how do you deal with proprietary media formats? Axel: hoping that being part of a larger organization can help get this information. Create an interface where we can easily create plugins so you can attach a closed source binary plugins licensed separately, as opposed to the open source core.
- Robert: does anyone have leverage on camera manufacturers?
- Carol: very cool tool! Two things: you are talking about camera manufacturers, being able to read their camera files with their SDKs. Are you thinking about providing the hooks since you can’t always open source the SDKs, they are proprietary code. The stuff is not “uncomplicated”. Also if your database is centrally located, gathering info on all projects, that’s going to be an issue with studios. Axel: this is just the current implementation, but can connect to any node, and host your own database. One of our clients are state run archives, they would never want to have a hosted solution, and will want to have their own. But to have all these archives connected would allow reusing assets. Also helps with sustainability, lots of emissions from on set / on location shoots, if we could shoot less, that would help with this, but need to be able to reuse assets. Only 1% of shot footage is used in the edit: maybe that footage isn’t useful to you, but it could be useful to someone else, and maybe still worth it years in the future now that digital footage has reached a quality level which would still make it useful years from now.
- Eric R: how do you secure this data transfer, even the metadata can be sensitive. Also, how is the backend modularized and easy to extend. Integrate proprietary camera SDKs, or proprietary storage systems. Axel: the system is generally agnostic to how you address your data, as long as the path is mounted in some way. Eric: it doesn’t have a concept of storage tiers for instance? Axel: no, it tries to stay “storage agnostic”, trying to bridge the gap with more naive users. All transfers are encrypted, but currently the database is not for performance reasons. But want to add end to end encryption while still meeting usability goals.
- Kimball: we will continue this discussion via email / Slack
- Robert: looking for feedback from experienced people. We have established ties to all the camera manufacturers, have gotten SDKs from most vendors, but not some of the largest ones.
- JC: can the transfer engine be swapped, is it modular, or is it tightly integrated. Axel: the peer to peer streaming hasn’t been implemented in WrangleBot yet, the original implementation was specific to the other app, not sure if that’s going to be the implementation for WrangleBot.