ASWF TAC Meeting - May 22, 2019

Voting member attendance

  • Daniel Heckenberg - Chairperson, Animal Logic Pty Ltd
  • Gordon Bradley, Autodesk
  • Mark McGuire, Blue Sky Studios, Inc.
  • Michael O’Gorman, Cisco Systems Inc.
  • Henry Vera, Double Negative
  • Jeff Bradley for Bill Ballew, DreamWorks Animation
  • Matt Kuhlenschmidt, Epic Games, Inc.
  • Brian Cipriano, Google, OpenCue Representative
  • Jim Jeffers, Intel Corporation
  • Larry Gritz, Sony Pictures Imageworks
  • Jean-Francois Panisset, VES Technology Committee
  • Cory Omand, The Walt Disney Studios
  • Kimball Thurston, Weta Digital Limited
  • Ken Museth, OpenVDB Representative
  • Michael Dolan, OpenColorIO Representative
  • Eric Enderton, NVIDIA
  • Cary Phillips, OpenEXR Representative

Apologies

Other Attendees

  • Bryan Stone (Linux Foundation)
  • Dan Bailey (ILM)
  • Doug Walker (Autodesk)
  • Jacob Palmer (Linux Foundation)
  • Jeff Bradley (Dreamworks)
  • Mike Dowlan (SPI)
  • Andrew Grimberg (Linux Foundation)

Agenda

  • Goals for TAC: Year 1 (0:00-0:05)
    • Timeline:
      • May: CI platform decision (based on at least one example setup)
      • June: CII badge. At least one project graduation.
      • July: Dependency management.
  • LF CommunityBridge Intro (Bryan Stone, Shubhra Kar) (0:05-0:20)
    • Video
    • Focus on funding diversity, security
      • Putting money in the hands of maintainers, user groups, meetups, causes that use open source technology
      • Be an intermediary to allow transparent access to funds
      • Sample Project: Chaos Group, how would they like to spend the funds, mentorship opportunities, smaller portion towards marketing
      • There are some fees associated with running the platform, but fees get credited back under $10m
      • Do vulnerability scanning
      • Mentorships, companies can sponsor mentors, paid internships
      • Key contributors can guide new contributors
      • Showcase individual contributors
      • Companies can provide direct funds as well as positions
      • Basically matchmaking
      • Static code analysis, integrating existing service offerings
      • Daniel: ASWF is looking for mechanism to find contributors / matchmaking. Taken early steps through GitHub “First Issues” mechanism.
      • Interest from our members in mentoring
      • Looking for ways to share and grow our security expertise (we are somewhat of an outlier from other projects due to primarily C++ nature of our projects)
      • Mike, Larry: specific resources for OpenColorIO. 2 types of resources we are looking for: people just getting into Open Source, also ASWF members who are looking to do more significant contributions.
      • Bryan: no barrier to entry, no downside to adding project to CommunityBridge. Just go to communitybridge.org to submit the project. Basic information: repo location, logo, project description. Also what is the project looking for, also valid for non-technical contributions (project management, documentation…). Covers both paid internships and voluntary contributions.
      • Larry: needs will vary a lot between ASWF projects (and potential ASWF projects). Some projects have wide appeal, some are very niche with very narrow technical focus. No clear if “soliciting help from the world” is going to really help or just “generate noise”. So may be best to concentrate on people working at companies already part of ASWF / in the industry.
      • Bryan: can be used as just a landing page that you can point people to, can post link to that page on relevant forums. Larry: are there tools to filter projects on CommunityBridge based on interest / relevance? Bryan: currently no good tools for filtering / grouping but on the roadmap.
      • Daniel: as we develop auxiliary projects (CI-related, CMake), we need a better documentation infrastructure.
      • Bryan: some organizations have created a single “project” for the entire organization, others break it up into individual projects and seek specific skill sets for these individual projects.
      • Daniel will create ASWF page on CommunityBridge
  • CI updates (0:20-0:30)
    • Last meeting in May, need decisions made before Board meeting next week
    • Platform decision (May)
    • Azure Pipelines testing
      • Andrew: GPU access, increase parallel processing in Azure. No group discount rates currently, at moment more processing time would require paying market rate for those resources. Haven’t found exactly what those costs would be.
      • No agent pools available with GPUs, we would spin up a “container group”, install Azure Agent.
      • Azure: GPU on Linux only
      • Since we can run our own agent pool out of other clouds, we should be able to run GPU systems on other public cloud providers if needed.
      • Could use MacStadium for Macs
      • Best approach is to do everything inside containers
      • Trying to get better information on pricing
      • Already have a LF organization on Docker Hub, could get ASWF specific org
      • Are we looking at testing on multiple GPU or driver versions? On Azure we only have access to a very specific GPU. Daniel: early on we would be happy with “any GPU”. There’s a good opportunity for us to use the CI to build the matrix of GPUs / drivers.
      • Larry: for now OpenColorIO just needs to be able to run “modern OpenGL”. Mac also needs OpenGL, but may have to switch to Metal.
      • Andrew: all Azure GPUs are Tesla. Larry: some projects (current or future) will want to run CUDA code.
      • Existing LF/ASWF CI infrastructure does have access to GPU on Linux/Windows on OpenStack public cloud provider
      • Will use containers to get RHEL/CentOS environment on the Ubuntu agent pool
      • Need to experiment, how to interact with component repositories for simpler experimentation
      • Jenkins system already has Docker workflows.
        • Could use existing Jenkins docker staging system with other CI systems
      • Container approach helps to not be strongly tied to any one CI solution.
  • Technical Project updates (0:30-0:50)
    • OpenVDB
      • Dan: 6.1 released recently, complete overhaul of CMake build system, much more modern now. Deprecating a lot of older functionality, with a more formal mechanism for deprecating older functionality.
      • Migration to CircleCI, in tandem with CMake overhaul. Aloys created prototype of how this would work on Azure.
      • CI supports WIndows and Linux, no Mac or Maya builds.
      • Cleaning up warnings.
      • Dealing with flow of issues reported by users on different platforms, newer compilers. Want to build wider range of configurations to catch those issues.
      • Looking at VFX Reference Platform CY2020.
      • CII badge hold ups:
        • User feedback mechanism
        • Static analysis (will come after move to Azure)
        • Security and vulnerability
      • Looking at likely 6.2 release around SIGGRAPH time with some big new features.
    • OpenColorIO
      • Michael: still working on Azure pipeline setup, not a ton of progress. Looking at using Docker images.
      • Actively working on correcting embedded external dependencies. Script to download dependencies into a directory in the build location which CMake can build and patch as needed. Want to make building OCIO as simple as possible while adhering to ASWF licensing requirements.
      • New feature from Autodesk merged last week: support for dynamic parameters in processors, can use GPU renderer with uniforms. Currently supports only display transformations but plan to expand.
      • Issues with Azure? Michael: there’s a ASWF Docker Hub account, can OCIO still use the ASWF Docker Hub before the repo has been moved to ASWF GitHub? Andrew: yes, but he would need admin privileges / keys to the ASWF Docker Hub repo (required to push images to Docker Hub). All access tokens had to be revoked 2 weeks ago due to security breach.
      • No authentication tokens needed to pull Docker images from Azure.
      • Larry: we should have a set of pre-built Docker images based on VFX Reference Platform years. Andrew: we probably need a repo for the Docker container configurations. Also build recipes for external components used by our projects.
    • OpenEXR
      • Larry: TSC on hiatus due to vacations, but pick up in 2 weeks. Should have been activity by next TAC meeting.
    • OpenCue
      • Brian: making progress on the transfers to ASWF. TSC now has 5 members with a good variety of people.
      • Finalizing repository migration.
      • Figuring out how CLAs are going to work
      • About ready to migrate to the ASWF mailing lists
      • Started on CII badge process (see TAC readme)
      • New TSC members joined a week ago so still ramping up.
      • Should be able to pull together initial roadmap and priority list soon.
      • Daniel: does OpenCue want to draw on experiences of other projects? Brian: work is required on how the internal scheduling algorithm works, so would be good to get input on that.
  • VFX Reference Platform 2020 Draft (0:50-0:55)
    • Details
    • VFX Platform 2020: main thing is Python 3. Larry asked about compiler versions. Daniel: AL feels we should move forward with compiler versions fairly aggressively.
    • Extension of platform to other operating systems: which version, which compiler?
    • Kimball: 10% performance boost from switching to LLVM8 using gcc headers, seems to generate better code. Larry: SPI builds with CLangs with old gcc headers for compatibility. Should VFX Platform tackle this, or should this be a recommended practice from ASWF. We should give people a bit more guidance / leaway, wasteful for everyone to have to figure this out separately.
    • Daniel: use of the RedHat Developer Toolset (DTS) compiler with peculiar approach of allowing newer C++ extensions while still being able to link to older system libraries gets complicated / fragile. Decoupling compiler version requirement vs binary compatibility would be useful for everything. Right now they build most things with DTS, but more modern compilers for in-house, performance critical projects.
    • Jeff: also run into problems mixing compilers, they prefer safe approach of sticking with recommended compiler.
  • Next meeting
    • 5 June 2019
    • Daniel on leave for next 3 weeks, will hand over chair duties to someone