ASWF TAC Meeting - 07 September, 2022

Video Conference Link

Voting member attendance

Observer member attendance

Other Attendees

David Morin, ASWF
Michelle Martineau, LF
Andrew Grimberg, LF RelEng
John Mertic, LF
Emily Olin, LF
Sean Wallitsch, AWS
Doug Walker, OCIO / Autodesk
Lee Kerley, Sony Imageworks
JT Nelson, Pasadena Open Source consortium / SoCal Blender group
Jason Scott, FuseFX / Rising Sun
Deke Kincaid, Digital Domain
Mathieu Mazerolle, Foundry/OpenAssetIO
Kevin Bjorke, Intel Corporation, covering for Sean McDuffee
Scott Wilson, Rust WG
Alexander Schwank, Apple
Mitch Prater, Laika
Nick Porcino, Pixar Animation Studios

Apologies

Sean McDuffee, Intel Corporation

Agenda

Scaling Up Projects Presentation (John Mertic)
- Working with projects for quite a bit, systematically if I look across all projects I’ve worked with, a lot of the challenges is once the project is started, have a lot of excitement, lots of initial work, but how do we grow this but not “kill” ourselves. A topical question for a number of the maintainers in ASWF.
- What does scaling mean?
  - Scaling means a project is able to:
    - Build a diverse and vibrant contributor / maintainer base
    - Enable 100s or 1000s of developers / users
    - Support a vendor ecosystem of 100s of products
  - How can a project do that?
    - Not have to do individual demos? Presentations?
  - Not every project will hit those metrics, but thinking along those metrics lets you think about how you invest your time. Using your time to address valuable goals is important.
  - In the process of writing a book which will ship in 2023/Q2 on those topics
- What does diversity mean?
  - Multiple organizations committing, no one organization more than 40% of committers
  - Committers from different races, genders, nationalities, locales
  - A healthy project is a diverse one. How can a project do this?
  - Example of Kubernetes contribution, percentage of contributions from Google is declining from 80% to 25%, they didn’t lower their commitment (more engineers), but rest of community was growing at faster pace, so overall share of commits is decreasing.
  - If Google walked away from the project, it would feel pain, but would no longer be catastrophic.
- First step is to measure
  - Being able to scale means knowing how the project operates. Without data, you don’t know where to begin.
  - Key data points to consider:
    - Rate of contributions
    - Number and diversity of contributors
    - Collaboration and engagements in mailing lists, discussion groups, chat channels. This is where potential new contributors first get involved with a project.
  - LFX Insights is a great tool to dig more into this
    - Larry (chat): How does it know which contributors are from which companies? Particularly if they are not using their work email for thief GitHub accounts?
    - Carol: I think it goes by the CLA
    - Andrew: it’s based upon information on your personal profile. There is a section that allows you to help identify what company you were working for and when
    - Larry: Profiles on GitHub, or LFX Insights?
    - Andrew: LF profile. Direct email accounts in LF profile, LinkedIn, GitHub and Gitlab are all used for sourcing this.
- Build an on-ramp for new contributors
  - Contributors often aren’t sure how to best engage. Open source projects can feel intimidating to them.
  - Some ideas to make that on-ramp easier:
    - Have a clear, concise contributors guide
    - Use the ‘good first issue’ tag on issues that are good ones for new contributors to tackle. Encourage projects to spend significant time on this.
    - Hold regular, open developer community meetings that encourage potential and new contributors to engage the maintainers.
    - Ensure maintainers are engaging potential contributors in a positive, encouraging way in pull requests, and responding in online forums.
- Mentorships
  - Offering paid mentorships helps draw new contributors into the project tackle key issues. Mentorships are also a great onramp for new to industry developers to learn more and explore.
- Recognize great contributors
  - Open source contributors and maintainers often feel underappreciated for the work they do. Additionally, contributors use the work they do in open source as a “portfolio” which helps grown their career:
    - Recognize contributors in release notes to formally thank them
    - Developer a badging program using a tool like credly to let contributors showcase their community status
    - Give them some swag! Leverage tools like Spreadshop and give them some codes for free merchandise.
    - Scott (chat); I like the idea of the merit badges. Although, one thing to be careful about is having metrics that make sense. For example, if the goal is to get an accepted PR, someone could make a really low effort PR that is likely going to be accepted, then “free swag time!”.
    - Andrew (chat): you can see an example of some of the merit badges here (scroll down a bit and look to the right). Badges are custom to the project.
- Communities have a lifecycle too
  - Sustainable open source projects aim to train the contributors of today to be the maintainers of tomorrow. With that, you need space for these new maintainers to step in for the maintainers of today.
  - Preparation for a project for this cycle should include
    - Create clear documentation on processes and roles for things like releases, code contributions, and general decision making.
    - Let new maintainers pair with existing maintainers to mentor them in the role
    - Make sure current maintainers can step away from the project with minimal negative impacts
  - Trying to lower the “bus factor”: if maintainers step away, how do you continue. Thinking about succession planning.
- Larry: there’s a tension between having lots of “good first issues” vs having taken care of all these small issues, don’t want to leave a lot of clutter around for people to do? John: every project is a bit different, “good first issues” can be things we would “love to do”. Run into this with organizations that want to open source a code base, but think they have a “lot of work to do before”: if you take that approach, you’ll never get to open source if you wait for it to be “perfect”. In the Linux development process, “release early / release often”, also makes it clearer what you are trying to accomplish. I tend to lean on the side of “don’t be overly self conscious about your code”.
- Kimball: some of the larger projects have a tiering of acceptance, your first PR may be in a “scratch” area. John: yes, could be in a feature branch, or work with someone else more experienced.
- What are the challenges someone wanting to use the code might have?
  - How can I get started quickly?
  - Where can I find some tips and tricks on common use cases?
  - How can I showcase my competence (and leverage that to build my career)
- Getting Started Guides
  - Helping developers get up and running quickly is the most important indicator of success
    - Make the instructions concise and easy to understand
    - Use videos to help
    - Point developers to other resources to help them on the next steps
    - Look at ONNX project as an example for their Getting Started guide
- Solid documentation
  - Good documentation unblocks developers and users to reduce frustration and become productive quickly.
  - Writing great technical documentation is a unique skill - consider bringing in someone.
    - HIre a technical writer. A great resource for finding writers is Write The Docs
    - Leverage a program such as Google Season of Docs
    - Use a tool like Read the Docs as a framework for making documentation easy to read and navigate
    - Not everyone is a native English speaker. Consider internationalization for the documentation.
    - Eric (chat): Is Read The Docs like a Doxygen alternative?
    - Scott: I think soft ot. But, from what I remember, it was built on the Sphinx Python documentation generator.
    - Kimball: sorta, it was based on Sphinx, but with Breathe you can actually put some of the doxygen output into there as well
- Outreach
  - There are millions of open source projects. Projects need to work to stand out
    - Write blog articles on a regular basis that outline development milestones and new feature / functionality
    - Create videos and share via social media of how to get started, sample use-cases
    - Present at events that are pertinent to your project
    - Do meetups and hackathons where potential contributors are.
  - This community is hubed around a few locations, so meetups could work. Also LF has platforms that help to run a meetup program. Great ways to connect with contributors.
- Training and Certification
  - Organizations that want to use your project need to know there is talent out there to help them out. They also need to enable their own staff.
    - Develop formal training programs that cover the key aspects of using the project
    - Build a certification program that lets individuals showcase their skills in the technology
  - LF Training is a great partner in developing training and certification programs.
  - “What should an OpenEXR developer know”?
  - Start working with OCIO on building a training course.
- A screen recording of simple “how to” can be a useful video. What are the 4 things someone new to a project needs to know. People just want to know what to do.
- Just having a template is not enough, have to have a structure, a “3 act play”, similar to how you need to structure a presentation. John: can get it even tighter / more concise, demo short, specific tasks that someone might want to do. A way to decomplex the situation. A 5 min “quick and dirty” video can be useful, following the lines of “release early, release often”. Would highly encourage to keep in mind what you are trying to achieve, and stick to the UNIX philosophy of small, single function tools”. Erik: Can be harder to do a concise 5 minutes than a 30 minute piece.
  - Joshua (chat): can we invite a very skilled YouTuber to do some behind the scenes intro / tutorial content with us?
  - Scott: maybe invite Corridor Digital to talk about some of the tech
  - Carol: not to promote my company here, but if folks are looking for examples… we do a decent amount of these types of videos, it could be something I could talk about at some point if interested.
  - JT: In local TV production to make a production easy to put together, we just do answer questions talking head session. It works every time. Decide on key questions then the people answering just do stream of consciousness. This also works for documentation. Just convert the video audio to text and edit.
- Productization is part of the cycle of a sustainable project
  - Successful projects depend on members, developers, standards and infrastructure to develop products that the market will adopt.
  - Helping define that ecosystem makes it easier to bring products to market and end-users to adopt.
- Case studies and User Stories
  - End-users look to other end-user experiences to help shape their decision making
- Build a conformance program
  - Conformance programs enable the project to showcase an ecosystem of products and services leveraging the project. If also lets project ensure that there is interoperability between vendor solutions which gives end-users choice.
    - Technical requirements defined by the community
    - Program is administered by the project staff to ensure non-bias
    - Vendors get use of exclusive marks and listing in a solution directory with equal footing with other solutions
    - Projects may choose to provide additional benefits as they make sense for the ecosystem
- Read more on project success
  - Case Studios - Linux Foundation
  - CNCF Project Journey Reports
- Kimball: something we struggle with is that we have focus on M&E, is that a limiting factor or an advantage? Some of these factors are broader in scope, requires thought. John: we are seeing Waifair and Ikea using ASWF projects, puts pressure on maintainers to support different users. Those are complex conversations, but you have to think about it for the long term direction of the project. Kevin Bjorke: when looking at metrics, are there metrics you see as red flags, “if it doesn’t get to this, it won’t fly”? John: biggest worry is committer diversity, a project typically starts from an organization so you’ll see majority of commits from that org at first, but if you don’t see diversity increase, that will likely be a problem in the future. So that’s one I really pay attention to. Also, on the maintainer side, are we seeing new names come up to lead contributors, or is it the same people on the “top 10 list”, it’s a sign that you are overtaxing your maintainers and not bringing in people who can support them. Early days of a project is a sprint, but then it has to turn into a marathon. Also don’t get too hung up on comparing your project to another. “Velocity” is unique to an industry and project, don’t get too hung up on that.
Feedback on annual project review process
- Best practices
- How different to status for Board meetings
- How different from Open Source Days
- Forum for project leads
- Want to try to get through backlog of projects for “formal” review
DPEL wants to monitor downloads from S3 bucket (Eric)
- Anyone have tools for this?
- Everything is currently in same bucket, so how to monitor individual assets
- Kimball: know how to get per-bucket metric, but not sure if you can break it up into individual assets
- Sean: reach out to me on Slack, I can answer such questions
John: review OpenSSF badging requirements
- Can we provide help from foundation
- First pass at last CI meeting, most of the way, but have to finish gold and get to silver
- Anyone interested can join CI WG at next meeting (next week)
- Larry: since that CI meeting, have been going through Silver and Gold for OSL, when I really went line by line, found several line items that I wasn’t sure how to do it, but could not understand how a URL would apply to the line item. So lots of things to discuss.
- John: got some guidance from John Wheeler to contribute.