PyTorch Tutorials Refresh - Behind the Scenes

Hi, I’m Jessica, a Developer Advocate on the Facebook Open Source team. In this blog, I’ll take you behind the scenes to show you how Facebook supports and sustains our open source products - specifically PyTorch, an open source deep learning library. With every new release version, PyTorch pushes out new features, updates existing ones, and adds documentation and tutorials that cover how to implement these new changes.

On May 5, 2020, PyTorch released improvements to Tutorials homepage with new content and a fresh usability experience out into the world (see the Twitter thread) for the community. We introduced keyword based search tags and a new recipes format (bite-sized, ready-to-deploy examples) and more clearly highlighted helpful resources, which resulted in the fresh homepage style you see today.

Something Went Wrong

We're having trouble playing this video.

Learn more

As the framework grows with each release, we're continuously collaborating with our community to not only create more learning content, but also make learning the content easier.

What Changed

The tutorials refresh project focused on re-envisioning the learning experience by updating the UX and updating the learning content itself.

Our 3 major goals for the refresh were:

• Reduce blocks of text and make it easy for users to find important resources (e.g. PyTorch Cheat Sheet, New to PyTorch tutorials)
• Improve discoverability of relevant tutorials and surface more information for users to know about the available tutorial content
• Create content that allows users to quickly learn and deploy commonly used code snippets

And we addressed these goals by:

• Adding callout blocks with direct links to highlight important resource such as the beginner tutorial, the PyTorch Cheat Sheet and new recipes
• Adding filterable tags to help users easily find relevant tutorials; and formatting the tutorials cards with summaries so users know what to expect without having to click in
• Creating a new learning format, Recipes, and 15 brand new recipes covering some of the most popular PyTorch topics such as interpretability and quantization as well as basics such as how to load data in PyTorch
• In summary:

Why We Made These Changes

Now, what drove these changes? These efforts were driven by feedback from the community; two sources of feedback were the UX research study and direct community interactions:

• UX Research study - Earlier in 2020, we conducted a UX research study of our website in collaboration with the Facebook UX Research team to understand how our developer community is using the website and evaluate ways we can improve it to better meet their needs
• In-person events and online feedback - The team gathered community feedback about existing tutorials to help identify learning gaps.

We used these channels of input to fuel revisioning our learning experience.

Rethinking the Learning Experience

Given the feedback from the UX Research study and the in-person workshop, we went back and rethought the current learning experience.

• 3 levels
  
  • Level 1: API docs. Currently these exist and they contain code snippets that provide an easily understandable (and reproducible) example that allows a user to implement that particular API
  • Level 2: The missing puzzle piece
  • Level 3: Tutorials ideally provide an end-to-end experience that provides users an understanding of how to take data, train a model and deploy it into a production setting using PyTorch. These exist, but needed to be pruned of outdated content and cleaned up to better fit this model
  • Realized we were missing something in between, content that was short, informative, and actionable. That’s how recipes were born. Level 2: Recipes are bite-sized, actionable examples of how to use specific PyTorch features, different from our tutorials

What Was the Process

Just as it took a large team effort, this was more of a marathon as opposed to a sprint. Let’s look at the process:

Timeline of the process:

Overall, the project took about 6 months, not including the UX research and prior feedback collection time. It started off with the kickoff discussion to align on the changes. We assessed the existing tutorials, pruned outdated content and decided on new recipe topics and assigned authors. In the meantime, marketing and documentation engineers collaborated with our web design team on the upcoming UI needs, created mocks to preview with the rest of the team and built out the infrastructure.

For logistics, we created a roadmap and set milestones for the team of authors. We held weekly standup meetings, and the team bounced ideas in chat. The changes were all made in a staging branch in GitHub, which allowed us to create previews of the final build. Next, the build process. Many of the recipe authors were first time content creators, so we held a live onboarding session where we discussed teaching mindset, writing with an active voice, outlining, code standards and requirements; and this was all captured in a new set of content creation documentation.

The bulk of the process was spent in building out the content, copy editing and implementing the UI experience.

With the product out the door, we took some time to perform a team retrospective - asking what went well? What went poorly? What can we do better next time? In addition, we continue to gather ongoing feedback from the community through GitHub issues.

Moving forward, we are brainstorming and forming a longer-term plan for the PyTorch learning experience as it relates to docs and tutorials.

Ways to Improve

Looking back on ways we could have improved:

• Timeline - Our timeline ended up taking longer than anticipated because it had been coupled with a previous version release and team members were serving double-duty working on release content, as well as tutorials refresh content. As version release approached, we took a step back and realized we needed more time to put out a more polished product.
• Testing - In software development, if there is an impending deadline, typically testing is the first thing to get squished; however, more focused testing will always save time in the bigger picture. For us, we would always welcome more time for more CI tests of the tutorial build, as well as beta tests of the user experience. Both of these are ongoing works in progress, as we continue to improve the tutorials experience overall.

What’s Next

So what’s next? We understand that this was just one change in a larger landscape of the overall PyTorch learning experience, but we are excited to keep improving this experience for you, our dedicated PyTorch user.

We would like to hear from you about your experience in the new tutorials. Found a tutorial you loved? Tweet about it and tag us (@PyTorch). Ran into an issue you can help fix? File an issue in https://github.com/pytorch/tutorials. We are excited to continue building the future of machine learning with you!

To learn more about Facebook Open Source, subscribe on YouTube, or follow us on Twitter and Facebook.