Python Developer’s Guide

This guide is a comprehensive resource for contributing to Python – for both new and experienced contributors. It is maintained by the same community that maintains Python. We welcome your contributions to Python!

Quick Reference

Here are the basic steps needed to get set up and contribute a patch. This is meant as a checklist, once you know the basics. For complete instructions please see the setup guide.

1. Install and set up Git and other dependencies (see the Git Setup page for detailed information).

2. Fork the CPython repository to your GitHub account and get the source code using:

   git clone https://github.com/<your_username>/cpython
   cd cpython

3. Build Python, on UNIX and Mac OS use:

   ./configure --with-pydebug && make -j

   and on Windows use:

   PCbuild\build.bat -e -d

   See also more detailed instructions, how to install and build dependencies, and the platform-specific pages for UNIX, Mac OS, and Windows.

4. Run the tests:

   ./python -m test -j3

   On most Mac OS X systems, replace ./python with ./python.exe. On Windows, use python.bat.

5. Create a new branch where your work for the issue will go, e.g.:

   git checkout -b fix-issue-12345 main

   If an issue does not already exist, please create it. Trivial issues (e.g. typo fixes) do not require any issue to be created.

6. Once you fixed the issue, run the tests, run make patchcheck, and if everything is ok, commit.

7. Push the branch on your fork on GitHub and create a pull request. Include the issue number using bpo-NNNN in the pull request description. For example:

   bpo-12345: Fix some bug in spam module

8. Add a News entry into the Misc/NEWS.d directory as individual file. The news entry can be created by using blurb-it, or the blurb tool and its blurb add command. Please read more about blurb in documentation.

Note

First time contributors will need to sign the Contributor Licensing Agreement (CLA) as described in the Licensing section of this guide.

Quick Links

Here are some links that you probably will reference frequently while contributing to Python:

• Issue tracker

• Buildbot status

• Where to Get Help

• PEPs (Python Enhancement Proposals)

• Git Bootcamp and Cheat Sheet

Status of Python branches

The main branch is currently the future Python 3.11, and is the only branch that accepts new features. The latest release for each Python version can be found on the download page.

Status:

features

new features, bugfixes, and security fixes are accepted.

prerelease

feature fixes, bugfixes, and security fixes are accepted for the upcoming feature release.

bugfix

bugfixes and security fixes are accepted, new binaries are still released. (Also called maintenance mode or stable release)

security

only security fixes are accepted and no more binaries are released, but new source-only versions can be released

end-of-life

release cycle is frozen; no further changes can be pushed to it.

Dates in italic are scheduled and can be adjusted.

By default, the end-of-life is scheduled 5 years after the first release, but can be adjusted by the release manager of each branch. All Python 2 versions have reached end-of-life.

See also the Development Cycle page for more information about branches.

Contributing

We encourage everyone to contribute to Python and that’s why we have put up this developer’s guide. If you still have questions after reviewing the material in this guide, then the Core Python Mentorship group is available to help guide new contributors through the process.

A number of individuals from the Python community have contributed to a series of excellent guides at Open Source Guides.

Core developers and contributors alike will find the following guides useful:

• How to Contribute to Open Source

• Building Welcoming Communities

Guide for contributing to Python:

Advanced tasks and topics for once you are comfortable:

• Silence Warnings From the Test Suite

• Fixing issues found by the buildbots

• Coverity Scan

• Helping out with reviewing open pull requests. See how to review a Pull Request.

• Fixing “easy” Issues (and Beyond)

It is recommended that the above documents be read as needed. New contributors will build understanding of the CPython workflow by reading the sections mentioned in this table. You can stop where you feel comfortable and begin contributing immediately without reading and understanding these documents all at once. If you do choose to skip around within the documentation, be aware that it is written assuming preceding documentation has been read so you may find it necessary to backtrack to fill in missing concepts and terminology.

Proposing changes to Python itself

Improving Python’s code, documentation and tests are ongoing tasks that are never going to be “finished”, as Python operates as part of an ever-evolving system of technology. An even more challenging ongoing task than these necessary maintenance activities is finding ways to make Python, in the form of the standard library and the language definition, an even better tool in a developer’s toolkit.

While these kinds of change are much rarer than those described above, they do happen and that process is also described as part of this guide:

• Adding to the Stdlib

• Changing the Python Language

Other Interpreter Implementations

This guide is specifically for contributing to the Python reference interpreter, also known as CPython (while most of the standard library is written in Python, the interpreter core is written in C and integrates most easily with the C and C++ ecosystems).

There are other Python implementations, each with a different focus. Like CPython, they always have more things they would like to do than they have developers to work on them. Some major examples that may be of interest are:

• PyPy: A Python interpreter focused on high speed (JIT-compiled) operation on major platforms

• Jython: A Python interpreter focused on good integration with the Java Virtual Machine (JVM) environment

• IronPython: A Python interpreter focused on good integration with the Common Language Runtime (CLR) provided by .NET and Mono

• Stackless: A Python interpreter focused on providing lightweight microthreads while remaining largely compatible with CPython specific extension modules

Key Resources

• Coding style guides

  • PEP 7 (Style Guide for C Code)

  • PEP 8 (Style Guide for Python Code)

• Issue tracker

  • Meta tracker (issue tracker for the issue tracker)

  • Experts Index

• Buildbot status

• Source code

  • Browse online

  • Snapshot of the *main* branch

  • Daily OS X installer

• PEPs (Python Enhancement Proposals)

• Where to Get Help

• Developer Log

Additional Resources

• Anyone can clone the sources for this guide. See Helping with the Developer’s Guide.

• Help with …

  • Exploring CPython’s Internals

  • Changing CPython’s Grammar

  • Design of CPython’s Compiler

  • Design of CPython’s Garbage Collector

• Tool support

  • gdb Support

  • Dynamic Analysis with Clang

  • Various tools with configuration files as found in the Misc directory

  • Information about editors and their configurations can be found in the wiki

• python.org maintenance

• Search this guide

Code of Conduct

Please note that all interactions on Python Software Foundation-supported infrastructure is covered by the PSF Code of Conduct, which includes all infrastructure used in the development of Python itself (e.g. mailing lists, issue trackers, GitHub, etc.). In general this means everyone is expected to be open, considerate, and respectful of others no matter what their position is within the project.

Full Table of Contents

• 1. Getting Started
  • 1.1. Install git
  • 1.2. Get the source code
  • 1.3. Compile and build
    • 1.3.1. UNIX
    • 1.3.2. Windows
  • 1.4. Install dependencies
    • 1.4.1. Linux
    • 1.4.2. macOS and OS X
  • 1.5. Regenerate configure
  • 1.6. Troubleshoot the build
    • 1.6.1. Avoid recreating auto-generated files
  • 1.7. Editors and Tools
  • 1.8. Directory structure
• 2. Where to Get Help
  • 2.1. Discourse
  • 2.2. Mailing Lists
  • 2.3. Ask #python-dev
  • 2.4. Zulip
  • 2.5. Core Mentorship
  • 2.6. Core Developers Office Hours
  • 2.7. File a Bug
• 3. Lifecycle of a Pull Request
  • 3.1. Introduction
  • 3.2. Quick Guide
  • 3.3. Step-by-step Guide
    • 3.3.1. Resolving Merge Conflicts
  • 3.4. Making Good PRs
  • 3.5. patchcheck
  • 3.6. Making Good Commits
  • 3.7. Licensing
  • 3.8. Submitting
  • 3.9. Converting an Existing Patch from b.p.o to GitHub
  • 3.10. Reviewing
    • 3.10.1. How to Review a Pull Request
  • 3.11. Leaving a Pull Request Review on GitHub
  • 3.12. Dismissing Review from Another Core Developer
  • 3.13. Committing/Rejecting
  • 3.14. Crediting
• 4. Running & Writing Tests
  • 4.1. Running
    • 4.1.1. Unexpected Skips
  • 4.2. Writing
  • 4.3. Benchmarks
• 5. Increase Test Coverage
  • 5.1. Common Gotchas
  • 5.2. Measuring Coverage
    • 5.2.1. Using coverage.py
    • 5.2.2. Using test.regrtest
  • 5.3. Filing the Issue
  • 5.4. Measuring coverage of C code with gcov and lcov
• 6. Helping with Documentation
  • 6.1. Python Documentation
  • 6.2. Helping with documentation issues
  • 6.3. Proofreading
  • 6.4. Helping with the Developer’s Guide
  • 6.5. Developer’s Guide workflow
• 7. Documenting Python
  • 7.1. Introduction
  • 7.2. Style guide
    • 7.2.1. Use of whitespace
    • 7.2.2. Footnotes
    • 7.2.3. Capitalization
    • 7.2.4. Affirmative Tone
    • 7.2.5. Economy of Expression
    • 7.2.6. Security Considerations (and Other Concerns)
    • 7.2.7. Code Examples
    • 7.2.8. Code Equivalents
    • 7.2.9. Audience
  • 7.3. reStructuredText Primer
    • 7.3.1. Paragraphs
    • 7.3.2. Inline markup
    • 7.3.3. Lists and Quotes
    • 7.3.4. Source Code
    • 7.3.5. Hyperlinks
    • 7.3.6. Sections
    • 7.3.7. Explicit Markup
    • 7.3.8. Directives
    • 7.3.9. Footnotes
    • 7.3.10. Comments
    • 7.3.11. Source encoding
    • 7.3.12. Gotchas
  • 7.4. Additional Markup Constructs
    • 7.4.1. Meta-information markup
    • 7.4.2. Module-specific markup
    • 7.4.3. Information units
    • 7.4.4. Showing code examples
    • 7.4.5. Inline markup
    • 7.4.6. Cross-linking markup
    • 7.4.7. Paragraph-level markup
    • 7.4.8. Table-of-contents markup
    • 7.4.9. Index-generating markup
    • 7.4.10. Grammar production displays
    • 7.4.11. Substitutions
  • 7.5. Building the documentation
    • 7.5.1. Using make / make.bat
    • 7.5.2. Using sphinx-build
  • 7.6. Translating
    • 7.6.1. Starting a new translation
    • 7.6.2. PEP 545 summary:
    • 7.6.3. How to get help
    • 7.6.4. Translation FAQ
• 8. Silence Warnings From the Test Suite
• 9. Fixing “easy” Issues (and Beyond)
• 10. Issue Tracking
  • 10.1. Using the Issue Tracker
    • 10.1.1. Checking if a bug already exists
    • 10.1.2. Reporting an issue
    • 10.1.3. Understanding the issue’s progress and status
  • 10.2. Disagreement With a Resolution on the Issue Tracker
  • 10.3. Helping Triage Issues
    • 10.3.1. Classifying Reports
    • 10.3.2. Reviewing Patches
    • 10.3.3. Finding an Issue You Can Help With
  • 10.4. Gaining the “Developer” Role on the Issue Tracker
  • 10.5. The Meta Tracker
• 11. Triaging an Issue
  • 11.1. Python triage team
  • 11.2. Becoming a member of the Python triage team
    • 11.2.1. GitHub Labels for PRs
  • 11.3. Fields in the Issue Tracker
    • 11.3.1. Title
    • 11.3.2. Type
    • 11.3.3. Stage
    • 11.3.4. Components
    • 11.3.5. Versions
    • 11.3.6. Priority
    • 11.3.7. Keywords
    • 11.3.8. Nosy List
    • 11.3.9. Assigned To
    • 11.3.10. Dependencies
    • 11.3.11. Superseder
    • 11.3.12. Status
    • 11.3.13. Resolution
    • 11.3.14. Mercurial Repository
  • 11.4. Generating Special Links in a Comment
  • 11.5. Checklist for Triaging
• 12. Following Python’s Development
  • 12.1. Mailing Lists
  • 12.2. Zulip
  • 12.3. IRC
  • 12.4. Blogs
  • 12.5. Standards of behaviour in these communication channels
  • 12.6. Setting Expectations for Open Source Participation
  • 12.7. Additional Repositories
• 13. Porting Python to a new platform
• 14. How to Become a Core Developer
  • 14.1. What it Takes
  • 14.2. What it Means
  • 14.3. Gaining Commit Privileges
    • 14.3.1. Mailing Lists
    • 14.3.2. Sign a Contributor Agreement
    • 14.3.3. Pull Request merging
  • 14.4. Responsibilities
• 15. Developer Log
  • 15.1. Procedure for Granting or Dropping Access
• 16. Accepting Pull Requests
  • 16.1. Assessing a pull request
  • 16.2. Updating NEWS and What’s New in Python
  • 16.3. Working with Git
    • 16.3.1. Seeing active branches
    • 16.3.2. Backporting changes to an older version
    • 16.3.3. Reverting a merged pull request
• 17. Development Cycle
  • 17.1. Branches
    • 17.1.1. In-development (main) branch
    • 17.1.2. Maintenance branches
    • 17.1.3. Security branches
    • 17.1.4. End-of-life branches
  • 17.2. Stages
    • 17.2.1. Pre-alpha
    • 17.2.2. Alpha
    • 17.2.3. Beta
    • 17.2.4. Release Candidate (RC)
    • 17.2.5. Final
  • 17.3. Repository Administration
    • 17.3.1. Organization Repository Policy
    • 17.3.2. Organization Owner Policy
    • 17.3.3. Current Owners
    • 17.3.4. Repository Administrator Role Policy
    • 17.3.5. Current Administrators
    • 17.3.6. Repository Release Manager Role Policy
• 18. Continuous Integration
  • 18.1. Checking results of automatic builds
  • 18.2. Stability
  • 18.3. Flags-dependent failures
  • 18.4. Ordering-dependent failures
  • 18.5. Transient failures
  • 18.6. Custom builders
• 19. Adding to the Stdlib
  • 19.1. Adding to a pre-existing module
  • 19.2. Adding a new module
    • 19.2.1. Acceptable Types of Modules
    • 19.2.2. Requirements
    • 19.2.3. Proposal Process
• 20. Changing the Python Language
  • 20.1. What Qualifies
  • 20.2. PEP Process
  • 20.3. Suggesting new features and language changes
• 21. Experts Index
  • 21.1. Stdlib
  • 21.2. Tools
  • 21.3. Platforms
  • 21.4. Miscellaneous
  • 21.5. Documentation Translations
• 22. gdb Support
  • 22.1. gdb 7 and later
  • 22.2. gdb 6 and earlier
  • 22.3. Updating auto-load-safe-path to allow test_gdb to run
• 23. Exploring CPython’s Internals
  • 23.1. CPython Source Code Layout
  • 23.2. Additional References
• 24. Changing CPython’s Grammar
  • 24.1. Abstract
  • 24.2. Checklist
• 25. Design of CPython’s Compiler
  • 25.1. Abstract
  • 25.2. Parsing
  • 25.3. Abstract Syntax Trees (AST)
  • 25.4. Memory Management
  • 25.5. Parse Tree to AST
  • 25.6. Control Flow Graphs
  • 25.7. AST to CFG to Bytecode
  • 25.8. Introducing New Bytecode
  • 25.9. Code Objects
  • 25.10. Important Files
  • 25.11. Known Compiler-related Experiments
  • 25.12. References
• 26. Design of CPython’s Garbage Collector
  • 26.1. Abstract
  • 26.2. Memory layout and object structure
  • 26.3. Identifying reference cycles
    • 26.3.1. Why moving unreachable objects is better
  • 26.4. Destroying unreachable objects
  • 26.5. Optimization: generations
    • 26.5.1. Collecting the oldest generation
  • 26.6. Optimization: reusing fields to save memory
  • 26.7. Optimization: delay tracking containers
• 27. Updating standard library extension modules
• 28. Changing Python’s C API
  • 28.1. The internal API
    • 28.1.1. With PyAPI_FUNC or PyAPI_DATA
    • 28.1.2. With the extern keyword
    • 28.1.3. Private names
  • 28.2. Public C API
  • 28.3. Limited API
    • 28.3.1. Guidelines for changing the Limited API
    • 28.3.2. Adding a new definition to the Limited API
• 29. Coverity Scan
  • 29.1. Access to analysis reports
  • 29.2. Building and uploading analysis
  • 29.3. Known limitations
    • 29.3.1. False positives
    • 29.3.2. Intentionally
  • 29.4. Modeling
  • 29.5. Workflow
    • 29.5.1. False positive and intentional issues
    • 29.5.2. Positive issues
  • 29.6. Contact
• 30. Dynamic Analysis with Clang
  • 30.1. What is Clang?
  • 30.2. What are Sanitizers?
  • 30.3. Clang/LLVM Setup
    • 30.3.1. Download, Build and Install
  • 30.4. Python Build Setup
    • 30.4.1. Building Python
    • 30.4.2. Blacklisting (Ignoring) Findings
• 31. Running a buildbot worker
  • 31.1. Preparing for buildbot worker setup
  • 31.2. Setting up the buildbot worker
    • 31.2.1. Conventional always-on machines
    • 31.2.2. Latent workers
  • 31.3. Buildbot worker operation
  • 31.4. Required Ports
  • 31.5. Required Resources
  • 31.6. Security Considerations
• 32. Core Developer Motivations and Affiliations
  • 32.1. Published entries
  • 32.2. Goals of this page
  • 32.3. Limitations on scope
• 33. Git Bootcamp and Cheat Sheet
  • 33.1. Forking CPython GitHub Repository
  • 33.2. Cloning a Forked CPython Repository
  • 33.3. Listing the Remote Repositories
  • 33.4. Setting Up Your Name and Email Address
  • 33.5. Enabling autocrlf on Windows
  • 33.6. Creating and Switching Branches
  • 33.7. Deleting Branches
  • 33.8. Renaming Branch
  • 33.9. Staging and Committing Files
  • 33.10. Reverting Changes
  • 33.11. Stashing Changes
  • 33.12. Committing Changes
  • 33.13. Pushing Changes
  • 33.14. Creating a Pull Request
  • 33.15. Updating your CPython Fork
  • 33.16. Applying a Patch from Mercurial to Git
  • 33.17. Downloading Other’s Patches
  • 33.18. Accepting and Merging a Pull Request
  • 33.19. Backporting Merged Changes
  • 33.20. Editing a Pull Request Prior to Merging
• 34. Appendix: Topics
  • 34.1. Basics for contributors
  • 34.2. Core developers
  • 34.3. Development workflow for contributors
  • 34.4. Documenting Python and style guide
  • 34.5. Issue tracking and triaging
  • 34.6. Language development in depth
  • 34.7. Testing and continuous integration