Compiling Python to C using setuptools and Cython

July 04, 2022

plum1 compiles an ordinary Python file into an extension module (in C) using Cython and setuptools' Cython integration. It's the first time I've encountered this, so here's a high-level description of how it works.

In, pass the ext_modules arg to setup, wrapping an Extension that points to the module you want to compile:

from setuptools import setup, Extension

    # ...
    ext_modules=[Extension("plum.function", ["plum/"])],

setuptools has built-in Cython integration. If it finds that Cython is installed, it'll use it to build extensions with .py and .pyx sources. Including Cython as a build dependency in pyproject.toml ensures that it's installed and ready to go:

requires = [
    # ...

What's a Python extension module?

An extension module is a program written in C (or C++) that uses Python's C API to hook into Python's run-time system. Interop works both ways: you can call Python objects from C and vice versa. A common reason for using extensions is improved speed. See the official docs on extension modules for more.

What's Cython?

Cython is a compiler for compiling programs written in Python and the Cython programming language into C extension modules. I'd recommend the reading through the rather friendly documentation as well.

  1. plum is a really cool library that brings Julia-style typed multiple dispatch to Python!