Sharing Declarations Between Cython Modules

Note

This page uses two different syntax variants:

• Cython specific cdef syntax, which was designed to make type declarations concise and easily readable from a C/C++ perspective.

• Pure Python syntax which allows static Cython type declarations in pure Python code, following PEP-484 type hints and PEP 526 variable annotations.

  To make use of C data types in Python syntax, you need to import the special cython module in the Python module that you want to compile, e.g.

  import cython
  

  If you use the pure Python syntax we strongly recommend you use a recent Cython 3 release, since significant improvements have been made here compared to the 0.29.x releases.

This section describes how to make C declarations, functions and extension types in one Cython module available for use in another Cython module. These facilities are closely modeled on the Python import mechanism, and can be thought of as a compile-time version of it.

Definition and Implementation files

A Cython module can be split into two parts: a definition file with a .pxd suffix, containing C declarations that are to be available to other Cython modules, and an implementation file with a .pyx/.py suffix, containing everything else. When a module wants to use something declared in another module’s definition file, it imports it using the cimport statement or using special cython.cimports package.

A .pxd file that consists solely of extern declarations does not need to correspond to an actual .pyx/.py file or Python module. This can make it a convenient place to put common declarations, for example declarations of functions from an external library that one wants to use in several modules.

What a Definition File contains

A definition file can contain:

• Any kind of C type declaration.

• extern C function or variable declarations.

• Declarations of C functions defined in the module.

• The definition part of an extension type (see below).

It cannot contain the implementations of any C or Python functions, or any Python class definitions, or any executable statements. It is needed when one wants to access cdef/@cfunc attributes and methods, or to inherit from cdef/@cclass classes defined in this module.

Note

You don’t need to (and shouldn’t) declare anything in a declaration file public in order to make it available to other Cython modules; its mere presence in a definition file does that. You only need a public declaration if you want to make something available to external C code.

What an Implementation File contains

An implementation file can contain any kind of Cython statement, although there are some restrictions on the implementation part of an extension type if the corresponding definition file also defines that type (see below). If one doesn’t need to cimport anything from this module, then this is the only file one needs.

The cimport statement

The cimport statement is used in a definition or implementation file to gain access to names declared in another definition file. Its syntax exactly parallels that of the normal Python import statement. When pure python syntax is used, the same effect can be done by importing from special cython.cimports package. In later text the term to cimport refers to using both cimport statement or cython.cimports package.

Pure PythonCython

from cython.cimports.module import name [as name][, name [as name] ...]

Here is an example. dishes.pxd is a definition file which exports a C data type. restaurant.pyx/restaurant.py is an implementation file which imports and uses it.

dishes.pxd

cdef enum otherstuff:
    sausage, eggs, lettuce

cdef struct spamdish:
    int oz_of_spam
    otherstuff filler

Pure PythonCython

dishes.py

import cython
from cython.cimports.dishes import spamdish, sausage

@cython.cfunc
def prepare(d: cython.pointer(spamdish)) -> cython.void:
    d.oz_of_spam = 42
    d.filler = sausage

def serve():
    d: spamdish
    prepare(cython.address(d))
    print(f'{d.oz_of_spam} oz spam, filler no. {d.filler}')

It is important to understand that the cimport statement can only be used to import C data types, C functions and variables, and extension types. It cannot be used to import any Python objects, and (with one exception) it doesn’t imply any Python import at run time. If you want to refer to any Python names from a module that you have cimported, you will have to include a regular import statement for it as well.

The exception is that when you use cimport to import an extension type, its type object is imported at run time and made available by the name under which you imported it. Using cimport to import extension types is covered in more detail below.

If a .pxd file changes, any modules that cimport from it may need to be recompiled. The Cython.Build.cythonize utility can take care of this for you.

Search paths for definition files

When you cimport a module called modulename, the Cython compiler searches for a file called modulename.pxd. It searches for this file along the path for include files (as specified by -I command line options or the include_path option to cythonize()), as well as sys.path.

Using package_data to install .pxd files in your setup.py script allows other packages to cimport items from your module as a dependency.

Also, whenever you compile a file modulename.pyx/modulename.py, the corresponding definition file modulename.pxd is first searched for along the include path (but not sys.path), and if found, it is processed before processing the .pyx file.

Using cimport to resolve naming conflicts

The cimport mechanism provides a clean and simple way to solve the problem of wrapping external C functions with Python functions of the same name. All you need to do is put the extern C declarations into a .pxd file for an imaginary module, and cimport that module. You can then refer to the C functions by qualifying them with the name of the module. Here’s an example:

c_lunch.pxd

cdef extern from "lunch.h":
    void eject_tomato(float)

Pure PythonCython

lunch.py

import cython
from cython.cimports.c_lunch import eject_tomato as c_eject_tomato

def eject_tomato(speed: cython.float):
    c_eject_tomato(speed)

You don’t need any c_lunch.pyx/c_lunch.py file, because the only things defined in c_lunch.pxd are extern C entities. There won’t be any actual c_lunch module at run time, but that doesn’t matter; the c_lunch.pxd file has done its job of providing an additional namespace at compile time.

Sharing C Functions

C functions defined at the top level of a module can be made available via cimport by putting headers for them in the .pxd file, for example:

volume.pxd

cdef float cube(float x)

Pure PythonCython

volume.py

def cube(x):
    return x * x * x

spammery.py

import cython
from cython.cimports.volume import cube

def menu(description, size):
    print(description, ":", cube(size),
          "cubic metres of spam")

menu("Entree", 1)
menu("Main course", 3)
menu("Dessert", 2)

Note

Type definitions of function cube() in volume.py are not provided since they are used from .pxd definition file. See Augmenting .pxd and GitHub issue #4388.

Sharing Extension Types

An extension type can be made available via cimport by splitting its definition into two parts, one in a definition file and the other in the corresponding implementation file.

The definition part of the extension type can only declare C attributes and C methods, not Python methods, and it must declare all of that type’s C attributes and C methods.

The implementation part must implement all of the C methods declared in the definition part, and may not add any further C attributes. It may also define Python methods.

Here is an example of a module which defines and exports an extension type, and another module which uses it:

shrubbing.pxd

cdef class Shrubbery:
    cdef int width
    cdef int length

Pure PythonCython

shrubbing.py

import cython

@cython.cclass
class Shrubbery:
    def __cinit__(self, w: cython.int, l: cython.int):
        self.width = w
        self.length = l

def standard_shrubbery():
    return Shrubbery(3, 7)

landscaping.py

from cython.cimports.shrubbing import Shrubbery
import shrubbing

def main():
    sh: Shrubbery
    sh = shrubbing.standard_shrubbery()
    print("Shrubbery size is", sh.width, 'x', sh.length)

One would then need to compile both of these modules, e.g. using

setup.py

from setuptools import setup
from Cython.Build import cythonize

setup(ext_modules=cythonize(["landscaping.py", "shrubbing.py"]))

Some things to note about this example:

• There is a cdef/@cclass class Shrubbery declaration in both shrubbing.pxd and shrubbing.pyx. When the shrubbing module is compiled, these two declarations are combined into one.

• In landscaping.pyx/landscaping.py, the cimport shrubbing declaration allows us to refer to the Shrubbery type as shrubbing.Shrubbery. But it doesn’t bind the name shrubbing in landscaping’s module namespace at run time, so to access shrubbing.standard_shrubbery() we also need to import shrubbing.

• One caveat if you use setuptools instead of distutils, the default action when running python setup.py install is to create a zipped egg file which will not work with cimport for pxd files when you try to use them from a dependent package. To prevent this, include zip_safe=False in the arguments to setup().

Versioning

.pxd files can be labelled with a minimum Cython version as part of their file name, similar to the version tagging of .so files in PEP 3149. For example a file called shrubbing.cython-30.pxd will only be found by cimport shrubbing on Cython 3.0 and higher. Cython will use the file tagged with the highest compatible version number.

Note that versioned files that are distributed across different directories will not be found. Only the first directory in the Python module search path in which a matching .pxd file is found will be considered.

The purpose of this feature is to allow third-party packages to release Cython interfaces to their packages that take advantage of the latest Cython features while not breaking compatibility for users with older versions of Cython. Users intending to use .pxd files solely within their own project need not produce these tagged files.