Building the Bootloader

PyInstaller comes with pre-compiled bootloaders for some platforms in the bootloader folder of the distribution folder. When there is no pre-compiled bootloader for the current platform (operating-system and word-size), the pip setup will attempt to build one.

If there is no precompiled bootloader for your platform, or if you want to modify the bootloader source, you need to build the bootloader. To do this,

• Download and install Python, which is required for running :command:waf,

• git clone or download the source (see the Download section on the web-site),

• cd into the folder where you cloned or unpacked the source to,

• cd bootloader, and

• make the bootloader with: python ./waf all,

• test the build by ref:running (parts of) the test-suite <running-the-test-suite>.

This will produce the bootloader executables for your current platform (of course, for Windows these files will have the .exe extension):

• ../PyInstaller/bootloader/OS_ARCH/run,

• ../PyInstaller/bootloader/OS_ARCH/run_d,

• ../PyInstaller/bootloader/OS_ARCH/runw (OS X and Windows only), and

• ../PyInstaller/bootloader/OS_ARCH/runw_d (OS X and Windows only).

The bootloaders architecture defaults to the machine’s one, but can be changed using the --target-arch= option – given the appropriate compiler and development files are installed. E.g. to build a 32-bit bootloader on a 64-bit machine, run:

python ./waf all --target-arch=32bit

If this reports an error, read the detailed notes that follow, then ask for technical help.

Supported platforms are

• GNU/Linux (using gcc)

• Windows (using Visual C++ or MinGW’s gcc)

• Mac OX X (using clang)

Contributed platforms are

• AIX (using gcc or xlc)

• HP-UX (using gcc or xlc)

• Solaris

For more information about cross-building please read on and mind the section about the virtual machines provided in the Vagrantfile.

Building for GNU/Linux

Development Tools

For building the bootloader you’ll need a development environment. You can run the following to install everything required:

• On Debian- or Ubuntu-like systems:

  sudo apt-get install build-essential
  

• On Fedora, RedHat and derivates:

  sudo yum groupinstall "Development Tools"
  

• For other Distributions please consult the distributions documentation.

Now you can build the bootloader as shown above.

Alternatively you may want to use the linux64 build-guest provided by the Vagrantfile (see below).

Building Linux Standard Base (LSB) compliant binaries (optional)

By default, the bootloaders on GNU/Linux are ”normal“, non-LSB binaries, which should be fine for all GNU/Linux distributions.

If for some reason you want to build Linux Standard Base (LSB) compliant binaries [1], you can do so by specifying --lsb on the waf command line, as follows:

python ./waf distclean all --lsb

LSB version 4.0 is required for successfully building of bootloader. Please refer to python ./waf --help for further options related to LSB building.

[1]

Linux Standard Base (LSB) is a set of open standards that should increase compatibility among GNU/Linux distributions. Unfortunately it is not widely adopted and both Debian and Ubuntu dropped support for LSB in autumn 2015. Thus PyInstaller bootloader are no longer provided as LSB binary.

Building for Mac OS X

On Mac OS X please install Xcode, Apple’s suite of tools for developing software for Mac OS X. This will get you the clang compiler. Any version suitable for your platform should be fine. Xcode can be also installed from your Mac OS X Install DVD.

Now you can build the bootloader as shown above.

Alternatively you may want to use the darwin64 build-guest provided by the Vagrantfile (see below).

Cross-Building for Mac OS X

For cross-compiling for OS X you need the Clang/LLVM compiler, the cctools (ld, lipo, …), and the OSX SDK. Clang/LLVM is a cross compiler by default and is available on nearly every GNU/Linux distribution, so you just need a proper port of the cctools and the OS X SDK.

This is easy to get and needs to be done only once and the result can be transferred to you build-system. The build-system can then be a normal (somewhat current) GNU/Linux system. [2]

[2]

Please keep in mind that to avoid problems, the system you are using for the preparation steps should have the same architecture (and possible the same GNU/Linux distribution version) as the build-system.

Preparation: Get SDK and Build-tools

For preparing the SDK and building the cctools, we use the very helpful scripts from the OS X Cross <https://github.com/tpoechtrager/osxcross> toolchain. If you re interested in the details, and what other features OS X Cross offers, please refer to it’s homepage.

Side-note: For actually accessing the OS X disk image file (.dmg), darling-dmg is used. It allows mounting .dmg s under GNU/Linux via FUSE.

For saving you reading OSXCross’ documentation we prepared a virtual box description performing all required steps. If you are interested in the precise commands, please refer to packages_osxcross_debianoid, prepare_osxcross_debianiod, and build_osxcross in the Vagrantfile.

Please proceed as follows:

1. Download XCode 7.3.x <https://developer.apple.com/downloads/index.action?name=Xcode%207.3 and save it to bootloader/sdks/osx/. You will need to register an Apple ID, for which you may use a disposable e-mail-address, to search and download the files.

   Please make sure that you are complying to the license of the respective package.

2. Use the Vagrantfile to automatically build the SDK and tools:

   vagrant up build-osxcross && vagrant halt build-osxcross
   

   This should create the file bootloader/sdks/osx/osxcross.tar.xz, which will then be installed on the build-system.

   If for some reason this fails, try running vagrant provision build-osxcross.

3. This virtual machine is no longer used, you may now want to discard it using vagrant destroy build-osxcross.

Building the Bootloader

Again, simply use the Vagrantfile to automatically build the OS X bootloaders:

export TARGET=OSX  # make the Vagrantfile build for OS X
vagrant up linux64 && vagrant halt linux

This should create the bootloaders in * ../PyInstaller/bootloader/Darwin-*/.

If for some reason this fails, try running vagrant provision linux64.

3. This virtual machine is no longer used, you may now want to discard it using:

   vagrant destroy build-osxcross
   

4. If you are finished with the OS X bootloaders, unset TARGET again:

   unset TARGET
   

If you don’t want to use the build-guest provided by the Vagrant file, perform the following steps (see build_bootloader_target_osx in the Vagrantfile):

mkdir -p ~/osxcross
tar -C ~/osxcross --xz -xf /vagrant/sdks/osx/osxcross.tar.xz
PATH=~/osxcross/bin/:$PATH
python ./waf all CC=x86_64-apple-darwin15-clang
python ./waf all CC=i386-apple-darwin15-clang

Building for Windows

The pre-compiled bootloader coming with PyInstaller are self-contained static executable that imposes no restrictions on the version of Python being used.

When building the bootloader yourself, you have to carefully choose between three options:

1. Using the Visual Studio C++ compiler.

   This allows creating self-contained static executables, which can be used for all versions of Python. This is why the bootloaders delivered with PyInstaller are build using Visual Studio C++ compiler.

   You can use any Visual Studio version that is convenient (as long as it’s supported by the waf build-tool).

2. Using the MinGW-w64 suite.

   This allows to create smaller, dynamically linked executables, but requires to use the same level of Visual Studio [3] as was used to compile Python. So this bootloader will be tied to a specific version of Python.

   The reason for this is, that unlike Unix-like systems, Windows doesn’t supply a system standard C library, leaving this to the compiler. But Mingw-w64 doesn’t have a standard C library. Instead it links against msvcrt.dll, which happens to exist on many Windows installations – but i not guaranteed to exist.

[3]

This description seems to be technically incorrect. I ought to depend on the C++ run-time library. If you know details, please open an issue.

3. Using cygwin and MinGW.

   This will create executables for cygwin, not for ‘plain’ Windows.

In all cases you may want

• to set the path to include python, e.g. set PATH=%PATH%;c:\python35,

• to peek into the Vagrantfile or ../appveyor.yml to learn how we are building.

You can also build the bootloaders for cygwin.

Build using Visual Studio C++

• With our wscript file, you don’t need to run vcvarsall.bat to ’switch’ the environment between VC++ installations and target architecture. The actual version of C++ does not matter and the target architecture is selected by using the --target-arch= option.

• If you are not using Visual Studio for other work, installing only the standalone C++ build-tools might be the best option as it avoids bloating your system with stuff you don’t need (and saves a lot if installation time).

  Hint

  We recommend installing the build-tools software using the chocolatey package manager. While at a first glance it looks like overdose, this is the easiest way to install the C++ build-tools. It comes down to two lines in an administrative powershell:

  … one-line-install as written on the chocolatey homepage
  choco install -y python vcbuildtools
  

• Useful Links:

  • Microsoft Visual C++ Build-Tools 2015

  • Microsoft Build-Tools for Visual Studio 2017.

After installing the C++ build-tool you can build the bootloader as shown above.

Build using MinGW-w64

Please be aware of the restrictions mentioned above.

If Visual Studio is not convenient, you can download and install the MinGW distribution from one of the following locations:

• MinGW-w64 required, uses gcc 4.4 and up.

• TDM-GCC - MinGW (not used) and MinGW-w64 installers

Note: Please mind that using cygwin’s python or MinGW when running ./waf will create executables for cygwin, not for Windows.

On Windows, when using MinGW-w64, add PATH_TO_MINGWbin to your system PATH. variable. Before building the bootloader run for example:

set PATH=C:\MinGW\bin;%PATH%

Now you can build the bootloader as shown above. If you have installed both Visual C++ and MinGW, you might need to add run python ./waf --gcc all.

Build using cygwin and MinGW

Please be aware that this will create executables for cygwin, not for ‘plain’ Windows.

Use cygwin’s setup.exe to install python and mingw.

Now you can build the bootloader as shown above.

Vagrantfile Virtual Machines

PyInstaller maintains a set of virtual machine description for testing and (cross-) building. For managing these boxes, we use vagrant.

All guests [4] will automatically build the bootloader when running vagrant up GUEST or vagrant provision GUEST. They will build both 32- and 64-bit bootloaders.

[4]

Except of guest osxcross, which will build the OS X SDK and cctools as described in section Cross-Building for Mac OS X.

All guests (except of darwin64), when building the bootloaders, are sharing the PyInstaller distribution folder and will put the built executables onto the build-host (into ../PyInstaller/bootloader/).

Most boxes requires two Vagrant plugins to be installed:

vagrant plugin install vagrant-reload vagrant-scp

Example usage:

vagrant up linux64      # will also build the bootloader
vagrant halt linux64    # or `destroy`

# verify the bootloader has been rebuild
git status ../PyInstaller/bootloader/

You can pass some parameters for configuring the Vagrantfile by setting environment variables, like this:

GUI=1 TARGET=OSX vagrant up darwin64

or like this:

export TARGET=OSX
vagrant provision linux64

We currently provide this guests:

linux64:

GNU/Linux (some recent version) used to build the GNU/Linux bootloaders.

• If TARGET=OS is set, cross-builds the bootloaders for OS X (see Cross-Building for Mac OS X).

• If TARGET=WINDOWS is set, cross-builds the bootloaders for Windows using mingw. Please have in mind that this imposes the restrictions mentioned above.

• Otherwise (which is the default) bootloaders for GNU/Linux are build.

darwin64:

Mac OS X ‘Yosemite’ – not actually used by the PyInstaller team, but provided for testing.

This guest, when building the bootloaders, does not put the built executables onto the build-host. You need to fetch them using:

vagrant plugin install vagrant-scp vagrant-reload # required only once
vagrant scp -a darwin64:/vagrant/PyInstaller/bootloader/Darwin-* \
               ../PyInstaller/bootloader/

This is due the fact that this machine doesn’t include the Virtualbox guest additions and thus doesn’t support shared folders.

windows10:

Windows 10, used for building the Windows bootloaders using Visual C++.

• If MINGW=1 is set, the bootloaders will be build using MinGW. Please be aware of the restrictions mentioned above.

Note

The Windows box uses password authentication, so in some cases you need to enter the password (which is Passw0rd!).

build-osxcross:

GNU/Linux guest used to build the OS X SDK and cctools as described in section Cross-Building for Mac OS X.