14

I have a fresh install of Debian 9.8, the current stable release version.

$ lsb_release --description
Description:    Debian GNU/Linux 9.8 (stretch)

This system includes Python 3.5.3:

$ python3 --version
Python 3.5.3

Unfortunately that version doesn't meet my needs. I want a more current version of Python. At the time of writing, the current version is 3.7.2.

Hopefully this will be helpful for a range of versions, but in the interest of limiting the scope to something we can be very precise about, this question is about the above combination of OS and python versions.

To further specify: I want to build from the python git source repository, using the default configuration: we'll run ./configure with no arguments except maybe --prefix to choose a custom install location. I want any needed dependencies to come from the standard Debian apt sources, if possible.

Finally and most important: For any additional software we install as a prerequisite, give some information about it. Don't just say “install this incomprehensible list of apt packages”.

For any dependency available from Debian's stretch repositories, I want the following info:

  • Whether the dependency can be considered optional or must be treated as required in order to successfully complete python's build/install steps.

  • A link to the project homepage (or next next best thing if there isn't a suitable homepage, e.g. a repo url) that apt package came from.

  • What feature/module in the resulting installation does the dependency map to? Sometimes this will be obvious because the names will be similar or identical between project name, package name, and the module it is used to build, but sometimes it won't.

With that kind of information, anyone reading this can have some understanding of what they are installing and why. The task of determining which dependencies their project will need will hopefully be made significantly easier.

  • Build instructions are in the documentation. Everything else is in the man pages or package info of what you're installing, – OrangeDog Mar 11 '19 at 12:54
  • 1
    The pages and solutions you guys are pointing to don't provide any info about the apt dependencies, which was the main thing my question is asking for. – Charlie Mar 11 '19 at 13:31
  • 1
    Everything you're asking for is in the package info: apt-cache show PKGNAME – OrangeDog Mar 11 '19 at 13:33
  • 1
    "The task of determining which dependencies their project will need will hopefully be made significantly easier." - just include them all, same as the system package does. – OrangeDog Mar 11 '19 at 13:40
  • 1
    @OrangeDog could you show me how? superuser.com/questions/1413089/… – Charlie Mar 11 '19 at 13:46
23

Install git

$ sudo apt install git

Install Python build dependencies

Python includes a set of modules that it builds by linking to other popular open source projects. Many of those projects are included in Debian by default as executable programs, but the development files (headers, libraries) necessary to link them into Python are packaged separately and not included in a default Debian install.

Some of these libraries are actually required for the python build/install steps to complete successfully:

| Library                            | Python Module | Dev Package  |
| ---------------------------------- | ------------- | ------------ |
| https://www.zlib.net/              | `zlib`        | `zlib1g-dev` |
| https://www.sourceware.org/libffi/ | `_ctypes`     | `libffi-dev` |

The entries in the 'Dev Package' column are the names of packages containing the development files our python build needs.

Each of these 'dev' packages has a corresponding binary package that is probably already installed. So you're likely not going to be adding new pieces of software to your system by installing these. You're just installing the files necessary to compile new pieces of software (in our case a newer python) so that it can use them. (Also note that these 'dev' packages all have their corresponding binary packages as dependencies, so installing a dev package will ensure that its binary package is also installed.)

So let's install them:

sudo apt install zlib1g-dev libffi-dev

Next we have the OpenSSL library. Python considers OpenSSL optional, but you probably want it. For example, even using Python's package installation tools can run into trouble when fetching https urls if SSL/TLS support is missing.

| Library                  | Python Module | Dev Package |
| ------------------------ | ------------- | ----------- |
| https://www.openssl.org/ | `_ssl`        | `libssl-dev |

Let's install it:

sudo apt install libssl-dev

The next set of packages fall more in the 'optional' category:

| Library                                              | Python Module(s)              | Dev Package        |
| ---------------------------------------------------- | ----------------------------- | ------------------ |
| http://www.bzip.org/                                 | `_bz2`                        | `libbz2-dev`       |
| https://www.gnu.org/software/ncurses/                | `_curses` and `_curses_panel` | `libncursesw5-dev` |
| https://www.gnu.org.ua/software/gdbm/                | `_dbm` and `_gdbm`            | `libgdbm-dev`      |
| https://tukaani.org/xz/                              | `_lzma`                       | `liblzma-dev`      |
| https://www.sqlite.org/                              | `_sqlite3`                    | `libsqlite3-dev`   |
| https://www.tcl.tk/software/tcltk/                   | `_tkinter`                    | `tk-dev`           |
| https://github.com/karelzak/util-linux               | `_uuid`                       | `uuid-dev`         |
| https://tiswww.case.edu/php/chet/readline/rltop.html | `readline`                    | `libreadline-dev`  |

Python can build and install without these, and your applications might not need them. On the other hand, if you install them, the 'make' step shouldn't report any modules it was unable to build, and the binaries are already included in a default Debian install.

The command to copy-paste if you'd like to go ahead and install all of the above 'optional' packages is:

sudo apt install libbz2-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev tk-dev uuid-dev libreadline-dev

Build and install python from source

  1. clone the python git repository.
$ git clone git@github.com:python/cpython.git
Cloning into 'cpython'...
[...]
Resolving deltas: 100% (592215/592215), done.
  1. Checkout the version you want to install (in this case, 3.7.2) using the git tag name.
$ cd cpython/
cpython$ git checkout v3.7.2
Note: checking out 'v3.7.2'.
[...]
HEAD is now at 9a3ffc0492... 3.7.2final
cpython$
  1. Choose an install prefix. That is, the path the compiled and linked project will be installed into. In this case I'll use $HOME/python/v3.7.2. I'm including the version number since I'll want to install other versions in the future, and keep them separate. Run the configure script with this path as the --prefix argument. (If you don't provide a --prefix argument, it will default to /usr/local).
cpython$ ./configure --prefix=$HOME/python/v3.7.2
[...]
checking for inflateCopy in -lz... yes
[...]
checking for openssl/ssl.h in /usr... yes
[...]

The configure script will check a bunch of things, many of which are unnecessary. The ones shown in the above output, though, can be considered necessary.

  1. Build python by running make. If we included the minimum set of dependencies necessary to build and install python, the output will tell you near the end what modules were not built:
cpython$ make
[...]
Python build finished successfully!
The necessary bits to build these optional modules were not found:
_bz2                  _curses               _curses_panel      
_dbm                  _gdbm                 _lzma              
_sqlite3              _tkinter              _uuid              
readline                                                       
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

The make output describes this as a list of 'optional' modules but again, at least for this particular version of python, it probably won't build and install successfully without the zlib, and _ctypes modules.

  1. Install python by running make install. We've already provided the install location back in the configure step. If your install location is a root-owned directory (e.g. /usr/local), prefix this command with sudo. Since in this case I'm installing to a user-owned directory, I don't want to do that.
cpython$ make install
Creating directory /home/python/v3.7.2/bin
Creating directory /home/python/v3.7.2/lib
[...]

  1. You should now have an executable you can run from the bin directory underneath the install prefix named python3. (Note that this installer doesn't put anything at bin/python; just bin/python3.)
$ $HOME/python/v3.7.2/bin/python3
Python 3.7.2 (v3.7.2:9a3ffc0492, Mar 10 2019, 19:35:56) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()
$
|improve this answer|||||
  • 1
    On Debian 10, building the optional _dbm module also requires installing libgdbm-compat-dev. – Louis Sep 27 '19 at 14:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.