Python Ubuntu Docker

/ Comments off

Answer (1 of 3): Yes, you can try Alternatively, you can always do “docker pull nitincypher/docker. Use the below-given command. Docker run -it -name apacheserver -p 8080:80 ubuntu:latest. This will Download the ubuntu docker image and run it in interactive mode with port 8080 which will redirect to port 80 of the Apache2 webserver. This is known as Patting. Docker command to Download ubuntu container. Docker SDK for Python¶ A Python library for the Docker Engine API. It lets you do anything the docker command does, but from within Python apps – run containers, manage containers, manage Swarms, etc. For more information about the Engine API, see its documentation.

About one year ago I’ve created the Ubuntu 16.04 PFNE Docker image. It’s time for a new version of the Ubuntu PFNE Docker image to support Network engineers learn Python and test automation.

Recently, Ubuntu announced that on the Ubuntu Docker Hub the 18.04 LTS version is using the minimal image.

With this change when launching a Docker instance using

you’ll have an instance with the latest Minimal Ubuntu.

While this is great, especially if you need to quickly pull an image, the fact stays that it doesn’t have preinstalled the necessary tools to test network automation, learn Python or run some QoS tests using packages like IPerf.

Based on My previous Ubuntu 16.04 PFNE Docker image, I’ve created the same using the new Ubuntu 18.04 LTS minimal image.

It contains all the tools found in Ubuntu 16.04 PFNE:

  • Openssl
  • Net-tools (ifconfig..)
  • IPutils (ping, arping, traceroute…)
  • IProute
  • IPerf
  • TCPDump
  • NMAP
  • Python 2
  • Python 3
  • Paramiko (python ssh support)
  • Netmiko (python ssh support)
  • Ansible (automation)
  • Pyntc
  • NAPALM

and two new additions:

  • Netcat
  • Socat

I’ve added these two because some blog followers asked me, after reading the Ubuntu image for eve-ng – Python For Network Engineers post, if I can add to image servers installation like web, ftp, etc.

Personally, I don’t think is needed to burden the image with these extra packages. You can have tools like Netcat testing various servers. This is one of the reasons I’ve added Netcat and Socat.

It’s easy for me to add them to this image or future ones (and I’ll do it if I get more requests), however I’m planning some articles on how to do your own Docker images and add whatever packages you need.

While writing this post, time to push it to Docker Hub :)

If you want to test the new Ubuntu 18.04 PFNE Docker image, please pull it from Docker Hub:

To start it use:

Let me know if you find this useful, happy testing and most important Never Stop Learning!

$ docker run -rm -v /home/realpython/code:/app rp python /app/headlines.py Understanding Python Versions and Docker Using Docker Running Python in a Docker Container Conclusion Further Reading The option -v /home/realpython/code:/app says that the local directory /home/realpython/code should be mounted as /app inside the container. There are two methods for installing Docker on Ubuntu 16.04. One method involves installing it on an existing installation of the operating system. The other involves spinning up a server with a tool called Docker Machine that auto-installs Docker on. When you’re choosing a base image for your Docker image, Alpine Linux is often recommended. Using Alpine, you’re told, will make your images smaller and speed up your builds. And if you’re using Go that’s reasonable advice. But if you’re using Python, Alpine Linux will quite often: Make your builds much slower. Make your images bigger. Waste your time. On occassion, introduce obscure. Docker SDK for Python. A Python library for the Docker Engine API. It lets you do anything the docker command does, but from within Python apps – run containers, manage containers, manage Swarms, etc.

When you’re choosing a base image for your Docker image, Alpine Linux is often recommended.Using Alpine, you’re told, will make your images smaller and speed up your builds.And if you’re using Go that’s reasonable advice.

But if you’re using Python, Alpine Linux will quite often:

  1. Make your builds much slower.
  2. Make your images bigger.
  3. Waste your time.
  4. On occassion, introduce obscure runtime bugs.

Let’s see why Alpine is recommended, and why you probably shouldn’t use it for your Python application.

Why people recommend Alpine

Let’s say we need to install gcc as part of our image build, and we want to see how Alpine Linux compares to Ubuntu 18.04 in terms of build time and image size.

First, I’ll pull both images, and check their size:

As you can see, the base image for Alpine is much smaller.

Docker Ubuntu Python3

Next, we’ll try installing gcc in both of them.First, with Ubuntu:

Docker Ubuntu Python Command

Note: Outside the very specific topic under discussion, the Dockerfiles in this article are not examples of best practices, since the added complexity would obscure the main point of the article.

To ensure you’re writing secure, correct, fast Dockerfiles, consider my Python on Docker Production Handbook, which includes a packaging process and >70 best practices.

We can then build and time that:

Now let’s make the equivalent Alpine Dockerfile:

And again, build the image and check its size:

As promised, Alpine images build faster and are smaller: 15 seconds instead of 30 seconds, and the image is 105MB instead of 150MB.That’s pretty good!

But when we switch to packaging a Python application, things start going wrong.

Let’s build a Python image

We want to package a Python application that uses pandas and matplotlib.So one option is to use the Debian-based official Python image (which I pulled in advance), with the following Dockerfile:

And when we build it:

Python Ubuntu Docker

The resulting image is 363MB.

Can we do better with Alpine? Let’s try:

And now we build it:

What’s going on?

Standard PyPI wheels don’t work on Alpine

If you look at the Debian-based build above, you’ll see it’s downloading matplotlib-3.1.2-cp38-cp38-manylinux1_x86_64.whl.This is a pre-compiled binary wheel.Alpine, in contrast, downloads the source code (matplotlib-3.1.2.tar.gz), because standard Linux wheels don’t work on Alpine Linux.

Why?Most Linux distributions use the GNU version (glibc) of the standard C library that is required by pretty much every C program, including Python.But Alpine Linux uses musl, those binary wheels are compiled against glibc, and therefore Alpine disabled Linux wheel support.

Python Ubuntu Docker Free

Most Python packages these days include binary wheels on PyPI, significantly speeding install time.But if you’re using Alpine Linux you need to compile all the C code in every Python package that you use.

Which also means you need to figure out every single system library dependency yourself.In this case, to figure out the dependencies I did some research, and ended up with the following updated Dockerfile:

And then we build it, and it takes…

… 25 minutes, 57 seconds! And the resulting image is 851MB.

Here’s a comparison between the two base images:

Base imageTime to buildImage sizeResearch required
python:3.8-slim30 seconds363MBNo
python:3.8-alpine1557 seconds851MBYes

Alpine builds are vastly slower, the image is bigger, and I had to do a bunch of research.

Can’t you work around these issues?

Build time

For faster build times, Alpine Edge, which will eventually become the next stable release, does have matplotlib and pandas.And installing system packages is quite fast.As of January 2020, however, the current stable release does not include these popular packages.

Even when they are available, however, system packages almost always lag what’s on PyPI, and it’s unlikely that Alpine will ever package everything that’s on PyPI.In practice most Python teams I know don’t use system packages for Python dependencies, they rely on PyPI or Conda Forge.

Image size

Some readers pointed out that you can remove the originally installed packages, or add an option not to cache package downloads, or use a multi-stage build.One reader attempt resulted in a 470MB image.

So yes, you can get an image that’s in the ballpark of the slim-based image, but the whole motivation for Alpine Linux is smaller images and faster builds.With enough work you may be able to get a smaller image, but you’re still suffering from a 1500-second build time when they you get a 30-second build time using the python:3.8-slim image.

But wait, there’s more!

Alpine Linux can cause unexpected runtime bugs

While in theory the musl C library used by Alpine is mostly compatible with the glibc used by other Linux distributions, in practice the differences can cause problems.And when problems do occur, they are going to be strange and unexpected.

Some examples:

  1. Alpine has a smaller default stack size for threads, which can lead to Python crashes.
  2. One Alpine user discovered that their Python application was much slower because of the way musl allocates memory vs. glibc.
  3. I once couldn’t do DNS lookups in Alpine images running on minikube (Kubernetes in a VM) when using the WeWork coworking space’s WiFi.The cause was a combination of a bad DNS setup by WeWork, the way Kubernetes and minikube do DNS, and musl’s handling of this edge case vs. what glibc does.musl wasn’t wrong (it matched the RFC), but I had to waste time figuring out the problem and then switching to a glibc-based image.
  4. Another user discovered issues with time formatting and parsing.

Most or perhaps all of these problems have already been fixed, but no doubt there are more problems to discover.Random breakage of this sort is just one more thing to worry about.

Don’t use Alpine Linux for Python images

Unless you want massively slower build times, larger images, more work, and the potential for obscure bugs, you’ll want to avoid Alpine Linux as a base image.For some recommendations on what you should use, see my article on choosing a good base image.

When you’re building a Docker image for your Python application, you’re building on top of an existing image—and there are many possible choices.There are OS images like Ubuntu, and there are the many different variants of the python base image.

Which one should you use?Which one is better?There are many choices, and it may not be obvious which is the best for your situation.

Docker Ubuntu Python Dockerfile

So to help you make a choice that fits your needs, in this article I’ll go through some of the relevant criteria, and suggest some reasonable defaults that will work for most people.

What do you want from a base image?

There are a number of common criteria for choosing a base image, though your particular situation might emphasize, add, or remove some of these:

  • Stability: You want a build today to give you the same basic set of libraries, directory structure, and infrastructure as a build tomorrow, otherwise your application will randomly break.
  • Security updates: You want the base image to be well-maintained, so that you get security updates for the base operating system in a timely manner.
  • Up-to-date dependencies: Unless you’re building a very simple application, you will likely depend on operating system-installed libraries and applications (e.g. a compiler).You’d like them not to be too old.
  • Extensive dependencies: For some applications less popular dependencies may be required—a base image with access to a large number of libraries makes this easier.
  • Up-to-date Python: While this can be worked around by installing Python yourself, having an up-to-date Python available saves you some effort.
  • Small images: All things being equal, it’s better to have a smaller Docker image than a bigger Docker image.

The need for stability suggests not using operating systems with limited support lifetime, like Fedora or non-LTS Ubuntu releases.

Why you shouldn’t use Alpine Linux

A common suggestion for people who want small images is to use Alpine Linux, but that can lead to longer build times, obscure bugs, and performance issues.

You can see the linked article for details, but I recommend against using Alpine.

Option #1: Ubuntu LTS, RedHat Universal Base Image, Debian

There are three major operating systems that roughly meet the above criteria (dates and release versions are accurate at time of writing; the passage of time may require slightly different choices).

Ubuntu Python Docker-compose

  • Ubuntu 20.04 was released in April 2020, and since it’s a Long Term Support release it will get security updates until 2025.It’s usable in Docker via the ubuntu:20.04 image.
  • RedHat Enterprise Linux 8 was released in May 2019, and will have full updates until 2024 and maintenance updates until 2029.The RedHat Universal Base Image allows you to use it as a Docker base image.
  • Debian 10 (“Buster”) was released on July 2019, and will be supported until 2024.It’s usable in Docker via the debian:10 image.

Previous versions of this article covered CentOS, but CentOS is no longer a long-term stable operating system.

None of these operating systems includes the latest version of Python, Python 3.9, so you’ll have to install it yourself.

Option #2: The Python Docker image

Another alternative is Docker’s own “official” python image, which comes pre-installed with multiple versions of Python (3.7, 3.8, 3.9, etc.), and has multiple variants:

  • Alpine Linux, which as I explained above I don’t recommend using.
  • Debian Buster, with many common packages installed. The image itself is large, but the theory is that these packages are installed via common image layers that other official Docker images will use, so overall disk usage will be low.
  • Debian Buster slim variant. This lacks the common packages’ layers, and so the image itself is much smaller, but if you use many other Docker images based off Buster the overall disk usage will be somewhat higher.

For comparison purposes, the download size of python:3.9-slim-buster is 41MB, and python:3.9-alpine is 16MB.Their uncompressed on-disk sizes are 114MB and 44MB respectively.

So which should you use?

If you’re a RedHat shop, you’ll want to use their image.

Python-docker Ubuntu 20.04

Otherwise, as of January 2021 ubuntu:20.04 has the most up-to-date system packages.In practice, Debian’s packages won’t make much of a difference to most users, and the Debian-based official Python Docker images also give you the full range of Python releases.The base OS of Ubuntu 20.04 includes Python 3.8, but 3.9 is available via focal-updates packages so it is installable.However, as of early February 2021 it was on version 3.9.0, not 3.9.1, and getting pip installed was a pain.

The official Docker Python image in its slim variant—e.g. python:3.9-slim-buster—is a good base image for most use cases. it’s 41MB to download, 114MB when uncompressed to disk, it gives you the latest Python releases, it’s easy to use and it’s got all the benefits of Debian Buster.

If you care about performance, you’ll want to use ubuntu:20.04.Having run some benchmarks comparing multiple Python builds, it turns out that switching to Ubuntu 20.04 can give you a 20% performance boost.As such, if Python performance matters to you, I would recommend using the ubuntu:20.04 image.It’s a bit more annoying to set up, and for some reason gets updates less often, but it will be faster.