Install with Docker

Want the easiest way to start contributing to AMO? Try our Docker-based development environment.

First you’ll need to install Docker. Please read their docs for the installation steps specific to your operating system.

There are two options for running docker depending on the platform you are running.

  • Run docker on the host machine directly (recommended)
    • Update default settings in Docker Desktop - we suggest increasing RAM limit to at least 4 GB in the Resources/Advanced section and click on “Apply and Restart”.

  • Run docker-machine which will run docker inside a virtual-machine

Historically Mac and Windows could only run Docker via a vm. That has recently changed with the arrival of docker-for-mac and docker-for-windows.

If your platform can run Docker directly either on Linux, with docker-for-mac or docker-for-windows then this is the easiest way to run addons-server.

If you have problems, due to not meeting the minimum specifications for docker-for-windows or you’d prefer to keep running docker-machine vms then docker-machine will still work just fine. See the docs for creating the vm here Creating the docker-machine vm

Note

If you’re on a Mac and already have a working docker-machine setup you can run that and docker-for-mac (but not docker-for-windows) side by side. The only caveat is it’s recommended that you keep the versions of Docker on the vm and the host in-sync to ensure compatibility when you switch between them.

Setting up the containers

Note

docker-toolbox, docker-for-mac and docker-for-windows will install docker-compose for you. If you’re on Linux and you need it, you can install it manually with:

pip install docker-compose

Note

On Windows, ensure that Docker Desktop is running as Linux Container. You can double-check this by ensuring you see “Switch to Windows containers…” in the resulting context menu when right-clicking on the Docker Desktop icon in the task-bar.

For more information see switching docker containers.

Failure to do so will result in errors in later steps like make initialize:

ValueError: Unable to configure handler 'statsd': [Errno -2] Name or service not known
Makefile-docker:71: recipe for target 'initialize_db' failed
make: *** [initialize_db] Error 1

Next once you have Docker up and running follow these steps on your host machine:

# Checkout the addons-server sourcecode.
git clone git://github.com/mozilla/addons-server.git
cd addons-server
# Download the containers
docker-compose pull  # Can take a while depending on your internet bandwidth.
# Start up the containers
docker-compose up -d
make initialize  # Answer yes, and create your superuser when asked.
# On Windows you can substitute `make initialize` for the command:
docker-compose exec web make initialize

Note

Docker requires the code checkout to exist within your home directory so that Docker can mount the source-code into the container.

Accessing the web server

By default our docker-compose config exposes the web-server on port 80 of localhost.

We use olympia.test as the default hostname to access your container server (e.g. for Firefox Accounts). To be able access the development environment using http://olympia.test you’ll need to edit your /etc/hosts file on your native operating system. For example:

[ip-address]  olympia.test

Typically the IP address is localhost (127.0.0.1) but if you’re using docker-machine see Accessing the web-server with docker-machine for details of how to get the ip of the Docker vm.

By default we configure OLYMPIA_SITE_URL to point to http://olympia.test.

If you choose a different hostname you’ll need to set that environment variable and restart the Docker containers:

docker-compose stop # only needed if running
export OLYMPIA_SITE_URL=http://[YOUR_HOSTNAME}
docker-compose up -d

Running common commands

Run the tests using make, outside of the Docker container:

make test
# or
docker-compose exec web pytest src/olympia/

You can run commands inside the Docker container by sshing into it using:

make shell
# or
docker-compose exec web bash

Then to run the tests inside the Docker container you can run:

pytest

You can also run single commands from your host machine without opening a shell on each container. Here is an example of running the pytest command on the web container:

docker-compose run web pytest

If you’d like to use a python debugger to interactively debug Django view code, check out the Debugging section.

Note

If you see an error like No such container: addonsserver_web_1 and your containers are running you can overwrite the base name for docker containers with the COMPOSE_PROJECT_NAME environment variable. If your container is named localaddons_web_1 you would set COMPOSE_PROJECT_NAME=localaddons.

Updating your containers

Any time you update Olympia (e.g., by running git pull), you should make sure to update your Docker image and database with any new requirements or migrations:

docker-compose stop
docker-compose pull
docker-compose up -d
make update_docker  # Runs database migrations and rebuilds assets.
# On Windows you can substitute `make update_docker` for the following two commands:
docker-compose exec worker make update_deps
docker-compose exec web make update

Gotchas!

Here’s a list of a few of the issues you might face when using Docker.

Can’t access the web server?

Check you’ve created a hosts file entry pointing olympia.test to the relevant IP address.

If containers are failing to start use docker-compose ps to check their running status.

Another way to find out what’s wrong is to run docker-compose logs.

Getting “Programming error [table] doesn’t exist”?

Make sure you’ve run the make initialize step as detailed in the initial setup instructions.

ConnectionError during initialize (elasticsearch container fails to start)

When running make initialize without a working elasticsearch container, you’ll get a ConnectionError. Check the logs with docker-compose logs. If elasticsearch is complaining about vm.max_map_count, run this command on your computer or your docker-machine VM:

sudo sysctl -w vm.max_map_count=262144

This allows processes to allocate more memory map areas.

Connection to elasticsearch timed out (elasticsearch container exits with code 137)

docker-compose up -d brings up all containers, but running make initialize causes the elasticsearch container to go down. Running docker-compose ps shows Exited (137) against it.

Update default settings in Docker Desktop - we suggest increasing RAM limit to at least 4 GB in the Resources/Advanced section and click on “Apply and Restart”.

Port collisions (nginx container fails to start)

If you’re already running a service on port 80 or 8000 on your host machine, the nginx container will fail to start. This is because the docker-compose.override.yml file tells nginx to listen on port 80 and the web service to listen on port 8000 by default.

This problem will manifest itself by the services failing to start. Here’s an example for the most common case of nginx not starting due to a collision on port 80:

ERROR: for nginx  Cannot start service nginx:.....
...Error starting userland proxy: Bind for 0.0.0.0:80: unexpected error (Failure EADDRINUSE)
ERROR: Encountered errors while bringing up the project.

You can check what’s running on that port by using (sudo is required if you’re looking at port < 1024):

sudo lsof -i :80

We specify the ports nginx listens on in the docker-compose.override.yml file. If you wish to override the ports you can do so by creating a new docker-compose config and starting the containers using that config alongside the default config.

For example if you create a file called docker-compose-ports.yml:

nginx:
  ports:
    - 8880:80

Next you would stop and start the containers with the following:

docker-compose stop # only needed if running
docker-compose -f docker-compose.yml -f docker-compose-ports.yml up -d

Now the container nginx is listening on 8880 on the host. You can now proxy to the container nginx from the host nginx with the following nginx config:

server {
    listen       80;
    server_name  olympia.test;
    location / {
        proxy_pass   http://olympia.test:8880;
    }
}

Persisting changes

Please note: any command that would result in files added or modified outside of the addons-server folder (e.g. modifying pip or npm dependencies) won’t persist, and thus won’t survive after the running container exits.

Note

If you need to persist any changes to the image, they should be carried out via the Dockerfile. Commits to master will result in the Dockerfile being rebuilt on the Docker hub.

Restarting docker-machine vms following a reboot

If you quit docker-machine, or restart your computer, docker-machine will need to start again using:

docker-machine start addons-dev

You’ll then need to export the variables again, and start the services:

docker-compose up -d

Hacking on the Docker image

If you want to test out changes to the Olympia Docker image locally, use the normal Docker commands such as this to build a new image:

cd addons-server
docker build -t addons/addons-server .
docker-compose up -d

After you test your new image, commit to master and the image will be published to Docker Hub for other developers to use after they pull image changes.