Considerations for Production Deployments

This document contains a number of suggestions for deploying MediaGoblin in actual production environments. Consider “Deploying MediaGoblin” for a basic overview of how to deploy MediaGoblin.

Deploy with paste

The MediaGoblin WSGI application instance you get with ./lazyserver.sh is not ideal for a production MediaGoblin deployment. Ideally, you should be able to use a systemd service file or an init script to launch and restart the MediaGoblin process.

We will explore setting up MediaGoblin systemd service files and init scripts, but first we need to create the directory that will store the MediaGoblin logs.

Create the directory for your log file:

Production logs for the MediaGoblin application are kept in the /var/log/mediagoblin directory. Create the directory and give it the proper permissions:

sudo mkdir -p /var/log/mediagoblin && sudo chown -hR mediagoblin:mediagoblin /var/log/mediagoblin

Use systemd service files

If your operating system uses systemd, you can use systemd service files to manage both the Celery and Paste processes. Place the following service files in the /etc/systemd/system/ directory.

The first file should be named mediagoblin-celeryd.service. Be sure to modify it to suit your environment’s setup:

# Set the WorkingDirectory, Environment and ExecStart values to match your environment.
# If using Debian/*buntu, mkdir and chown are located in /bin/mkdir and /bin/chown, respectively.
# If using Fedora/CentOS/Red Hat, mkdir and chown are located in /usr/bin/mkdir and /usr/bin/chown, respectively.

[Unit]
Description=Mediagoblin Celeryd

[Service]
User=mediagoblin
Group=mediagoblin
Type=simple
WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin
# Create directory for PID (if needed) and set ownership
ExecStartPre=/bin/mkdir -p /run/mediagoblin
ExecStartPre=/bin/chown -hR mediagoblin:mediagoblin /run/mediagoblin
# Celery process will run as the `mediagoblin` user after start.
Environment=MEDIAGOBLIN_CONFIG=/srv/mediagoblin.example.org/mediagoblin/mediagoblin_local.ini \
            CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery
ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/celery worker \
              --logfile=/var/log/mediagoblin/celery.log \
              --loglevel=INFO
PIDFile=/run/mediagoblin/mediagoblin-celeryd.pid

[Install]
WantedBy=multi-user.target

The second file should be named mediagoblin-paster.service:

# Set the WorkingDirectory, Environment and ExecStart values to match your environment.
# If using Debian/*buntu, mkdir and chown are located in /bin/mkdir and /bin/chown, respectively.
# If using Fedora/CentOS/Red Hat, mkdir and chown are located in /usr/bin/mkdir and /usr/bin/chown, respectively.
[Unit]
Description=Mediagoblin

[Service]
Type=forking
User=mediagoblin
Group=mediagoblin
Environment=CELERY_ALWAYS_EAGER=false
WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin
# Start mg-paster process as root, then switch to mediagoblin user/group
PermissionsStartOnly=true
ExecStartPre=-/bin/mkdir -p /run/mediagoblin
ExecStartPre=/bin/chown -hR mediagoblin:mediagoblin /run/mediagoblin

ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
              /srv/mediagoblin.example.org/mediagoblin/paste_local.ini \
              --pid-file=/var/run/mediagoblin/mediagoblin.pid \
              --log-file=/var/log/mediagoblin/mediagoblin.log \
              --daemon \
              --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
ExecStop=/srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
             --pid-file=/var/run/mediagoblin/mediagoblin.pid \
             /srv/mediagoblin.example.org/mediagoblin/paste_local.ini stop
PIDFile=/var/run/mediagoblin/mediagoblin.pid

[Install]
WantedBy=multi-user.target

Enable these processes to start at boot by entering:

sudo systemctl enable mediagoblin-celeryd.service && sudo systemctl enable mediagoblin-paster.service

Start the processes for the current session with:

sudo systemctl start mediagoblin-paster.service
sudo systemctl start mediagoblin-celeryd.service

If either command above gives you an error, you can investigate the cause of the error by entering:

sudo systemctl status mediagoblin-celeryd.service  or
sudo systemctl status mediagoblin-paster.service

The above systemctl status command is also useful if you ever want to confirm that a process is still running. If you make any changes to the service files, you can reload the service files by entering:

sudo systemctl daemon-reload

After entering that command, you can attempt to start the Celery or Paste processes again.

Use an init script

If your system does not use systemd, you can use the following command as the basis for an init script:

CELERY_ALWAYS_EAGER=true \
 /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
 /srv/mediagoblin.example.org/mediagoblin/paste.ini \
 --pid-file=/var/run/mediagoblin.pid \
 --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543

The above configuration places MediaGoblin in “always eager” mode with Celery, this means that submissions of content will be processed synchronously, and the user will advance to the next page only after processing is complete. If we take Celery out of “always eager mode,” the user will be able to immediately return to the MediaGoblin site while processing is ongoing. In these cases, use the following command as the basis for your script:

CELERY_ALWAYS_EAGER=false \
 /srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
 /srv/mediagoblin.example.org/mediagoblin/paste.ini \
 --pid-file=/var/run/mediagoblin.pid \
 --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543

Members of the MediaGoblin community have provided init scripts for the following GNU/Linux distributions:

Debian
Arch Linux

You can reference these scripts to create an init script for your own operating system. Similar scripts will be in your system’s /etc/init.d/ or /etc/rc.d/ directory, but the specifics of an init script will vary from one distribution to the next.

Separate celery

MediaGoblin uses Celery to handle heavy and long-running tasks. Celery can be launched in two ways:

  1. Embedded in the MediaGoblin WSGI application [1]. This is the way ./lazyserver.sh does it for you. It’s simple as you only have to run one process. The only bad thing with this is that the heavy and long-running tasks will run in the webserver, keeping the user waiting each time some heavy lifting is needed as in for example processing a video. This could lead to problems as an aborted connection will halt any processing and since most front-end web servers will terminate your connection if it doesn’t get any response from the MediaGoblin WSGI application in a while.

  2. As a separate process communicating with the MediaGoblin WSGI application via a broker. This offloads the heavy lifting from the MediaGoblin WSGI application and users will be able to continue to browse the site while the media is being processed in the background.

To launch Celery separately from the MediaGoblin WSGI application:

  1. Make sure that the CELERY_ALWAYS_EAGER environment variable is unset or set to false when launching the MediaGoblin WSGI application.

  2. Start the celeryd main process with

    CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd

If you use our example systemd service files, Celery will be set to the “CELERY_ALWAYS_EAGER=false” value by default. This will provide your users with the best user experience, as all media processing will be done in the background.

Set up sentry to monitor exceptions

We have a plugin for raven integration, see the “raven plugin” documentation.