Deploying MediaGoblin¶
This deployment guide will take you step-by-step through setting up your own instance of MediaGoblin.
MediaGoblin most likely isn’t yet available from your operating system’s package manager, however, a basic install isn’t too complex in and of itself. We recommend a setup that combines MediaGoblin, virtualenv and Nginx on a .deb or .rpm-based GNU/Linux distribution.
Experts may of course choose other deployment options, including Apache. See our Deployment wiki page for for more details. Please note that we are not able to provide support for these alternative deployment options.
Note
These tools are for site administrators wanting to deploy a fresh install. If you want to join in as a contributor, see our Hacking HOWTO instead.
Note
Throughout the documentation we use the sudo command to indicate that
an instruction requires elevated user privileges to run. You can issue
these commands as the root user if you prefer.
If you need help configuring sudo, see the
Debian wiki or the
Fedora Project wiki.
Prepare System¶
Dependencies¶
MediaGoblin has the following core dependencies:
Python 3.7+
SQLite or PostgreSQL
Python Imaging Library (PIL or Pillow)
These instructions have been tested on Debian 11, Debian 12, Ubuntu 20.04 LTS, Ubuntu 22.04 LTS and Fedora 39.
Issue the following commands:
# Debian
sudo apt update
sudo apt install automake git nodejs npm python3-dev \
python3-gst-1.0 python3-lxml python3-pil virtualenv
# Fedora
sudo dnf install automake gcc git-core make nodejs npm \
libffi-devel python3-devel python3-lxml python3-pillow \
virtualenv
For a production deployment, you’ll also need Nginx as frontend web server and RabbitMQ to store the media processing queue:
# Debian
sudo apt install nginx-light rabbitmq-server
# Fedora
sudo dnf install nginx rabbitmq-server
Configure PostgreSQL¶
Note
MediaGoblin currently supports PostgreSQL and SQLite. The default is a local SQLite database. This will “just work” for small deployments. For medium to large deployments we recommend PostgreSQL. If you don’t want/need PostgreSQL, skip this section.
These are the packages needed for PostgreSQL:
# Debian
sudo apt install postgresql python3-psycopg2
# Fedora
sudo dnf install postgresql postgresql-server python3-psycopg2
Fedora also requires that you initialize and start the PostgreSQL database with a few commands. The following commands are not needed on a Debian-based platform, however:
# Fedora
sudo /usr/bin/postgresql-setup initdb
sudo systemctl enable postgresql
sudo systemctl start postgresql
The installation process will create a new system user named postgres,
which will have privileges sufficient to manage the database. We will create a
new database user with restricted privileges and a new database owned by our
restricted database user for our MediaGoblin instance.
In this example, the database user will be mediagoblin and the database
name will be mediagoblin too. We’ll first at the user:
sudo --login --user=postgres createuser --no-createdb mediagoblin
Then we’ll create the database where all of our MediaGoblin data will be stored:
sudo --login --user=postgres createdb --encoding=UTF8 --owner=mediagoblin mediagoblin
Caution
Where is the password?
These steps enable you to authenticate to the database in a password-less manner via local UNIX authentication provided you run the MediaGoblin application as a user with the same name as the user you created in PostgreSQL.
More on this in Drop Privileges for MediaGoblin.
Drop Privileges for MediaGoblin¶
MediaGoblin does not require special permissions or elevated access to run. As such, the preferred way to run MediaGoblin is to create a dedicated, unprivileged system user for the sole purpose of running MediaGoblin. Running MediaGoblin processes under an unprivileged system user helps to keep it more secure.
The following command will create a system account with a username of
mediagoblin.
If you are using a Debian-based system, enter this command:
# Debian
sudo useradd --system --create-home --home-dir /var/lib/mediagoblin \
--group www-data --comment 'GNU MediaGoblin system account' mediagoblin
# Fedora
sudo useradd --system --create-home --home-dir /var/lib/mediagoblin \
--group nginx --comment 'GNU MediaGoblin system account' mediagoblin
This will create a mediagoblin user and assign it to a group that is
associated with the web server. This will ensure that the web server can
read the media files that users upload (images, videos, etc.)
Many operating systems will automatically create a group
mediagoblin to go with the new user mediagoblin, but just to
be sure:
sudo groupadd --force mediagoblin
sudo usermod --append --groups mediagoblin mediagoblin
No password will be assigned to this account, and you will not be able to log in as this user. To switch to this account, enter:
sudo su mediagoblin --shell=/bin/bash
To return to your regular user account after using the system account, type
exit or Ctrl-d.
Create a MediaGoblin Directory¶
You should create a working directory for MediaGoblin. This document
assumes your local git repository will be located at
/srv/mediagoblin.example.org/mediagoblin/.
Substitute your preferred local deployment path as needed.
Setting up the working directory requires that we first create the directory with elevated privileges, and then assign ownership of the directory to the unprivileged system account.
To do this, enter the following commands, changing the defaults to suit your particular requirements:
# Debian
sudo mkdir --parents /srv/mediagoblin.example.org
sudo chown --no-dereference --recursive mediagoblin:www-data /srv/mediagoblin.example.org
# Fedora
sudo mkdir --parents /srv/mediagoblin.example.org
sudo chown --no-dereference --recursive mediagoblin:nginx /srv/mediagoblin.example.org
Install MediaGoblin and Virtualenv¶
We will now switch to our ‘mediagoblin’ system account, and then set up our MediaGoblin source code repository and its necessary services. You should modify these commands to suit your own environment.
Switch to the mediagoblin unprivileged user and change to the
MediaGoblin directory that you just created:
sudo su mediagoblin --shell=/bin/bash
$ cd /srv/mediagoblin.example.org
Note
Unless otherwise noted, the remainder of this document assumes that all
operations are performed using the unprivileged mediagoblin
account, indicated by the $ prefix.
Clone the MediaGoblin repository and set up the git submodules:
$ git clone --depth=1 https://git.sr.ht/~mediagoblin/mediagoblin \
--branch stable --recursive
$ cd mediagoblin
Set up the environment:
$ ./bootstrap.sh
$ ./configure
$ make
Create and set the proper permissions on the user_dev directory.
This directory will be used to store uploaded media files:
$ mkdir --mode=2750 user_dev
This concludes the initial configuration of the MediaGoblin environment. In the future, you can upgrade MediaGoblin according to the “Upgrading MediaGoblin” documentation.
Configure Mediagoblin¶
Edit site configuration¶
Edit mediagoblin.ini and update email_sender_address to the
address you wish to be used as the sender for system-generated emails.
You’ll find more details in “Configuring MediaGoblin”.
Note
If you’re changing the MediaGoblin directories or URL prefix, you
may need to edit direct_remote_path, base_dir, and
base_url.
Configure MediaGoblin to use the PostgreSQL database¶
If you are using PostgreSQL, edit the [mediagoblin] section in your
mediagoblin.ini and remove the # prefix on the line containing:
sql_engine = postgresql:///mediagoblin
This assumes you are running the MediaGoblin application under the
same system account and database account; both mediagoblin.
Update database data structures¶
Before you start using the database, you need to run:
$ ./bin/gmg dbupdate
to populate the database with the MediaGoblin data structures.
Create an admin account¶
Create a MediaGoblin account with full administration access. Provide your own email address and enter a secure password when prompted:
$ ./bin/gmg adduser --username you --email you@example.com
$ ./bin/gmg makeadmin you
Test the Server¶
At this point MediaGoblin should be properly installed. You can test the deployment with the following command:
$ ./lazyserver.sh --server-name=broadcast
You should be able to connect to the machine on port 6543 in your browser to confirm that the service is operable. You should also be able to log in with the admin username and password.
Type Ctrl-c to exit the above server test.
The next series of commands will need to be run as a privileged user.
To return to your regular user account after using the system account,
type exit or Ctrl-d.
Deploy MediaGoblin¶
Nginx as a reverse-proxy¶
This configuration example will use Nginx, however, you may use any webserver of your choice. If you do not already have a web server, consider Nginx, as the configuration files may be more clear than the alternatives.
Create a configuration file at
/srv/mediagoblin.example.org/nginx.conf and create a symbolic link
into a directory that will be included in your nginx configuration
(e.g. “/etc/nginx/sites-enabled or /etc/nginx/conf.d) with the
following commands:
# Debian
sudo ln --symbolic /srv/mediagoblin.example.org/nginx.conf /etc/nginx/sites-enabled/mediagoblin.conf
sudo rm --force /etc/nginx/sites-enabled/default
sudo systemctl enable nginx
# Fedora
sudo ln -s /srv/mediagoblin.example.org/nginx.conf /etc/nginx/conf.d/mediagoblin.conf
sudo systemctl enable nginx
You can modify these commands and locations depending on your
preferences and the existing configuration of your Nginx instance. The
contents of this /srv/mediagoblin.example.org/nginx.conf file
should be modeled on the following:
server {
#################################################
# Stock useful config options, but ignore them :)
#################################################
include /etc/nginx/mime.types;
autoindex off;
default_type application/octet-stream;
sendfile on;
# Gzip
gzip on;
gzip_min_length 1024;
gzip_buffers 4 32k;
gzip_types text/plain application/x-javascript text/javascript text/xml text/css;
#####################################
# Mounting MediaGoblin stuff
# This is the section you should read
#####################################
# Change this to allow your users to upload larger files. If
# you enable audio or video you will need to increase this. This
# is essentially a security setting to prevent *extremely* large
# files being uploaded. Example settings include 500m and 1g.
client_max_body_size 100m;
# prevent attacks (someone uploading a .txt file that the browser
# interprets as an HTML file, etc.)
add_header X-Content-Type-Options nosniff;
server_name mediagoblin.example.org www.mediagoblin.example.org;
access_log /var/log/nginx/mediagoblin.example.access.log;
error_log /var/log/nginx/mediagoblin.example.error.log;
# MediaGoblin's stock static files: CSS, JS, etc.
location /mgoblin_static/ {
alias /srv/mediagoblin.example.org/mediagoblin/mediagoblin/static/;
}
# Instance specific media:
location /mgoblin_media/ {
alias /srv/mediagoblin.example.org/mediagoblin/user_dev/media/public/;
}
# Theme static files (usually symlinked in)
location /theme_static/ {
alias /srv/mediagoblin.example.org/mediagoblin/user_dev/theme_static/;
}
# Plugin static files (usually symlinked in)
location /plugin_static/ {
alias /srv/mediagoblin.example.org/mediagoblin/user_dev/plugin_static/;
}
# Forward requests to the MediaGoblin app server.
location / {
proxy_pass http://127.0.0.1:6543;
# On Debian and derivatives the below proxy_set_header lines can be replaced by:
# include proxy_params;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
The first four location directives instruct Nginx to serve the
static and uploaded files directly rather than through the MediaGoblin
process. This approach is faster and requires less memory.
Note
The user who owns the Nginx process, normally www-data or nginx,
requires execute permission on the directories static,
public, theme_static and plugin_static plus all their
parent directories. This user also requires read permission on all
the files within these directories. This is normally the default.
Nginx is now configured to serve the MediaGoblin application. Perform a quick test to ensure that this configuration works:
sudo nginx -t
If you encounter any errors, review your Nginx configuration files, and try to resolve them. If you do not encounter any errors, you can start your Nginx server (may vary depending on your operating system):
sudo systemctl restart nginx
Now start MediaGoblin to test your Nginx configuration:
sudo su mediagoblin --shell=/bin/bash
$ cd /srv/mediagoblin.example.org/mediagoblin/
$ ./lazyserver.sh --server-name=main
You should be able to connect to the machine on port 80 in your browser to confirm that the service is operable. If this is the machine in front of you, visit <http://localhost/> or if it is a remote server visit the URL or IP address provided to you by your hosting provider. You should see MediaGoblin; this time via Nginx!
Try logging in and uploading an image. If after uploading you see any “Forbidden” errors from Nginx or your image doesn’t show up, you may need to update the permissions on the new directories MediaGoblin has created:
# Debian
sudo chown --no-dereference --recursive mediagoblin:www-data /srv/mediagoblin.example.org
sudo find /srv/mediagoblin.example.org -type d -exec chmod 755 {} \;
sudo find /srv/mediagoblin.example.org -type f -exec chmod 644 {} \;
sudo find /srv/mediagoblin.example.org/mediagoblin/user_dev/crypto -type d -exec chmod 750 {} \;
sudo find /srv/mediagoblin.example.org/mediagoblin/user_dev/crypto -type f -exec chmod 640 {} \;
sudo find /srv/mediagoblin.example.org/mediagoblin/bin -type f -exec chmod 750 {} \;
# Fedora
sudo chown --no-dereference --recursive mediagoblin:nginx /srv/mediagoblin.example.org
sudo find /srv/mediagoblin.example.org -type d -exec chmod 755 {} \;
sudo find /srv/mediagoblin.example.org -type f -exec chmod 644 {} \;
sudo find /srv/mediagoblin.example.org/mediagoblin/user_dev/crypto -type d -exec chmod 750 {} \;
sudo find /srv/mediagoblin.example.org/mediagoblin/user_dev/crypto -type f -exec chmod 640 {} \;
sudo find /srv/mediagoblin.example.org/mediagoblin/bin -type f -exec chmod 750 {} \;
Note
If you see an Nginx placeholder page, you may need to remove the
Nginx default configuration, or explictly set a server_name
directive in the Nginx config.
Type Ctrl-c to exit the above server test and exit or
Ctrl-d to exit the mediagoblin shell.
Run MediaGoblin as a system service¶
To ensure MediaGoblin is automatically started and restarted in case
of problems, we need to run it as system services. If your operating
system uses Systemd, you can use Systemd service files to manage
both the Celery and Paste processes as described below.
In the Systemd configuration below, MediaGoblin log files are kept in
the /var/log/mediagoblin directory. Create the directory and give
it the proper permissions:
sudo mkdir --parents /var/log/mediagoblin
sudo chown --no-dereference --recursive mediagoblin:mediagoblin /var/log/mediagoblin
Place the following service files in the /etc/systemd/system/
directory. The first file should be named
mediagoblin-paster.service. Be sure to modify it to suit your
environment’s setup:
# Set the WorkingDirectory and Environment values to match your environment.
[Unit]
Description=Mediagoblin
[Service]
Type=simple
User=mediagoblin
Group=mediagoblin
Environment=CELERY_ALWAYS_EAGER=false
WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin
ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/paster serve \
/srv/mediagoblin.example.org/mediagoblin/paste.ini \
--log-file=/var/log/mediagoblin/mediagoblin.log \
--server-name=main
[Install]
WantedBy=multi-user.target
The second file should be named mediagoblin-celeryd.service:
# Set the WorkingDirectory and Environment values to match your environment.
[Unit]
Description=MediaGoblin Celery
After=rabbitmq-server.service
[Service]
User=mediagoblin
Group=mediagoblin
Type=simple
WorkingDirectory=/srv/mediagoblin.example.org/mediagoblin
Environment=MEDIAGOBLIN_CONFIG=/srv/mediagoblin.example.org/mediagoblin/mediagoblin.ini \
CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery
ExecStart=/srv/mediagoblin.example.org/mediagoblin/bin/celery worker \
--logfile=/var/log/mediagoblin/celery.log \
--loglevel=INFO
[Install]
WantedBy=multi-user.target
For details on this approach with a separate Celery process, see Background Media Processing.
Enable these processes to start at boot by entering:
sudo systemctl enable mediagoblin-paster.service
sudo systemctl enable mediagoblin-celeryd.service
Start the processes for the current session with:
sudo systemctl start mediagoblin-paster.service
sudo systemctl start mediagoblin-celeryd.service
If either command above gives you an error, you can investigate the cause of the error by entering either of:
sudo systemctl status mediagoblin-celeryd.service
sudo systemctl status mediagoblin-paster.service
Or view the full logs with:
sudo journalctl -u mediagoblin-paster.service -f
sudo journalctl -u mediagoblin-celeryd.service -f
The above systemctl status command is also useful if you ever want to
confirm that a process is still running.
Assuming the above was successful, you should now have a MediaGoblin server that will continue to operate, even after being restarted. Great job!
If you have a moment, please send us an email about your experience installing MediaGoblin. We’d love to know what worked well, what didn’t work so well and anything that could be improved.
Restarting MediaGoblin¶
To restart MediaGoblin after making configuration changes, run:
sudo systemctl restart mediagoblin-celeryd.service
sudo systemctl restart mediagoblin-paster.service
If you make any changes to the “.service” files, you must first issue a daemon-reload command to refresh Systemd and then restart MediaGoblin with:
sudo systemctl daemon-reload
sudo systemctl restart mediagoblin-celeryd.service
sudo systemctl restart mediagoblin-paster.service
What next?¶
This configuration supports upload of images only, but MediaGoblin also supports other types of media, such as audio, video, PDFs and 3D models. For details, see “Media Types”.
See “Further Considerations for Production Deployments” for more information and other issues you may want to consider.
For other settings and configuration options, see “Configuring MediaGoblin”.
To enable and configure plugins, see “Plugins”.
