Performance and Mutagen¶

Mutagen¶

TL;DR: If you're on macOS or Windows just enable mutagen. ddev config global --mutagen-enabled. You'll be glad you did.

The Mutagen asynchronous caching feature offers advanced performance experiences and is recommended for most projects. It's now the preferred way to get great webserving performance on macOS and Windows. Unlike the NFS feature, it requires no pre-configuration or installation. You do not need to (and should not) install mutagen. It can also be significantly faster than NFS and massively faster than plain vanilla Docker or Colima. In addition, it makes filesystem watchers (fsnotify/inotify) work correctly.

Mutagen can offer massive webserver performance speedups on macOS and traditional Windows; it works fine (and has automated tests) on Linux or Windows WSL2, but the speedup you see may not be worth turning it on, since Linux/WSL2 are already so fast.

Docker bind-mounts (the traditional approach to getting your code into the DDEV web container) can be slow on macOS and Windows, even with NFS. The reason is that every file access has to be checked against the file on the host, and Docker's setup to do this on macOS and Windows offers is not very performant. (On Linux and Linux-like systems, Docker provides native file-access performance.)

Mutagen works by decoupling reads and writes inside the container from reads and writes on the host. If something changes on the host, it gets changed "pretty soon" in the container, and if something changes inside the container it gets updated "pretty soon" on the host. This means that the webserver inside the web container does not have to wait for slow file reads or writes, and gets near-native file speeds. However, it also means that at any given moment, the files on the host may not exactly match the files inside the container, and if files are changed both places, conflicts may result.

Another major advantage of Mutagen over NFS is that it supports filesystem notifications, so file-watchers on both the host and inside the container will be notified when changes occur. This is a great advantage for many development tools, which had to poll for changes in the past, but now will be notified via normal inotify/fsnotify techniques.

If you trouble with the Mutagen feature, please try to recreate it and report via one of the support channels.

Enabling Mutagen¶

To begin using Mutagen, just ddev stop and then ddev config --mutagen-enabled and start the project again. If the mutagen binary needs to be downloaded, it will be downloaded automatically.

To stop using Mutagen on a project, ddev mutagen reset && ddev config --mutagen-enabled=false.

You can also enable mutagen globally (recommended) with ddev config global --mutagen-enabled

Note that the nfs-mount-enabled feature is automatically turned off if you're using mutagen.

You can run mutagen on all your projects, there's no limit. To configure it globally, ddev config global --mutagen-enabled, but you cannot disable mutagen on individual projects if it's enabled globally (the global configuration wins).

Caveats about Mutagen Integration¶

Most people have an excellent experience with Mutagen, but it's good to understand how it works and what the trade-offs are:

• Not for every project: Mutagen is not the right choice for every project. If filesystem consistency is your highest priority (as opposed to performance) then there are reasons to be cautious, although people have had excellent experiences: there haven't been major issues reported, but two-way sync is a very difficult computational problem, and problems may surface.
• Only one mutagen version on machine please: DDEV installs its own mutagen. You do not need to install mutagen. Multiple mutagen versions can't coexist on one machine, so please stop any running mutagen. On macOS, killall mutagen. If you absolutely have to have mutagen installed via homebrew or another technique (for another project) make sure it's the same version as you get with ddev version.
• Works everywhere, best on macOS and Windows: This is mostly for macOS and traditional Windows users. WSL2 is already the preferred environment for Windows users, but if you're still using traditional Windows this makes a huge difference. Although DDEV with mutagen is fully supported and tested on traditional Windows and Linux/WSL2, enabling mutagen on Linux/WSL2 may not be your first choice, since it adds some complexity and very little performance.
• Increased disk usage: Mutagen integration increases the size of your project code disk usage, because the code exists both on your computer and also inside a docker volume. (As of v1.19+, this does not include your file upload directory, so normally it's not too intrusive.) So take care that you have enough overall disk space, and also (on macOS) that you have enough file space set up in Docker Desktop. For projects before v1.19, if you have a large amount of data like user-generated content that does not need syncing (i.e. fileadmin for TYPO3 or sites/default/files for Drupal), you can exclude specific directories from getting synced and use regular docker mount for them instead. See below for Advanced Mutagen configuration options. As of v1.19, this is handled automatically and these files are not mutagen-synced.
• If your project is likely to change the same file on both the host and inside the container, you may be at risk for conflicts.
• Massive changes to either the host or the container are the most likely to introduce issues. This integration has been tested extensively with major changes introduced by ddev composer and ddev composer create but be aware of this issue. Changing git branches, npm install, yarn install, or a script that deletes huge sections of the synced data are related behaviors that should raise caution. If you ddev stop and then change a git branch and then ddev start you are almost certain to get misbehavior, because mutagen didn't know you made those changes while it wasn't running, so tries to merge the results. If you have to do this, do a ddev mutagen reset before restarting the project, so that only the host side will have contents.
• Mutagen is asynchronous: If you make a massive change on either the host or inside the container, you may not see the results for a little while. In studying situations like this, use ddev mutagen monitor to watch what's going on on your computer.
• ddev mutagen sync: You can cause an explicit sync with ddev mutagen sync and see syncing status with ddev mutagen status. Note that both ddev start and ddev stop automatically force a mutagen sync.
• Composer: If you do composer actions inside the container (with ddev ssh) you'll probably want to do a ddev mutagen sync to make sure they get synced as soon as possible, although most people won't ever notice the difference and mutagen will get it synced soon enough.
• Big git operations (like switching branches) are best done on the host side, rather than inside the container, and you may want to do an explicit ddev mutagen sync command after doing something like that. Do them with the project running, rather than when it is stopped.
• Project with users who don't want mutagen: If you share a project with some users (perhaps on macOS) that want mutagen and other users (perhaps on WSL2) that don't want or need it, then don't check in the mutagen_enabled: true in the .ddev/config.yaml. Instead, you can either use global mutagen configuration or add a not-checked-in project-level .ddev/config.mutagen.yaml that just has mutagen_enabled: true in it. Then only users that have that will have mutagen enabled.
• Mutagen restrictions on Windows symlinks: On macOS and Linux (including WSL2) the default .ddev/mutagen/mutagen.yml chooses the posix-raw type of symlink handling (See mutagen docs). This basically means that any symlink created will try to sync, regardless of whether it's valid in the other environment. However, Mutagen does not support posix-raw on traditional Windows, so ddev uses the portable symlink mode. So on Windows with Mutagen symlinks have to be strictly limited to relative links that are inside the mutagen section of the project.
• Backups!!!: Keep backups.

Syncing after git checkout¶

In general, it's best practice on most projects to do significant git operations on the host, but they can be disruptive to the sync. It's easy to add a git post-checkout hook to do a ddev mutagen sync operation though. Add the file .git/hooks/post-checkout to your project and set it to be executable (chmod +x .git/hooks/post-checkout):

#!/bin/bash
ddev mutagen sync || true

Syncing after yarn, npm, pnpm actions¶

Actions by those programs can also set off massive filesystem changes.

You should run ddev mutagen sync in order to get things into sync, or simply wait.

Advanced Mutagen configuration options¶

The Mutagen project provides extensive configuration options that are documented on the mutagen.io site.

Each project by default already has a .ddev/mutagen/mutagen.yml file with basic defaults which you can override if you remove the #ddev-generated line at the beginning of the file.

Remember if you edit the .ddev/mutagen/mutagen.yml file:

• Remove the #ddev-generated line
• Execute a ddev mutagen reset to avoid the situation where the docker volume still has files from an older configuration.

The most likely thing you'll want to do is to exclude a path from mutagen syncing, which you can do in the paths: section of the ignore: stanza in the .ddev/mutagen/mutagen.yml.

It is possible to exclude mutagen syncing from a path and then bind-mount something from the host or a different volume on that path with a docker-compose.*.yaml file. So if you have an extremely heavyweight subdirectory in your project (lots of fonts or user-generated content for example), you could exclude that subdirectory in the .ddev/mutagen/mutagen.yml and then add a docker-compose.exclude.yaml.

For example, if you want the stored-binaries subdirectory of the project to be available inside the container, but do not need mutagen to be syncing it, you can use normal docker bind-mounting for that subdirectory with this procedure:

1. Take over the .ddev/mutagen/mutagen.yml by removing the #ddev-generated line
2. Add /stored-binaries to the excluded paths:

    ignore:
      paths:
        - "/stored-binaries"

1. Add a .ddev/docker-compose.bindmount.yaml something like this:

version: "3.6"
services:
  web:
    volumes:
      - "./stored-binaries:/var/www/html/stored-binaries" 

Troubleshooting Mutagen Sync Issues¶

• Please make sure that DDEV projects work without mutagen before troubleshooting mutagen. ddev config --mutagen-enabled=false && ddev restart.
• DDEV's mutagen may not be compatible with an existing mutagen on your system. Please make sure that any mutagen installs you have are not running, or stop them. You may want to brew uninstall mutagen-io/mutagen/mutagen mutagen-io/mutagen/mutagen-beta to get rid of brew-installed versions.
• DDEV's mutagen is installed in ~/.ddev/bin/mutagen. You can use all the features of mutagen by running that, including ~/.ddev/bin/mutagen sync list and ~/.ddev/bin/mutagen daemon stop. You can run the script diagnose_mutagen.sh to gather some information about the setup of mutagen. Please report its output when creating an issue or otherwise seeking support.
• Try ~/.ddev/bin/mutagen daemon stop && ~/.ddev/bin/mutagen daemon start to restart the mutagen daemon if you suspect it's hanging.
• Use ddev mutagen reset if you suspect trouble (and always after changing the .ddev/mutagen/mutagen.yml. This restarts everything from scratch, including the docker volume that's used to store your project inside the container.)
• ddev mutagen monitor can help watch mutagen behavior. It's the same as ~/.ddev/bin/mutagen sync monitor <syncname>
• ddev debug mutagen will let you run any mutagen command using the binary in ~/.ddev/bin/mutagen.
• If you're working on the host and expecting things to show up immediately inside the container, you can learn a lot by running ddev mutagen monitor in a separate window as you work. You'll see when mutagen responds to your changes and get an idea about how much delay there is.
• Consider ddev stop before massive file change operations (like moving a directory, etc.)
• If you get in real trouble, ddev stop, reset your files with git, and then ddev mutagen reset to throw away the docker volume (which may already have incorrect files on it.)
• If you're having trouble, we really want to hear from you to learn and try to sort it out. See the Support channels.

Advanced Mutagen Troubleshooting¶

Most people get all the information they need about mutagen by running ddev mutagen monitor to see the results. However, Mutagen has full logging. Here's how to run it:

• killall mutagen
• export MUTAGEN_LOG_LEVEL=debug or export MUTAGEN_LOG_LEVEL=trace
• ~/.ddev/bin/mutagen daemon run
• Work with your project various actions and watch the output.

When you're done, you can ddev poweroff and <ctrl-c> the running mutagen daemon to get back to normal.

Mutagen Strategies and Design Considerations¶

Mutagen provides enormous speed boosts in everyday usage, but of course it's trying desperately under the hood to keep everything that changes in the container updated in the host, and vice versa.

DDEV mounts a fast Docker volume onto /var/www/html inside the web container and then delegates to the mutagen daemon (on the host) the job of keeping all the contents of the project on the host in sync with the contents of the docker volume.

The strategy in the DDEV integration is to try to make sure that at key points everything is completely in sync (consistent). Consistency is a really high priority for this integration.

The life cycle of the mutagen daemon and sync sessions are something like this:

1. On ddev start the mutagen agent will be started if it's not already running.
2. If there is already a sync session for this project it's stopped and recreated.
3. On ddev stop and ddev pause the sync session is flushed (made completely consistent) and then terminated.

In addition, a synchronous flush is performed after any ddev composer command, because composer may cause massive changes to the filesystem inside the container, and those need to be synced before operation continues.

If you need to reset everything for a project, you can do it with ddev mutagen reset which starts the mutagen session from scratch and removes the docker volume so it can be recreated from scratch.

Interaction with other usages of Mutagen¶

DDEV requires and provides a specific version of Mutagen, which you can see with ddev version.

Mutagen does not guarantee interoperability between different mutagen versions, so you may have trouble if you have another version of mutagen installed. You can find out what version of mutagen you may have installed outside of DDEV with mutagen version.

You'll want your system version of mutagen to be the same as the one provided with DDEV. If you're using mutagen for anything else, see the Mutagen installation instructions and install the required version.

Using NFS to Mount the Project into the Web Container¶

NFS (Network File System) is a classic, mature Unix technique to mount a filesystem from one device to another. It provides significantly improved webserver performance on macOS and Windows. DDEV-Local supports this technique, but it does require a small amount of pre-configuration on your host computer. DDEV-Local doesn't make changes to your computer's configuration without your involvement and approval, so this is done with a setup script that you run and that asks you for your sudo password.

The steps to set up NFS mounting on any operating system are:

1. Make sure DDEV-Local is already working and you can use it.
2. Configure the NFS server and exports files using the provided scripts for each operating system.
3. Test that NFS is working correctly by using ddev debug nfsmount in a project directory. The first line should report something like "Successfully accessed NFS mount of /path/to/project"
4. Enable NFS mounting globally with ddev config global --nfs-mount-enabled (You can also configure NFS mounting on a per-project basis with ddev config --nfs-mount-enabled in the project directory, but this is unusual. If nfs mounting is turned on globally it overrides any local project settings for NFS.)
5. ddev start your project and make sure it works OK. Use ddev describe to verify that NFS mounting is being used. The NFS status is near the top of the output of ddev describe.

Note that you can use the NFS setup described for each operating system below (and the scripts provided) or you can set up NFS any way that works for you. For example, if you're already using NFS with vagrant on macOS, and you already have a number of exports, the default export here (your home directory) won't work, because you'll have overlaps in your /etc/exports. Or on Windows, you may want to use an NFS server other than Winnfsd, for example the Allegro NFS Server. The setups provided below and the scripts provided below are only intended to get you started if you don't already use NFS.

Note that NFS does not really add to performance on Linux, so it is not recommended.

nfs.server.mount.require_resv_port = 0
nfs.server.verbose = 3

$ ddev debug nfsmount
Successfully accessed NFS mount of /Users/rfay/workspace/d8composer
TARGET    SOURCE                                                FSTYPE OPTIONS
/nfsmount :/System/Volumes/Data/Users/rfay/workspace/d8composer nfs    rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.65.2,mountvers=3,mountproto=tcp,local_lock=all,addr=192.168.65.2
/nfsmount/.ddev