Drupal 8 and DRUD Tech's ddev Local Development Environment

Submitted by nigel on Saturday 10th March 2018
Command Line

My favourite presentation at DrupalCamp London 2018 was Turbocharge Your Agency given by DRUD Tech co-founder Kevin Bridges and Jeffrey A. "jam" McGuire. They talked at length about the container solution ddev which enables developers to manage their local environments and thus optimise their time and in addition save their configuration under source control. Furthermore the demonstration showed that it was possible to flip between different configurations with different Drupal versions easily and quickly. The solution found here uses Docker and is wrapped by the lightning fast Golang.

I've been thinking for a while about moving from my existing hand crafted Virtual Machine towards Docker. My VM's configuration isn't under source control management, and has built up over the years into quite a complex beast. I have Jenkins, phing, composer, Drupal console, Mail Catcher, Compass, node, npm, Python, nginx, Apache, MariaDB and MongoDB that I can remember off the top of my head. It's becoming a difficult proposition to keep it all working. When I need to run a different version of PHP I manually switch simlinks around. Oh for something easier. 

So I elected to investigate whether Drud Tech's offering could possibly help out. The demonstration given by their guys at DrupalCamp London 2018 showed the ease of moving between D7 and D8 repos. However they were fresh builds and for the tool to be of value to me, it would have to work from pre-existing codebases. 

Installing a VirtualBox VM
ddev VM
ssh port forward

This was purely a Proof of Concept so as a consequence I needed to keep anything I did in a ring-fenced environment. Therefore a VirtualBox Ubuntu 16.04LTS image was used as a starting point. I won't provide screenshots of the entire build process since I have previous blogs to cover this. 

Once the build has completed and I have logged in I issue a ifconfig command to determine the IP address which is (second screenshot). This will need to be used with the network configuration. 

When I was building the VM I kept the default network configuration which is NAT. I have moved away from using Bridged Adapter. A bridged adapter on the face of it is very convenient - it automatically gets an IP address via DHCP from my ISP provided router at home, in the range. However this falls foul at client sites with security policies prohibiting unknown MAC Addresses, and Internet cafes that don't use the same IP range - it simply won't work. 

The solution is to use NAT and use the port forwarding facility. Navigate to Settings -> Network -> Advanced -> Port Forwarding and add values for ssh as per the third screenshot above.  

I can now ssh into the ddev VM.
$ ssh -p 2222 nigel@
The authenticity of host '[]:2222 ([]:2222)' can't be established.
ECDSA key fingerprint is SHA256:VRNxWpagSl2DGz7dmgRjBqdDdadXITdPPkItrMR9k38.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[]:2222' (ECDSA) to the list of known hosts.
nigel@'s password: 
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-87-generic x86_64)
 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage
137 packages can be updated.
69 updates are security updates.
Last login: Mon Mar  5 21:50:35 2018
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
Install Docker
Don't be tempted to install the Docker version that is bundled in the Ubuntu default repos. Follow the instructions on the Docker site for installing the CE version of Docker from their private repo.

nigel@ddev:~$ sudo apt-get update
[sudo] password for nigel: 
Hit:1 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:2 http://gb.archive.ubuntu.com/ubuntu xenial InRelease
Hit:3 http://gb.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:4 http://gb.archive.ubuntu.com/ubuntu xenial-backports InRelease
Reading package lists... Done 
Install packages to allow apt to use a repository over HTTPS
nigel@ddev:~$ sudo apt-get install \
>     apt-transport-https \
>     ca-certificates \
>     curl \
>     software-properties-common
Reading package lists... Done
Building dependency tree       
Reading state information... Done
apt-transport-https is already the newest version (1.2.25).
ca-certificates is already the newest version (20170717~16.04.1).
curl is already the newest version (7.47.0-1ubuntu2.6).
software-properties-common is already the newest version (
0 to upgrade, 0 to newly install, 0 to remove and 2 not to upgrade.
That's good. Nothing to do there. Now let's add Docker’s official GPG key:
nigel@ddev:~$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
Now I need to verify that we have the key with the fingerprint 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88, by searching for the last 8 characters of the fingerprint.
nigel@ddev:~$ sudo apt-key fingerprint 0EBFCD88
pub   4096R/0EBFCD88 2017-02-22
      Key fingerprint = 9DC8 5822 9FC7 DD38 854A  E2D8 8D81 803C 0EBF CD88
uid                  Docker Release (CE deb) <docker@docker.com>
sub   4096R/F273FCD8 2017-02-22
All good. Next the following command to set up the stable repository.
nigel@ddev:~$ sudo add-apt-repository \
>    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
>    $(lsb_release -cs) \
>    stable"
Update apt again
nigel@ddev:~$ sudo apt-get update
Hit:1 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:2 http://gb.archive.ubuntu.com/ubuntu xenial InRelease
Hit:3 http://gb.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:4 http://gb.archive.ubuntu.com/ubuntu xenial-backports InRelease
Get:5 https://download.docker.com/linux/ubuntu xenial InRelease [65.8 kB]
Get:6 https://download.docker.com/linux/ubuntu xenial/stable amd64 Packages [3,329 B]
Fetched 69.1 kB in 0s (140 kB/s)
Reading package lists... Done
Again we are looking good - we can see it's picked up the Docker repo ok. Let's install the latest version of Docker.
nigel@ddev:~$ sudo apt-get update
Let's check it's working..
nigel@ddev:~$ sudo docker run hello-world
[sudo] password for nigel: 
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
ca4f61b1923c: Pull complete 
Digest: sha256:083de497cff944f969d8499ab94f07134c50bcf5e6b9559b27182d3fa80ce3f7
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
This message shows that your installation appears to be working correctly.
Success! Now we need docker-compose installing. Again, don't be tempted to install from the standard Ubuntu repos, because if you do with Ubuntu 16.04LTS you'll end up with docker-compose version 1.8.0 which does not meet the ddev requirements. So do it the proper way!
nigel@ddev:~$ sudo curl -L https://github.com/docker/compose/releases/download/1.19.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   617    0   617    0     0    795      0 --:--:-- --:--:-- --:--:--   795
100 8288k  100 8288k    0     0   355k      0  0:00:23  0:00:23 --:--:--  196k
Now make it executable
nigel@ddev:~$ sudo chmod +x /usr/local/bin/docker-compose
Let's see the version
nigel@ddev:~$ docker-compose --version
docker-compose version 1.19.0, build 9e633ef
Neat! We are almost there. Now we need to make sure Docker starts every time we boot the machine.
nigel@ddev:~/projects/meedjum$ sudo systemctl enable docker
Synchronizing state of docker.service with SysV init with /lib/systemd/systemd-sysv-install...
Executing /lib/systemd/systemd-sysv-install enable docker
Before I am done here, I need to add myself to the Docker group so I can execute Docker commands (or rather ddev can on my behalf)
nigel@ddev:~/projects/meedjum$ sudo usermod -aG docker $(whoami)
Rebooting will now serve two purposes - check that the above command works which will start Docker, and make sure my new group status has been picked up.
nigel@ddev:~/projects/meedjum$ sudo su
root@ddev:/home/nigel/projects/meedjum# init 6
Installing ddev

There are two ways of installing ddev - by downloading from the repository's releases - or use the install script in the ddev documentation. I opted for the latter. 

nigel@ddev:~$ curl https://raw.githubusercontent.com/drud/ddev/master/install_ddev.sh | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2545  100  2545    0     0   5821      0 --:--:-- --:--:-- --:--:--  5823
ddev_linux.v0.15.0.tar.gz: OK
Download verified. Ready to place ddev in your /usr/local/bin.
Running "sudo mv /tmp/ddev /usr/local/bin/" Please enter your password if prompted.
[sudo] password for nigel: 
Bash completion for ddev was not installed. You may manually install /tmp/ddev_bash_completion.sh in your bash_completions.d directory.
ddev is now installed. Run "ddev" to verify your installation and see usage.
Installed - now check our installation
nigel@ddev:~$ ddev
This Command Line Interface (CLI) gives you the ability to interact with the ddev to create a development environment.
  ddev [command]
Available Commands:
  auth-pantheon Provide a machine token for the global pantheon auth.
  config        Create or modify a ddev project configuration in the current directory
  describe      Get a detailed description of a running ddev project.
  exec          Execute a shell command in the container for a service. Uses the web service by default.
  help          Help about any command
  hostname      Manage your hostfile entries.
  import-db     Import the database of an existing project to the dev environment.
  import-files  Import the uploaded files directory of an existing project to the default public upload directory of your project.
  list          List projects
  logs          Get the logs from your running services.
  pull          Import files and database using a configured provider plugin.
  remove        Remove the development environment for a project.
  restart       Restart the development environment for a project.
  ssh           Starts a shell session in the container for a service. Uses web service by default.
  start         Start a ddev project.
  stop          Stop the development environment for a project.
  version       print ddev version and component versions
  -h, --help          help for ddev
  -j, --json-output   If true, user-oriented output will be in JSON format.
Use "ddev [command] --help" for more information about a command.
We have it!
Getting Started with an Existing Drupal 8 Project
This blog has been built using Drupal 8 - so it made sense to use this for my first project. Let's create a directory for my projects and git clone. At this point I had already created a new .ssh key for my ddev VM and added it to my GitHub profile.
nigel@ddev:~$ mkdir projects
nigel@ddev:~$ cd projects/
nigel@ddev:~/projects$ git clone git@github.com:sanddevil/meedjum.git
Cloning into 'meedjum'...
Now let's cd into the project and config ddev for our Drupal 8 through its interactive shell.
nigel@ddev:~/projects/meedjum$ ddev config
Creating a new ddev project config in the current directory (/home/nigel/projects/meedjum) 
Once completed, your configuration will be written to /home/nigel/projects/meedjum/.ddev/config.yaml
Project name (meedjum): 
The docroot is the directory from which your site is served. This is a relative path from your project root (/home/nigel/projects/meedjum) 
You may leave this value blank if your site files are in the project root 
Docroot Location (docroot): 
Found a php codebase at /home/nigel/projects/meedjum/docroot. 
Project Type [php, drupal6, drupal7, drupal8, wordpress, typo3, backdrop] (php): drupal8
Configuration complete. You may now run 'ddev start'. 
ddev is suggesting I should run start, so why not?
nigel@ddev:~/projects/meedjum$ ddev start
Network ddev_default created 
Starting environment for meedjum... 
ddev needs to add an entry to your hostfile.
It will require root privileges via the sudo command, so you may be required
to enter your password for sudo. ddev is about to issue the command: 
    sudo /usr/local/bin/ddev hostname meedjum.ddev.local 
Please enter your password if prompted. 
Running Command  Command=sudo /usr/local/bin/ddev hostname meedjum.ddev.local
[sudo] password for nigel: 
Pulling db (drud/mariadb-local:v0.8.0)... 
Creating ddev-meedjum-db ... done
Pulling dba (drud/phpmyadmin:v0.2.0)... 
Creating ddev-meedjum-web ... done
Creating ddev-meedjum-dba ...  
Creating ddev-meedjum-web ...  
Network ddev_default is external, skipping 
Pulling ddev-router (drud/ddev-router:v0.5.0)... 
Creating ddev-router ... done
Successfully started meedjum 
Your project can be reached at http://meedjum.ddev.local and https://meedjum.ddev.local 
Installing Composer
Drupal 8 requires composer and that needs installing on our clean system. There are plenty of tutorials online on how to achieve this, so I'll whizz through this quickly.
nigel@ddev:~$ sudo apt-get update
Hit:1 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:2 http://gb.archive.ubuntu.com/ubuntu xenial InRelease
Hit:3 http://gb.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:4 http://gb.archive.ubuntu.com/ubuntu xenial-backports InRelease
Hit:5 https://download.docker.com/linux/ubuntu xenial InRelease
Reading package lists... Done
Install the dependencies
nigel@ddev:~$ sudo apt-get install curl php-cli php-mbstring git unzip
Reading package lists... Done
Download composer
nigel@ddev:~$ curl -sS https://getcomposer.org/installer -o composer-setup.php
Install it globally
nigel@ddev:~$ sudo php composer-setup.php --install-dir=/usr/local/bin --filename=composer
All settings correct for using Composer
Composer (version 1.6.3) successfully installed to: /usr/local/bin/composer
Use it: php /usr/local/bin/composer
Now check it's been installed correctly
nigel@ddev:~$ composer
  / ____/___  ____ ___  ____  ____  ________  _____
 / /   / __ \/ __ `__ \/ __ \/ __ \/ ___/ _ \/ ___/
/ /___/ /_/ / / / / / / /_/ / /_/ (__  )  __/ /
\____/\____/_/ /_/ /_/ .___/\____/____/\___/_/
Composer version 1.6.3 2018-01-31 16:28:17
Building Out the Drupal 8 Project

The ddev start command creates a new subdirectory called .ddev in the project's root directory. Within the directory we have config.yaml and  docker-compose.yaml. The config file contains the configuration of our ddev project such as the version of PHP, webpage url, type of project (drupal8) amongst other items. The docker yaml file contains the container configuration. 

Interaction with the ddev project can be accomplished in two ways, either with 

ddev exec command

or by spawning a shell with

ddev ssh

and using exit to terminate the interactive session. 

Our first requirement is to get dependencies into our D8 project since it is vendor-ised and uses composer as its dependency management.

Ok my Drupal core was on 8.4.4 and 8.4.5 and then 8.5 had been released so it needed updating, and in my case would update to 8.4.5 initially
nigel@ddev:~/projects/meedjum/docroot$ composer update --with-dependencies
Gathering patches for root package.
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 51 installs, 0 updates, 0 removals
Gathering patches for root package.
Gathering patches for dependencies. This might take a minute.
Now since there were schema changes, those need to be applied.
nigel@ddev:~/projects/meedjum/docroot$ ddev exec drush updatedb
Performing node_update_8401                                          [ok] 
Cache rebuild complete.                                              [ok] 
Finished performing updates.                                         [ok] 
The following updates are pending:
node module : 
  8401 -   Run a node access rebuild, if required. 
Do you wish to run all pending updates? (y/n): y

This is where things started to go awry. I already have a fully constructed settings.php and settings.local.php. The settings.php file contains my Memcached settings. However the memcache(d) PHP libraries aren't shipped in the ddev product. Obviously by that fact ddev doesn't support Memcached. 

When I attempted a drush status command I was inundated with errors.
nigel@ddev:~/projects/meedjum/docroot/sites/default$ vi settings.local.php
nigel@ddev:~/projects/meedjum/docroot/sites/default$ ddev exec drush status
Drupal\memcache\MemcacheException: No Memcache extension found in /var/www/html/docroot/modules/contrib/memcache/src/DrupalMemcacheFactory.php on line 162 #0 /var/www/html/docroot/modules/contrib/memcache/src/DrupalMemcacheFactory.php(62): Drupal\memcache\DrupalMemcacheFactory->initialize() 
As a consequence I commented out the Memcache settings
// Memcache
//Memcache configuration
//$settings['memcache']['servers'] = ['' => 'default'];
//$settings['memcache']['bins'] = ['default' => 'default'];
//$settings['memcache']['key_prefix'] = 'badzillad8';
//$settings['cache']['default'] = 'cache.backend.memcache';
Importing the Database
The next problem is ddev looks for the db credentials in settings.php whereas I have my credentials in my settings.local.php file. I changed the credentials in the settings.local.php file and I copied my database file to the /tmp directory.
nigel@ddev:~/projects/meedjum/docroot/sites/default$ ddev import-db  --extract-path /tmp
Provide the path to the database you wish to import. 
Import path: /tmp/db-2018-03-10.sql.gz
Failed to import database for meedjum: failed to write settings file for meedjum: Failed to get Drupal settings file path: settings files already exist and are being managed by the user 
More errors! The import failed. I thought it could be because it required the credentials in settings.php but that didn't work either. So I decided to use Drush instead:
nigel@ddev:~/projects/meedjum/docroot/sites/default$ ddev exec drush sql-cli < /tmp/db-2018-03-10.sql
HTTP Port Forwarding
HTTP Port Forward
Localhost domain
Error 40x
Just like our earlier forwarding of the ssh port so we can log into the VM, we also have to forward http port 80 traffic. To do this, we go back to the virtual machine network configuration, and set up a redirect from port 9090 (somewhat arbitrary) on the host to port 80 on the guest. See the first screenshot. This will take us to the the default website on the VM. We need to explicitly tell a browser which site it is - to do that we add an entry on the machine where we will be using our browser - which in my case will be my MacBook.

So I have added the entry to /etc/hosts of the working ddev domain name and the localhost IP address       localhost broadcasthost
::1             localhost meedjum.ddev.local
Now we need to reboot our VM and then login back in and issue a ddev start on our project. We can also check whether everything is good with a docker ps command.
root@ddev:/home/nigel# init 6
Connection to closed by remote host.
Connection to closed.
Nigels-MacBook-Pro:tmp nigel$ ssh -p 2222 nigel@
nigel@ddev:~$ cd projects/meedjum/
nigel@ddev:~/projects/meedjum$ ddev start
Starting environment for meedjum... 
nigel@ddev:~/projects/meedjum$ docker ps
CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS                             PORTS                                                                                      NAMES
5fd9a60cb90c        drud/ddev-router:v0.5.0           "/app/docker-entrypo…"   10 seconds ago      Up 10 seconds (healthy)  >80/tcp,>443/tcp,>8025/tcp,>8036/tcp   ddev-router
cea77b7cbc6b        drud/nginx-php-fpm-local:v1.1.0   "/start.sh"              7 hours ago         Up 11 seconds (healthy)            443/tcp,>80/tcp,>8025/tcp                                    ddev-meedjum-web
2dc08e116820        drud/phpmyadmin:v0.2.0            "/run.sh phpmyadmin"     7 hours ago         Up 11 seconds (health: starting)>80/tcp                                                                      ddev-meedjum-dba
f239a52ae806        drud/mariadb-local:v0.8.0         "/docker-entrypoint.…"   7 hours ago         Up 12 seconds (healthy)  >3306/tcp                                                                    ddev-meedjum-db
We can now point a web browser at the url we typed into our /etc/hosts file - see third image above. Ok - so we are close but not quite there yet! Looks like this could be the old trusted host pattern issue since it doesn't know about my domain meedjum.ddev.test.
Add ddev Domain to Trusted Hosts in Settings.php
...or rather in my case my settings.local.php file. I added the ddev domain name to the trusted hosts pattern array
['trusted_host_patterns'] = [
Copying the Assets
No images
Adding the trusted host entry now means the web site loads successfully but of course the assets are missing and need copying down from my prod site. ddev provides a command for this:
nigel@ddev:~/projects/meedjum/docroot$ ddev import-files --src=/tmp/files.tar.gz
Successfully imported files for meedjum 
100% my fault, but my file structure ended up as sites/default/files/files because I'd tarballed my prod files directory within the tarball, instead of just its contents. So I moved everything back one directory.
nigel@ddev:~/projects/meedjum/docroot$ mv sites/default/files/files/* sites/default/files/.
nigel@ddev:~/projects/meedjum/docroot$ mv sites/default/files/files/.htaccess sites/default/files/.
nigel@ddev:~/projects/meedjum/docroot$ rmdir sites/default/files/files
Completed Project
Complete Blog
Successful Blog

Ok so in the interim I fixed the missing Font Awesome library - not something I'd noticed before, but obviously it's been gitignored and it isn't in the composer.json file. Anyhoo the site is now fully functional and you can see from the URL bar I need to append the port :9090 as per my previous port forwarding fix. 


Playing with ddev was an interesting experience, not without challenges, but ultimately rewarding. Here are some of the points I made as I went along. 

  • The ddev exec command is run as root. This is a problem because it basically means using composer with it - such as ddev exec composer update - would trash the file permissions of my codebase. Ownerships would become root, and that would lead to problems down the line. Composer should never be run as root. So the net effect is composer needs to be installed on the host system. Ideally I wanted a totally un-poluted filesystem without installing anything on the host beyond ddev. It also means since composer has a dependency on PHP, that needs to be installed, and now we have a system with different PHP CLI and FPM versions. 
  • I couldn't get the ddev import-db command to work. A minor inconvenience and of course it is easy to  use alternatives with the command line or drush. 
  • Now I've got a working system - what next? I have a welter of phing targets to deploy my code, usually invoked by Jenkins. I could add phing at project level but how would I get Jenkins docker-ised up and running across all projects? I sense that could be tricky. 
  • I can see ddev's value. I would place it in a small agency with both legacy D6 and D7 support projects alongside new D8 and WP builds. I can't see it as an enterprise solution - there is too much missing such as Memcache and the previously mentioned phing and Jenkins. 

But all in all I really enjoyed my Saturday watching the football whilst playing with ddev and writing this blog - despite my teams losing!