Docker for Mac may provide a really bad performance, depending on the current setup and used software. I provide here some tipps how to make your local development environment much, much faster.
The history
For a very long time, I used a local setup using a lamp stack based on Homebrew, had some scripts that made it easy to switch between php versions. Unfortunately this setup broke in early 2018, when Homebrew decided to change their architecture. Another backdraw of this approach was, that it is nearly impossible the share a local development setup. Each setup was slightly different, mostly due to personal taste.
At this time some colleagues startet to discover Docker and build a setup, which was working quite stable. But in the last weeks, the performance issues began to annoy me more and more. Especially in bigger projects and with some time pressure on them, it was hard to wait for response in the browser. It was about 8 – 10 seconds per request.
My first searches did not have a usable outcome and raised more questions, that I got answers.
It started with a tweet …
Mid January 2019 Tomas Norre Mikkelsen asked on Twitter, which setups for local development are currently recommended:
Dear #TYPO3 Extension Developers, What do you do to have multiple TYPO3 Versions running, to easier debug when issues are reported?
Using
* the TYPO3 Homestead by @TuurlijkNiet ?
* #ddev?
* something else ?— Tomas Norre Mikkelsen (@tomasnorre) 14. Januar 2019
Besides ddev and native setups also plain docker-compose files were recommended. Sasche Egerer was so nice to share his setup on Github. This file and my following research brought some very nice findings regarding the performance.
Filesystem synchronisation
What I already knew was, that the bottle neck on Mac for Docker are the filesystem I/O requests on volumes of the type “bind”. A volume of this type is often configured like this
services: php: [...] volumes: - .:/usr/local/apache2/htdocs:cached [...]
There must be a layer between MacOS and the Docker containers, because docker cannot access the fs directly. (On Linux based systems host and guests shares the same resources).
To improve the situation, Docker introduced the osxfs
caching in version 17.04. There are three different access and caching mechanisms available:
- delegated
- cached and
- consistent
With the last one, one single I/O request takes about 100ms! The first and second caching mechanism provide a far better performance, but are still far from optimal. For a lot of more details, see the links in the “Links” section below.
So, when optimizing the performance of Docker for Mac, the focus must lie on reducing the I/O requests to host system.
Concrete findings for TYPO3
Database
For whatever reason, we had all the mysql specific files laying on a so called “bind-mount”. So every SQL statement had to go through the I/O requests to the hosts file system. It was very easy to remove this limitation: I just followed the setup of Saschas file and introduced a volume for database data.
services: mysql: image: "in2code/mysql" ports: - "3306:3306" environment: MYSQL_ROOT_PASSWORD: <your_password> MYSQL_DATABASE: <your_databasename> volumes: - mysql:/var/lib/mysql volumes: mysql: driver: local
Before that setup an sql import of slightly over 600MB took about 4 minutes and 40 seconds. After that change the same import just lasted 1 minute and 20 seconds. Ok, an initial database import does not happen so often, but gives already a hint about the (possible) performance gain.
The difference between a volume and a bind mount is, that the volume is completely managed by Docker. A bind mount is managed by the host system and can be altered at any time, so Docker for Mac needs to take care to synchronize the files between host and container.
Unfortunately the performance gain while developing based on that change was not as high as expected … “only 1 – 2 seconds per request”. But what other possibilities are there for optimizing?
Project files
As already shown in the first example, the project files are located in a bind mount: - .:/usr/local/apache2/htdocs:cached
. All files within that directory must be synchronized and kept up to date between host and guest. For the files we usually work on, while developing TYPO3 or a website, this is not really a problem. The php files are auto-loaded, cached by the OpCache and so on. But this bind mount also contains the directory typo3temp
, which contents are read and changed very, very often through TYPO3.
So my question was: Would it be possible to move just this directory into a volume and to leave the rest in the bind mount? Like normal unix filesystems also work … just add an additional mount to an arbitrary directory? Guess what? It is possible!
version: '2.3' services: php: build: "in2code/php-7.0" ports: - "9000" volumes: - .:/usr/local/apache2/htdocs:cached - type: volume source: typo3temp target: /usr/local/apache2/htdocs/Web/typo3temp volume: nocopy: true environment: - TYPO3_CONTEXT=Development volumes: typo3temp: driver: local
In the volumes section an additional volume of type
volume
for the source typo3temp
is created. It is then mapped to the target
directory /usr/local/apache2/htdocs/Web/typo3temp
, which is a subdirectory of the bind mount one line above.
And this is the change that really, really improved the performance. This one reduced the response time to about 2 seconds per page request in the frontend. Also the backend, especially the page tree become much more responsive. I did not do any concrete measurements, but now it feels like working on a normal remote system.
Possible issues
- Error: Volume must be a string
If you get an error message like this, you need to adjust the version of docker-compose in the first line of thedocker-compose.yml
. It must be set toversion: 2.3
at least. - Error: Could not create typo3temp/<what_ever_path>
In this case you need just to delete the directory on the host computer. Afterwards, it will work fine. - Performance gain not that high
A colleague reported, that he had to restart Docker4Mac on his computer to get the full performance win.
DDEVs approach
DDEV faces the same performance issue on Mac. Since DDEV version 1.5.0 there is an experimental feature for Docker for Mac, which copies all project files at ddev start
(except the ones in the .git
directory) to the container. Saying that, means also that the ddev start
command takes a (little) bit longer depending on the size of your project.
The mechanism is based on the docker image docker-bg-sync
of Cameron Eagans using the Unison file synchronizer.
The big advantage of this approach is, that it will work for any php project and is not TYPO3 specific.
Credits
I want to thank my supporters via patreon.com, who make this blog post possible. Unfortunately three major sponsorships ended in December 2018. I want to thank
- Thomas Deuling of coding.ms (https://www.typo3-adventskalender.de/)
- Maximilian Grimm (no link for credits available :-/ ) and
- Olivier Dobberkau (dkd.de)
for their long-lasting and recurring support in the last year.
If you appreciate my blog and want to step in to this (financial) gap, you can say “Thank You!”. You find all possibilities behind that button:
I found the blog post image on pixabay . It was published by Pexels under the CC0 public domain license. It was modified by myself using pablo on buffer.
Further reading
(1) https://docs.docker.com/docker-for-mac/osxfs-caching/
(2) https://stories.amazee.io/docker-on-mac-performance-docker-machine-vs-docker-for-mac-4c64c0afdf99
(3) https://docs.docker.com/compose/compose-file/#volumes
(4) https://docs.docker.com/storage/
Thank you for your blog post and sharing your knowledge. I was already aware that excluding of the typo3temp folder will increase the speed.
I always to that if do functional tests. But i did not do that in the basic setup as people using it would wonder where the files of the folder are if they are running into problems or just want to debug something.
I would also suggest to use a `tmpfs` volume instead as this should increase the speed a bit more.
I’ve just pushed a change of my dummy setup containing this change. https://github.com/sascha-egerer/docker-example-setups/commit/76d1e8a2005b3922c5ac718c969bb35849ff860a
Thanks for your post!
Since a few years already I am happy user of „Docker Machine“ and „Docker Machine NFS”. With Docker Machine the Docker Deamon is executed on a virtual machine using a hypervisor of your choice (e.g. VirtualBox, Parallels, …). To speed up everything a NFS mount is used to „synchonize“ the files.
The following commands will create a new VM with 60 GB HDD and 3 GiB RAM using VirtualBox.
docker-machine create default --virtualbox-disk-size 60000 --virtualbox-memory 3072
docker-machine nfs
To start your Docker Compose project:
eval $(docker-machine env)
docker-compose up
Docker Machine NFS can be installed using brew:
brew install docker-machine-nfs
Pingback: Links der Woche… - TYPO3 Blogger | TYPO3 Blogger
Thank to the post and the initiative from Sascha.
This tweak makes me love my old MacBook again. ;)
5