DDEV multisite setup with Acquia pull - drupal

Question

I've just gotten DDEV setup and I have multisite working by manually running ddev import-db --target-db=[db-name]. It's working just fine but I would like to figure out how to get database pulls from Acquia to work where I can specify the site to pull from.
I have this script working but is there a way to do this with DDEV commands that would be a little cleaner?
First I modified acquia.yaml to this:
environment_variables:
project_id: mysite.dev
uri: mysite.com
db_name: mysite_us
#uri: mysite.ca
#db_name: mysite_canada
#uri: mysite.co.uk
#db_name: mysite_unitedkingdom
# etc etc
db_pull_command:
command: |
# set -x # You can enable bash debugging output by uncommenting
ls /var/www/html/.ddev >/dev/null # This just refreshes stale NFS if possible
pushd /var/www/html/.ddev/.downloads >/dev/null
acli remote:drush -n ${project_id} -- sql-dump --extra-dump=--no-tablespaces --uri=${uri} >${db_name}.sql
Then I wrote the following script which i call like:
./ddev-refresh-db.sh mysite_us mysite.com
#!/bin/bash
site="$1"
uri="$2"
ddev pull acquia
ddev import-db --target-db=${site} --src=.ddev/.downloads/${site}.sql
ddev drush --uri=${uri} cr
However this still requires us to change the site and URI in the acquia.yaml file before running this command.
Is there a way to pass a variable through to ddev pull acquia ? And also a way to mimic what this script is doing with a real DDEV command?

Answer 1

Here's a more complete answer for Acquia multisite pull, pulling all sites. As of DDEV v1.18.0, the ddev pull itself really isn't robust enough to pull multiple sites, because it assumes one database and one set of files. This works where #kelly howard's answer in https://stackoverflow.com/a/68553116/215713 is inadequate. (In her example, she pulls just one of the multisites, and it works great for that situation.)
But here we'll put all the logic in a DDEV custom command and pull all databases and files for any named site, so ddev acquiapull <sitename>
Place this file in the project as .ddev/commands/web/acquiapull
#!/bin/bash
# This DDEV custom command is set up to pull database and files from Acquia for several subsites.
# Usage: `ddev acquiapull [ --skip-db ] [ --skip-files ] <site1> <site2>
# Example: `ddev acquiapull subsite1`
# This assumes that each subsite has its own database (named for the site)
# and that each subsite has its own files in sites/<sitename>/files
# To use it set up the needed ACQUIA_API_KEY and ACQUIA_API_SECRET in global
# or project config, just as described in
# https://ddev.readthedocs.io/en/stable/users/providers/acquia/
acquia_project_id=myprojectid.dev
tmpdir=/tmp #inside web container
set -eu -o pipefail
while :; do
case ${1:-} in
-h | -\? | --help)
show_help
exit
;;
-y|--yes)
SKIP_CONFIRMATION=true
;;
--skip-files)
SKIP_FILES=true
;;
--skip-db)
SKIP_DB=true
;;
--) # End of all options.
shift
break
;;
-?*)
printf 'WARN: Unknown option (ignored): %s\n' "$1" >&2
;;
*) # Default case: No more options, so break out of the loop.
break ;;
esac
shift
done
# Map sitename to database name
function target_db_name() {
site_name=$1
echo $site_name
}
# Map sitename to files dir
function target_files_dir() {
site_name=$1
echo "sites/${site_name}/files"
}
# Get the files from upstream and load them.
function files_pull() {
#set -x # You can enable bash debugging output by uncommenting
set -eu -o pipefail
site_name=$1
files_dir=$(target_files_dir $1)
mkdir -p ${DDEV_DOCROOT}/${files_dir}/
echo "Using drush rsync to update files for ${site_name}..."
drush rsync --alias-path=~/.drush -q -y -r ${DDEV_DOCROOT} --verbose #${acquia_project_id}:${files_dir}/ ${DDEV_DOCROOT}/${files_dir}/
}
# Get the db from upstream and load it
function db_pull() {
#set -x # You can enable bash debugging output by uncommenting
set -eu -o pipefail
site_name=$1
target_db=$(target_db_name ${site_name})
echo "Downloading ${site_name} database..."
acli remote:drush -n ${acquia_project_id} -- sql-dump --uri=${site_name} --extra-dump=--no-tablespaces >${tmpdir}/${site_name}.sql
echo "Loading ${site_name} into database '${target_db}'..."
mysql -uroot -proot -e "CREATE DATABASE IF NOT EXISTS ${target_db}; GRANT ALL ON ${target_db}.* TO 'db'#'%'"
mysql -uroot -proot ${target_db} <${tmpdir}/${site_name}.sql
drush -r root --uri=${site_name} cr
}
# Handle initial authentication via Acquia secrets and ssh
function authenticate() {
if [ -z "${ACQUIA_API_KEY:-}" ] || [ -z "${ACQUIA_API_SECRET:-}" ]; then echo "Please make sure you have set ACQUIA_API_KEY and ACQUIA_API_SECRET in your project or global config" && exit 1; fi
if ! command -v drush >/dev/null; then echo "Please make sure your project contains drush, ddev composer require drush/drush" && exit 1; fi
ssh-add -l >/dev/null || (echo "Please 'ddev auth ssh' before running this command." && exit 1)
acli auth:login -n --key="${ACQUIA_API_KEY}" --secret="${ACQUIA_API_SECRET}"
acli remote:aliases:download -n >/dev/null
}
# Main script
authenticate || (printf "Failed to authenticate" && exit $?)
if [ $# -eq 0 ]; then
printf "Usage: ddev acquiapull [ --skip-db ] [ --skip-files ] <sitename>"
exit 1
fi
if [ ${SKIP_CONFIRMATION:-} != "true" ]; then
echo "This will overwrite your database and files for sites $*. OK?"
select yn in "Yes" "No"; do
case $yn in
No ) exit;;
esac
done
fi
for subsite in $*; do
echo "Pulling subsite: $subsite"
if [ "${SKIP_DB:-}" != "true" ]; then
db_pull ${subsite} || (printf "Failed to pull db for ${subsite}" && exit $?)
else
echo "Skipping db pull for ${subsite}"
fi
if [ "${SKIP_FILES:-}" != "true" ]; then
files_pull ${subsite} || (printf "Failed to pull files for ${subsite}" && exit $?)
else
echo "Skipping files pull for ${subsite}"
fi
done

Answer 2

Thanks to the guidance from #rfay I set up a set of files in .ddev/providers for each country. Each one is structured like this:
environment_variables:
uri: mysite.be
db_name: belgium
auth_command:
command: |
<no changes>
db_pull_command:
command: |
# set -x # You can enable bash debugging output by uncommenting
ls /var/www/html/.ddev >/dev/null # This just refreshes stale NFS if possible
pushd /var/www/html/.ddev/.downloads >/dev/null
acli remote:drush -n ${ACQUIA_PROJECT_ID} -- sql-dump --extra-dump=--no-tablespaces --uri=${uri} >${db_name}.sql
Then I created a custom command in .ddev/commands/host that has the contents of my script. There are more cases in the real script to cover all the countries.
#!/usr/bin/env bash
## Description: Refresh a database from Acquia and run post-db commands
## Usage: refresh-db [dbname]
## Example: "ddev refresh-db france"
site="$1"
case $site in
canada)
uri="mysite.ca"
;;
australia)
uri="mysite.com.au"
;;
belgium)
uri="mysite.be"
;;
brazil)
uri="mysite.com.br"
;;
*)
site="db"
uri="mysite.com"
;;
esac
ddev pull ${site} -y 2>/dev/null # suppress pull failed message since it really didn't
ddev import-db --target-db=${site} --src=${DDEV_APPROOT}/.ddev/.downloads/${site}.sql
ddev drush --uri=${uri} cr
ddev drush --uri=${uri} -y pmu simplesamlphp_auth
ddev drush --uri=${uri} -y config-set system.performance css.preprocess 0
ddev drush --uri=${uri} -y config-set system.performance js.preprocess 0
I tried to handle the db import during the db_pull_command as suggested but I couldn't get past database permission errors for importing a DB that I had not already imported using ddev import-db. However with the custom command I can also incorporate the post-db-import steps that normally would only run against the default DB if done through config.yaml.
The other change I made was to move the project ID into the web environment settings in global_config.yaml file. This way if we want to change the environment we want to pull from, we just make an edit to the project ID there and don't have to edit the provider files.
I'm not experienced with contributing back to open source projects but if this can be helpful to others I'd love to work with someone to do that pull request on the documentation or wherever it belongs.

Answer 3

I'm going to go ahead and answer in general, but you can add a full answer when you get this sorted out. (I don't have access to an Acquia multisite.)
You're on the right track, but you can do all of this in the pull script. The problem you're having is that ddev just assumes a single database, and you have multiple.
Here's a strategy for your acquia.yaml:
Create all the databases. You can use mysql -e "CREATE DATABASE IF NOT EXISTS <dbname>;, use several lines or a for loop.
Pull all the databases. You can do this with separate acli lines, or use a for loop.
Import the databases that aren't the primary db using the mysql command. mysql <dbname> < <dbname.sql Again, this can be a few lines or a for loop. (You can also just import the primary db and it will just be re-imported by ddev, no harm done if it's not large.)
Thanks for the great question, and I hope you'll give a full answer here. Your answer could also be incorporated into https://ddev.readthedocs.io/en/stable/users/providers/acquia/ - you could do a PR there by clicking the pencil link at the upper right.