I've begun migrating a lot of our development environments to Vagrant. So far, this has been great for almost everything, but our first Drupal migration is unusable. It's unbelievably slow. Our Wordpress, CakePHP and Node.js sites all perform very adequately or better, but not Drupal. This think is just awful.
The box is a Veewee-created Ubuntu 12.04 64bit machine. It's the same base box we use for all of our web-based projects so nothing unique there. In my sites directory, I have a canonical directory (sites/my-site/) with all of the site resources and a symlink to that canonical directory with the domain name (sites/dev.mysite.com -> /vagrant/www/sites/my-site) that is evidently required for some module that the team is using.
This is a mixed Windows/OSX dev team and it's slow across both platforms. The only semi-unconventional snippet from my Vagrantfile is this:
config.vm.forward_port 80, 8080
config.vm.share_folder( "v-root", "/vagrant", ".", :extra => 'dmode=777,fmode=777' )
# Allows symlinks to the host directory.
config.vm.customize ["setextradata", :id, "VBoxInternal2/SharedFoldersEnableSymlinksCreate/v-root", "1"]
Vagrant::Config.run do |config|
config.vm.provision :shell, :path => "provision.vm.sh"
end
My shell provisioner only does a couple of things:
Installs drush
Creates the aforementioned symlink to the canonical site directory
Writes out an Nginx server block
If necessary, creates a settings.php file.
Is there anything I can do to improve performance? Like, a lot?
UPDATE
I've narrowed this down to a point where it looks like the issue is the remote database. To compare apples to apples with no project baggage, I downloaded a fresh copy of Drupal 7.21 and performed a standard install from the Vagrant web server against 3 different databases:
A new database created on the same Vagrant VM as the webserver (localhost)
A new database created on the shared dev server used in the original question (dev)
A new database created on an EC2 instance (tmp)
Once that was done, I logged in to the fresh Drupal install and loaded the homepage (localhost:8080) 5 times. I then connected to each database and loaded the same page, the same way. What I found was that the page loaded 4-6x slower when Drupal was connected to the remote database.
Remember, this is a fresh (standard) install. There is no project baggage.
I am hit by similar problem, too. It seems that VirtualBox shared folder can be very slow for project tree with +1000 files.
Switching to NFS might be the solution.
The issue is almost certainly either skip_name_resolve (being needed in my.cnf) or VirtualBox's poor handling of shared directories with large numbers of files. Both are easy to track down with strace -c, but you may find it easier just to correct them one at a time and see which one fixes your performance issues.
If you're still seeing slowness after both of these changes, let me know and we can debug it further.
I got here via google for similar, so I'm replying in the hopes others find this useful.
If you're using the precise32 vagrant box as your starting point, it's worth noting that the box by default has only 360MB of RAM.
Up the ram (at least in Vagrant V2 with VirtualBox) like so
config.vm.provider :virtualbox do |vb|
vb.customize ["modifyvm", :id, "--memory", "1024"]
end
This made Drupal much more responsive for me.
It's just a PHP/MySQL app so there's not much special about Drupal besides how it has been customized. You may have done some of this, but here are some suggestions to isolate the issue.
Check the Drupal dblog for errors.
Check your nginx & php logs for errors.
Consider how many active modules you are running (over 100? That would be a very heavy install)
Install a fresh Drupal instance & compare. This may isolate the problem to your instance and not Drupal in general.
If you find that it is your instance of Drupal
Install the devel module and enable memory reporting so you know how much memory is being used per page load, as well as to have a base line for improvement.
Make sure you have APC or another PHP opcache installed, and make sure the hit rate is good. If you weren't running it before, note the memory usage difference reported by devel.
run something like xhprof or disable suspicious modules till you find the major offenders.
enable mysql slow & index log to find potential issues, then add indexes or take other action appropriately
If your other apps are running fine, I suspect there is a problem with a particular module, or you have a fat Drupal install in general that needs some optimizing or more memory.
I tried pretty much everything to get my slow Vagrant to speed up and finally stumbled on this in the Issues tracker of the project.
config.vm.provider "virtualbox" do |v|
v.memory = 1024
v.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
v.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
end
I had previously tried NFS to no avail; this happened to be the silver bullet.
Since Vagrant 1.5 you can use rsync as a mechanism to sync a folder to the guest machine. Because rsync copies the files directly onto the remote filesystem, performance is noticeably better than NFS and VM shared folders.
You can read more about it here: http://www.vagrantup.com/blog/feature-preview-vagrant-1-5-rsync.html.
I just was trying to solve this issue myself. I tried the suggestions here and at Rails Windows Vagrant very slow response time. No real luck, I shaved 200 ms off 1800 ms response time on a warm request with no real data rendered. This with Ruby on Rails, not Drupal. The problem is the same, though.
Switching the shared folder to Rsync gave me a response time of ~280ms on that same request.
Vagrantfile:
config.vm.synced_folder '.', '/vagrant', type: 'rsync',
rsync__exclude: '.git/'
Usage:
$ vagrant up
$ vagrant rsync-auto
The latter command will watch your working dir and sync changed automatically.
See https://www.vagrantup.com/docs/synced-folders/rsync.html and https://www.vagrantup.com/docs/cli/rsync-auto.html
Latency is a big issue with database connections in any server environment. Even just running encryption on the DB connections is going to be a substantial performance issue, though it's presumably needed under these conditions.
What's your ping time to the database? If you've got at least one round trip for each query you run, then that's going to add up. Plus a bit of time for encryption. Worse again. if you don't use persistent database connections.
I'd think about where you do your caching. Eg cache in memcached on the VM instead of in the DB.
I run into the same problem. These advises will be especially helpful for those who uses Windows host machine. You will not be able to get decent performance without NFS supoort (for windows it is a big issue), so:
Do not use synced folder at all.
config.vm.synced_folder "../data", "/vagrant", disabled: true
Setub samba server in the guest VM + network drive on Windows host.
There are a lot of articles how to do it, e.g.: https://www.liberiangeek.net/2014/07/ubuntu-tips-create-samba-file-server-ubuntu-14-04/
If the NFS shares with Vagrant are still too slow for you, you can do the contrary:
Instead of installing a NFS server on your host machine, you can install it on the VM guest: http://guillaumeduveau.com/en/drupal-lightning-fast-synced-folders-in-vagrant-virtualbox/
I started to get slow performance on a drupal site once I installed nodejs and gulp. I had to do this because the drupal bootstrap 4 barrio sass subtheme requires nodejs/gulp. Then I ran into issues with vagrant on Windows and npm install commands. All npm install commands fail because they create sym links and Windows OS does not recognize these links. I had to create a sym link to the sites node_modules folder over to my vagrant home directory. npm install comands work after doing this. But then I started noticed the very slow response on this website. My other site run fast.
Related
We have one particular site that is Symfony and uses the e-commerce bundle Sylius.
Our developers are trying to use Vagrant so we can have similar dev environments. We use Puphpet to generate the Vagrant instance and share the config file.
If we are working on the site/repo natively or on a staging server, all runs fine. Pages load in around 2-3 seconds.
When we are using Vagrant / Virtualbox, it's 30-35 seconds per page load.
So far we've tried
Allocating up to 6GB to the box
Giving up to 4 processors to the box
Turning on NFS for file sync
Turning off all other programs on computers running Vagrant / Virtualbox (chat, other browsers, etc)
None of those things made an impact on page load time.
I can provide 2 things. One is the load trace from Symfony: https://nimbus.everhelper.me/client/notes/share/708707/mvw707mckzm2wq4rlkzc
Since there is so much code to the puphpet config, I put it in a pastebin here: http://pastebin.com/7ciVA5FL
What is OS on a host machine?
My guess would be that file system is slow. Try to run an app outside of shared folder on the guest machine. If it will be fast, then you'll spot a problem at least.
NFS on *nix or mac should be fast enough, are you sure you've succeed to turn it on?
I had this pain once, and finally started to use unison instead of native vagrant's file sharing system (https://www.cis.upenn.edu/~bcpierce/unison/)
Have your tried:
http://www.whitewashing.de/2013/08/19/speedup_symfony2_on_vagrant_boxes.html
or http://jeremybarthe.com/2015/02/02/speed-up-vagrant-environment-symfony2/
I think the first one is already included in Sylius, but not sure.
Also, dynamic image resize/crop may be reading/writing in the host file system and maybe there's a way to also change that (using symlinks or similar)?
vagrant-winnfsd works fine for me for getting NFS to work on Windows.
I want to start a new WordPress project with another developer. The decisions we made are:
We want to use Bedrock as the WP structure
we want to use Sage as the WP theme
We put the project in a GIT repository
I now ask myself if we should use Trellis, Valet or Docker.
My personal opinion is that Trellis / Docker is a bit too much for a project with two developers working on it. Additionally my experience with Vagrant is not very positive, as it was very slow when I used it. My favorite would be Valet, because it's so slim. The repository I would use is Beanstalk, from there I would trigger my deployments to my test and live system.
Additionally I am not 100% sure if my server to which I want deploy my project also needs Docker installed - does anybody knows that? And what happens when my server runs on Apache and not Nginx?
Now that Docker has native Mac and Windows apps you wouldn't need Vagrant for local dev, and running a series of Docker containers is much faster than a full-fledged VM with Vagrant+VirtualBox. Right now I have MariaDB + PHP-FPM + Nginx + WordPress + PHPMyAdmin, and the whole thing is really fast relative to my previous experience with Vagrant. Faster as in: faster initial install, faster to start/stop, faster to make changes and have them reflected after a restart. I just replaced MySQL with MariaDB in a matter of minutes (mostly fumbling with having the proper syntax in my docker-compose file).
The beauty of Docker appears precisely when you want to switch components (say Apache vs. Nginx). In WordPress' case, they provide images on Docker Hub that either include Apache or PHP-FPM (in the latter case you just add a Nginx container to your stack).
That said I just got started with Docker and there are some kinks to figure out, but it's worth figuring out.
I haven't deployed with Docker yet but I plan to test that next once I've got local dev fully working as intended. It's optional though, you could always deploy with Git webhooks or whatever you were using up to now.
I got tremendous performance issues with Vagrant and Magento. I hope you might be able to help me out on this.
Vagrant: 1.7.1
Wordpress: 4.2.1
Magento: 1.9.1
Git version 1.7.10.4
PHP 5.4.39-0+deb7u2 (cli)
nginx/1.2.1
Vagrantfile:
Vagrant.configure(2) do |config|
config.vm.box = "puphpet/debian75-x64"
config.vm.network "private_network", type: "dhcp"
# config.vm.network :forwarded_port, host: 8080, guest: 80
config.vm.synced_folder "./www", "/vagrant/www", type: "rsync", rsync__exclude: ".git/"
config.vm.provider "virtualbox" do |v|
v.memory = 2048
v.cpus = 4
end
config.vm.provider :virtualbox do |vb|
vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
vb.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
end
# config.vm.synced_folder "./www", "/vagrant/www", group: "www-data", owner: "www-data"
end
OK, so far so good.
I cloned my GIT Repo into the shared folder. I'm using a Wordpress Installation with magento integration (MWI Plugin).
As long as I am not activating the Magento Plugin all fine, Pageload is about 2 sec as soon as I use the Plugin things going up to 60 seconds.
I have no idea what is going wrong here, as for the git origin (live server) everything is fine.
I read a few posts about the synced folder topic. I sticked with rsync for now, seems to be the fastest option. I tried NFS as well. No real difference here.
I'm happy for any hints on this.
Thanks a lot,
Steven
You’re not alone. All Magento sites out of the box run slow. But your slow Magento site may be running even slower than the standard.
Some of the reasons may be :
No Full Page Cache (FPC). If you’re not running a copy of Magento Enterprise or are not on Magento Community 1.5+, chances are that you don’t have FPC activated. This one tool single handedly speeds up Magento sites by leaps and bounds.
Memory Limit Too Low. There is a setting in your configuration files which sets the maximum amount of memory you can dedicate to PHP processes. Since Magento is a big memory hog, having this value be greater than 128mB can significantly bump up the time it takes Magento to perform operations.
Indexes are not updated. If your indexes have not been updated manually through the admin panel, you’re causing your Magento site to figure out calculations every time a customer sees a page.
Not Using Memcached. Memcached is a memory object cacheing tool which in short means it can store user sessions and things that get queried in a cache.
Memcached can store and recall these things so much faster than your server can.
Compilation Turned Off. This is a simple setting in the admin panel which compiles files into more easily accessible blocks of HTML. Lots of companies have this turned off because it breaks the site.
Developers Hacked The Core. “Hacking the core” means that your developers manually overwrote overwrote Magento files. Theoretically, this should never be the case, but it’s worth checking. Most tips on improving Magento speed overlook this key pitfall. Sometimes someone gets hired that gets the job quickly but doesn’t think too far ahead. Don’t let it slow you down – if you’re having speed issues, check if your core is hacked.
Bad Hosting. Cheap hosting companies who don’t advertise they have a custom Magento setup are generally bad at hosting Magento. Spend the extra money and find someone who supports what you’re trying to do.
More
I have a virtual machine I manage using Vagrant. When provisioning, I get some updates for the system (Ubuntu) using apt-get, install Node.js using nvm, and then run npm install for various modules. Now strange things happen:
If I try to create and provision the VM in physical network A (at one place), provisioning fails. Either creation already fails and Vagrant tells me that the machine went to an invalid state, or the VM crashes when npm starts to do its work.
If I try to do the exactly same thing on physical network B (at another place), everything is fine.
As the only difference between the two places (I can think of) is the physical network, I wonder how this can happen. I have tried it with two different MacBooks, one running Mavericks, the other running Mountain Lion. The effect happens on both and also when using different Vagrant / VirtualBox versions, so apparently it's actually a problem of the environment.
Any idea what might cause issues like that?
I know this thread is a bit older, but I've got the same problem and I found a solution.
Here is a official issue and a solution https://github.com/mitchellh/vagrant/issues/2786
There are issues related "NAT Networking" and massive DNS querys. I had several npm install running in my provisioner.
My solution, as described at the official github issue:
config.vm.provider :virtualbox do |vb|
vb.customize ['modifyvm', :id, '--natdnshostresolver1', 'on']
end
I hope this helps.
Can I install Drupal on a server with 256MB ram ?
If yes, how many Drupal websites ? The server is supposed to only run such websites.
Can I install Drupal on a server with
256MB ram ?
No problem. Take a Debian GNU/Linux minimal install. Add apache2, php and mysql. Install drupal, You are done.
If yes, how many Drupal websites ?
As much as fit on your disk drives.
All other depends on what modules you need, the request load, the response payload, etc.
Exact same question
Can drupal run on a Debian Linux vserver with 256MB?
Drupal need a LAMP server, lamp can be of any configuration min-max, number sites depend your safe available HDD space.
But i think. its enough to run server to host drupal sites.
Drupal is very processor intensive in my experience. RAM is generally not an issue. I use a shared hosting server with 32 MB RAM to run the latest version of Drupal.
You might be running into problems because of using PHP settings. Try adding memory_limit = 128M to your php.ini file.
If you are still facing problems, then you'd have to find out which modules are eating up your RAM and disable them. If a single module is doing this, then it may be buggy, so you are better off without it.
The solution was not about Drupal configuration but Apache configuration.
I had to change these MPM parameters:
http://articles.slicehost.com/2010/5/19/configuring-the-apache-mpm-on-ubuntu