The Setup:
I'm setting up a Wordpress-powered application using Elastic Beanstalk from Amazon Web Services. All development is being done locally under a MAMP apache2/php5 server environment with a GIT repository controlling the entire application root.
Deployment Workflow:
After committing any code changes (edits, new plugins, etc) to the repo the application is deployed using AWS EB CLI's eb deploy command which pushes the latest version out to any running EC2 instances managed by Elastic Beanstalk.
My Issue:
Sometimes the code changes aren't exactly syncing up between my development/production environments and I'm not sure how to overcome it. Especially when trying to install and setup plugins like W3 Total Cache or WP Super Cache.
Since my local environment doesn't have things like a memcahced server installed, but my production environment does (ElastiCache) I'm unable to save the proper settings file and deploy it for use in my production environment. These plugins won't allow me to select the needed services because it sees them as not available...
It seems I can only get W3 Total Cache to work if I install it directly onto a live production environment, which seems like a bad idea.
Given the above:
Am I going about deployments the wrong way?
Should plugins like W3 Total Cache be installed and configured on
local development environments and pushed to production environments?
I cannot comment on the issues specific to Elastic Beanstalk, but based on experience I can make a suggestion about the second part of your issue statement:
You are better off running a development environment that mirrors your production environment as closely as possible. I suggest that you convert from MAMP to a VM environment like VirtualBox. You might want to check out puphpet.com for help in getting it set up. It requires some startup effort, but gives you an environment similar to or the same as your production servers. For example, you could run memcached yourself so you could actually test it with W3 Total Cache.
As for your second question, just installing a plugin in the production environment without testing it beforehand has obvious risks (but then again clients do that all the time). I would prefer to test first. To a certain extent it probably depends on how critical it is if the site experiences downtime or weirdness.
I would suggest you to create another environment on Beanstalk.
It's easy, fast and more reliable than a VM in your case because it will allow you to test your deployment process as well.
I usually have 3 environment for a every website. Each environment is on its own branch. If your configuration is different between environment (url and database access for example), just store your wp-config and other config files into S3 (you may not want production password in your git repository), and through ebextensions you can download them into your website automatically.
I use AWS Beanstalk that way for 16 websites and some are wordpress one. All with autoscaling and able to get thousands of users simultaneously.
Don't hesitate to ask me for further details.
Related
I am currently working on a Drupal project. Being a developer, I have often heard disagreements on whether it is better to develop on a local stack or on a shared development server. To anyone here with Drupal experience, do you have any insight or advice for me?
This is nothing Drupal-specific.
Everybody should work on their local machine. In a local development environment. There are thousands of options to choose from to make your own computer run a website locally. Start with MAMP maybe.
Use Git (and the Gitflow Workflow) to commit your changes to a repo where your co-workers can pull them from into their local environment.
Use deployment routines or webhooks to have your changes pulled onto the live server on every release automatically.
When developing things always move in two directions:
Database copies move down (from live to dev to local).
Code changes move up (from local to dev to live).
What should be good when multiple persons work on one and the same code instance at once? Nothing. Imagine all the conflicts that can arise from two persons working on one file at the same time. Insane!
Forgive me if this is an obvious question, but I am trying to figure out what the best way is of handling autoscaling of EC2 instances running WordPress such that their themes and plugins (along with their associated configurations) are preserved.
I am already able to decouple the data and content layers via RDS and S3, respectively, but I am struggling how to preserve the themes and plugins through an EC2 instance autoscaling event.
My EC2 instances are configured as follows:
EC2 bootstrap script installs WordPress onto blank Amazon Linux AMI
EC2 runs behind an ELB
Database is on RDS
Web content is on S3 (using W3 Total Cache plugin)
Plugins/themes are installed on the local EC2 filesystem
To preserve themes and/or plugins through an EC2 autoscaling event, I could:
Install the themes/plugins I need first, then upload the /wp-content/plugins and /wp-content/themes dirs to S3, downloading them automatically each time an EC2 instance restarts via the bootstrap script. DISADVANTAGES: need to update S3 every time i make a config change, not all plugins are installed neatly within the /themes subdir, and changes to one instance don't flow to all (need to restart the cluster everytime a change is made).
Install the themes/plugins I need first, then take an AMI snapshot of the entire instance. Use this AMI as a template when launching new instances. DISADVANTAGES: need to update the AMI every time i make a config change (seems tiresome), and changes to one instance don't flow to all.
Create a symbolic link out of the /wp-content/plugins and /wp-content/themes dirs, pointing to an EFS filesystem that is mounted on all EC2 instances. DISADVANTAGES: EFS can be a bit slow, not all plugins are installed entirely within the /themes subdir.
Anybody have any experience with this? Am I over-engineering this? Perhaps the themes/plugin files don't really change much throughout the lifespan of your WordPress blog (ie, once you're set up, you don't really find yourself changing them much), in which case maybe Option 1 (zip to S3 and download via bootstrap script) is the best option for me, and Option 3 (EFS) is over-engineered.
I would love to get your take on this if you have experience with this conundrum!
Thanks in advance!
You can take a look at this link:
https://cloudonaut.io/wordpress-on-aws-smooth-and-pain-free/
It provides a CloudFormation template that sets up an ASG backed by EFS, installs wordpress and some plugins there, uses RDS for their database, sets up CloudFront as CDN and a few other goodies.
I tweaked their template for our use case and added an extra ASG with spot instances, replaced all the VPC stuff with references to my VPC template and tweaked the LaunchConfig so it automatically sets up the S3 Offload plugin with a bucket created on my template. It also automatically sets up the certificate for the ELB and a few other goodies.
I thought that would be the end and that I would be able to forget about WordPress and leave another team to work on it. Wrong. They complained it felt sluggish and some plugins failed to install with timeouts (and installing them manually using wp-cli took way too long, one of them up to 2 minutes and a half).
So here my 2 cents: set up RDS and CloudFront, use a reserved instance for wordpress and offload your static assets to S3 using a plugin. Once the site is completely set up bake an AMI or take a snapshot of your EBS and set alarms so in case the instance breaks down you can quickly spin up a new one with your AMI/snapshot.
Either that or have your dev team baking AMIs from a dev environment so you can set up an ASG in your production environment with those.
I got to a point where I believe trying to set up WordPress for a non-dev team (meaning, they can install/upload plugins and themes from the browser) in an ASG is just madness without support from a dev team (baking AMIs, updating stacks). You could, of course, automate all of this. You could, of course, develop a full new site using anything else. Your call.
/rant
As we all know, we can run a meteor app by just typing meteor in a terminal.
By default it will start a server and use port 3000.
So why do I need to deploy it using MUP etc.
I can configure it to use port 80 or use nginx to route to port 80 for the app. So the port is not the point.
Edit:
Assume meteor is running on a VPS or cloud server with public IP address, not a personal computer.
MUP does a few extra things you can do yourself:
it 'bundles' the code into a single file, using meteor build bundle
the javascript is one file, and css another; it's minified, and obfuscated so it's smaller and faster to load, and less easy to decipher on the client.
some packages are also meant to be removed when running in production. For example meteorToys, the utility toolset to look up collections and much more, is not bundled into the production bundle, as per the instructions in its package. This insures you don't deploy code with security vulnerabilities (Meteor toys basically opens up client side delete / updates etc... if you're not careful)
So, in short, it installs a minimal version of your site, making sure that what's meant for development only doesn't get push to a production environment.
EDIT: On other reason to do this, is that you don't need all the Meteor build tools on your production server; that can add up to a lot of stuff, especially if you keep caches going for a while...
I believe it also takes care of hooking up to a remote MongoDB Instance (at least it used to be the case on the free meteor site) which is more scalable and fault tolerant than running on the same instance as the web server, as well as provision storage etc... if needed.
basically, to deploy a Meteor app yourself manually, you need to:
on your dev box:
meteor build bundle your app to a tar file (using the architecture flag corresponding to the OS you will use)
on the server:
install node v0.10 (or whatever is the current version of node required by Meteor)
you might have to install Fiber#1.0.5 (but I believe this is now part of meteor install already)
untar the bundle, get into bundle/programs/server/ and run npm install
run the server with node main.js in the bundle folder.
The purpose of deploying an application is that you are situating your project on hardware outside of your local machine. For example if you deploy an application on Heroku app you create a repository on heroku's systems and that code based is used to serve your application off of their servers.
If you just start an application on your personal system, you will suffer a lack of network and resource availability as well as under use of computer time at non-peak hours as your system will need to remain attentive for additional users without having alternative tasks. Hosting providers provide resources as needed, and their diverse client base allows their systems to work around the clock on a global scale.
I have an app running on my own digitalocean VM that I'm trying to play around with to figure out how to run a meteor production server. I deployed it with meteor build, but now I'm a bit unsure about how to push updates. If I build a new tarball on my own machine, I will loose file references that my users have made to files in bundle/uploads, because the remote filesystem isn't incorporated into my local project. I can imagine some hacky ways to work around this, but besides hosting the files on s3 or another 3rd party server, is there any way to "hot code push" into the deployed app without needing to move files around on my server?
Am I crazy for wondering what the meteor equivalent of git push/pull is in production, or just ignorant?
You can use dokku (https://github.com/progrium/dokku). DigitalOcean allows you to create an instance pre-installed with dokku too.
Once you've set up your ssh keys, set the environment variables, ROOT_URL, PORT and MONGO_URL you can add that server as a git remote and simply git push to it.
Dokku will automatically build up the Meteor app and have it running, and keep it up to date whenever you git push.
I find Dokku is very convenient. There's also flynn and deis which are able to do the same in multi tenant environment with way more options.
Just one thing to keep in mind with this is to push the guys who own the repo to keep the Node version in the buildpack up to date. Meteor is a bit overzealous when it comes to using the latest version of Node and refusing older versions.
Meteor does lack a bit in this department. I can't remember where I may have heard this, but I believe they intend on adding this very popular Meteor deployment package to their library. Short of switching to a more compatible host, I'm not aware of any better solutions.
I worked with Plone 2 several years ago, but its workflow and security systems (not to mention the new theming engine) seem the best fit for a current project. Plone 4 looks much better from a usability standpoint, but in its drive not to force a particular setup on users, it has become a bit of a maze as to how best to host it on our public-facing Ubuntu servers. I’ve got Plone installed using the Unified Installer (in /home/plone/Plone/zinstance), and have been running it in the foreground for development, but I’m now going to need to set things up for the move to making this website live.
I want to achieve a development instance on the project server that we can use for our own customisations (with additional local instances on our desktops for testing before checking into the development instance), then a site that our client can test things on before they go live, and finally the live website.
In particular:
I’m assuming that the live site will be based on ZEO, while testing and development will be standalone; does that make sense?
At present, everything lives in /home/plone/Plone, while all our other (generally PHP CMS) sites are hosted from /srv/<domain>/www; everything for the domain is then backed up in one go. What’s the best way to layout the different Plone instances to fit into this?
What’s the best way to move our changes to buildouts and products through the cycle from development to testing and then to a live site? We currently have a basic deployment process using Mercurial for other systems.
Current best practice is to use buildout to define a fully reproducible deployment.
Generally, you use a production.cfg and a development.cfg configuration that share configuration via inclusion, to tweak the deployment to the specific needs of either environment; you can expand on this model as needed. A big project might have a staging.cfg to deploy new features to a test environment first, for example.
Whether or not you use ZEO in development depends entirely on your use cases. Most development setups certainly do not need it, but in a large deployment that includes async workers you may find you need a ZEO server when developing anyway. Again, buildout will make switching your development environment around for changing needs easier, especially when combined with supervisord to manage the various daemons used in the setup.
For the biggest deployment I manage, we use a combination of subversion and git, but only for historic reasons. This is all being moved to one git repository. In the end, the repository will hold the complete buildout with it's various configurations, development.cfg, staging.cfg and per-production-machine configuration files (one for each server in the production cluster). For a 3-machine ZEO setup, that'd be one zeo.cfg and instances-1.cfg and instances-2.cfg, for example. The core development eggs are stored in the same repository in src/.
Development takes place exclusively in per-feature and per-issue branches. Once complete, a branch is merged into the staging branch, and the staging server is updated and restarted. Once the customer has signed off on each merged branch, and a roll-out has been planned, we merge the approved branches to the main branch and tag a release. The production cluster machines are then switched to that new tag and restarted.
We also use Jenkins for continuous integration testing; both the main and the staging branches are put through their tests at least once a day, letting us catch problems early.
All daemons are managed by a dedicated supervisord, which is started via a crontab entry (#reboot), not the native OS init.d structure. This way we can manage the running daemons entirely with buildout. The buildouts even generate logrotate configurations and munin monitoring plugins; these are symlinked into the OS locations as needed.
Because a buildout is entirely self-contained, it doesn't matter where you set up the production buildout on a machine. Do pick something consistent, document it and stick to that. Make sure you have sufficient disk space for your daemon needs and otherwise not worry too much about it.
ZEO or not - for production or development - depends on personal needs and preferences. Many devs use ZEO for production and development and many do not. ZEO is needed for scaling (multiple app servers)
Nobody cares...install Plone where you want it or need it...re-running buildout after moving an installation fixes relocation issues. It is your system, not ours...you decide.
Usually buildout configurations live in a repository and can be pulled on the production server from the repository....otherwise you need to copy over the related buildout files yourself some. Typical installation works like this:
run virtualenv
checkout the buildout configuration from git/subversion
run bootstrap
run buildout