I like to use a simple Git workflow for static web sites but I build Joomla and Wordpress sites a semi-regular basis too. However I am at a loss as to how to use Git with with database driven site development.
For a static site I would 'Push' to dev.websitename.com, then push to www.websitename.com once the dev site checks out. How would I mimic that process with database driven site like wordpress or joomla.
Thanks in advance for you insight!
You can definitely use Git with your website code, such as changes to your WordPress theme/plugin, exactly as you would if you are developing a static website.
However, you wouldn't use it for your database. Git provides version control for code, while WordPress and Joomla already manage content stored in the database. Plus, Git wouldn't understand a database, so it wouldn't have any advantage over a periodic backup, which you should already be doing. Take a look at running a dev copy of your site for how to download your database directly from your server.
By the way, if you use Git with WordPress/Joomla, you should add e.g. cache, logs, tmp to .gitignore. There are also lots of tutorials out there--try searching e.g. http://google.com/search?q=wordpress+git.
In addition Chris, you may want to embark on your Git workflow without the handy script approach (at least initially). The script approach and using Git hooks can sure seem sexy (well, because they are) and handy too, but initially why not go with a more manual cmd line approach, which will also help you familiarize yourself with Git.
Once you've got your repo setup (GitHub, Bitbucket, somewhere else) and you've pushed your latest to it and are ready to deploy to production or staging, just login to your host and from wherever you've initialized the git repo (site root, example: /site) just do a:
git pull origin master
This will fetch and merge your code. Good idea to test this on a dev/staging environment and if the merge goes well then do it in production.
Related
For the continuous integration and deployment of websites, I am using this pipeline:
But for many CMS like wordpress, prestashop, magento and others, the configuration of the website and the installation of plugins is done in the back-office of the deployed website.
For now, I am building the docker image on top of the CMS base image, then replacing all the /var/html directory with the files in github. Then Kubernetes is deploying the containers and plug a database and a persistent storage
Hence, this is breaking my pipeline: imagine that someone is installing and configuring a plugin in the back-office, then someone else is doing a modification on a file and pushes it to github. The github repo doesn't have the info that a plugin was installed and will build and deploy a new image without it.
How to integrate all the modifications done in the back-office in my github repository?
The solution we use is an override of the DB class.
So we monitor a number of tables (Configuration, module, hook, etc ...) and we store all queries about it in a sql file.
So during commit, we also have a .sql actions to perform on the database side.
Once deployed, either you manually execute the sql, or a script detect that new SQL are present and executes.
In this way we are always up to date.
This solution we developed in the form of Prestashop modules to track all actions.
Regards
My (by any means not ideal) working solution:
Create plugins folder outside docker and symlink this folder in dockered /wp-content/plugins
recreate above in production
Then installing new plugin doesn't break CI flow but requires two independent installations and configurations, if you (or dev team) need to install something new.
So you basically treat plugin files same way as you already do it with DB.
Total newbie question but what is the best practice when it comes to using SSH with Git? I'm working on a WordPress project. In the root I have gulp and other dev files/folders like SASS and Scripts that I don't need on the server and in the same project I have my WordPress folder that contains a theme and a few custom plugins. As you can imagine when the theme or any of the plugins are ready to be deployed I don't want to pull everything in my repository on the server. So far as a newbie I've always just pull and pushed the entire repository and used FTP to upload what I need to the server, so how is this done with SSH and Git and is there a better way to have my setup?
EDIT: To make my question a little bit more clear let me give you an example of what I think my issue is. In my main project folder, I have a SASS folder next to my WordPress folder. All I really need to deploy to the server is the WordPress folder. My build process that happens on my dev machine combines all of the SASS files into a single CSS that is then placed into the WordPress folder. I need the SASS folder to be tracked by Git so that any other developer can pull them and continue developing so I can't have git ignore it. However none of those SASS files need to be on the server for WordPress to work either. I just simply need to deploy the WordPress folder and everything that's in it.
I understand the idea of creating a bare repository on the server and using post-receive hook to point the git folder sitting outside your web root to point to where the web root is. But that's basically how GIT and SSH work and that's not answering my concern.
Not with Git
Git is not designed to pull specific files or directories only. It's a directed acyclic graph with binary blobs as objects and sometimes multiple objects get compacted into a single larger object.
Due to Git design, your specific request is not possible.
Alternatives
post-receive hook
If your website only contains simple static files then it's okay to push to a git repository over SSH. In reality, it's unlikely your repository will be large as long as you don't have non-text files.
Take for example the following setup.
/var/lib/www - apache web dir which is the cloned copy of www.git
/var/lib/www.git - a bare git repository.
/var/lib/www.git/hooks/post-recieve - A server side git hook. It can be a shell script that pulls the www repository when this repository is updated.
Sample post-recieve hook script:
#!/bin/bash
cd /home/sam/sandbox/git-hooks/www
unset GIT_DIR
git fetch origin master
git reset --hard origin/master
Zip up build in a tar.gz
At the end of your build you can zip up your files in a tar.gz. This file should be hosted somewhere (perhaps GitHub releases if you're using GitHub). Some enterprises use on premise artifact hosting like Nexus or Artifactory.
The idea being: you have a tested artifact that has a specific sha256sum. The artifact you test is the exact same artifact which eventually goes to production.
Diving into more detail such as continuous integration, continuous delivery, and the software development life cycle might be out of scope for your question.
No best practice.
Git is for source control, not for deployment. There is no best practice for using git this way because git is not a deployment tool. You also don't need git history on your server. In fact, you don't need git at all unless you insist on using it for deployment. You are welcome to use it this way but it's not ideal because of exactly the kind of problem you're asking about.
What is the best practice?
There are a number of tools you could use to handle your deployments. Most of the tools generally let you set up a series of steps that let you deploy the code you want into the environment you want. You could go with simple tools such as Phing or Deployer in the PHP world, or something more sophisticated like Puppet or Chef if you have more complex needs. You could just write your own bash scripts if what you need is really very simple. I recommend Phing or Deployer given the info you've provided. https://deployer.org/ https://www.phing.info/
You'll just configure whichever tool you want to ssh into your target box and copy over only the files you want into the directory you want on the server, in whatever way you would like to do that. Usually, you have the script copy files into a temp dir, tarball them up, ssh them over and untar them. After that, you'll usually do some additional work on the server to move files around, change symlinks, whatever else you might need to do.
What about compiled SASS, ES6 js files, or modern static stuff?
All you need to do is add steps to the handle the static files and where you want them to go. Include the generated static files in your tarball when you push stuff up, and put them in the right directories in the server once you untar it.
When you configured your SASS compiler, and whatever other pre-compiled static code you may have - you configured it to create a destination file. That is, the file(s) of actual CSS and JS that they generate. That's all you need to bring along - and if you have the destination directory set to be inside your wordpress theme, you may not even have to pay all that much special attention to it's handling. You may need to move them somewhere else once they are on the server but that all depends on the specific setup in your server, which I think is outside the scope of this question.
Additional Notes:
You didn't ask about this but I thought it was worth mentioning, that you shouldn't be sending the entire wordpress repository every time you update. Just like you don't need the uncompiled SASS code, you also don't need to be repackaging core WordPress. You don't even need to be commiting core WordPress, its a dependency and you don't need to be changing it.
All that should be getting committed by you is your theme and plugin code, and the uncompiled static files. Compiled static files and external dependencies like the WordPress core don't belong in your git history. For deployment purposes, WordPress should already be installed. The stuff in your tarballs should just be plugins and themes, and additional static files if they aren't already in there for some reason.
TLDR;
Don't use git for this. Use a tool like Phing or Deployer. Build your static files into your theme, and create phing/deployer scripts that tarball up only the code you want, SSH's it over to your server, and untars it into the directories you want. If you have some special location on the server for your static files, just make sure to add steps in your script for that.
So, based on your question and comment, there are three computers involved. There is a web server (when you say "server", I take it as a Web server in this scenario, or the server computer that runs a Web server program). There is another server where your git repo is hosted. And, there is your dev workstation. Is this correct?
It seems like, you have a cloned git repo on your Web server. Your current practice/workflow appears to be (1) (based on your expression "SSH'ed into my server") you log into the web server via SSH (just like Telnet) from your workstation (SSH is just a protocol, which can be used for different purposes). (2) you pull from your repo on hosted service (e.g., github), and (3) deploy it to your "www" directory on the same server. Is this correct?
(I can think of an alternative scenario based on your use of the word "FTP", etc., but let's focus on the above scenario, for now.)
Now, your question is, whenever you "pull" (on your Web server), you feel like you are pulling everything from your repo on your hosted service. And, is there a better way? Am I understanding your question correctly?
If so, as another commenter suggested, git (and, any version control system, in general) is very good at fetching "deltas" only. If you are worried about "fetching everything" every time you pull (the step (2) above), then your worry is unfounded.
Now, the question is, why do you have a git repo on your Web server, if that is indeed the case? This is a pretty legit setup and I've done this before (e.g., on EC2). But, as a best practice, people generally don't do that on "production" servers. It's because you have to "build" your web app, and you really don't want to do that on production servers.
The next question is, what do you exactly do in Step (3)? The build process (whatever process you use) typically generates an "output" which can be directly deployed to the web server. (The convention is the output is generally a single folder, "public", "www", "dist", or whatever, or a single file (e.g., tar.gz, zip, jar, war), etc.) Regardless of whether you build the deployable output on your dev workstation (or, a build machine) or on your Web server, you don't generally do "deltas" in this context. Even if you've only changed a single file (say, a CSS file), you generally build the whole output again (instead of, say, just replacing the changed CSS file only). When you use FTP to upload files, etc., you can selectively upload certain files and/or directories, etc., but as a general practice, we don't do that. We always build the complete output from scratch and deploy it to the Web server. (This is mainly to reduce the potential deployment errors and increase the reliability.)
So, to answer your question, (A) If you are pulling git repo on your Web server, you should really change that practice, and move the build process to your dev computer or a dedicated build machine. (BTW, services like github, gitlab, TFS, ... provide the build service for you.) (B) If you are currently selectively FTP'ing your web app files to your Web server, then you should really consider adopting some kind of formal build, and deployment, process moving forward.
After your SASS build process is done use scp or rsync to move the files to the prod server:
scp -r /[local wordpress dir]/wp-content/themes/your-theme/ username#your.prod.server.com:/path/to/dir/wordpress/wp-content/themes/
scp -r /[local wordpress dir]/wp-content/plugins/* username#your.prod.server.com:/path/to/dir/wordpress/wp-content/plugins/
I am working in a project and using git ssh with bitbucket following is the process i am using it may work for you also if not please correct me :
Step 1 ->I have setup git and create repo in bit-bucket.
Step 2 ->And setup project with my local and linked with my repo.
Step 3 ->connect my server using ssh.
Step 4 ->Work in my local and commit and push all changes in my git repo.
Step 5 ->Run git pull on ssh so all changes deployed in my server.
I am using above process and i love this process.i have used .gitignore file that is not required for push on my repo.
Thanks
I have a wordpress site that I am finally getting into git. Previously I was editing changes locally and uploading via ftp. I want to come up with a strategy to be able to do the following:
Update my git repository with any changes made on the ftp server (plugin updates etc). That I will then sync that down into my local copy
Push locally tested changes into git and then deploy via ftp (I know how to do this part)
I don't know the best way to approach 1 and I'm very new to using git (usually use svn).
Thanks
Using a conventional development workflow on Wordpress is challenging.
Because WP puts too much in the database, including site config, it makes syncing changes from one environment to the other a real pain.
We have a number of WP installs (against my better judgement) and organise our workflow like this.
You might lighten this process if you're not part of a dev team and are going alone.
Local Dev
codebase changes only. Including shortcode development and templating. This lives under a GIT workflow (branches, tags, etc). Plugin/framework updates are installed here, and tested before adding to the gitball.
Beta Live
All content is built here, with any code based changes being requested as needed. Testing is carried out here, responsive, UAT etc.
Code changes are pushed up via deployment scripts which trigger either a GIT pull, or by syncing files via ssh:rsync. heres a good resource on how to use git hooks for this.
Live
For live deployment, we use a WP database migration plugin to get content across, and the git repository to push tagged releases up to the live server. We also use the sync tool to pull down up to date copies of the database to our local dev machines.
I'm sure there are lots of other opinions on what's best, and how to implement them.
My opinion
The fact is is that WP is a bad choice for websites that require a lot of under the hood attention. It was never designed for this, and consequently is a real pain to force into a proper development cycle. If I must use a CMS, I generally use Drupal for sites that need more functionality, its a steeper learning curve, but more than worth it.
You could follow a git flow pattern. This consist of three main branches:
master
release
develop
Preform all of your development in the develop branch which you would build and test locally. Then when you are ready to test on a server merge your changes into the release branch for testing. When everything checks out you can then merge into your master branch. From here you can either manually ftp the files from the master branch like you have been or set up a deployment hook to automate deployment to your server when changes are detected.
As your project grows you can implement a couple of other branches to further maintain your build. You can base a new branch feature off of develop when you need to add a new feature then merge it back into develop following the flow back up again. As well as a hotfix branch to take care of any bugs "on the fly" straight off the master branch.
Here is a good example of the git flow pattern:
[http://nvie.com/posts/a-successful-git-branching-model/][1]
This article goes into a lot more detail about the git flow pattern and is an awesome resource!
Be sure that whenever you start a new development phase you merge from master to develop so you know you are starting with the most recent version of your project.
I've developed WP sites and themes before, but mostly independently, and mostly live via FTP.
This seems to be frowned upon, and it also seems like a stupid way to continue if I want to start with preprocessors, which look amazingly useful and fun.
So local development makes sense, with MAMP for the WordPress requirements, and something like codekit because I am no terminal ninja. But I'm also going to working within a team -- so, test server that everyone can access?
I used Logmein to get into someone else's system years ago, but I can't imagine that's standard practice.
This seems like a basic thing, and I'm sure there's knowledge that I'm missing, but how do people set something like this up? What are best practices for team projects?
Working with WP is tough on a team. I can explain what I typically do when working within a team:
Git
USE IT!! This is especially when two or more people are working on the same project. Once you start using it, you will use it even when it is just you working on a project.
A team I freelance with uses Bitbucket as it provides free private repos, and a program called Sourcetree (made by the Bitbucket team), which is a decent git GUI. The GitHub app is pretty decent as well, however Sourcetree has a few more options.
Wordpress Workflow
Each team member works locally using MAMP, because using CodeKit or Grunt and LiveReload just makes the world a better place, and load repsonse times are way shorter.
Just the WP theme folder that is being developed is pushed with git to the repo.
We include a sql export folder (named 'db') within the theme folder, typically structured like this:
THEME FOLDER
|-(theme files and folders)
|
|-db
|-/-local
|-|-/-team_member_name_1
|-|-|-/-(local sql exports)
|-|-/-team_member_name_2
|-|-|-/-(local sql exports)
|-|-dev
|-|-/-(sql exports from development site)
|-|-production
|-|-/-(sql exports from production site)
The local folder has a folder for each team member to put their exports in. These can then be merged as needed for the dev site. (Note: The db folder is never put on the production server-it's just used to pass sql exports back and forth with git)
It seems like no matter what, importing and exporting sql databases are the only hurdle in a streamlined workflow. We just try to keep doing it to a minimum.
So our basic flow of a project is:
Setup and install a WP dev site (www.devcompanyurl.com/wpdevsite)and database
Setup and copy of the site locally using MAMP
Connect theme files to CodeKit/Grunt
Establish project repo, use git to push a master set of theme files that the rest of the team will download and work from.
Then it's just a matter of pushing, pulling, and merging files, and doing the occasional .sql import.
Hope this helps!
I have been developing a Drupal 6 site on my PC using XAMPP. I'm done now, and everything looks peachy.
Problem is, I need to put all my content (including custom modules and themes) up onto a staging server which only has a fresh Drupal 6 install on it. I can't imagine having to set up all my custom content types and whatnot all over again on the staging server.
So I ask, how does one go about doing what I need to do? Which is essentially duplicating my Drupal install from my PC, to the staging server.
The staging server is running Linux, and I develop on a Windows PC, if that helps.
Thanks in advance.
Copy up all the files from development to live, and mysqldump your database and run that on the live server. Then all you have to do is change the settings.php file to point at the right database, if for some reason 'localhost' is not also your mysql database.
The quickest solution is probably the backup_migrate module. It is only a tool to copy your database. You could also use phpmyadmin or similar instead if you wanted. The backup_migrate module do have some good defaults settings as to which tables to skip (like cache tables). All the settings etc. that is not defined in code is stored in your db. So you only need to copy the db to be set. You can choose to exclude some tables, like the node or user table if you don't want to bring over your test data.
If you don't use subversion, then you gotta manually copy the files (rsync, scp, whatever) and the db (mysqldump).
what we usually do is have a hierarchy of independent subversion repos as follows:
core
sites/all/modules/contributed
sites/all/modules/custom
sites/all/themes/ (we develop our own and don't use contributed themes)
sites/all/libraries
then we use the svn:externals properties so that if you check out "core" you get every associated repo.
we got about 2 main developers with 4 other guys that may also contribute code to the site. each have their own local dev environment and we all got a common sandbox - where we make sure the stuff we wrote doesn't break someone else's module (it has happened before!).
we use svn commit hooks to update the beta/staging/sandbox site upon commit.
with all that setup, [re]deploying a site is a simple matter of going to the proper folder and issuing a "svn co http://repolocation/reponame ." and then updating the DB.
two last things to consider:
we are moving from svn to git
the features module will allow you to save changes you make to your own modules (views, content types, etc) and package all that into a deployable module so you don't have to duplicate your efforts. we are also looking into using this for ourselves.
I hope this helps you.
I second using backup_migrate. It's great.
When I'm installing a fresh site from development to production, I:
backup the site using backup_migrate module
copy all the files up to the server
edit the sites/default/settings.php to have the right database path and account info
do an import of the last backup_migrate dump (usually using mysql < backupfilename.sql, unless I already have drupal setup and have backup_migrate installed, then I use the GUI
But take a look here for the official version:
http://drupal.org/node/776864
Now, you didn't ask, but when the site is live and users are contributing content, moving future development versions of your site from development/staging to production without blowing away live content is a whole different problem, and one that Drupal doesn't have a good answer for...
Andy-