I was wondering how teams that develop sites using Drupal (or any other CMS) integrate version control, subversion, git or similar, into their workflow. You'd obviously want your custom code and theme files under version control but when you use a CMS such as Drupal a lot of the work consists of configuring modules and settings all of which is stored in the database.
So when you are a team of developers, how do you collaborate on a project like this? Dumping the database into a file and putting that file under version control might work I guess, but when the site is live the client is constantly adding content which makes syncing a bit problematic.
I'd love to know how others are doing this.
You are correct that this is an issue for Drupal--version control works fine until you turn the site over to your client or open it up to users.
Your question seems like a more specific version of this one, which touched on version control in the Drupal workflow. You may find some answers there that help.
For some projects, I have exported all of the views to code, using that feature of the Views module, and I have one project where all of the blocks have been exported, as well. (Although that was a development exercise and not a customary thing to do with blocks.)
Take a look at the work that Development Seed is doing to work around this problem. They are leading the development of the Context, Features, and Spaces modules that work together to store configuration data in modules (outside of the DB) so that it can be versioned with the code.
There is a Drupal group called Packaging & Deployment for discussing the various solutions that are being developed for this issue.
Right now there are a lot of efforts towards creating something that will handle the dev -> production difficulties with drupal in relation to the database. Features, that flaminglogos mentioned is one, but I feel that is more focused on creating stand alone projects, ie ones that would be installed on many sites.
For simple maintaining you dev and prod databases I'd take a look at http://drupal.org/project/deploy and http://drupal.org/project/dbscripts. They support syncing and merging db side drupal config data.
I can't guarantee they are ready for prime time though...
There is a lot of effort of shipping the next drupal version with configuration in code. That's is the key to have it in a version system.
For now you can use the features module, with that you can export things like content types, views, etc. to code, and then compare, version and revert it as you need.
Related
I'm very new to Drupal, so please don't be too mad in case I have any major misunderstandings :) I've tried searching for a similar problem, but is just couldn't find a suitable solution for my case.
We're currently setting up a Drupal 9 project, which will perspectively have a shared development environment and a production environment as well as a local instance to develop on. I'd wish to have a way to synchronize those instances to have the same configuration, content types and optionally even content.
At the moment, I'm developing a theme locally, which means I have installed a Drupal instance inside a XAMPP server. That theme is versioned by git, so it is migratable to another developer without a problem.
For migrating the structure and content (which is obviously saved in the database), I tried using Backup & Migrate, but there were two issues I was facing: The D9 version is not fully supported yet, so an installation via composer fails with default security settings, and there seems to be an already multiple times reported bug when trying to backup the entire site. You can workaround it by backing up the database and the files separately, but this is pretty inconvenient due to other issues (but let's keep it a little short...).
I also tried to export the whole database, which is actually working (after this little fix), but the overhead seems a little high for me. Especially when I just want to copy new content types from dev to prod environment without users, content and so on, for instance.
So, to finally come to an end, is there any best practice for this case? Or should I even consider to go a whole other way?
Thanks in advance!
I definitely wouldn't recommend using Backup & Migrate for this - that's so Drupal 7! Drupal 9 has better tools that are baked into core!
There are many possible ways to import/export Config and Content entities across environments, but I'll share what I believe to be the current best practices.
For Configuration, Drupal 9 has a built-in Configuration Management system that makes it quite easy to migrate Config across environments. It can be used within the Drupal UI, and also integrates with Drush (a command-line tool for Drupal).
Essentially, the Config system exports all Config settings as standardized YAML files, which can easily be included in your Git repository. This will make it incredibly easy to set up another Drupal environment that is identical in its Config settings.
More info on Configuration Management can be found here.
For Content, Drupal 9 has a built-in Migrate API, which facilitates migrations from any data source into a Drupal 9 environment. That means you could set up a migration that would allow you to migrate your Content entities across environments.
I've only ever used this for migrating Content (migrated from Drupal 7), but I believe it's also possible to use this to migrate Config as well.
If you decide to use the Migrate API, you may (depending on the setup of your content) need to install Migrate Tools and Migrate Plus.
More info on the Migrate API can be found here.
I've never worked with any CMS and I simply wanted to play with such ones. As originally I come from .NET roots, so I was thinking about choosing Orchard Core CMS.
Let's imagine very simple scenario, together with my colleague I'd like to create a blog. As I'm used to work with web based systems and applications for a business for me it's kinda normal to work with code repository, having multiple environments dev/test/stage/prod, implementing CI / CD, adjusting database via migrations or scripts.
Now the question is do I need all of this with working on our blog with a usage of CMS.
To be more specific I can ask few questions:
Shall I create blog using CMS locally (My PC) -> create few articles and then deploy it to the web or I should create a blog over the internet and add articles in prod environment directly.
How to synchronize databases between environments (dev / prod).
I can add, that as I do not expect many visitors on a website I was thinking to use Orchard Core CMS together with SQLite. Also I expect that I can customize code, add new modules, extend existing ones etc. - not only add content (articles). You can take that into consideration in answering the question
So basically my question is what should be the workflow of a person who want to create / administer and maintain CMS (let it be blog) as a single person or as a team.
Shall I work and create content locally, then publish it and somehow synchronize both application and database (database is my main question mark - also in a context how to do that properly using SQLite).
Or simply all the changes - code + content should be managed directly on a server let's call it production environment.
Excuse me if question is silly and hard to understand, but I'm looking for any advice as I really didn't find any good examples / information about that or maybe I'm seeking in totally wrong direction.
Thanks in advance.
Great question, not at all silly ;)
When dealing with a CMS, you need to think about the data/content in very different terms from the code/modules, despite the fact that the boundary between them is not always completely obvious.
For Orchard, the recommendation is not to install modules in production, but to have a dev - staging - production type of environment: install new modules on a dev environment, test them in staging, and then deploy to production when it's safe to do so. Depending on the scale of the project, the staging may be skipped for a more agile dev to prod setting but the idea remains the same, and is not very different from any modular application.
Then you have the activation and configuration of the settings of the modules you deploy. Because in a CMS like Orchard, those settings are considered data and stored in the database, they should be handled like content. This includes metadata such as the very shape of the content of your site: content types are data.
Data is typically not deployed like code is, with staging and prod environments (although it can, to a degree, more on that in a moment). One reason for this is that a CMS will often feature user-provided data, such as reviews, ratings, comments or usage stats. Synchronizing all that two-ways is very impractical. Another even more important reason is that the very reason to use a CMS is to let non-technical owners of the site manage content themselves in a fast and direct manner.
The difference between code and data is also visible in the way you secure their changes: for code, usual source control is still the rule, whereas for the content, you'll setup database backups.
Also important to mention is the structure of the database. You typically don't have to worry about this until you write your own modules: Orchard comes with a rich data migration feature that makes sure the database structure gets updated with the code that uses it. So don't worry about that, the database will just update itself as you deploy code to production.
Finally, I must mention that some CMS sites do need to be able to stage contents and test it before exposing it to end-users. There are variations of that: in some cases, being able to draft and preview content items is enough. Orchard supports that out of the box: any content type can be marked draftable. When that is not enough, there is an optional feature called Deployments that enables rich content deployment workflows that can be repeated, scheduled and validated. An important point concerning that module is that the deployment only applies to the subset of the site's content you decide it should apply to (and excludes, obviously, stuff like user-provided content).
So in summary, treat code and modules as something you deploy in a one-way fashion from the dev box all the way to production, with ordinary source control and deployment methods, and treat data depending on the scenario, from simple direct in production database instances with a good backup policy, to drafts stored in production, and then all the way to complex content deployment rules.
We are a software development company and are using Wordpress for static portion of the web site. Naturally, all our workflow is built around version control: multiple developers -> continious integration -> staging -> deployment.
Our challenge with integrating Wordpress into our workflow is that its database is stuck like a bone in the throat: you cannot put it into the version control, easily roll back, promote from staging to production etc.
I am wondering what people do in similar situations? I would like to find a way to integrate WP into the development workflow and not the other way around :-)
Clarification we want to "develop" and test pages on the staging system and when ready then move them over to the production as part of the version upgrade process. We don't want to do full replication of the staging database to production.
That's a common question and one that I've worked on tackling. I've written some code to address these issues albeit the code's not ready for distribution. Basically the idea is to create scripts to import the content and then version control the scripts. (Actually my approach uses a custom import/export format designed to be easy to hand-modify, but the idea is similar.)
Anyway, there are some related questions over on StackOverflow's sister site WordPress Answers:
Questions tagged with the term [staging]
Questions tagged with the term [deploy]
UPDATE
Per the clarification, this would probably be helpful too:
Is there any way to draft a revision of a published page or post? What workarounds have you used?
Hope this helps.
-Mike
I've just hit the same problem. For now we are using MySQL dump files to export/import database content, but it gets ugly with several people working on the database changes.
Since the team that works on the project is all internal and consists of just a few people, I'm thinking into the direction of locking the database dump file in VCS. Subversion had this functionality built-in, but we are using git, which, I think, is conceptually opposite of any kind of locking.
Probably we'll have a workaround script with pre-commit hook to check for the existence of a lock file next to the dump. The person who committed the lock file will be the only one allowed to commit the dump. Once he finishes the work, he will need to commit the removal of the lock file.
It sounds ugly, I know. But I've thought about it for a while and don't see an elegant solution yet.
If you're only using WordPress for static content, then any tool/methodology for version controlling databases should work - for example, work the mysql command line tools into your CI and deployment routines.
I really like drupal somehow. But what disturbs me most is that i can't figure out a clear way of deployment. Drupal stores a lot of stuff inside the database (views, cck, workflow, trigger etc) that needs to be updated.
I've seen some modules that could be used for this task (eg features) and I'm not sure if they are sufficient. Yet they are only for drupal6 and i currently have to work on a drupal5 site where updating is not yet an option.
Any ideas?
This is a weakness. Drupal doesn't have the developer tools built in that make development and deployment easy like Rails does (for example). One problem is Drupal isn't aware of it's environment natively. Secondly, there are too many different methods and modules that require special care. It can get very confusing. But things are getting better with drush and drush make.
I'm assuming here that you have a development environment on your local machine and a live or staging server you upload to.
The first thing you have to do is work out how to get your database fixture and your code to and from your server to your development environment very quickly. You need to make this proceedure as painless as possible so you can keep different versions of your site in sync without much effort. This will mean you will hopefully be able to manage less change every time you deploy. Hopefully...
Moving the database around isn't too hard. You could use phpMyadmin or mysqldump but the backup migrate module is my favorite tool.
To upload code from your local repository or site can be done in a few ways. If you use a version control system like git, you can commit on your local machine and check out again on the staging server. There are also special deployment tools like capistrano you should take a look at. (if you know this stuff already it may benefit others to read). If you're using FTP you should probably try something different.
If you're working with a site that is still in production, you can afford to make small incremental changes to your local site, then repeat on the live site and down load the new version of the database when your changes are in place. This does mean you double handle the database but can be a safe way of doing things. It keeps both your database closer to each other and minimises risk.
You can also export views backup to your server in either your code or importing them into your live site. There is a hack to get around deploying cck changes here: http://www.tinpixel.com/node/53 it works OK but cannot truly manage changes like rollbacks. (Respect to the guy who wrote that)
You can also use hook_updateN to capture changes and then run update.php to apply them. I worked on a d5 site with dozens of developers and this was the only way to keep things moving forward. This may be a good option if your site is live or if you need all database schema changes captured in a version control system (so you can roll back).
Also: Take a look at drush and drush make. These tools can be of great benefit. I can't remember how much support is for d5.
One final method of dealing with this is not to use cck or views (and use hook updates). But this is really only suitable for enterprise sites where you have big developer resources. This may seem like a strange suggestion but it can negate this whole problem completely.
Sorry I could not give you a clear answer. This is because one does not exist yet. You'll end up finding your own rhythm once you get into. Just keep backups of your database if you can roll back to them easy enough.
We have a series of ASP.Net applications that have been written over the course of 8 years. Mostly in the first 3-4 years. They have been running quite well with little maintenance, but new functionality is being requested and we are running into IDE and platform issues. The apps were written in .Net 1.x and 2.x and run in separate spaces but are presented as a single suite of applications which use a common navigation toolbar (implemented as a user control). Every time we want to add something to a menu in the nav we have to modify it in all the apps which is a pain. Also, the various versions of Crystal reports and that we used tables to organize the visual elements and we end up with a mess, especially with all the multi-platform .Net versions running. We need to streamline the suite of apps and make it easier to add on new apps without a hassle. We also need to bring all these apps under one .Net platform and IDE.
In addition, there is a WordPress blog styled to match the style of the application suite "integrated" into the UI and a link to a MediaWiki Wiki application as well.
My current thinking is to use an open source content management system (CMS) like Joomla (PHP based unfortunately, but it works well) as the user interface framework for style templating and menu management. Joomla's article management would allow us to migrate the Wiki content into articles which could be published without interfering with the .Net apps. Then essentially use an IFrame within an "article" to "host" the .Net application, then...
Upgrade the .Net apps to VS2010, strip out all the common header/footer controls and migrate the styles to use the style sheets used in the CMS.
As I write this, I certainly realize this is a lot of work and there are optimization issues which this may cause as well as using IFrames seems a bit like cheating and I've read about issues with IFrames.
I know that we could use .Net application styling, but it seems like a lot more work (not sure really). Also, the use of a CMS to handle the blog and wiki also seems appealing, unless there is a .Net CMS out there that can handle all of these requirements.
Given this information, I am looking to know if I am totally going in the wrong direction? We tried to use open source and integrate it over time, but not this has become hard to maintain. Am I not aware of some technology out there that will meet our requirements? Did we do this right and should we just focus on getting the .Net streamlined? I understand that no matter what we do, it's going to be a lot of work. The communities considerable experience would be helpful. Thanks!!
PS - A complete rewrite is not an option.
Hmm, we're in the midst of a project to do something that sounds familiar. We're using www.sitecore.net CMS but you could use the Open Source alternative Umbraco again both of these will have a learning curve, but they're .Net apps and aren't targetted specifically at blogs. SiteCore ultimately can use normal .Net user controls if you want, though it's slightly against their model, but it works.
One thing I'll warn you of is SiteCore Must be the root of your website, it has to control the root of the domain (it has a urlrewriting module that needs to be at the root) and you can tell it to exclude certain folders where your applications might live. You can obviously put your navigation in a folder under the root of the site. Also note SiteCore's a .Net 3.5 application running under the 2.0 runtime.
Are your sub-applications.. Actual seperate applications in virtual dirs or something I'm guessing?
Depending on the nature of the .Net apps, you may find DotNetNuke to be a useful choice.
It's a CMS where you write widgets ('modules') in .Net, then add them to the pages of the CMS. In your case, you'd wrap your existing functionality in such widgets. I've done exactly this several times, and now that I'm used to it it's no big deal.
The downside is you have to learn to swim in the DNN environment, which (like any CMS) has a bit of a learning curve.
I'd have to know a lot more about your existing apps to be sure this is a plausible option. If it looks appealing, you should probably contact someone who's dealt with a situation like yours (such as myself) and go into detail. It's very easy to find yourself in a dead end with these CMS frameworks.
Edit: Like a product mentioned in a different answer, DNN has to control the top level of its subdomain -- all requests begin by going through Default.aspx and are then dispatched in various ways.