In the publishing scenario I have, we have multiple deployers pushing content to both file system and database (broker). Pages and Binaries are put on the file system, everything else in the Broker. We have one of the deployers putting the content into the database. Is this the recommended best practice?
If the storage configurations in all deployers also put the content into the database, how does Tridion handle this? Could this cause duplicate entries, locking failures etc?
I'm afraid at the time of writing I don't have access to an environment to test how this would work.
SDL best practice is to have a one-to-one relationship between a deployer and a publication; that means so long as two deployers do not publish the same content (from the same publication) then they will not collide providing, if a file system, there is separation between the deployed sites e.g. www/pub1 & www/pub2.
Your explanation of your scenario needs some additional information to make it complete but it sounds most likely that there are multiple broker databases (albeit hosted on a single database server). This is the most common setup when dealing with multiple file systems on webservers, combined with a single database server.
I personally do not like this set up as I think it would be better to host file system content in a shared location & share single DB. Or better still deploy everything to the database and uses something like DD4T/CWA.
I have seen (and even recommended based on customer limitations) similar configurations where you have multiple deployers configured as destinations of a given target.
Only one of the deployers can write to the database for the same transaction, otherwise you'll have concurrency issues. So one deployer writes to the database, while all others write to the file system.
All brokers/web applications are configured to read from the database.
This solves the issue of deploying to multiple servers and/or data centers where using a shared file system (preferred approach) is not feasible - be it for cost or any other reason).
In short - not a best practice, but it is known to work.
Julian's and Nuno's approaches cover most of the common scenarios. Indeed a single database is a single point of failure, but in many installations, you are expected to run multiple schemas on the same database server, so you still have a single point of failure even if you have multiple "Broker DBs".
Another alternative to consider is totally independent delivery nodes. This might even mean running a database server on your presentation box. These days it's all virtual anyway so you could run separate small database servers. (Licensing costs would be an important constraint)
Each delivery server has it's own database and file system. Depending on how many you want, you might not want to set up multiple destinations/deployers, so you deploy to one, and use file system replication and database log shipping to mirror the content to the rest.
Of course, you could configure two deployment systems (or three) for redundancy, assuming you can manage all the clustering etc.
OK - to come clean - I've never built one like this, but I'm fairly sure elements of this kind of design will become more common as virtualisation increases, and licensing models which support it. (Maybe we have to wait for Tridion to support an open source database!)
Related
We have a large, complex Kentico build which uses Kentico's Continuous Integration locally, and Kentico's Staging module to push Kentico object changes through various environments.
We have a large internal dev team and have found that occasionally (probably due to Git merging issues) certain staging tasks aren't logged. When dealing with large deployments this is often not obvious until something breaks on the target server.
What I'd like is to write a custom module which can pull certain data from a target server (e.g. a collection of serialized web parts). I can then use this to compare with the source server to identify where objects are not correctly synchronized. I'd hoped this might be possible using the web services already exposed by Kentico which handle the staging sync tasks.
I've been hunting through a few namespaces in the Kentico API (CMS.Synchronization, CMS.Synchronization.WSE3 etc.) but it's not clear if what I'm trying to do is even possible. Has anyone tried anything similar. If so, could you point me in the right direction?
Instead of writing your own code/tool for this I'd suggest taking advantage of what someone else has already done. This is like Red Gate's SQL Compare for Kentico BUT on steroids. It compares, database data, schema AND file system changes on staging and target servers.
Compare for Kentico
I would like to create and archive a collection of demo ASP.NET web form applications that show projects with certain features in the sense "this feature can be implemented like this" -- to be presented to a potential customer.
Before the presentation, I would like to get the selected set of demo and install them easily to the notebook. Each of the demos will be "frozen". The target notebook is not the customer's one. It is one of our ones that is bring to the customer for the presentation. This way, it can be prepared in the sense that a named MS SQL instance with the fixed name can be ready, etc.
Can you share some experience with such situation? (I do not want to have marked this question as of opinionated; so please, if you have some explicit links to the related documents or explicit suggestions...)
Here are some other facts and initial ideas:
Each of the demo projects uses two databases: xxx_users (the standard ASP.NET authentication...), and xxx_application (and possibly xxx_external) where xxx is a prefix for the specific project.
The demo application is expected to be compiled (binary only, no sources needed for the presentation).
The Web.config files can use the local\SQLINSTANCEFORDEMOS in connection strings.
The SQL instance has a fixed name, fixed administrator account (like sa) and fixed password for the logging to the SQL instance. This way, it can be included in the Web.config files.
The sample data can be fairly big (not extremely tiny).
The application will use its own SQL tables in the xxx_application database.
The application will simulate the outer database that is accessed from the web application can be simulated by xxx_external database.
This way, I should be able to create and archive SQL backups of xxx_users, xxx_application, and xxx_external databases, plus the archive of the web app binary.
Have you ever encountered this situation? Is the approach reasonable? Could you share some better ideas?
I have an Azure website running on several instances on Basic compute mode. I want to synchronize local data (just a few small numbers like number of online users, total app users, not whole tables or files) between the instances. How can I achieve this? I've seen Refreshing Data on all Azure Instances but it didn't help much. My current naive approach is writing these values to my database, and in my application code, keeping local changes, and syncing the actual data from the database every few minutes, which is OK for my application. But I'm looking for a more elegant way to achieving sync between instances. Is this possible? If yes, how?
There is not an API that will support syncing state between instances. However, here are a few ideas you might consider.
Local File System
The web site instances share a common file system. So, you could just write data to a .txt file in your App_Data folder and all the instances will use that same file. This would at least eliminate the dependency on a database for such simple data.
Caching
Another option is to write the data to a cache. Your options for this would be to use the Managed Cache service or the new Redis Cache (in preview).
Storage Tables
This may not be any better than your current solution. But, it is cheap and very efficient.
I maintain a web application (ASP.NET/IIS7/SQL2K8/Win2K8) that needs to access documents, actually hundreds of thousands of documents, and growing. Currently, they are all on a Windows 2K8 Server fileshare, being accessed by UNC path (SMB). The files are in a single flat directory and I'm trying to plan how to best improve this solution. I don't want to use the SQL Filestream attribute as it would be significant effort to migrate it all into that, and would really lock in to SQL Server. I also need to find a way to replicate the data for disaster recovery, so perhaps a solution can help with that too.
Options could be:
Segment files into multiple directories?
Application would add metadata for which directory it's on (or segment by other means)
Segment files into separate servers? (virtualize)
Backup becomes more complicated.
Application would add metadata for which server it's on
NAS Storage
SAN Storage
Put a service (WCF) in front of the files and have the app talk to the service
bonus of being reusable across many applications
Assuming I'm going to store on filesystem and not in database (I've read those disccusions here), which would be a more scalable solution?
You've got a couple issues:
- managing a large volume of (static?) files
- preparing for backups and disaster recovery of said files
I'll throw this out there, even though I'm not a fan of the answer, but you might poke around with the free SharePoint 2010 Foundation that's included with server 2k8. If you're having issues with finding the documents you need (either by search, taxonomy via tagging or other metadata) as well as document expiration and you don't want to buy a full blown document management system, this might be a solution. Of course it introduces new problems...
If your only desire is to have these files available to spit out on the web, then the file store like you're using now really is the simplest solution. For DR/redundancy purposes, I'd look at a) running them on a raid/SAN of some sort and b) auto-syncing them with the cloud (either azure or amazon). For b) you can get apps that make the cloud appear as a mapped drive and then use an rsync type software to keep the cloud up to date.
If you want to build something new and cool, you might think about moving the entire file archive into the cloud and just write a table in a db to manage the file name, old location, new cloud location and a redirector code that can provide the access tokens to requestors.
3 different approaches... your choice.
We have a situation where users are allowed to upload content, and then separately make some changes, then submit a form based on those changes.
This works fine in a single-server, non-failover environment, however we would like some sort of solution for sharing the files between servers that supports failover.
Has anyone run into this in the past? And what kind of solutions were you able to develop? Obviously persisting to the database is one option, but we'd prefer to avoid that.
At a former job we had a cluster of web servers with an F5 load balancer in front of them. We had a very similar problem in that our applications allowed users to upload content which might include photo's and such. These were legacy applications and we did not want to edit them to use a database and a SAN solution was too expensive for our situation.
We ended up using a file replication service on the two clustered servers. This ran as a service on both machines using an account that had network access to paths on the opposite server. When a file was uploaded, this backend service sync'd the data in the file system folders making it available to be served from either web server.
Two of the products we reviewed were ViceVersa and PeerSync. I think we ended up using PeerSync.
In our scenario, we have a separate file server that both of our front end app servers write to, that way you either server has access to the same sets of files.
The best solution for this is usually to provide the shared area on some form of SAN, which will be accessible from all servers and contain failover.
This also has the benefit that you don't have to provide sticky load balancing, the upload can be handled by one server, and the edit by another.
A shared SAN with failover is a great solution with a great (high) cost. Are there any similar solutions with failover at a reasonable cost? Perhaps something like DRBD for windows?
The problem with a simple shared filesystem is the lack of redundancy (what if the fileserver goes down)?