I've read it's good practice to split a BizTalk solution into multiple projects, and have seen some debate as to the exact nature of the split, e.g. ...
- could be split by artifact, i.e. Schemas, Orchestrations, Maps, etc.
- could be split by function
But what's the benefits / con's ??
BizTalk solutions typically include schemas, maps, and orchestrations. Solutions can also include supporting components, Business Rules, definitions of port-based routing and transformations, trading partners, and several other types of artifacts.
Effectively managing all of these artifacts has many benefits – far more benefits than drawbacks.
The benefits include:
Separation of concerns based on the
logical grouping of artifacts (by
functionality or artifact type for
example). This approach reduces the
possibility of modifying aspects of
your solution that aren’t related to
the problem you’re working on at the
time.
Easier to test – you can compile and
deploy just the components you’re
modifying.
Easier to split work among a group of
developers.
Easier to manage when the solution
gets larger – it can take several
minutes to load large BizTalk
solutions in Visual Studio.
Supports more advanced approaches
related to ESB-style solutions (very
loose coupling). Depending on your
overall approach, you can create a
solution that is very modular – to
the point that modules can operate
and be updated completely
independently of each other.
Makes it possible to version
artifacts separately.
Facilitates more fine-grained control
over security and memory utilization
by grouping related functions such
that you deploy them for a particular
host instance, for example (you can
also administer fine grained .NET
security policies more easily than
you can with a solution that deploys
a few assemblies).
The main drawback to splitting your solution over several projects or solutions surfaces when you are debugging your solution. Debugging BizTalk solutions is not straightforward for many developers that are new to BizTalk and having to narrow-down bugs across solutions does not make the job any easier. However, you can mange this issue by more effectively arranging your solution and using standards around naming, directory structure, arrangement of namespaces, and related methods to make it easier to figure out where to look.
Other drawbacks include:
More assemblies to sign and deploy
into the GAC
Inter-dependencies between
projects can result in deployment
errors that can be difficult to
track down in poorly organized
solutions.
You should dedicate some time at the beginning of a project – ideally during design – to setup the basic organization of your solution. A one-size-fits-all approach does not exist – you need to think about how you want to manage the solution during development, deployment and maintenance in the context of the functionality that the solution provides to your organization or clients.
A good place to start is to divide your solution based on artifact type or functional areas. As you grow your solution, you’ll get a better understanding about how artifacts relate to each other, how you want to manage strong naming, security, and physical deployment and be in a better position to arrange your solution more effectively. You need to be careful with this approach since you could end up having to rearrange large parts of the solution, which can be disruptive if your project’s timelines are tight.
Related
We have a modular application, each app creates its own tables (typically one two) and manage the data.
We use Flyway in our main application but also need it for our modules. However, if we add the patches to our main application, ALTER TABLE queries won't work for some deployments if the corresponding module is not installed.
One way to solve this issue is to perform the schema evolution with multiple Flyway operations, each module gets its own Flyway and manage itself. However since Flyway creates tables for managing the state we ended up too much tables since we have ~20 modules right now.
What's the elegant way to solve this issue?
I would say having the migrations managed by the unit of software it is supporting is the cleanest and trumps "too many tables". In terms of neat organisation of those tables, you can silo those using a schema (if you RDBMS supports those) and Flyway lets you name the table that is used per migration managed application.
The key thing here is "modules". From your description, it sounds like not all applications are made up of the same modules. I would ask you, if we go to the effort of making our modules descrete to create decoupled / reusable software - why should database schemas of those modules be treated any different?
To counter your concern about "too many tables", lets try and debunk the costs of that.
Volume. Your RDBMS is made to handle thousands, there is no cost there.
Operational. Flyway does all the management here, they are effectively opaque to you.
Performance. They are a deployment concern, not a runtime liability.
Organisational. Hide them / name them with the methods mentioned above.
Our natural urge is to aggregate related things but doesn't always lead to the best outcome, so we must be pragmatic. In this situation, good/flexible design trumps aggregation.
Let's say I have a Visual Studio solution with a Web project, a BLL project, and a DAL project. I'm trying to follow the repository pattern keeping my SQL code in the DAL with an interface that is referenced by the BLL.
I have a handful of common solutions for things such as error handling, usage logging, and other things that can be considered utility functions (i.e. not in the business spec). I'm keeping these in a Common project.
Here are a few ideas I've had with regards to structuring the Common project...
Bundle SQL with logic in a given class
Create a layered solution within the Common project
Discard the Common project and put utility functions in with BLL/DAL
Is one of these ideas better/worse than the other? Does anyone have a better solution?
It's worth noting that these utility functions will be reused in a variety of other applications.
Instead of creating a Utilities project which will be used have you thought about creating something that can provide a service? You might want to look at Aspect Oriented Programming. Red flags went up when I saw you listing off your examples error handling, logging, etc. Those scream AOP.
But if you want to stick with your layout.
I think I would go with 2, assuming that means restructuring the utilities project to be more Cohesive.
I don't understand (please clarify and I will edit my post)
Bundle SQL with logic in a given class
As for:
Discard the Common project and put utility functions in with BLL/DAL
I would be against doing so. If this logic is truly going to be repeated there is no need to push it back into your projects. This will lead to duplicate code and increased maintenance.
Side Note:
Just as a lessons learned, the only way Utilities projects work, are if you are the only developer or it is well documented and well designed. Sometimes utilities are too programmer specific, or are written in a way that only benefits a particular coders style.
I have seen countless times people rework their infrastructure pulling out all kinds of utilities, only to see their utilities project never get used. Make sure the "utilities" you are creating are truly useful to other people.
For our company I'm creating a big Extranet website which will feature a set of sub-applications. I'm a bit puzzled by what should be the right setup of the solution and projects.
I have one web application that we call the Portal. It contains the authentication/authorization classes, masterpages, navigation/url routing classes and theme definitions. It will also contain some basic overviews for our customers to get a quick idea of their project status.
In the coming year we are going to develop and integrate more applications with the portal. Think of it as detailed overviews and tools called Feature A, B and C. Over the years we will improve these applications and release new versions. These web applications should fit into the navigation of the Portal seamlessly. I'd like them to reuse the masterpages and themes.
What is the proper way to setup this solution?
How can I tie the applications together, re-use the master pages and keep it maintainable?
How can I share certain webcontrols (ASCX) in a proper way?
We are using TFS, perhaps any branching/merging ideas?
Edit:
I'd like to create it in ASP.Net WebForms, we have the most experience in that area. But basically we can use anything, we've got the webserver under our own control (as long as it is Microsoft oriented, not switching to php or something like that :))
What is the proper way to setup this solution?
The proper way... There are so many. I have seen a lot of applications, and a lot of different setups (a lot of which that I would deem "proper"). What you're really asking is for the best way for your situation.
Because you're building a portal, you'll have the luxury of feature separation which will help you as you develop additional features for your application.
I would setup a single website with a separate folder for each feature. Making it a single website will allow all features to share the same masterpages, usercontrols, and configuration file - without having to do anything special. (On that note, I would put all your master pages in a folder by themselves, and create another folder for your usercontrols).
How can I tie the applications together, re-use the master pages and keep it maintainable?
Again... folders are the best option here. Folders will help separate each feature, making the application easy to manage.
How can I share certain webcontrols (ASCX) in a proper way?
First of all, ascx files are not webcontrols. They are usercontrols. WebControl is a class for creating server controls that reside in a separate assembly. Regarding usercontrols, as I said above, if you put them in a separate folder, they're always in one place and available throughout the application.
We are using TFS, perhaps any branching/merging ideas?
There really isn't anything special you need to do here. There are a lot of different paths you can take regarding branching:
One is to create a branch for every release.
Another is to create a branch for every new feature you add (in your case, this is pretty much the same as the first option).
Yet another is to create a branch for each developer.
When I decide how I am going to branch my code, I think about what will protect me the most. In your case, you need to plan for bug fixes in between feature releases so maybe one branch after each release makes the most sense (call it your dev branch). Given the separation of features, though, one feature may not effect the rest of the application. You may not need this kind of branching to be safe.
As Brian says when making an API public you should commit to it as much as possible, which means it should change as little as possible after the initial release. However to make something that stable requires lots of effort up front so if you aren't ready to commit to the API you should instead internalize it as much as possible and for that reason you might want to combine things more than separating them.
However, I'm not going to suggest an architecture that fits your application based on a 5 paragraph description. What you need to do is to weight pros and cons of having a few big projects vs. having a bunch of loosely coupled small projects. I mean, the more planning you do up front, the easier you will have it down the line, provided you stick with the plan.
So contrary to Brians answer, I wouldn't recommend you make your entire system "as loosly coupled as possible", only that you make it as loosly coupled as it needs to be. ;) Loosely coupled code can cause as much trouble as tightly coupled code, if you are abusing it.
See:
1. What is better, many small assemblies, or one big assembly?
2. Specific down-sides to many-‘small’-assemblies?
In the end, only you know how much you want to focus on each of the "...bilities", maintainability, extensibility, reliability etc. So get your priorities and goals straight and plan accordingly.
Regarding branching strategies you could read the TFS Branching Guideline 2.0 which have a good introduction to various branching strategies ranging from basic to advanced. Even if you don't use TFS this is a good guide to read (I use SVN at the moment). Since I currently work in small teams with 1-4 devs, I tend to use a strategy that is between basic and standard. Not that I'm recommending this for you, but that whats works best for our team.
As for sharing code between projects. In SVN we can use "externals" which means that the shared file will appear in several folders so when you change one copy and commit the change to svn, all the other copies will be updated on the next svn update. However, I can't remember if TFS have something similar.
Note: Beware of externals in SVN... they can cause... problems. ;)
My advice is to try to avoid sharing aspx, ascx and master pages as much as possible. It usually hurts a lot more than it helps. Instead try to use inheritance or other alternatives to achieve your goal.
ASP.NET MVC 2.0 has a concept called "Areas" where you build subsections of an application in isolation from the rest. As far as I know these areas can be maintained in separate projects from the "main" application. It sounds a lot like what you are requesting so maybe you should look into that.
Hope it makes sense. ;)
I would look at making your system as loosely coupled as possible. As/when you add more applications, your website will become less and less reliable (since no component will be up 100% of the time, combining these will reduce your overall reliability). So you should build your system to cater for the non-major services being down (I believe the Amazon homepage, for example, has 100-ish services contributing to it, and as such it's built to be fault-tolerant)
Your APIs between services should remain as stable as possible, such that the implementations can change without breaking the coupling. You should also investigate automated testing of this at the web level (perhaps Selenium or similar?) since testing the individual services will give you little coverage re the overall behaviour.
You might find it useful to look at implementing a custom VirtualPathProvider. On my last project we had multiple ASP.NET sites which needed to share theme files (master pages, user controls, images, style sheets) so I created a VirtualPathProvider which allowed us to map a virtual folder (e.g. /Themes) to any physical folder on the hard drive (e.g. C:\Shared\SiteThemes).
It's not trivial but didn't take too long and hasn't caused any problems. Incidentally it turned out to be a great way to overcome the maximum component limit in WiX... Note that you can't precompile sites that use a VirtualPathProvider.
Use MVC Concepts from now. they give more extendability and flexibility for a robust applications.
You might look at using SharePoint. It's a pretty decent platform for ASP.NET application delivery, particularly if they coexist in an intranet environment; it gives you a lot of stuff for free.
Of course, it has very rough elbows, so to speak, so proceed with caution.
I wouldn't think of the applications as seperate but as modules of the overall portal.
I would recommend you look into MEF as this would seem to be a perfect fit.
http://blogs.msdn.com/hammett/archive/2009/04/23/mef-and-asp-net-mvc-sample.aspx
Do you have one solution with the web application project, class libraries, database project and tests? Or, do you segment it into multiple solutions? Why?
I'm asking because we're trying to streamline this scenario for Visual Studio 2010 and I'd like to get input from the community on how you'd prefer to work.
I tend (but not always) to have one solution per job, but I import existings projects from other solutions, such as my WebControlLibrary where I keep common user controls and classes, etc.
My actual solution for the job I then tend to break down into the Web Application, Business Logic Layer, Data Access Layer and Entity Layer, i.e.:
Solution
...MyCompany.WebControlLibrary
...Project
...Project.BusinessLogic
...Project.DataAccess
...Project.Entities
...Project.Scripts
...Project.Testing
...Project.Deployment
If a project requires something such as a mobile device, I'll always put that in a new solution, but it may perhaps share some projects of the current solution, i.e.
MobileSolution
...MobileProject
...Project.Entities
...MobileProject.BusinessLogic
The more 'stuff' you have combined, the slower Visual Studios becomes at building. You can obviously stop certain projects building by default, but that's when you have to start creating your own build configurations. If you are going to be creating large applications, I'd suggest breaking down into multiple solutions. I find it much easier to flick between solutions that to keep changing build configurations.
Another option is that when you build your projects you can reference their DLLs. I prefer to import said projects into my solution as you never have to worry about referencing the creating build configuration i.e. selecting the DLL from the Debug or Release folder.
Stand alone libraries can be their own solutions. References for those libraries can be made into the project that you're working with. Related items like the web application, the test setups, and specific libraries such as data access or business rules can be setup as projects within one solution. It really all comes down to how much you want to break things out for resuability.
This depends a little on the job the project performs.
For ease of use it's simple to have a solution that just contains all the projects required. If it's a large solution this can hamper you later on when the IDE starts to get slow and build times rocket through the roof.
Let's say one of the projects is a library used by your company to take card payments and interface with 3d secure. You present you're own GUI page to take the details etc.
If you had numerous sites that all take card payments you would greatly benefit by having this project in a separate solution and referencing the compiled dll. Any changes you require you would need to open up the solution, make the change, build it, go to the solution you're working in and test it. Sounds like a pita and you find it's just simpler to have it all in one big solution. But then if you have this library in every solution and make a generic change to it you need to repeat that change through out.
So you just need to make a decision on wether you're developing a separate project in the same solution or something that might be used elsewhere. If you needed more functionality than the library provides you could implement a partial class in your project and extend the library in that way. Or perhaps a wrapper class will suffice. But then you know you're not affecting the other sites that use this library and you are keeping your solution smaller and more manageable with a smaller memory print during development.
I have a web application that is arranged into data, business and UI projects. As the system evolves changes are deployed by building all three projects and deploying them in one package. This has worked well and has allowed the illusion of “three tiers” without tackling the communications, versioning issues of truly separate systems.
So along comes a request for XML summaries of some of the data and my thoughts turn to a fancy WCF service that, one day, could be my “Web API” (ahh… the mind.. what a evil little monkey it is). So, assuming this survives the “is that really the best idea?” test here is my question:
What structure have you had the most success with when posed with two
evolving “clients” serving content from a single evolving “model”?
James, your question is rather board as there are a large number of variables that go into choosing the right type of architecture for your needs. I would recommend reading the patterns & practices Application Architecture Guide 2.0 to better understand the options and pick the best one that suits your individual needs.