I'm new to Biztalk. I wish to structure my artifacts into applications.
Should I go with one application per artifact type - for example 3 seperate applications, like 'BusinessProcess.Schemas', 'BusinessProcess.Maps', 'BusinessProcess.Maps', with all the assemblies related to these (typically just one), or do you think it best to just have one 'BusinessProcess' application, which would contain all assemblies related to the business process?
Should I go with one application per artifact type
I think in general, you mean one project per artifact type? Although artifacts common to more than one application should be split out into a separate application / solution.
The rationale for splitting BizTalk projects up into component projects (and assemblies) is similar to any other .Net project - separation of concerns, isolation for testing purposes, independent versioning, etc.
Most of these are addressed in Erik Westermann's great answer here
One point that to elaborate on is that the dependency chain usually looks something like this:
Schemas (Internal / Canonical, External)
^
Maps
^
Ports + Pipelines
^
Orchestrations
(plus other miscellaneous .Net assemblies with helpers)
On a server with multiple applications deployed, at some point you'll likely need to have cross-application dependencies (especially if you have chosen to couple to definite schemas and not loosely couple e.g. via multipart messages), usually on Internal / Canonical Schemas which are common to more than one project.
Unless these schemas are separated from the other assemblies, any time a monolithic application changes, you'll need to go through the pain of uninstalling / reinstalling all dependent apps every time any change is made to the monolithic app. Generally because the whole enterprise is dependent on the Canonical schemas, they are carefully deliberated and thus fairly stable, whereas an app's maps and orchestrations are more likely candidates for frequent change.
Here's a structure for this scenario:
Separate Common / Base Solution Projects:
Common internal schemas (event / EDA Meta information , internal
errors etc)
Canonical Domain schemas (i.e. the internal / enterprise
model of Invoices, Claims, Transactions etc of your business)
Helper assemblies for interacting with the above
Your application solution references the above, and then the commonly accepted separation is along the lines of the artifact types:
External system schemas (if more than one, then each in its own assembly)
Maps
Pipelines
Orchestrations
It's really a judgment call. If you have large app with lots of Schemas, Maps, Orchestrations, sure, it makes sense to split them out.
However, small apps, 10 or so artifacts, not so much.
For a new BizTalk Developer, I'd stick with stick with one "MyBusinessProcess" app. At some point, you'll see an when a multi-Project solution starts to make sense, then do it.
Related
Multiple Asp.Net Framework 4.6.2 MVC/Razor projects within a single solution.
They use Areas, and each project will implement one or more Areas, distinct from the other.
One project acts as the "primary", and includes the global.asax.
The "subordinate" apps can know about the "primary", but not vice versa.
Is it possible to deploy these to the same web site? (on-prem VM).
There would be some collisions, such as the _Layout.cshtml.
Some how I'd have to be able to setup dependency injections for the classes in the "subordinate" project assembly.
Thanks
-John
I don't think this is possible easily, because so much of an MVC site is pre-compiled into DLLs, and yet some of it (e.g. view files) must be actually physically deployed outside the DLL. You are likely to run into conflicts or confusion I think. It's certainly not a reliable, definable process that you could count on in production.
Here are some alternative suggestions to solve the underlying problem, based on what you described in the comments. These are just ideas, some of which I've done or seen done, and might not suit you, but any of them should be a lot easier to manage than trying to bodge the deployment process:
1) Try building the areas in separate projects as you do now, but without all the generic MVC stuff that surrounds it. I.e. Do it as if they were libraries only containing the files specific to their functionality. Then you can package them up as Nuget packages. When they're ready, install the packages into the "main" site to create the combined solution. That way you won't run into conflicts of layout pages etc. If one of the sub-projects changes, then the main site can just upgrade the NuGet package. You can create private NuGet feeds either via Visual Studio Online (if you have a subscription) or on the simplest level via a shared network folder. I think other online vendors offer private feeds as well. We've done that quite successfully to apply some default styles, scripts, layouts etc to all our various MVC apps.
2) Re-architect your solution as a series of separate apps (that's the bit you've already got), but with a central service that provides federated authentication/authorisation (e.g. ADFS, IdentityServer or similar), and a web API (or APIs) that the other sites can call to access other centralised services and functions programatically. I'm pretty confident you can use routing config and shared sessions to make it all hang together as if it was actually one site.
3) Build it all out as just one site, but with clear separation for the various teams about the back-end services, classes etc that they are responsible for, so they don't interfere with each other's code. Each team can check-in code to different branches in source control and only merge to the shared branches for integration/system testing and deployment. You can use a continuous integration server to test the build process after each check-in, and if you have robust code review and gated check-ins you can stop people changing stuff that they shouldn't. You also have the flexibility to release with/without updates from the various areas, if the development lifecycles are going at different speeds.
4) Some combination of all of the above!
I am trying to implement modular design in an asp.net project dividing the application into different modules like HR, Inventory Management System etc. Since I am trying to keep different modules independent of each other, I separated these modules in such a way that each module is a separate Visual studio solution having UI, BLL, DAL and even a separate database schema.
Up till now I thought this as a common practice for developing Management systems and ERPs but I am searching the web for last three days but hardly found any help full stuff regarding developing modular applications. Most of what I found is mere theory explaining the concepts of cohesion and coupling but not real world scenarios. So I wonder
Is it the right approach of separating modules?
How the real world modular applications are developed?
How should the different modules communicate with each other yet they stay independent of each other.
I think there should be a core application which makes use of these modules, how should the core application communicate with these modules?
There is some data, entities , objects which are common to each module, should I put them in the core modules in order for other modules to use them (I think this will make the modules coupled to core) or should every modules maintain its own copy of data + define those object, (which I think voilates DRY)
Any thoughts, links are warmly welcome.
This is a personal opinion and is debatable.
I separated these modules in such a way that each module is a separate Visual studio solution having UI, BLL, DAL and even a separate database schema.
Sounds like a total overkill. Abstraction over abstraction makes your application pain in the neck to maintain, support, and enhance. Is it that large that you need to separate modules into separate solutions?
Is it the right approach of separating modules?
No, I think it is a total over-engineering. I would suggest using projects to separate modules. And not separate solutions. The problem with solution is that it will require external dependencies management tool, which requires a lot of effort to bring in and later maintain.
How the real world modular applications are developed?
Using abstraction (interfaces and abstract classes) and separate projects.
How should the different modules communicate with each other yet they stay independent of each other.
By using interfaces, DI, IOC, TDD
I think there should be a core application which makes use of these modules, how should the core application communicate with these modules?
Core does not communicate with modules. In fact it should ideally not depend on any other project/library. This makes it simple to reference and use in large solutions.
There is some data, entities , objects which are common to each module, should I put them in the core modules in order for other modules to use them (I think this will make the modules coupled to core) or should every modules maintain its own copy of data + define those object, (which I think voilates DRY)
I would highly recommend using a single copy from the Core project. See this questions for details of why.
This is one of those topics that is entirely subjective for the most part, but you may wish to consider a SOA (Service Oriented Architecture).
Using SOA, you can define a service (for this example, I'll stick to web services, though other service types exist depending on requirements) for each business area - an HR web service, a projects web service, a finance web service and so forth.
You can then bring all these together with a front end system that will communicate with and utilise these services, that would normally be your core application, though depending on your needs and requirements you may opt for multiple front end systems.
For the front end system I would recommend using ASP.NET MVC which has the concept of areas and will let you separate the front end into specific areas - an HR area, a projects area, a finance area and so forth that will contain the models and views for each specific area.
Doing this will let you build in a modular manner, you can build your first web service, say, the HR web service, that has methods for getting relevant HR data and so forth, and then build the HR area of your MVC application. Expanding then simply depends on building the web service, and creating the front end in the MVC application. There is nothing stopping say the HR area then accessing the finance web service if it needs finance information, but it still keeps everything in distinct independent modules.
Using this method can also be helpful in aiding future interoperability - it may be that other systems in the company will find it useful to interact with certain web services. For example, in a previous role it was useful for the companies engineering software to integrate with the projects team web service as it allowed for engineering related information to be linked to it's related project.
If the system grows in terms of resource requirements it should also be fairly scalable as it is trivial to say, offload the projects web service to another service if it starts eating a lot of system resources. It also allows you to switch modules out if need be - if you ever decided to move to say, a Linux/Java platform, you could trivially move by porting module by module with no real interruption of the overall system.
But of course, as I say, this is simply one such option and much of it depends on the specifics of your circumstances.
It is too late to answer but it seems interesting.
Since I am trying to keep different modules independent of each other, I separated these modules in such a way that each module is a separate Visual studio solution having UI, BLL, DAL and even a separate database schema.
It depends on your scale of application. If you create a very small-simple application with a little functionality, then it is safe to has a combined assembly. Or if you want, just separate the UI with other module. At least it can help you to emphasize SOC. Keep in mind that loading multiple assembly can be slower than a single assembly.
Is it the right approach of separating modules?
Module separation always has a drawback, that it is require mapping. It means slower performance in general (maybe negligible, but still there is), and slower development time. If your application will be large and complex enough, it is worth it, since you can create modular unit tests for each module.
How the real world modular applications are developed?
No exact practice though, every problem needs a solution. You won't need a heavy multi-threading or dependency injection architecture for a simple calculator application.
How should the different modules communicate with each other yet they stay independent of each other.
Using interface. You can make the implementation different later on. Example is, you currently use C# Winform for your application, communicate to the BLL using interface. Later on, you want to migrate to ASP.Net, then you just change the implementation, but keep the interface to communicate with the BLL the same.
I think there should be a core application which makes use of these modules, how should the core application communicate with these modules?
There is some data, entities , objects which are common to each module, should I put them in the core modules in order for other modules to use them (I think this will make the modules coupled to core) or should every modules maintain its own copy of data + define those object, (which I think voilates DRY)
I assume it is an enterprise level application which share the same modules / data such as employee. If it is really need to behave uniformly, then you should provide the very basic logic at the core Level. At the application / implementation level, you may has different implementation to fulfill each requirement.
Do not force to uniform all of the business logic to the core. If a specific application need a different implementation, it is hard to make the core configurable.
We have different web based products. All the products share same underlying authentication and authorization mechanism. All are on same database server and are ultimately published to same server.
Each project has its own namespace, folder structure and pages. Still due to the fact that authentication and authorization is shared, we use login and other pages across all the projects.
Also to make look and feel uniform across the projects/products we use same master pages.
Currently we have a separate project which contains code, markup and scripts etc. for shared things. We copy the markup and other things to all the projects to build and run them. It is really a hell. We have to include/exclude the files, change namespaces etc. all the times and over that make sure that shared things are at same version in all dependent projects.
What would be the best methodology to handle all this in a way that we don't go to asylum?
We are on ASP.Net 4.0, Visual Studio 2010, Telerik 2013 Q1 release.
You have several options to improve your situation. The best option for your will likely depend on more than the information you have provided, however the following may be worth investigation.
Decouple authorisation system. If more than one application is using a single common authorisation code base then you may want to consider decoupling the functionality into a standalone (probably web service based) application. Authorisation through such an architecture is tricky, and easy to get wrong from a security point of view, but is achievable. The authorisation code base will then only need to be maintained in one location which will inevitably reduce deployment and building mishaps.
Extended configuration management. Does your application have any configuration management capability? If not, it should. It may well solve your problems with regards to includes and excludes and namespace chopping, especially when combined with point 3.
Improved version control management. It sounds as if you possibly aren't making the most of your version control system. Although you allude to versions in your question, if you were maintain different branches of a common trunk for your different applications the chopping up of namespaces and includes and excludes would probably be reduced or even not necessary since customisations could co-exist.
I have done some investigation around SOA and Component Based Architecture, and it seems one of the key differences is that a service should be independently deployable.
If I have services that are jar files, and these jar files don't start living until they are deployed as part of a larger application's ear file - is this really SOA? I.e. these service are jar files that cannot be deployed as independent services that can run independently of the ear file.
A service is an independent unit in terms of deployment but also versioning, scaling and data. The API (whether it is HTTP, messaging, events etc.)is "published" and public which many times comes with implication on security, backward compatibility etc.
In any event,I don't think it matters what you call them or even if you have an SOA or not. The point is to get an architecture that will serve your current needs and will enable you to evolve it when future needs will arise
If your code is communicating in process them I think you are just using components. A real SOA would have code communicating via some other mechanism like http. Not saying one is better than the other it just depends on the problem you are solving.
I'm doing some work with several shared .NET assemblies and a generic web application that I would like to handle better in our CC.NET/NAnt build environment.
Currently, we have several .NET assemblies (shared common code that we use in client projects) that exist in different .NET solutions within different repositories in our SCM (Vault incidentally). They are all configured under CC.NET separately so we have a decent amount of control over their build and deployment at present.
We have developed a CMS system that uses some of the .NET assemblies and includes a common administration website project and a template website example project. Out of this one solution we have the following elements that need to managed separately:
Admin interface is not tied to .NET so it is template based and we are developing a PHP backend for it currently.
CMS shared assembly build on top of our other common company wide assemblies.
Control over functionality within each major CMS build/release.
I'd like the build output of this solution to be a Visual Studio template, which we can use to develop other client sites and better manage version changes within the CMS itself, as we add features to the codebase.
I have a rough approach for all this and think it is achievable, however, I wanted to open this topic up for discussion and see what everyone else is doing when it comes to managing the build and deployment of multiple solutions.
Main considerations for us are:
Do we make use of the integration queue functionality in CC.NET to ensure a build order and pull together the assemblies we need for the CMS at build time?
Debugging within a CMS client site i.e. stepping into the shared assemblies' code when the client solution is a version of the base CMS system and therefore separate.
Developing and extending the CMS when it uses shared assemblies i.e. do we add the assembly projects to the trunk solution during development (across source control repositories) and then rely on the build to pull it together or do we use a different approach entirely?
Any other issues people might have experienced that could change our way of thinking?
Hopefully this question isn't too vague and some of you will have dealt with these issues. Look forward to hearing everyones experiences.
Many thanks!
Tim
I unfortunately cannot answer all of your points, but let me start with this one:
Do we make use of the integration queue functionality in CC.NET to
ensure a build order and pull together
the assemblies we need for the CMS at
build time?
The short answer is -yes, you should. The queue attribute ensures a build order within the running instance of CC.NET and is gives you serialization of the builds that depend on each other. For specifying which projects depend on each other, you should use project triggers. Do not rely on the queuePriority for this task.
You shold most likely pull the pieces you need to do the build at build time. Unless you have some time constraints on your individual builds.
Re:
Developing and extending the CMS when it uses shared assemblies i.e. do we add the assembly projects to the trunk solution during development (across source control repositories) and then rely on the build to pull it together or do we use a different approach entirely?
I'm fundamentally against distributing binaries in the trunk unless it's some libraries that does not need to be updated/changed on a frequent basis. If you build the shared assemblies yourself, you should consider pulling them from the artifacts on the build server(s).