I was reading through Nevatech Sentinet the last week and I'm currently asking myself the following question: "When NevaTech Sentinet exists, with all these named features, why should anyone use BizTalk with ESB Toolkit and extend it with Sentinet?"
Does I see there something wrong, but Sentinet is able to handle everything and more what BizTalk with ESB Toolkit is also able to do?
Essentially you are talking about 2 very different products here.
Sentinet is a very good tool if you are thinking about centralized API management. It is very good in what it does in that niche.
BizTalk on the other hand is a ESB, using a publish/subscribe architecture. BizTalk also has various ways of connecting to non-trivial systems like SAP, DB2, Siebel, MSMQ, etc... It can also do EDI/AS2/X12, flat file parsing and so on.
You can set it up as a ESB and/or message broker/hub/etc...
In your specific case (I'm guessing web services related?) it might seem that BizTalk and Sentinet are similar, yet the two are very different and actuallly complement each other rather nicely.
As you see these are 2 completely diverse products and together they actually might be a perfect match in your case.
Some more clarification after your comment:
Using BizTalk does not necessarily mean you have to use ESB Toolkit. BizTalk can perfectly act as an ESB without the ESB Toolkit.
The benefits of using Sentinet is definitely API management. What you are focusing on when "doing" API management with Sentinet is to form a single layer of API's within Sentinet. Often you would point all your clients (both internal or external) to Sentinet, where you would host all of your services virtually. You gain a lot of control that way and can add security, versioning, load balancing, SLA reporting, etc... to your existing services without any hassle.
Another thing Sentinet is quite good at is low-latency services. This is something BizTalk is not particulary good at, since it will persist everything to it's database to prevent losing messages. (I once used it in a POC and setup a virtual service calling an existing, external service with additional enrichment and easily covered 200+ trx/seconds).
BizTalk on the other hand is middleware. That's a whole other playing field.
It's very good in connecting different systems to each other using different protocols, mapping messages to other formats (xml, flat file or EDI), adding business logic to your flows, integration patters, long running flows, loose coupling, etc... you wouldn't want to use it as a virtual service since it will persist everything to it's message box!
Hopefully now you see they are both quite a different tool set.
They do play along nicely next to each other though: hosting your BizTalk web services in a virtual web service on Sentinet has a lot of advantages, especially in a fast-paced environment.
One addition to #Peter's great answer:
I almost always install the ESB ToolKit, if nothing else for the centralized exception handling (EsbExceptionDb). You may or may not want to use the itinerary and other services in the toolkit, but the exception DB is very handy when trying to debug and potentially resubmit messages.
Related
I have been researching workflow foundation for a week or so now, but have been aware of it and the concepts and use cases for it for many years, just never had the chance to dedicate any time to going deeper.
We now have some projects where we would benifit from a centralized business logic exposed as services as these projects require many different interfaces on different platforms I can see the "Business Logic Silos" occuring.
I have had a play around with some proof of concepts to discover what is possible and how it can be achieved and I must say, its a bit of a fundamental phase shift for a regular C# developer.
There are 3 things that I want to achieve:
Runtime instanciated state machines
Customizable by the user (perform different tasks in different orders and have unique functions called between states).
WCF exposed
So I have gone down the route of testing state machine workflows, xamlx wcf services, appfabric hosted services with persistance and monitoring, loading xamlx services from the databse at runtime, etc, but all of these examples seem not to play nicely together. For example, a hosted state machine service, when in appfabric, has issues with the sequence of service method calls such as:
"Operation 'MethodName' on service instance with identifier 'efa6654f-9132-40d8-b8d1-5e611dd645b1' cannot be performed at this time. Please ensure that the operations are performed in the correct order and that the binding in use provides ordered delivery guarantees".
Also, if you call instancial workflow services at runtime from an sql store, they cannot be tracked in appfabric.
I would like to Thank Ron Jacobs for all of his very helpful Hands On Labs and blog posts.
Are there any examples out there that anyone knows of that will tie together all of these concepts?
Am I trying to do something that is not possible or am I attempting this in the right way?
Thanks for all your help and any comments that you can make to assist.
Nick
Regarding the error, it seems like you have modified the WF once deployed (is that #2 in your list?), hence the error you mention.
Versioning (or for this case, modifying a WF after it's been deployed) is something that will be improved in the coming version, but I don't think it will achieve what you need in #2 (if it is what I understood), as the same WF is used for every instance.
I have some internal-facing ASP.NET web services that have had numerous API additions over the years. Some of the original web methods, while still available for consumption, have recommended replacements available. I would like to steer consuming clients toward using these new methods so I can retire and eventually remove their elders.
If this were a client API rather than a web service API, I'd just mark the offending methods with the obsolete attribute. But .NET attributes do not get serialized and are not visible to consuming developers when they add or refresh web references.
What techniques are recomended for obsoleting ASP.NET web methods? Is there anything built into the tooling (VS2005-2010)? I don't want to break any of the existing clients, so I can't simply remove the web methods outright or change their internal behavior to reprot their usage as erroneous.
Tim, the short answer to this is unfortunately that you have to contact those clients and communicate the change with them and agree on timelines etc. There might be something that you can do to smooth the process over for them, particularly if they are not IT savvy clients and had to get their applications built by external contractors.
You can butter this up any way you like for them really, from the system is going to be replaced, to we are doing it bigger, better and faster.
Additionally you can build in code to slow them down, NOT RECOMMENDED, but then when they inquire you can give them the, we don't support that system any longer, it has been replaced by system 'X'.
If the new methods you are talking about are still just web-methods, you can just point the old ones to the new ones, and let the clients use the old one.
Another option is to identify the clients stuck on the old methods, get their IP addresses and lock it down so only they can use it, this way you ensure new clients will not attempt to connect to the old methods.
Other than that, I cannot think of anything that will not be a pain or difficult for both yo and the client.
My project is converting a legacy fat-client desktop application into the web. The database is not changing as a result. Consequently, we are being forced to call external web services to access data in our own database. Couple this with the fact that some parts of our application are allowed to access the database directly through DAOs (a practice that is much faster and easier). The functionality we're supposed to call web services for are what has been deemed necessary for downstream, dependent systems.
Is this really how SOA is supposed to work? Admittedly, this is my first foray into the SOA world, but I have to think this is the complete wrong way to go about this.
I agree that it's the wrong approach. Calling your own database via a webservice should raise red flags in a design review, and a simple DAO is the way to go (KISS principle).
Now, if it's data that truly needs to be shared across your company (accounts, billing, etc) THEN it's time to consider a more heavy-duty solution such as SOAP or REST. But your team could still access it directly, which would be faster.
My team had the same thing happen with a web service that we wanted to call in batch mode. Rather than call our own SOAP endpoint, we instead set it up to call a POJO (plain old java object) interface. There's no XML transformation or extra network hop through an SOA appliance.
It's overkill to put an XML interface between MVC layers when your team owns the whole application. It may not be traditional SOA... but IMO it's traditional common sense. ;)
I've seen people try to jam SOA at too low a level and this may be such a case. I would certainly not equate DAO and SOA at the same level.
I agree with #ewernli
What is SOA "in plain english"?
IMHO, SOA makes sense only at the enterprise-level, and means nothing for a single application.
If I'm reading into your question correctly, your web services are for C/R/U/D data into the database. If so, providing C/R/U/D services directly to the database and its tables are likely too low level to be SOA services.
I'd look for services at a higher level and try to determine whether they are interesting at to the enterprise. If so, those are your services. I'd also ask myself whether my former desktop app is providing services (i.e. should you be looking to make your new app an SOA service itself rather than trying to force an SOA architecture into the desktop app at a low level.
Consequently, we are being forced to
call external web services to access
data in our own database.
Man, that gotta hurt. As far as services in SOA go,
a service is a repeatable logical manifestation of a business task - that means you are not implementing SOA if you are not 'service enabling' business processes. If you are putting some web services to select data out of your data base, all you got is a bunch of webservices, which would slowdown your applications which could have been faster by conventional data access patterns (like DAO)
When you equate SOA with Web services there is a risk of replacing existing APIs with Web services without proper architecture. This will result in identifying many services that are not business aligned.
Also, service orientation is a way of integrating a business as a group of linked services - so ask yourself is the organization making use of these atomic services to achieve further benefits?
Do a google search for SOA anti-patterns and you will find what are the different ways to end up with a pile of web-services instead of SOA.
SOA... SOA... is the bane of my existence, for just this reason. What, or what not, constitutes SOA? I support SOA products in my day job, and some people get it, some don't. SOA.. SOA is about wrapping discrete business services in XML. ZIP+4 validation services. Payment gateways. B2B messaging.
SOA CAN be used to decouple desktop apps from backend databases. Sometimes it doesn't make sense, sometimes it does. What almost NEVER makes sense is low-latency high-query-count logic. If you ever have to use an application in France directly connected to a database in California, you'll get what I mean. SOA pretty much forces you to then smartly about how you model and return your data (look into SDO - Service Data Objects). The devil's in the details though. Marshalling data to/from XML can be costly.
Good SOA design is all about separation of behavior and data.
I repeat behavior and data need to be separate or else you will have lots or problems whether its CORBA/SOAP/REST/XMLRPC or even plain old in-the-same-JVM-method calls.
Lots of people will talk about service end points, message handling, and contracts making SOA one of the more soporific areas of computing when its surprisingly not complicated.
If you are doing Java its really easy. Make POJOs for your domain objects with no weird state behavior and no weird collaborators and then make Service classes with the behavior. More often then not you can just use your DAO as the service (I mean you should have a thin layer over the DAO but if you don't need one....).
OOP lovers will disagree of this separation of data and behavior but this design pattern scales extremely well and is infact what most functional programming languages like Erlang do.
That being said if you are making a video game or something very state based then this design philosophy is a bad idea. BTW SOA is about as vacuous as the term enterprise.
Which part do you think is wrong? The part that you have to hit the web service, or the part you are hitting the database directly?
SOA is more of an API design guideline, not a development methodology. It's not an easy thing to implement, but the reward of reusability is often worth it.
See Service-Oriented Architecture expands the vision of Web services or any technical book on SOA. Simply wrapping function calls with web call does not make it a Service Oriented Architecture. The idea of the SOA is to make reusable services, and then you make higher level services (like website) by compositing or orchestrating underlying low-level services. At the very low level, you should focus on things like statelessness, loose coupling, and granularity. Modern frameworks like Microsoft's WCF supports wiring protocols like SOAP, REST, and faster binary side by side.
If your application is designed to run over the Internet, you should be mindful of the network latency issues. In a traditional client-server application that is deployed on a LAN, because the latency is sub 10 msec, you could hit the database every time you need the data without interrupting the user experience. However, on the Internet, it is not uncommon to have 200 msec latency if you go across proxies or oceans. If you hit the database 100 times, and that will add up to 20 seconds of pause. In SOA, you would try to pack the whole thing into a single document, and you exchange the document back and forth, similar to the way tax is filed using Form 1040 if you live in the US.
You may say that the latency issue is irrelevant because the web service is only consumed by your web application layer. But you could hit the web service from the browser using AJAX reload the data, which should give the user shorter response time.
"What makes a good BizTalk project" is a question I was asked recently by a client's head of IT. It's rather open ended, so rephrasing it slightly to :
"what are you top ten best practices for a BizTalk 2006 and onwards projects - not limited to just technical practices, eg organisational"
I wrote an article called "Top 10 BizTalk Server Mistakes" that covers some key best practices in terms is usable information rather than a simple list. Here's the listing:
Using orchestrations for everything
Writing custom code instead of using existing adapters
Using non-serializable types and wrapping them inside an atomic transaction
Mixing transaction types
Relying on Public schemas for private processing
Using XmlDocument in a pipeline
Using ‘specify now' binding
Using BizTalk for ETL
Dumping debug/intermediate results to support debugging
Propagating the myth that BizTalk is slow
...and the link to the complete article: [Top 10 BizTalk Server Mistakes] (http://artofbabel.com/columns/top-x/49-top-10-biztalk-server-mistakes.html)
The key point is to emphasize to the client that BizTalk is a swiss army knife for interop... an expensive swiss army knife. A programmer can wire up two enterprise systems with a WCF application as fast as you can with BizTalk. The key things to include/require when using BizTalk is to:
Have more than simple point integrations. If this is all you have, fine, see the rest.
Have all or a portion if a process that is valuable going theough BizTalk so that you can instrument it with BAM and provide process monitoring to the organization... maybe even some BI.
If you are implementing a one to many or many to one scenario, use of the BizTalk ESB patterns will pay deividends in th elong run
When there are items that need to be regularly tweaked - threshholds, URI'ss, etc... use of the Business Rules Engine can provide an easily maintainable solution.
When endpoints might be semi connected, BizTalk bakes in queueing of messages for no extra effort.
Complicated correlations or ordering of messages.
Integrating with exisitng enterprise systems can be simplified with the adapter packs provided as part of BizTalk. This alone can save big bucks. Asking Oracle, PeopleSoft or Siebel folks about XML and Web Services can be a challenging experience. The adapters get you and BizTalk through the enterprise apps' front door and reduces the work for themsignifcantly.
There are more I just can't think of at midnight.
Any of these items make BizTalk a winning candidate because so much of it is given to you with the platform. If you are not being required to provide any of these, you should really attempt to deliver some of these beneftis in highly visible way to the client. if you don't it's just an expensive and under utilized swiss army knife.
I'll start with Environment and Deployment Planning. Especially testing deployment and matching your QA/Stage (whatever the pre-production environment is) to the production environment so you don't find out some weirdness at midnight when you are trying to go live.
I'm implementing a high traffic client web application that uses a lot of REST API's for its data access layer from the cloud database. I said client because it implements REST and not provides it.
REST APIs are implemented server side as well as client side and I need to figure out a good solution for caching. The application is running on a web farm so it I'm leaning toward a distributed caching like memcached. This caching solution will need to be like a proxy layer between my application and REST APIs and support both client side as well as server side.
For example if I make a call to update a record I would update through REST and I'd like to keep updated record in the cache so next calls to that record won't need extra call to the outside REST services.
I want to minimize REST calls as much as possible and would need to keep the data accurate as much as I can, but it doesn't need to be 100% accurate.
What is the best solution for this caching proxy? Is it a standalone application that runs on one of the servers with local cache, or built into current solution using distributed caching? what are you ideas, suggestion or concerns
Thank you,
You hit the nail on the head. You need a caching layer that acts as a proxy to your data.
I suggest that you create a layer that abstracts the concept of the cloud a way a bit. Your client shouldn't care where the data comes from. I would create a repository layer that communicates with the cloud and all other data. Then you can put a service layer on top of that that your client would actually call into. Inside this service layer is where you would implement things like your caching layer.
I used to always suggest using MemCached or MemCached Win32 depending on your environment. MemCached win32 works really well if you are in a windows world! Look to the Enyim client for MemCached win32...it is the least problematic of all the other ports.
If you are open to it though and you are in a .net world then you might try Velocity. MS finally got the clue that there was a hole in their caching framework in that they needed to support the farm concept. Velocity last time I checked is not out of beta yet...but still worth a look.
I generally suggest using the repository and service layer concepts from day one...even though you don't need it. The flexibility it provides for your application is worth having as you never know which direction your application will need to be pulled in. Needing to scale is usually the best reason to need this flexibility. But usually when you need to scale you need to scale now and refactoring in a repository layer and services layer while not impossible is usually semi-complex to do down the road.