Related
Our company has people in every catastrophic event here in the U.S. and parts of Canada. An example is they were quite prevalent in Katrina immediately after the event.
We are constructing an application to improve their job in the field which may be either ASP.NET or WPF, and the disconnect requirement makes us believe it will be a WPF application. Our people need to be able to create their jobs, provide all of the insurance and measurement data, and save it as if in the database whether or not the internet is available.
The issue we are trying to get our heads around is that when at catastrophic events our people need to be able to use our new application even when the internet is not available. (They were offline for 3 days in Katrina)
Has anyone else had to address requirements like this and suggestions on how they approached functioning on small-footprint devices while saving data as if they were still connected to the backend services and database? We also have to incorporate security into this as well, and do it well enough that their entered data loads into the connected database without issues.
Our longterm goal is to also provide this application for Android and IPad Tablet devices as well as laptops. Our initial desire for ASP.NET was it gave us an immediate application for the tablet environment. In the old application they have, they run a local server, run remote connections on the tablets and run the application through terminal server. Not pretty. Not pretty.
I feel this is a serious question that is not subjective so hopefully this won't get deleted.
Our current architecture on the server side is Entity Framework with a repository pattern, WCF services to satisfy CRUD requests returning composite data transfer objects, and a proxy for use by the clients.
I'm interested in hearing other developers' input and this design puzzle.
Additional Information Added to the Discussion
Lots of good information provided!!! I'll have to look at Microsoft Sync for sure. For the disconnected database I would be placing only list tables (enumerations) in the initial database. Jobs and, if needed, an item we call dry books, will be added for each client we are helping. (though I hope the internet returns by the time we are cleaning and drying out the homes) These are the tables that would then populate back to the host once we have a stable link. In the case of Katrina we also lost internet connectivity in our offices which meant the office provided no communication relief for days as well.
Last night I realized that our client proxy is the key to everything working! The client remains unaware of the fact that it is online or offline and leaves the synchronization process within that library. We are discovering how much data we are talking about today. I also want to make it clear that ASP.NET was a like-to-have but a thick client (actually WPF with XAML) may end up being our end state.
Now -- for multiple updates. The disconnected work will be going to individual homes by a single franchise. In fact our home office dispatches specific franchises to specific events. So we have a reduced likelihood (if any) of the problem of multiple people updating a record. The reason is that they are creating records for each job (person's home/office/business) and only that one franchise will deal with it. Of course this also means that if they are disconnected for days that the device that creates the job (record of who, where, condition, insurance company, etc) is also the only device that knows of the job. But that can be lived with. In fact we may be able to have a facility to sync the franchise devices on a hub.
I'm looking forward to hearing additional stories of how you've implemented your disconnected environment.
Thanks!!!
Looking at new technology from Microsoft
I was directed to look at a video from TechEd 2012 and thought I might have an answer. The talk was on using ASP.NET and MVC4 along with 2 libraries for disconnected behavior. At first I thought it would be great but then as it continued it worried me quite a bit.
First the use of a javascript backend to support disconnected I/O does not generate confidence. As a compiler guy (and one who wrote two interpretive languages) I really do not like having a critical business model reliant upon interpretive javascript. And script at that! It may be me but it just makes me shudder.
Then they show their "great"(???) programming model having your ViewModel exist as just javascript. I do not care for an application (asp.net and javascript) that can be, and may as well be (for lack of intellisense ) written in notepad.
No offense meant to any asp lovers, but a well written C# program that has been syntactically and type checked gives me stronger confidence in software than something written with a hope and prayer that a class namespace has been properly typed without any means of cross check. I've seen too many hours of debugging looking for a bug that ended up in a huge namespace with transposed ie in it's name. I ran my thought past the other senior developers in my group and we are all in consensus on this technology.
But we continue to look. (I feel this is becoming more of a diary than a question) :)
Looks like a perfect example for Microsoft Sync Framework
http://msdn.microsoft.com/en-us/sync/bb736753.aspx
A comprehensive synchronization platform that enables collaboration
and offline access for applications, services, and devices with
support for any data type, any data store, any transfer protocol, and
any network topology.
I often find that building a lightweight framework to fit my specific needs is more beneficial to me than using an existing one. However, always look at what's available and weigh the pros and cons before making that decision.
I haven't use the Microsoft Sync Framework, but it sounds like that's a good one to research first. If you have Sql Server Standard (or some other version other than the Express version) then replication might also be an option.
If you want to develop your own homegrown solution, then be sure to put lastupdated and dateadded fields on any tables that need to stay in sync. It doesn't 'sound' like your scenario will be burdened by concurrency issues (i.e. if person A and B both modify a field at the same time, who wins?). If that's the case then developing your own lightweight solution will be pretty straightforward.
As Jeremy pointed out, you will need a way to get the changes. In addition to using a web service, you can also use WCF which is similar to a web service in some ways. But my personal bias would be towards just accessing a SQL server remotely over the internet. The downside of that solution is added security concerns, while the upside is decreased development overhead (i.e. faster/easier development now and less maintenance over time). Also, the direct SQL solution is also assuming that this is an internal application... that you're in charge of all development and not working with 3rd parties who need access to your data and wouldn't be allowed to access it this way.
Not really a full answer but too much for a comment.
I have two apps one that synchs one way and the other two way.
I do a one way synch to client for disconnected operation. At the server full SQL Server and at the client Compact Edition. TimeStamp is a prefect for finding any rows that needs to be synched. I also don't copy the whole database as some of the largest table are non nonessential. The common use is the user marks identified records they want to synch.
If synch does what you need great +1 for Jakub. For me I don't have the option to synch the whole MSSQL both based on size and security.
Have another smaller application that synchs two way but in this case it has regions and update are only within the region. So a region only synchs their data and in disconnected mode they can only add new records. Update to an existing records must be performed in connected mode. That was mangeable. In that case MSSQL for the master and used XML for the client.
No news to you but the hard part of a raw synch is that two parties may have added or revised the same record.
My project is converting a legacy fat-client desktop application into the web. The database is not changing as a result. Consequently, we are being forced to call external web services to access data in our own database. Couple this with the fact that some parts of our application are allowed to access the database directly through DAOs (a practice that is much faster and easier). The functionality we're supposed to call web services for are what has been deemed necessary for downstream, dependent systems.
Is this really how SOA is supposed to work? Admittedly, this is my first foray into the SOA world, but I have to think this is the complete wrong way to go about this.
I agree that it's the wrong approach. Calling your own database via a webservice should raise red flags in a design review, and a simple DAO is the way to go (KISS principle).
Now, if it's data that truly needs to be shared across your company (accounts, billing, etc) THEN it's time to consider a more heavy-duty solution such as SOAP or REST. But your team could still access it directly, which would be faster.
My team had the same thing happen with a web service that we wanted to call in batch mode. Rather than call our own SOAP endpoint, we instead set it up to call a POJO (plain old java object) interface. There's no XML transformation or extra network hop through an SOA appliance.
It's overkill to put an XML interface between MVC layers when your team owns the whole application. It may not be traditional SOA... but IMO it's traditional common sense. ;)
I've seen people try to jam SOA at too low a level and this may be such a case. I would certainly not equate DAO and SOA at the same level.
I agree with #ewernli
What is SOA "in plain english"?
IMHO, SOA makes sense only at the enterprise-level, and means nothing for a single application.
If I'm reading into your question correctly, your web services are for C/R/U/D data into the database. If so, providing C/R/U/D services directly to the database and its tables are likely too low level to be SOA services.
I'd look for services at a higher level and try to determine whether they are interesting at to the enterprise. If so, those are your services. I'd also ask myself whether my former desktop app is providing services (i.e. should you be looking to make your new app an SOA service itself rather than trying to force an SOA architecture into the desktop app at a low level.
Consequently, we are being forced to
call external web services to access
data in our own database.
Man, that gotta hurt. As far as services in SOA go,
a service is a repeatable logical manifestation of a business task - that means you are not implementing SOA if you are not 'service enabling' business processes. If you are putting some web services to select data out of your data base, all you got is a bunch of webservices, which would slowdown your applications which could have been faster by conventional data access patterns (like DAO)
When you equate SOA with Web services there is a risk of replacing existing APIs with Web services without proper architecture. This will result in identifying many services that are not business aligned.
Also, service orientation is a way of integrating a business as a group of linked services - so ask yourself is the organization making use of these atomic services to achieve further benefits?
Do a google search for SOA anti-patterns and you will find what are the different ways to end up with a pile of web-services instead of SOA.
SOA... SOA... is the bane of my existence, for just this reason. What, or what not, constitutes SOA? I support SOA products in my day job, and some people get it, some don't. SOA.. SOA is about wrapping discrete business services in XML. ZIP+4 validation services. Payment gateways. B2B messaging.
SOA CAN be used to decouple desktop apps from backend databases. Sometimes it doesn't make sense, sometimes it does. What almost NEVER makes sense is low-latency high-query-count logic. If you ever have to use an application in France directly connected to a database in California, you'll get what I mean. SOA pretty much forces you to then smartly about how you model and return your data (look into SDO - Service Data Objects). The devil's in the details though. Marshalling data to/from XML can be costly.
Good SOA design is all about separation of behavior and data.
I repeat behavior and data need to be separate or else you will have lots or problems whether its CORBA/SOAP/REST/XMLRPC or even plain old in-the-same-JVM-method calls.
Lots of people will talk about service end points, message handling, and contracts making SOA one of the more soporific areas of computing when its surprisingly not complicated.
If you are doing Java its really easy. Make POJOs for your domain objects with no weird state behavior and no weird collaborators and then make Service classes with the behavior. More often then not you can just use your DAO as the service (I mean you should have a thin layer over the DAO but if you don't need one....).
OOP lovers will disagree of this separation of data and behavior but this design pattern scales extremely well and is infact what most functional programming languages like Erlang do.
That being said if you are making a video game or something very state based then this design philosophy is a bad idea. BTW SOA is about as vacuous as the term enterprise.
Which part do you think is wrong? The part that you have to hit the web service, or the part you are hitting the database directly?
SOA is more of an API design guideline, not a development methodology. It's not an easy thing to implement, but the reward of reusability is often worth it.
See Service-Oriented Architecture expands the vision of Web services or any technical book on SOA. Simply wrapping function calls with web call does not make it a Service Oriented Architecture. The idea of the SOA is to make reusable services, and then you make higher level services (like website) by compositing or orchestrating underlying low-level services. At the very low level, you should focus on things like statelessness, loose coupling, and granularity. Modern frameworks like Microsoft's WCF supports wiring protocols like SOAP, REST, and faster binary side by side.
If your application is designed to run over the Internet, you should be mindful of the network latency issues. In a traditional client-server application that is deployed on a LAN, because the latency is sub 10 msec, you could hit the database every time you need the data without interrupting the user experience. However, on the Internet, it is not uncommon to have 200 msec latency if you go across proxies or oceans. If you hit the database 100 times, and that will add up to 20 seconds of pause. In SOA, you would try to pack the whole thing into a single document, and you exchange the document back and forth, similar to the way tax is filed using Form 1040 if you live in the US.
You may say that the latency issue is irrelevant because the web service is only consumed by your web application layer. But you could hit the web service from the browser using AJAX reload the data, which should give the user shorter response time.
Closed. This question needs to be more focused. It is not currently accepting answers.
Closed 9 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
Can someone explain in plain english what is SOA all about? I hear SOA here, SOA there but I cannot understand exacly what it is and what is used for. Was it some simple concept and later evolved into something huge or what?
All documents, including wiki are a bit abstract or maybe I'm an idiot and don't get it. Is there an idiot's guide on this?
What exactly is there behind these three letters?
SOA is a new badge for some very old ideas:
Divide your code into reusable modules.
Encapsulate in a module any design decision that is likely to change.
Design your modules in such a way that they can be combined in different useful ways (sometimes called a "family" or "product line").
These are all bedrock software-development principles, many of them first articulated by David Parnas.
What's new in SOA is
You're doing it on a network.
Modules are communicating by sending messages to each other over the network, rather than by more tradtional programming-language mechanisms like procedure calls. In particular, in a service-oriented architecture the parts generally don't share mutable state (global variables in a traditional program). Or if they do share state, that state is carefully locked up in a database which is itself an agent and which can easily manage multiple concurrent clients.
You might find this article (What is SOA? - SOA and Web Services Explained ) helpful.
A little teaser:
SOA is a style of architecting applications in such a way that they are composed of discrete software agents that have simple, well defined interfaces and are orchestrated through a loose coupling to perform a required function.
There are 2 roles in SOA- a service provider and a service consumer. A software agent may play both roles. SOA is not an entirely new concept – however, this article mainly focuses on SOA as implemented with web services.
I see many answers explaining a Service Oriented Architecture (SOA) using even more advanced words and technical terms. I'd like to give a shot at explaining it for the layman, using an analogy in plain english.
But first a description of a SOA
SOA could be described in three layers as seen in the picture below. On one side we have the Provider and on the other side we have the Consumer, separated by a Bridge where the two sides communicate.
The consumer uses a number of Applications necessary for it's business and the provider uses Components that provide these applications with information. They communicate through a set of Services using a common architecture.
The analogy
Imagine a house on the country side, that in many ways is part of a larger community, like a city or town. The city has it's own complex systems for providing water and electricity, handling sanitation, providing transportation and other utilities. The House is the consumer in this model, the City (or community) is the provider and the pipes, sewers, powerlines, optical fibers etc. is the Infrastructure in which they communicate.
This model could loosely be compared to a SOA. The people in the house uses a number of different "applications" like radiators, computers, toilets, lamps, underfloor heating, bathtubs etc. These applications don't care how the city generates the water, creates the electricity or handles the waste as long as it works. The components of the city are generators, water pumps and sanitation areas. It provides the house with all these needs but it's up to the house to use it in what ever way it sees fit.
I hope this gave at least someone a better picture of a SOA.
Let's assume you have four cooks. In SOA, you assume they hate each other, so you strive to let them have to talk to each other as little as possible.
How do you do that? Well, you will first define the roles and interface -- cook 1 will make salad, cook 2 will make soup, cook 3 will make the steak, etc.. Then you will place the dishes well organised on the table (so these are the interfaces) and say, "Everybody please place your creation into your assigned dishes. Don't care about anybody else.".
This way, the four cooks have to talk to each other as little as possible, which is very good in software development -- not necessarily because they hate each other, but for other reasons like physical location, efficiency in making decisions etc.
It also means you can recombine the dishes (services) as you like. For example, you might just use the dessert to service a cafe, or just take the soup and combine it with a bread you bought from another company to provide a cheaper menu, or let other restaurants use your salads to combine with their dishes, etc.
One of the most successful implementation of SOA was at Amazon. Because of their design, they could re-package their whole infrastructure and sell it as Amazon Web Service.
*This is only one aspect of SOA.
SOA is an architectural style but also a vision on how heterogeneous application should be developped and integrated. The main purpose of SOA is to shift away from monolithic applications and have instead a set of reusable services that can be composed to build applications.
IMHO, SOA makes sense only at the enterprise-level, and means nothing for a single application.
In many enterprise, each department had its own set of enterprise applications which implied
Similar feature were implemented several times
Data (e.g. customer or employee data) need to be shared between
several applications
Applications were department-centric.
With SOA, the idea is to have reusable services be made available enterprise-wide, so that application can be built and composed out of them. The promise of SOA are
No need to reimplement similar features over and over (e.g.
provide a customer or employee service)
Facilitates integration of applications together and the access
to common data or features
Enterprise-centric development
effort.
The SOA vision requires an technological shift as well as an organizational shift. Whereas it solves some problem, it also introduces other, for instance security is much harder with SOA that with monolithic application. Therefore SOA is subject to discussion on whether it works or not.
This is the 1000ft view of SOA. It however doesn't stop here. There are other concepts complementing SOA such as business process orchestration (BPM), enterprise service bus (ESB), complex event processing (CEP), etc. They all tackle the problem of IT/business alignement, that is, how to have the IT be able to support the business effectively.
SOA is acronym for Service Oriented Architecture.
SOA is designing and writing software applications in such a way
that distinct software modules can be
integrated seamlessly with high degree
of re-usability.
Most of the people
restrict SOA as writing client/server
software-web-services. But it is too
small context of SOA. SOA is much
larger than that and over the past few
years web-services have been primary
medium of communcation which is
probably the reason why people think
of SOA as web-services in general
restricting the boundaries and meaning
of SOA.
You can think of writing a database-access module which is so independent that it can work on its own without any dependencies. This module can expose classes which can be used by any host-software that needs database access. There's no start-up configuration in host-application. Whatever is needed or required is communicated through classes exposes by database-access module. We can call these classes as services and consider the module as service-enabled.
Practicing SOA gives high degree of
re-usability by enforcing DRY [Don't
repeat your self] which results into
highly maintainable software.
Maintainability is the first thing any
software architecture thinks of - SOA
gives you that.
As far as I understand, the basic concept there is that you create small "services" that provide something useful to other systems and avoid building large systems that tend to do everything inside the system.
So you define a protocol which you will use for interaction (say, it might be SOAP web services) and let your "system-that-does-some-business-work" to interact with the small services to achieve your "big goal".
I would suggest you read articles by Thomas Erl and Roger Sessions, this will give you a firm handle on what SOA is all about. These are also good resources, look at the SOA explained for your boss one for a layman explanation
Building a SOA
SOA Design Pattern
Achieving integrity in a SOA
Why your SOA should be like a VW Beetle
SOA explained for your boss
WCF Service Performance
what tends to happen in large organizations is that over time everything is either monolithic or disparate systems everywhere or a little of both. Someone eventually comes in and says we've got a mess. Now, you want to re-design (money to someone) everything to be oriented in a sort of monotlithic depends on who you pay paradigm but at the same time be able to add pieces and parts independently of the master/monolith.
So you buy Oracle's SOA and Oracle becomes the boss of all your parts. All the other players coming in have to work with SOA via a service (web service or whatever it has.) The Oracle monolith takes care of everything (monolith is not meant derogatory). Oh yeah, you got ASP.NET MVC on the front or something else.
main thing is moving things in and out of they system without impact and keeping the vendor Oracle SOA, Microsoft WCF, as the brains of it all. everything's all oop/ood like, fluid, things moving in and out with little to no impact, even human services, not just computers.
To me it just means a bunch of web services (or whatever we call them in the future) with a good front end. And if you own the database just hit the database and stop worrying about buzzwords. it's okay.
Only one suggestion:-
Read SOA Concepts, Technology and Design by Thomas Erl.
It has very beautifully given the details about SOA in plain English and with case studies.
Well You see.. SOA stands for Service Oriented Architecture.... In simplest words, you write a piece of code that is very generic i.e. it does some thing that can be used in a lot of applications ... may be something like a address book or may be a calculator. and you launch this code on the IIS. So you provide a service through your code. So you are a service provider. Now someone wants to use a similar code then he does not have to write the code again. He simply uses your code maybe through a web service. Hence he becomes a service consumer. Hence making a program using such services is called SOA. And the loose coupling is there as the service provider and consumer may be interacting even if they are using diff programming languages.
Hope you understand.
from ittoolbox blogs.
The following outlines the similarities and differences to past design techniques:
• SOA versus Structured Programming
o Similarities: Most similar to subroutine calls where parameters are passed and the operation of the function is abstracted from the caller - e.g. CICS link and execute and the COBOL CALL reserved word. Copybooks are used to define data structure which is typically defined as an XML schema for services.
o Differences: SOA is loosely coupled implying changes to a service have less impact to the consumer (the "calling" program) and services are interoperable across languages and platforms.
• SOA versus OOA/OOD
o Similarities: Encapsulation, Abstraction and Defined Interfaces
o Differences: SOA is loosely coupled with no class hierarchy or inheritance, Low-level abstractions - class level versus business service
• SOA versus legacy Component Based Development (CBD) - e.g. CORBA, DCOM, EJB
o Similarities: Reuse through assembling components, Interfaces, Remote calls
o Differences: Wide adoption of standards, XML Schemas vs. Marshaled Objects, Service Orchestration, Designing for reuse is easier, services are business focused vs. IT focused, business services are course grained (broad in scope)
• SOA (for integration) versus Enterprise Application Integration (EAI)
o Similarities: Best practices (well defined interfaces, standardized schemas, event driven architecture), reusable interfaces, common schemas
o Differences: Standards, adoption, and improved tools
Reading the responses above, it sounds to me that SOA is what developers (good ones at least) have been doing from day one.
It could also stand for "Struct of Arrays" (as opposed to "Array of Structs") which is a common topic in parallel (especially SIMD) programming, but I'm guessing that's not what you mean here!
SOA is a buzzword that was invented by technology vendors to help sell their Enterprise Service Bus related technologies. The idea is that you make your little island applications in the enterprise (eg: accounting system, stock control system, etc) all expose services, so that they can be orchestrated flexibly into 'applications', or rather become parts of aggregate enterprise scoped business logic.
Basically a load of old bollocks that nearly never works, because it misses the point that the reasons why technology is the way it is in an organisation is down to culture, evolution, history of the firm, and the lock in is so high that any attempt to restructure the technology is bound to fail.
Have a listen to this week's edition of the Floss Weekly podcast, which covers SOA. The descriptions are pretty high level and don't delve into too many technical details (although more concrete and recognizable examples of SOA projects would have been helpful.
A traditional application architecture is:
A user interface
Undefined stuff (implementation) that's encapsulated/hidden behind the user interface
If you want to access the data programmatically, you might need to resort to screen-scraping.
SOA seems to me to be an architecture which focus on exposing machine-readable data and/or APIs, instead of on exposing UIs.
SOA or Service-Oriented Architecture is a software architecture pattern in which applications or systems are constructed from underlying (and usually distributed) software services that conform to a specific set of characteristics, namely:
Interface, Policy and Contract based
Location transparency
Autonomous
Abstract
Reusable
Composable
Stateless
Discoverable
Extensible
Loosely coupled
The primary goal of SOA is sofware development agility, i.e. the ability to respond the change easily, and cheaply, thus allowing businesses to rapidly respond to changing markets.
Services are typically (but by no means exclusively) implemented as web services, i.e. they operate over the ubiquitous web HTTP protocol, and are implemented either using XML-based SOAP or the lightweight (and more popular) REST paradigm.
Depends on who you are!
If you're an business owner, SOA is a solution to increase your incomes and business agility. If you're an entreprise architect, SOA is a way to draw nice and clean piece of software on a blank canvas. If you're an architect SOA is the solution to design loosely coupled services over an integration platform, to just plug services into outlets. If you're a developper SOA is a programming paradigm where a service is in the center of the design and the code.
You should read 100-SOA-Questions [pdf]
Cheers
Service Oriented Architecture (SOA) is a software architectural style that builds applications as a collection of pluggable parts, each of which can be reused by other applications.
I frequently hear Service-Oriented Architecture (SOA) being tossed around as a buzzword among non-technical customers or program managers with little concern or understanding for what it actually entails (example: "Can I buy a SOA?"). There's also a lot of misinformation about SOA (example: "Only web apps can use SOA") and a general lack of understanding for its capabilities (example: "SOA can make your make all of your data work together").
What are some key facts that you, as someone who understand the technical side of SOA, use to educate program managers on the appropriate use and understanding of SOA? What's the best way to set the record straight with non-technical folks?
For non technical people I would use the following concept. The whole professional world is service oriented.
Instead of baking a cookie by
youself, you go to the baker.
Instead of trying to cure yourself,
you go to the doctor.
Instead of writing a program, you
ask a programmer to do this for
you.
This implies two major advantages:
Each one does his job better than if
we all were trying to solve all our
tasks separately.
There is a way, which allows non
professionals to communicate with
those, who will solve our task (in
real world such way is money and
business contracts)
In the world of software such architecture is implemented by defining specialized services (applications) which are dedicated to perform specific tasks and by defining protocols, which are solving problem of communications between such applications.
When such architecture is deployed, you get some benefits, which can be also mapped to the real world:
If doctor is unavailable, you cannot
be cured but at least you can get a
cookie from the bakery! In software this means one failed service does not break the whole system.
Usually doctors and bakers do not share the same room and this allows them to operate better. Just like in software you can place each service on its own hardware.
For software world this means, better availability, maintainability, reuse, and reduced costs.
Good luck!
"SOA is like hiring new employees when the job gets too large for the current team." Each part of the whole system is analogous an employee. Managers understand employees ;)
Maybe you have some applications in your company to use as a demonstration.
Try to show them the big picture with lots of loosely dependent services with some common needs/features created by various teams, and pulling out those embedded but commonly used features and use them as service providers.
The other thing that came into my mind is to show them the various connectors that the services can use to communicate (maybe there are some really old screenscraping legacy apps). Also, the message bus concept with normalizing and transaction handling needs to be clarified. In my opinion, non-technical people should see this whole SOA concept as loosely coupled services talking to each other with any kind of messages, where services are written/managed/governed by different teams (so formal service declarations and SLAs can come handy).
Try to avoid mentioning vendors, if possible. Or mention lots of vendors and technologies for each part in order to show them the various options.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I'm in the market for a good open source network based Pub/Sub (observer pattern) library. I haven't found any I like:
JMS - tied to Java, treats message contents as dumb binary blobs
NDDS - $$, use of IDL
CORBA/ICE - Pub/Sub is built on-top of RPC, CORBA API is non-intuitive
JBOSS/ESB - not too familiar with
It would be nice if such a package could to the following:
Network based
Aware of payload data, users should not have to worry about endian/serialization issues
Multiple language support (C++, ruby, Java, python would be nice)
No auto-generated code (no IDLs!)
Intuitive subscription/topic management
For fun, I've created my own. Thoughts?
You might want to look into RabbitMQ.
As pointed-out by an earlier post in this thread, one of your options is OpenSplice DDS which is an Open Source implementation of the OMG DDS Standard (the same standard implemented by NDDS).
The main advantages of OpenSplice DDS over the other middleware you are considering can be summarized as:
Performance
Rich support for QoS (Persistence, Fault-Tolerance, Timeliness, etc.)
Data Centricity (e.g. possibility of querying and filtering data streams)
Something that I'd like to understand is what are your issues with IDL. DDS uses IDL as language-independent way of specifying user data types. However DDS is not limited to IDL, you could be using XML, if you prefer. The advantage of specifying your data types, and decoupling their representation from a specific programming language, is that the middleware can:
(1) take away from you the burden of serializing data,
(2) generate very time/space efficient serialization,
(3) ensure end-to-end type safety,
(4) allow content filtering on the whole data type (not just the header like in JMS), and
(5) enable on-the wire interoperability across programming languages (e.g. Java, C/C++, C#, etc.)
Depending on the system or application you are designing, some of the properties above might not be useful/relevant. In that case, you can simply generate one, a few, "DDS Type" which is the holder of you serialized data.
If you think about JMS, it provides you with 5 different topic types you can use to send your data. With DDS you can do the same, but you have the flexibility to define exactly the topic types.
Finally, you might want to check out this blog entry on Scala and DDS for a longer discussion on why types and static-typing are good especially in distributed systems.
-AC
We use the RTI DDS implementation. It costs $$, but it supports many quality of service parameters.
There is a free DDS implementation called OpenDDS, but I've not used it.
I don't see how you can get around the need to predefine your data types if the target language is statically typed.
Look a bit deeper into the various JMS implementations.
Most of them are not Java only, they provide client libraries for other languages too.
Suns OpenMQ have at least a C++ interface, Apache ActiveMQ provides client side libraries for many common languages.
When it comes to message formats, they're usually decoupled from the message middleware itself. You could define your own message format. You could define your own XML schema and send XML messages. You could send BER encoded ASN.1 using some 3. party library if you want.
Or format and parse the data with a JSON library.
You might be interested in the MUSCLE library (disclaimer: I wrote it, so I may be biased). I think it meets all of the criteria you specified.
https://public.msli.com/lcs/muscle/
Three I've used:
IBM MQ Series - Too Expensive, hard to work with.
Tico Rendezvous - (renamed now to EMS?) Was very fast, used UDP, could also be used with no central server. My favorite but expensive and requires a maint fee.
ActiveMQ - I'm using this currently but finding it crashes frequently. Also it's requires some projects ported from Java like spring.net. It works but I can't recommend it due to stability issues.
Also used MSMQ in an attempt to build my own Pub/Sub, but since it doesn't handle it out of the box your stuck writing a considerable amount of code.
There is also OpenSplice DDS. This one is similar to RTI's DDS, except that it's LGPL!
Check it out:
IBM Webpshere MQ, and the licence is not too expnsive if you work on a corporate level.
You might take a look at PubSubHubbub. It's a extension to Atom/RSS to alow pubsub through webhooks. The interface is HTTP and XML, so it's language-agnostic. It's gaining increasing adoption now that Google Reader, FriendFeed and FeedBurner are using it. The main use case is blogs and stuff, but of course you can have any sort of payload.
The only open source implementation I know of so far is this one for the Google AppEngine. They say support for self-hosting is coming.