I'm developing a wrapper around an existent RESTful API. I basically have to do some preprocessing, calling the underlying API, and some preprocessing, with a little bit of cache in the middle. The API is specially designed for RESTful access via http.
My question is, should I refactor the API so I can invoke it via code, or should I make local http calls to it. This second option seems nice since it increases decoupling, but I'm afraid that creating the http requests / responses can seriously affect performance. I've heard though that couchDB does something like that (its api is RESTful and accessed via http).
No one can answer this for you, as it will depend hugely on how your current RESTful API is implemented. For example, you can write a relatively short C program that will listen on a socket and handle HTTP requests - if it does RESTful things in response to the different HTTP methods then it's an implementation of a RESTful API and can have very very little overhead over just calling the underlying functions directly (without HTTP). On the other hand, you can write your program as this bloated, heavy Java EE monster - in that case, the overhead may be quite large.
Thus, skaffman was correct to say "Measure it and see" - this really is the only way to get a meaningful answer.
All that said, if you are asking this question, odds are good that you're not really facing a Google-scale problem, so if the refactoring is going to be a lot of work and just intercepting HTTP requests is easy for you then I'd suggest you first get the functionality you need with the HTTP wrapper and only once you have a working product start worrying about performance optimization.
Look at section 5.1.6 in the REST Dissertation about layered systems. What you are actually describing fits very nicely into this idea of a layered architecture. Effectively you are building a HTTP proxy that will intercept the incoming requests, do some work and then and then pass it along to the next layer.
I'd refactor it. You used to have some set of functionality exposed by a RESTful API. You've now got a set of functionality exposed by a RESTful API and by your wrapper. You should refactor the code so that it can do both. It should be easy if your code is reasonably well organized.
When in doubt, err on the side of doing less work. Write the wrapper and test it. Refactor if you have to.
Related
I need to build an LDAP proxy that I can program to inspect and modify the LDAP requests and responses - some of the LDAP requests/responses will simply be passed through, but for others I might want to send two different requests to the server and then combine the results (that's just one example - there will be other use cases).
I've looked at the proxying options documented for OpenLDAP's slapd, and I see that it has quite flexible configuration and 'overlays', but no capability to insert custom code.
So I think that's not a solution, unless slapd's source code is easy to modify, to insert my own modules plus hooks to/from the existing code (?)
An alternative would be to start with a friendly TCP/IP framework library (or even a complete TCP/IP proxy). Then I can link to an ASN.1 decoding/encoding library, and write the rest myself.
I'd prefer to avoid having to write (& learn) all the TCP/IP connection/message handling and event loop myself.
So I'm looking for the most complete starting point that does the hard work and gives me the flexibility to write what I need. Typical lazy/greedy approach :-)
Must be open source, ideally in C or C++, and I'll probably be targetting RHEL/CentOS 8 in a container.
JMS or messaging is really good in tying up disparate applications and form the infrastructure of many ESB and SOA architectures.
However say Application A needs an immediate response from a service on Application B e.g. Needs the provisioning details of an Order OR Needs an immediate confirmation on some update. Is Messaging the right solution for that from a performance point of view? Normally the client would connect to a MoM on a Queue - then a listener which has to be free will pick up the message and forward to the server side processor - which will process the response and send it back to a Queue or Topic and the requesting client will follow the same process and pick it up. If the message size is big the MoM will have to factor that in as well.
Makes me wonder if Http is a better solution to access such solutions instead of going via messaging route? I have seen lots of applications use MoM like AMQ or TIBCO Rvd to actually use for immediate Request/Response - but is that bad design or is it some fine tuning or setting that makes it same as Http.
It really depends on your requirements. Typically messaging services will support one or all of the following:
Guaranteed Delivery
Transactional
Persistent (i.e. messages are persisted until delivered, even if the system fails in the interrim)
An HTTP connection cannot [easilly] implement these attributes, but then again, if you don't need them, then I suppose you could make the case that "simple" HTTP would offer a simpler and more lightweight solution. (Emphasis on the "simple" because some messaging implmentations will operate over HTTP).
I don't think Request/Response implemented over messaging is, in and of itself, bad design. I mean, here's the thing.... are you implementing both sides of this process ? If not, and you already have an established messaging service that will respond to requests, all other considerations aside, that would seem to be the way to go... and bypassing that to reimplement using HTTP because of some desgin notion would need some fairly strong reasoning behind it, in my view.
But the inverse is true as well. If you an HTTP accessible resource already, and you don't have any super stringent requirements that might otherwise suggest a more robust messaging solution, I would not force one in where it's not warranted.
If you are commensing totally tabula-rasa and you must implement both sides from scratch..... then..... post another question here with some details ! :)
To get a search website built quickly I plan to split the work between two teams: One to build the search engine and one to build web UIs (mobile/desktop). My plan is to build the search engine as a set of REST services based on .NET 3.5. UIs may be built using some other technology.
Questions: is the REST interface likely to be a performance bottleneck? How best to avoid this?
REST is unlikley to be a bottleneck in this scenario. It wasn't clear from your post whether you were making REST calls directly from your HTML UI on the client, or whether you were making server-to-server REST calls on the back end. So I'll cover both cases below.
If your REST calls are being made between your client UI and your servers, then using REST or another HTTP remoting approach matters relatively little-- the time it takes to execute the search on the back end and then send the results back down to the client should dwarf the impact of the REST call itself. If you want to improve perf, focus on client-side networking tricks (e.g. HTTP Compression, proper caching headers, etc) and optimizing your search engine itself.
If your architecture is one tier of servers (hosting your web UI) calling another tier (your serach engine), then calling between those tiers over REST also shouldn't add too much to your overall latency. This is because (same as above) running the search and sending results back down to the client will usually take a few hundred milliseconds at least, and the overhead of the back-end REST call (if done properly) will usually be 50ms or less.
That said, it's easy to mess up the client end of server-to-server HTTP calls. For example, many HTTP Client libraries (including .NET's) will by default limit the number of concurrent client connections, which makes sense if you're building an actual client app but will kill your scalability if used from a "client" that's actually a server serving hundreds of users concurrently. Other potential problems include authentication issues, proxy problems, DNS, etc. So be careful to build and configure your REST client code carefully, and be sure to load-test with a few hundred concurrent users!
No. REST is not (and generally cannot be) a bottleneck. REST is HTTP without the fancy HTML page. It's cheaper and faster than a regular web page.
I think it should not effect your performance, but to have a proper use of REST service .Net has ASP.Net MVC which supports REST fully.
Do remember to read through this link http://www.ytechie.com/2008/10/aspnet-mvc-what-about-seo.html
I'm implementing a high traffic client web application that uses a lot of REST API's for its data access layer from the cloud database. I said client because it implements REST and not provides it.
REST APIs are implemented server side as well as client side and I need to figure out a good solution for caching. The application is running on a web farm so it I'm leaning toward a distributed caching like memcached. This caching solution will need to be like a proxy layer between my application and REST APIs and support both client side as well as server side.
For example if I make a call to update a record I would update through REST and I'd like to keep updated record in the cache so next calls to that record won't need extra call to the outside REST services.
I want to minimize REST calls as much as possible and would need to keep the data accurate as much as I can, but it doesn't need to be 100% accurate.
What is the best solution for this caching proxy? Is it a standalone application that runs on one of the servers with local cache, or built into current solution using distributed caching? what are you ideas, suggestion or concerns
Thank you,
You hit the nail on the head. You need a caching layer that acts as a proxy to your data.
I suggest that you create a layer that abstracts the concept of the cloud a way a bit. Your client shouldn't care where the data comes from. I would create a repository layer that communicates with the cloud and all other data. Then you can put a service layer on top of that that your client would actually call into. Inside this service layer is where you would implement things like your caching layer.
I used to always suggest using MemCached or MemCached Win32 depending on your environment. MemCached win32 works really well if you are in a windows world! Look to the Enyim client for MemCached win32...it is the least problematic of all the other ports.
If you are open to it though and you are in a .net world then you might try Velocity. MS finally got the clue that there was a hole in their caching framework in that they needed to support the farm concept. Velocity last time I checked is not out of beta yet...but still worth a look.
I generally suggest using the repository and service layer concepts from day one...even though you don't need it. The flexibility it provides for your application is worth having as you never know which direction your application will need to be pulled in. Needing to scale is usually the best reason to need this flexibility. But usually when you need to scale you need to scale now and refactoring in a repository layer and services layer while not impossible is usually semi-complex to do down the road.
I'm working on an app which will, like most apps, have a whole boat load of buisness logic, almost all of which will need to be executed both on the server and the Flash-based client… And I'm trying to figure out the best (read: least complex) way to implement the rules engine.
These are the parameters of the problem:
The rules engine must both run in a web browser (ie, in Flash Player) and on the server. Duplicating the logic (eg, by writing a "server" version and a "client" version) would be an unacceptable risk.
The input/output data is fairly complex, so serialization is a nontrivial problem. We are currently using AMF for all of our serialization needs, and using another protocol would add significant complexity… So it should probably be avoided.
It is infeasible to implement a "rules description language". Experimentation has shown that rules are sufficiently complex that any such language would need to be Turing complete… Which would also add a significant amount of complexity.
The rules engine will not need to make some, but not very many, service calls.
Currently, the best contenders are:
Writing the code in ActionScript, then running it on the server. In theory it's possible to start up an AVM instance, get it long-polling a gateway, then pass data back and forth that way… But that seems less than ideal. Is there a "good" way of doing this?
Writing the code in Haxe. I don't know anything about Haxe's AMF support, so that could be a deal-breaker.
Something involving Tamarin. Seems like a viable option, but I haven't done enough research to tell either way.
So, what do you think? Are any of these options clearly better than others? Is there something I haven't though of that's worth considering?
Finally, thanks for reading this wall of text :)
How much data are you talking about? You can use Air if you want to run it on the server and access a queue or something.