In an application .Net XCC being used to make communication with marklogic module database to execute module, function and adhoc queries etc.
I want to replace the same XCC calls with REST calls so that we can run application in marklogic 9 as .Net XCC has been deprecated in Marklogic 9.
I have tried in built rest api in marklogic. It only allows to execute module exiting in module database.
Is there any online source stuffs available or anything that could help us.
Any help would be appreciated.
Thanks,
ArvindKr
There is /v1/invoke to invoke modules in the modules database attached to the REST app-server you are addressing, but also /v1/eval that allows running ad hoc queries.
HTH!
If you're going to replace XCC.NET with RESTful calls, try out XQRS, it allows you to build services in XQuery in a manner similar to JAX-RS for Java.
I only consider the following for cases such as yours, where compatibility with legacy code is useful or required and where other options are exausted. This is not an elegant approach, but it may be useful in special cases.
The XDBC protocol (which is what XCC uses) is supported natively on the exactly same app servers and ports which the REST API is exposed. You can see this on port 8000 in a default install. The server literally cannot tell a 'REST Application' and an 'XCC Application' apart except by the URI requested in the request (and in some cases additional headers like cookies). REST and XDBC are both HTTP based, and at the HTTP layer are very similar to the extent that they can share the same ports and configurations.
XDBC is 'passed through' the REST processing via the XML Rewriter. XDBC uses /eval and /invoke while REST uses /v1/eval and /vi/invoke. If you look at the default rewriter.xml for port 8000 you can see how the routing is made. While the XDBC protocol is not formally published its not difficult to 'reverse engineer' by looking at the XCC code (public java source) and the rewriter. For example its not difficult to construct URL and payload data to do a basic eval or invoke call. You should be able to replicate existing XCC.NET client behaviour exactly by using the /eval and /invoke endpoints (look for the xdbc attribute set in the rewriter.xml, this causes the request handling to use pure XDBC protocol and behaviour.
Another alternative, if you cannot solve the external variables problem is to write new 'REST Friendly' apis that then xdmp:invoke() on the legacy APIS passing in the appropriate namespaces. An option is to put the legacy code in an entirely seperate modules DB and then replicate the module URIs exactly with the new code. If you don't need to maintain co-existing versions then you modify the old code to remove the namespaces from the parameters or assign local variable aliases.
We use nomad to deploy our applications - which provide gRPC endpoints - as tasks. The tasks are then registered to Consul, using nomad's service stanza.
The routing for our applications is achieved with envoy proxy. We are running central envoy instances loadbalanced at IP 10.1.2.2.
The decision to which endpoint/task to route is currently based on the host header and every task is registered as a service under <$JOB>.our.cloud. This leads to two problems.
When accessing the service, the DNS name must be registered for the loadbalancer IP which leads to /etc/hosts entries like
10.1.2.2 serviceA.our.cloud serviceB.our.cloud serviceC.our.cloud
This problem is partially mitigated by using dnsmasq, but it is still a bit annoying when we add new services
It is not possible to have multiple services running at the same time which provide the same gRPC service. If we e.g. decide to test a new implementation of a service, we need to run it in the same job under the same name and all services which are defined in a gRPC service file need to be implemented.
A possible solution we have been discussing is to use the tags of the service stanza to add tags which define the provided gRPC services, e.g.:
service {
tags = ["grpc-my.company.firstpackage/ServiceA", "grpc-my.company.secondpackage/ServiceB"]
}
But this is discouraged by Consul:
Dots are not supported because Consul internally uses them to delimit service tags.
Now we were thinking about doing it with tags like grpc-my-company-firstpackage__ServiceA, ... This looks really disgusting, though :-(
So my questions are:
Has anyone ever done something like that?
If so, what are recommendations on how to route to gRPC services which are autodiscovered with Consul?
Does anyone have some other ideas or insights into this?
How is this accomplished in e.g. istio?
I think this is a fully supported usecase for Istio. Istio will help you with service discovery w/ Consul and you can use route rules to specify which deployment will provide the service. You can start explore from https://istio.io/docs/tasks/traffic-management/
We do something similar to this, using our own product, Turbine Labs.
We're on a slightly different stack, but the idea is:
Pull service discovery information into our control plane. (We use Kubernetes but support Consul).
Organize this service discovery information by service and by version. We use the tbn_cluster, stage, and version (like here).
Since version for us is the SHA of the release, we don't have formatting issues with it. Also, they don't have to be unique, because the tbn_cluster tag defines the first level of the hierarchy.
Once we have those, we use UI / API to define all the routes (e.g. app.turbinelabs.io/stats -> stats_service). These rules include the tags, so when we deploy a new version (deploy != release), no traffic is routed to it. Releases are done by updating the rules.
(There's even some nice UI affordances for updating those rules for the common case of "release 10% of traffic to the new version," like a slider!)
Hopefully that helps! You might check out LearnEnvoy.io -- lots of tutorials and best practices on what works with Envoy. The articles on Service Discovery Integration and Incremental Blue/Green Releases may be helpful.
In order to test how a REST service performs under load I would like to send production data to two service end points, one production service and one test service. I would like the responses from the test service to be ignored so the client has no idea it's request is being sent to 2 services. I would like something in between the client and the service so that I do not need to make any changes to either, just have some sort of additional 'filter' service that the traffic will go through which will then pass the request on untouched but also send a duplicate request to the test service. Someone suggested that I might be able to do this using Apache Camel but it's a bit overwhelming and I don't want to get started learning Camel if my aim is not possible. Has anyone achieved anything similar, either using Camel or some other method?
This can be done by defining a jetty or a cxf endpoint that acts as a proxy routing the request to two other http endpoints ie, the REST services.
from("jetty:http://somehost:8282/xxx").
to("http://prod:8181/rest/service/xyz").
to("http://test:8182/rest/service/xyz");
The client can fire the load to http://somehost:8282/xxx. The client will not be aware that it is being routed to two services.
Note: The test will not yeild a real load result if you are routing the client request to two services via the above route or other proxy/router methodologies. Because some latency will be introduced by the proxy/route itself. Instead i suggest to use some load test tool to generate the load and directly fire to the prod and test endpoints separately.
For example, apache ab benchmark tool. Take a look at this and this.
I'm looking into building an ASP.NET MVC application that exposes (other than the usual HTML pages) JSON and XML REST services, as well as Web Sockets.
In a perfect world, I would be able to use the same URLs for the Web Sockets interface as I do for the other services (and determine which data to return by what the user agent requests) but, knowing that IIS wasn't built for persistent connections, I need to know if there's a way that I can accept (and possibly even handshake) the Web Sockets connection and then pass the connection off to another service running on the server.
I do have a workaround in mind if this isn't possible that basically involves using ASP.NET to check for the Web Sockets connection upgrade headers, and responding with a HTTP/1.1 302 Found that points to a different host that has my Web Sockets service configured to directly listen to the appopriate endpoint(s).
If I completely understand your goal, I believe you can use the IIS7/7.5 Application Request Routing module to accomplish this.
Here's a quick reference: http://learn.iis.net/page.aspx/489/using-the-application-request-routing-module/
Rather than 302 responses you could use ISAPI_rewrite to direct to an appropriate endpoint (and manipulate the HTTP header to get it there)
http://www.isapirewrite.com/docs/
Otherwise no, IIS cannot natively pass off an HTTP connection. The current MSFT method is to use a 302 or something else that is intercepting the raw socket and performing header manipulation prior to sending to IIS (or whatever other application)
It strikes me that this would be a better question to ask Microsoft than to ask us. Web Sockets is new technology, and rather than looking for a hack, you might want to ask Microsoft how they plan to support it. IIS is their software. Poke around on http://iis.net (maybe in http://forums.iis.net) and see what you learn.
The way to do this is to use a unique Session ID that is associated with the Http Session. From the description, it seems like you might want to scope this to a single HttpApplication instance, but this is not necessary (you may also persist a session across many application instances). Anyway, this session ID needs to be attached somehow to each Http Request (either with a cookie, querystring, static variable with the HttpApplication instance, form data). Then you persist the identifying information about the Http session somewhere with the ID.
This identifying information may vary depending on your needs but could entail the entire http request or just some stripped down representation that serves your particular purpose.
Using this SessionID somewhere in the Http request allows you to restore whatever information you need to call and interact with the appropriate services. The instances of the services may also need to be scoped to the session as well.
Basically, what I am suggesting is that you NOT directly pass the Http connection to an external process, but instead pass the necessary data to the external process, and allow create a mechanism for sending callback data. I think looking into the mediator pattern may be helpful for you in understanding what I mean here. http://en.wikipedia.org/wiki/Mediator_pattern . I hope this helps.
After reading a lot about the differences between REST and SOAP, I got the impression that REST is just another word for HTTP. Can someone explain what functionality REST adds to HTTP?
Note: I'm not looking for a comparison of REST versus SOAP.
No, REST is the way HTTP should be used.
Today we only use a tiny bit of the HTTP protocol's methods – namely GET and POST. The REST way to do it is to use all of the protocol's methods.
For example, REST dictates the usage of DELETE to erase a document (be it a file, state, etc.) behind a URI, whereas, with HTTP, you would misuse a GET or POST query like ...product/?delete_id=22.
HTTP is a protocol used for communication, usually used to communicate with internet resources or any application with a web browser client.
REST means that the main concept you are using while designing the application is the Resource: for each action you want to perform you need to define a resource on which you often do only CRUD operation, which is a simple task. For that it's very convenient to use four verbs used in HTTP protocol against the four CRUD operations (GET for Read, POST is for CREATE, PUT is for UPDATE and DELETE is for DELETE).
That's unlike the older concept of RPC (Remote Procedure Call), in which you have a set of actions you want to perform as a result of the user's call. if you think for example on how to describe a Facebook like on a post, with RPC you might create services called AddLikeToPost and RemoveLikeFromPost, and manage it along with all your other services related to FB posts, thus you won't need to create special object for Like.
With REST you will have a Like object which will be managed separately with Delete and Create functions. It also means it will describe a separate entity in your DB. That might look like a small difference, but working like that would usually yield a much simpler code and a much simpler application. With that design, most of the app's logic is obvious from the object's structure (model), unlike RPC with which you would usually have to explicitly add a lot more logic.
Designing a RESTful application is often a lot harder because it requires you to describe complicated things in a simple manner. Describing all functionalities using only CRUD functions is tricky, but after doing that your life would be a lot simpler, and you will find that you write a lot shorter methods.
One more restraint REST architecture presents is not to use a session context when communicating with a client (stateless), meaning all the information needed to understand who is the client and what he wants is passed with the web message. Each call to a function is self-descriptive, there is no previous conversation with the client which can be referenced in the message. Therefore, a client could not tell you "give me the next page" since you don't have a session to store what is the previous page and what kind of page you want, the client would have to say "my name is Yuval, get me page 2 of a specific post in a specific forum". This means a bit more data would have to transfer in the communication, but think of the difference between finding a bug reported from the "get me the next page" function in oppose to "get me page 2 of question ID 2190836 in stack overflow".
Of course there is a lot more to it, but to my humble opinion these are the main concepts in a teaspoon.
REST enforces the use of the available HTTP commands as they were meant to be used.
For example, I could do:
GET
http://example.com?method=delete&item=xxx
But with rest I would use the "DELETE" request method, removing the need for the "method" query param
DELETE
http://example.com?item=xxx
HTTP is an application protocol. REST is a set of rules, that when followed, enable you to build a distributed application that has a specific set of desirable constraints.
If you are looking for the most significant constraints of REST that distinguish a RESTful application from just any HTTP application, I would say the "self-description" constraint and the hypermedia constraint (aka Hypermedia as the Engine of Application State (HATEOAS)) are the most important.
The self-description constraint requires a RESTful request to be completely self descriptive in the users intent. This allows intermediaries (proxies and caches) to act on the message safely.
The HATEOAS constraint is about turning your application into a web of links where the client's current state is based on its place in that web. It is a tricky concept and requires more time to explain than I have right now.
HTTP is a contract, a communication protocol and REST is a concept, an architectural style which may use HTTP, FTP or other communication protocols but is widely used with HTTP.
REST implies a series of constraints about how Server and Client should interact. HTTP is a communication protocol with a given mechanism for server-client data transfer, it's most commonly used in REST API just because REST was inspired by WWW (world wide web) which largely used HTTP before REST was defined, so it's easier to implement REST API style with HTTP.
There are three major constraints in REST (but there are more):
Interaction between server and client should be described via hypertext only.
Server and client should be loosely coupled and make no assumptions about each other. Client should only know resource entry point. Interaction data should be provided by the server in the response.
Server shouldn't store any information about request context. Requests must be independent and idempotent (means if same request is repeated infinitely, exactly same result is retrieved)
And HTTP is just a communication protocol (a tool) that can help to achieve this.
For more info check these links:
https://martinfowler.com/articles/richardsonMaturityModel.html
http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
REST = Representational State Transfer
REST is a set of rules, that when followed, enable you to build a distributed application that has a specific set of desirable constraints.
REST is a protocol to exchange any(XML, JSON etc ) messages that can use HTTP to transport those messages.
Features:
It is stateless which means that ideally no connection should be maintained between the client and server.
It is the responsibility of the client to pass its context to the server and then the server can store this context to process the client's further request. For example, session maintained by server is identified by session identifier passed by the client.
Advantages of Statelessness:
Web Services can treat each method calls separately.
Web Services need not maintain the client's previous interaction.
This in turn simplifies application design.
HTTP is itself a stateless protocol unlike TCP and thus RESTful Web Services work seamlessly with the HTTP protocols.
Disadvantages of Statelessness:
One extra layer in the form of heading needs to be added to every request to preserve the client's state.
For security we need to add a header info to every request.
HTTP Methods supported by REST:
GET: /string/someotherstring
It is idempotent and should ideally return the same results every time a call is made
PUT:
Same like GET. Idempotent and is used to update resources.
POST: should contain a url and body
Used for creating resources. Multiple calls should ideally return different results and should create multiple products.
DELETE:
Used to delete resources on the server.
HEAD:
The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The meta information contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request.
OPTIONS:
This method allows the client to determine the options and/or requirements associated with a resource, or the capabilities of a server, without implying a resource action or initiating a resource retrieval.
HTTP Responses
Go here for all the responses.
Here are a few important ones:
200 - OK
3XX - Additional information needed from the client and url redirection
400 - Bad request
401 - Unauthorized to access
403 - Forbidden
The request was valid, but the server is refusing action. The user might not have the necessary permissions for a resource, or may need an account of some sort.
404 - Not Found
The requested resource could not be found but may be available in the future. Subsequent requests by the client are permissible.
405 - Method Not Allowed
A request method is not supported for the requested resource; for example, a GET request on a form that requires data to be presented via POST, or a PUT request on a read-only resource.
404 - Request not found
500 - Internal Server Failure
502 - Bad Gateway Error
Not quite...
http://en.wikipedia.org/wiki/Representational_State_Transfer
REST was initially described in the
context of HTTP, but is not limited to
that protocol. RESTful architectures
can be based on other Application
Layer protocols if they already
provide a rich and uniform vocabulary
for applications based on the transfer
of meaningful representational state.
RESTful applications maximise the use
of the pre-existing, well-defined
interface and other built-in
capabilities provided by the chosen
network protocol, and minimise the
addition of new application-specific
features on top of it.
http://www.looselycoupled.com/glossary/SOAP
(Simple Object Access Protocol) The
standard for web services messages.
Based on XML, SOAP defines an envelope
format and various rules for
describing its contents. Seen (with
WSDL and UDDI) as one of the three
foundation standards of web services,
it is the preferred protocol for
exchanging web services, but by no
means the only one; proponents of REST
say that it adds unnecessary
complexity.
REST is a specific way of approaching the design of big systems (like the web).
It's a set of 'rules' (or 'constraints').
HTTP is a protocol that tries to obey those rules.
From You don't know the difference between HTTP and REST
So REST architecture and HTTP 1.1 protocol are independent from each
other, but the HTTP 1.1 protocol was built to be the ideal protocol to
follow the principles and constraints of REST. One way to look at the
relationship between HTTP and REST is, that REST is the design, and
HTTP 1.1 is an implementation of that design.
HTTP is a communications protocol that transports messages over a network.
SOAP is a protocol to exchange XML-based messages that can use HTTP to transport those messages.
Rest is a protocol to exchange any(XML or JSON) messages that can use HTTP to transport those messages.
REST is not necessarily tied to HTTP. RESTful web services are just web services that follow a RESTful architecture.
What is Rest -
1- Client-server
2- Stateless
3- Cacheable
4- Layered system
5- Code on demand
6- Uniform interface
REST APIs must be hypertext-driven
From Roy Fielding's blog here's a set of ways to check if you're building a HTTP API or a REST API:
API designers, please note the following rules before calling your creation a REST API:
A REST API should not be dependent on any single communication protocol, though its successful mapping to a given protocol may be dependent on the availability of metadata, choice of methods, etc. In general, any protocol element that uses a URI for identification must allow any URI scheme to be used for the sake of that identification. [Failure here implies that identification is not separated from interaction.]
A REST API should not contain any changes to the communication protocols aside from filling-out or fixing the details of underspecified bits of standard protocols, such as HTTP’s PATCH method or Link header field. Workarounds for broken implementations (such as those browsers stupid enough to believe that HTML defines HTTP’s method set) should be defined separately, or at least in appendices, with an expectation that the workaround will eventually be obsolete. [Failure here implies that the resource interfaces are object-specific, not generic.]
A REST API should spend almost all of its descriptive effort in defining the media type(s) used for representing resources and driving application state, or in defining extended relation names and/or hypertext-enabled mark-up for existing standard media types. Any effort spent describing what methods to use on what URIs of interest should be entirely defined within the scope of the processing rules for a media type (and, in most cases, already defined by existing media types). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]
A REST API must not define fixed resource names or hierarchies (an obvious coupling of client and server). Servers must have the freedom to control their own namespace. Instead, allow servers to instruct clients on how to construct appropriate URIs, such as is done in HTML forms and URI templates, by defining those instructions within media types and link relations. [Failure here implies that clients are assuming a resource structure due to out-of band information, such as a domain-specific standard, which is the data-oriented equivalent to RPC’s functional coupling].
A REST API should never have “typed” resources that are significant to the client. Specification authors may use resource types for describing server implementation behind the interface, but those types must be irrelevant and invisible to the client. The only types that are significant to a client are the current representation’s media type and standardized relation names. [ditto]
A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]
REST is a light version of SOAP, with no sugars or colors agents added.
The goal of both SOAP and REST is to allow the communication between Information Systems that may be written in different languages and use different communiation protocols.
While SOAP uses API contracts to expose it's services and define the way a client can call a service, what Parameters should be sent and what results are to be expected, REST on the other hand has uses no API contracts, for a client to know what services exist and how to call them it should look into the Rest API documentation (this can be defined in a yml file with a OpenAPI or Swagger).
Second SOAP is verbose, it reyes on XML to send a request and describe the services, parameteres and the results returned. On the other hand REST relyed on simple JSON Objects to send requests and receive results. JSON is simple to undersand, lightweight and does not use too much bandwith when sending requests or receiving results.