linkerd Dtab expression not transforming as expected - linkerd

I've got linkerd running locally per this getting started page, and the basic proxy example works. My use case requires me to proxy web requests to one of several .NET WebApi services, so I made a little sample WebApi project running locally that has two routes:
localhost:58371 returns hello world
localhost:58371/api/values returns ["value1", "value2"]
Mapping requests with matching routes
If I leave my linkerd.yaml file like it comes out-of-the-box:
routers:
- protocol: http
dtab: |
/svc => /#/io.l5d.fs
then mapping routes directly works:
curl -H "Host: web" http://localhost:4140/ --> hello world
curl -H "Host: web" http://localhost:4140/api/values --> ["value1", "value2"]
Mapping mismatched routes
Now, I would like to map localhost:4140/tacos --> localhost:58371/api/values. So, I update my .yaml file to:
routers:
- protocol: http
dtab: |
/svc => /#/io.l5d.fs;
/tacos => /api/values;
and restart linkerd.
However, http://localhost:4140/ always seems to resolve to http://localhost:58371/tacos, not http://localhost:58371/api/values. What am I not understanding?
I've read through a bunch of example dtab transformations on linkerd's site, and I've played around with a bunch of different configurations in my yaml file. Surely it's just something silly that I'm missing because this seems like a really simple use case.

By default, linkerd uses the value of the HTTP Host header associated with the request to route traffic. In your example, you're setting Host: web, so all traffic is going to the "web" service that's found in service discovery, regardless of the request URI's path.
So rather than sending your request to localhost:4140/tacos, you should send it to -H 'Host: tacos' localhost:4140/api/values. You would also need to adjust your dtab as follows:
/svc => /#/io.l5d.fs;
/svc/tacos => /svc/web;
That will route all traffic with Host: tacos to the "web" service that's found in service discovery. This is just the default configuration, however.
If you're interested in routing based on the URI path instead of the HTTP Host header, you can use linkerd's path identifier. Something like:
routers:
- protocol: http
identifier:
kind: io.l5d.path
segments: 1
dtab: |
/svc => /#/io.l5d.fs;
With that configuration, a request to localhost:4140/api/values would be routed to the "api" service in service discovery, and a request to localhost:4140/tacos would be routed to the "tacos" service, and both of those routing decisions can be changed by modifying your dtab.
I should note that linkerd does not do arbitrary HTTP URI path rewriting based on dtab rules. Since it's a proxy, it expects to proxy the request without modification, with a few exceptions (such as the consume option on the path identifier). You could always write a linkerd plugin though that would fit your specific use case.

Related

Nginx Health Check - Post Endpoint

Is it possible for to configure NGINX such that the healthcheck calls a post endpoint on a server and match the response?
For example, I'd like to do a healthcheck using an existing post endpoint in a node - this would mean having to set a RequestBody in order to allow the check.
My options now are to expose an endpoint that does that for me and just go through that, but I am not completely sure if that's the most optimal way.

Tomcat not handling encoded URL

I'm hitting a problem with Tomcat 8.5 when using URL that needs to be encoded. The URL contains [ and ] characters, and those are correctly being encoded as %5B and %5D. I send the request with curl e.g. curl http://somewhere.com/foo%5Bbar%5D but Tomcat throws a 400 error stating that "The server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing).".
In the Tomcat access log the URL is report in its decoded form along with the 400 response.
I do have an nginx reverse proxy in between, but I don't think that would be decoding the URL.
Any ideas what's wrong?
So it turned out to be a know problem with the nginx ingress in that when it has to rewrite the URL is decodes it and sends the decoded URL downstream to Tomcat which then blows up.
There's various discussions on this, but as I needed to fix this as part of the K8S ingress definition I used the info described here. I've not sure where the upstream_balancer thing comes from but must be some black magic, but it does seem to fix the problem. I'm sure there are other approaches that would also work.

REST API generating absolute URLs behind reverse proxy

Situation
I have a REST API in place that returns JSON objects to its clients. Part of said JSON responses are absolute URLs to other "linked" resources/objects of the form "http://myservice.com/servicePath/apiVersionA/123". In the example URL "servicePath" is the web root of my service, "apiVersionA" is the specific version of the REST API being used, and "123" is the identifier of the actual resource. The fact that my API is returning absolute resource URLs in the response body is prescribed by the protocol I'm using, I am not at liberty to change that.
Problem
The problem I'm facing now are reverse proxies, and more specifically how I assume they handle request patterns like the following:
Client sends request "GET http://myproxy.com/rest/apiVersionA/123" to the Reverse Proxy.
Reverse Proxy forwards the request to its final destination: "GET http://myservice.com/servicePath/apiVersionA/123". Please note that the proxy modifies both the host name and the URL path ("rest" is changed to "servicePath") of the request.
AFAIK, the usual suspects (i.e. HTTP headers "Forwarded", "X-Forwarded-For", "X-Forwarded-Host" etc.) only contain either IP addresses or domain names, but not the entire URL path. So right now in the best case scenario I can generate URLs like "http://myproxy.com/apiVersionA/123" - notice the missing "rest" URL path element that is required by the proxy. This URLs will thus not be accepted by the proxy in a subsequent request made by the client, e.g. to retrieve a resource that is referenced in the response message.
Questions
Is there an obvious proxy-safe HTTP header I am missing that would deliver the original request URL to the actual REST API service?
If not, is there another way how I can generate correct absolute URLs in my service?
So far I have thought about using a custom HTTP header (which might get removed by an intermediate proxy), or by using something like https://httpd.apache.org/docs/2.4/mod/mod_substitute.html on the reverse proxy (which I'd say is not possible as I do not control the networking infrastructure of my customers or any intermediaries). Another "non-solution" that I came up with is to document this as part of my API: "do not use custom URL path elements on your HTTP proxies". This would allow me to generate URLs using just X-Forwarded-Host and X-Forwarded-Proto.
Any help is appreciated!
After some more research it became apparent that there is indeed no standard HTTP header that can do the job. I have thus proceeded with implementing a combination of "Forwarded/X-Forwarded-For" and Microsoft's proprietary "X-Forwarded-PathBase" (https://microsoft.github.io/reverse-proxy/articles/transforms.html) header. This works as intended - customers that cannot or do not want to use the proprietary header still get back valid absolute URLs via the reverse proxy, as long as they do not implement request path mapping in their proxy configuration.

How to force apigee proxy to rewrite response URLs from target URL to proxy URL with base path

I've created proxy for my REST API:
Default Target Endpoint URL: http://products.example.com
Default Proxy Endpoint Base Path: /products/v2
Proxy URL: http://example.apigee.net/products/v2
When you call target endpoint directly, example:
POST http://products.example.com/items
you'll get response with location header like this
http://products.example.com/items/123131
But if I go through proxy:
POST http://example.apigee.net/products/v2/items
then location header for newly created resource still points to target URL:
http://products.example.com/items/3423423
But I expected to get
http://example.apigee.net/products/v2/items/3423423
The question is: how to configure proxy to rewrite URLs in response to Proxy URLs?
Not sure if this helps much but I raised a very similar question to yours at How do you correctly return Location headers and do HATEOAS with Apigee?. We ended up needing to write our own policy which does a simple search and replace on the response received from the target server. It would be nice if something was provided out of the box.
Apigee proxy allows you to add policies in the response part of a flow. Example below:
<Flows>
<Flow name="myFlow">
<Description></Description>
<Request/>
<Response>
<Step>
<Name>ModifyLocationHeaderPolicy</Name>
</Step>
<Response/>
</Flow>
</Flows>
I suggest you to add a javascript or python or java callout policy which will read the Location header of the response and re-write it. Something to the effect like below in Javascript
locationHeader = context.getVariable("message.header.Location");
// modify it
context.setVariable("message.header.Location", locationHeader);

Symfony2 Routes only for sub-requests

I am creating an advanced app that uses websocket instead of ajax for dynamic interaction. My WebSocket messages are handled like HTTP Requests, they contain a json-encoded array of path and parameters, which will be converted to a Request. Now the HttpKernel handles this request like every other HTTP request (as sub-request). The only problem is, that the routes for websocket messages are public avaible.
Has anyone an idea how to allow only internal access for a route in this situation?
This answer explains why the firewall configuration can't be used to block routes by name as it uses the RequestMatcher which allows only path regexes and not route names.

Resources