Path conflict in Ocelot routing configuration - .net-core

How would you set re-reoute following two set of similar routes to different machines. Please advise.
Problem/Question: The conflicting situation is between /customers/1 & /customers/1/products where each one is going to a different machine.
- machine name: customer
GET /customers
GET /customers/1
POST /customers
PUT /customers1
DELETE /customers/1
- machine name: customerproduct
GET /customers/1/products
PUT /customers/1/products
ocelot.json
{
"ReRoutes": [
{
"DownstreamPathTemplate": "/customers/{id}",
"DownstreamScheme": "http",
"DownstreamHostAndPorts": [
{
"Host": "customer",
"Port": 80
}
],
"UpstreamPathTemplate": "/customers/{id}",
"UpstreamHttpMethod": [ "Get", "Post", "Put", "Delete" ]
},
{
"DownstreamPathTemplate": "/customers/{id}/products",
"DownstreamScheme": "http",
"DownstreamHostAndPorts": [
{
"Host": "customerproduct",
"Port": 80
}
],
"UpstreamPathTemplate": "/customers/{id}/products",
"UpstreamHttpMethod": [ "Get", "Put" ]
}
],
"GlobalConfiguration": {
"BaseUrl": "http://localhost:80"
}
}

Just adding an answer based on the OP's own solution in the comments.
The reason this was not working is because of order of evaluation. Ocelot will evaluate paths in the order they are specified, unless the priority property is specified on the routes (see docs https://ocelot.readthedocs.io/en/latest/features/routing.html#priority)
So the in the following routing config:
{
"ReRoutes":[
{
"UpstreamPathTemplate":"/customers/{id}", // <-- will be evaluated first
...
},
{
"UpstreamPathTemplate":"/customers/{id}/products",
...
},
...
],
...
}
the first path will be evaluated and Ocelot will match to this path even if the upstream call specifies the /products sub-path.
To fix this, either change the ordering so that the more specific path is evaluated first:
{
"ReRoutes":[
{
"UpstreamPathTemplate":"/customers/{id}/products", // <-- will be evaluated first
...
},
{
"UpstreamPathTemplate":"/customers/{id}",
...
},
...
],
...
}
or use the priority property, with the priority being used to indicate the desired order of evaluation:
{
"ReRoutes":[
{
"UpstreamPathTemplate":"/customers/{id}",
"Priority": 0,
...
},
{
"UpstreamPathTemplate":"/customers/{id}/products", // <-- will be evaluated first
"Priority": 1,
...
},
...
],
...
}

Related

Ocelot - unable to re-route

I have ASP.NET Web API microservices and currently I am implementing API gateway for routing and authentication.
From my 3 APIs, one of them is for User (crud) operations, so I decided to use it for a gateway, as well. And while I was implementing the routing I came across a strange problem.
With this (test) configuration (again, using my User API running on port 7268)
{
"Routes": [
//User API
{
"UpstreamPathTemplate": "/User/Login",
"UpstreamHttpMethod": [ "Post" ],
"DownstreamScheme": "https",
"DownstreamHostAndPorts": [
{
"Host": "localhost",
"Port": 7124
}
],
"DownstreamPathTemplate": "/api/Product",
"DownstreamHttpMethod": "Get"
}
],
"GlobalConfiguration": {
"BaseUrl": "https://localhost:7268"
}
}
... I am able to get list of products from the Product API on port 7124, when I navigate to /User/Login.
Now if I change the configuration to access any endpoint of the same User API (which matches the BaseUrl path), it does not work. Example:
{
"Routes": [
//User API
{
"UpstreamPathTemplate": "/User/Login",
"UpstreamHttpMethod": [ "Post" ],
"DownstreamScheme": "https",
"DownstreamHostAndPorts": [
{
"Host": "localhost",
"Port": 7268
}
],
"DownstreamPathTemplate": "/api/User/Login",
"DownstreamHttpMethod": "Post"
}
],
"GlobalConfiguration": {
"BaseUrl": "https://localhost:7268"
}
}
It gives me 404 Not Found and the following message in the console:
Is this because I am using an existing API to re-route itself and it is expected behavior, or I am doing something wrong?

Ocelot Swagger MMLib.SwaggerForOcelot showing "No operations defined in spec!"

I am using Ocelot gateway and for swagger document using "MMLib.SwaggerForOcelot" library.
For some swagger Key, swagger UI is showing "No operations defined in spec!" and swagger JSON is coming without paths like
{
"openapi": "3.0.1",
"info": {
"title": "Admin API",
"version": "v1"
},
"paths": {},
"components": {
"schemas": {}
}
}
Ocelot Configuration Route is
{
"DownstreamPathTemplate": "/api/admin/v{version}/{everything} ",
"DownstreamScheme": "http",
"DownstreamHostAndPorts": [
{
"Host": "localhost",
"Port": 5000
}
],
"UpstreamPathTemplate": "/api/admin/v{version}/{everything}",
"UpstreamHttpMethod": [],
"QoSOptions": {
"ExceptionsAllowedBeforeBreaking": 3,
"DurationOfBreak": 1000,
"TimeoutValue": 900000
},
"SwaggerKey": "AdminAPI"
}
and Swagger Configuration is
"SwaggerEndPoints": [
{
"Key": "AdminAPI",
"Config": [
{
"Name": "Admin API",
"Version": "v1",
"Url": "http://localhost:5000/swagger/v1/swagger.json"
}
]
}
]
after reviewing the MMLib.SwaggerForOcelot source code, it looks like something to do with the version in the downstream path, any clue on how this can be fixed?
The issue is that MMLib.SwaggerForOcelot is not considering {version} while doing Ocelot transformation.
RouteOptions has a property TransformByOcelotConfig which is set to true by default, so once swagger JSON is obtained from downstream, the transformation will be done.
here, it will try to find the route from the route configuration like below and if not found it will delete the route from swagger JSON
private static RouteOptions FindRoute(IEnumerable<RouteOptions> routes, string downstreamPath, string basePath)
{
string downstreamPathWithBasePath = PathHelper.BuildPath(basePath, downstreamPath);
return routes.FirstOrDefault(p
=> p.CanCatchAll
? downstreamPathWithBasePath.StartsWith(p.DownstreamPathWithSlash, StringComparison.CurrentCultureIgnoreCase)
: p.DownstreamPathWithSlash.Equals(downstreamPathWithBasePath, StringComparison.CurrentCultureIgnoreCase));
}
The problem is StartsWith will return false since swagger JSON path will be like
/api/admin/v{version}/Connections
and route config is like
/api/admin/v{version}/{everything}
and version will replace with the version given in swagger options so that it will become
/api/admin/v1/{everything}
Fix to this problem will be
Either set "TransformByOcelotConfig":false in swagger option
"SwaggerEndPoints": [
{
"Key": "AdminAPI",
"TransformByOcelotConfig":false,
"Config": [
{
"Name": "Admin API",
"Version": "v1",
"Url": "http://localhost:5000/swagger/v1/swagger.json"
}
]
}
]
Or Change the route, just to have {everything} keyword
{
"DownstreamPathTemplate": "/api/admin/{everything} ",
"DownstreamScheme": "http",
"DownstreamHostAndPorts": [
{
"Host": "localhost",
"Port": 5000
}
],
"UpstreamPathTemplate": "/api/admin/{everything}",
"UpstreamHttpMethod": [],
"QoSOptions": {
"ExceptionsAllowedBeforeBreaking": 3,
"DurationOfBreak": 1000,
"TimeoutValue": 900000
},
"SwaggerKey": "AdminAPI"
}

Logic App workflow creating or updating documents in CosmosDB fails with conflict

I'm in the process of creating a Logic App which should copy all documents from specific containers in the database to a different database and its corresponding collections.
For this, I've created a workflow which retrieves all documents, loops through them and tries to do a Create or Update document on the target location.
As you can see on the image, this looks fairly straightforward.
I've specified the PartitionKey in the header and Upsert is set to true, so if a document already exists it'll be updated, otherwise created.
Originally, the body was filled with #items('[blurred]_-_For_each_document').
However, I got an error stating there was a conflict on the id.
I've also tried to remove all 'system' properties like so:
#removeProperty(removeProperty(removeProperty(removeProperty(removeProperty(items('[blurred]_-_For_each_document'), 'id'), '_rid'), '_self'), '_ts'), '_etag')
but it appears not having an id in place isn't valid, so now I've got this as I'm only interested in having the contents of the 'actual' document:
#removeProperty(removeProperty(removeProperty(removeProperty(items('[blurred]_-_For_each_document'), '_rid'), '_self'), '_ts'), '_etag')
This still fails.
The raw input looks a bit like this:
{
"method": "post",
"headers": {
"x-ms-documentdb-raw-partitionkey": "1050",
"x-ms-documentdb-is-upsert": "True"
},
"path": "/dbs/[myDatabase]/colls/[myCollection]/docs",
"host": {
"connection": {
"name": "/subscriptions/[subscriptionGuid]/resourceGroups/[resourcegroup]/providers/Microsoft.Web/connections/[connectionName]"
}
},
"body": {
"id": "2faee185-4a51-4797-bff9-3ce23a603690",
"MyPartitionKeyNumber": 1050,
"SomeValue": false,
}
}
With the following response:
{
"code": "Conflict",
"message": "Entity with the specified id already exists in the system., ... ResourceType: Document, OperationType: Upsert ..."
}
I know for a fact, the id doesn't exist, but changed the step to do a PUT anyway.
{
"method": "put",
"headers": {
"x-ms-documentdb-raw-partitionkey": "1050",
"x-ms-documentdb-is-upsert": "True"
},
"path": "/dbs/[myDatabase]/colls/[myCollection]/docs",
"host": {
"connection": {
"name": "/subscriptions/[subscriptionGuid]/resourceGroups/[resourcegroup]/providers/Microsoft.Web/connections/[connectionName]"
}
},
"body": {
"id": "2faee185-4a51-4797-bff9-3ce23a603690",
"MyPartitionKeyNumber": 1050,
"SomeValue": false,
}
}
With a response I'm expecting to see:
{
"statusCode": 404,
"message": "Resource not found"
}
I've also tried to do a Delete document before running the Create or Update document step, but got the same error(s).
There's some post here on Stack Overflow stating the x-ms-documentdb-raw-partitionkey should be x-ms-documentdb-partitionkey (without the raw-part), but doing so results in the following error message:
PartitionKey extracted from document doesn't match the one specified in the header
For completeness sake, these are the relevant steps of my workflow:
"[blurred]_-_Get_all_documents": {
"type": "ApiConnection",
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['documentdb_3']['connectionId']"
}
},
"method": "get",
"path": "/v2/dbs/#{encodeURIComponent('myDatabase')}/colls/#{encodeURIComponent(items('[blurred]_-_For_each_collection'))}/docs"
},
"runAfter": {}
},
"[blurred]_-_For_each_document": {
"type": "Foreach",
"foreach": "#body('[blurred]_-_Get_all_documents')?['value']",
"actions": {
"[blurred]_-_Create_or_Update_document_on_blurred2": {
"type": "ApiConnection",
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['documentdb_4']['connectionId']"
}
},
"method": "post",
"body": "#removeProperty(removeProperty(removeProperty(removeProperty(items('[blurred]_-_For_each_document'), '_rid'), '_self'), '_ts'), '_etag')",
"headers": {
"x-ms-documentdb-raw-partitionkey": "#{items('[blurred]_-_For_each_document')['MyPartitionKeyNumber']}",
"x-ms-documentdb-is-upsert": true
},
"path": "/dbs/#{encodeURIComponent('myDatabase')}/colls/#{encodeURIComponent(items('[blurred]_-_For_each_collection'))}/docs"
},
"runAfter": {}
}
},
"runAfter": {
"[blurred]_-_Get_all_documents": [
"Succeeded"
]
}
}

Request probleme with Google Cloud Datastore and Filter

I'm currently doing some tests on google datastore, but I'm having a problem with my queries.
If I believe in the documentation https://cloud.google.com/datastore/docs/concepts/queries we can realize a filter on several columns with the instruction EQUALS.
But when testing, I get an error from the API.
While searching on Datastore's github, I found this reference: https://github.com/GoogleCloudPlatform/google-cloud-dotnet/issues/304 which corresponds to my problem, except that for my case the query to the look good.
Here is the request sent:
{
{
"kind": [{
"name": "talk.message"
}],
"filter": {
"compositeFilter": {
"op": "AND",
"filters": [{
"propertyFilter": {
"property": {
"name": "Conversation"
},
"op": "EQUAL",
"value": {
"stringValue": "2f16c14f6939464ea687d316438ad4cb"
}
}
},
{
"propertyFilter": {
"property": {
"name": "CreatedOn"
},
"op": "LESS_THAN_OR_EQUAL",
"value": {
"timestampValue": "2019-03-15T10:43:31.474166300Z"
}
}
},
{
"propertyFilter": {
"property": {
"name": "CreatedOn"
},
"op": "GREATER_THAN_OR_EQUAL",
"value": {
"timestampValue": "2019-03-14T10:43:31.474175100Z"
}
}
}
]
}
}
}
}
And here is the answer from the API:
{Grpc.Core.RpcException: Status(
StatusCode=FailedPrecondition,
Detail="no matching index found. recommended index is:
- kind: talk.message
properties:
- name: Conversation
- name: CreatedOn"
)
According to the documentation, this should be good... but it's not !
What am I missing ?
Your query includes both an EQUALS (on Conversation) and a non-EQUALS filter (on CreatedOn), therefore you need a composite index to fulfil the query. So your query is valid, but it needs a composite index to be able to run the query.

Pact verify provider, what does Pact::UnexpectedIndex mean?

I am using Pact for Consumer Driven Contracts testing.
In my usecase, my consumer "some-market-service-consumer" is using provider "market-service". The contract "produced" at some-market-service-consumer, looks like this:
{
"provider": {
"name": "market-service"
},
"consumer": {
"name": "some-market-service-consumer"
},
"interactions": [
{
"description": "Get all markets",
"request": {
"method": "GET",
"path": "/markets"
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json; charset=utf-8"
},
"body": {
"markets": [
{
"phone": "HBiQOAeQegaWtbcHfAro"
}
]
},
"matchingRules": {
"$.headers.Content-Type": {
"regex": "application/json; charset=utf-8"
},
"$.body.markets[*].phone": {
"match": "type"
},
"$.body.markets": {
"min": 1,
"match": "type"
}
}
}
}
],
"metadata": {
"pact-specification": {
"version": "2.0.0"
},
"pact-jvm": {
"version": "3.3.6"
}
}
}
On provider-site, I am using pact-provider-verifier-docker¹. Here is my test-result:
WARN: Ignoring unsupported matching rules {"min"=>1} for path $['body']['markets']
.....
## -1,7 +1,7 ##
{
"markets": [
... ,
- Pact::UnexpectedIndex,
+ Hash,
]
}
Key: - means "expected, but was not found".
+ means "actual, should not be found".
Values where the expected matches the actual are not shown.
It seems, as if the testing works fine - "phone" is tested valid.
But at this point, I have no clue, what (expected) "Pact::UnexpectedIndex" means, and why it fails. Where does this expectation come from, how to fix it?
¹ In special, my own version, which uses most recent external dependencies.
As you can see in this test case here the Pact::UnexpectedIndex is used to indicate than an array is longer than was expected. I think what it's trying to say is that there is an extra hash at the end of the array. (I agree that it is not clear at all! I wrote this code, so I apologise for its confusing nature. It turns out that the hardest part of writing pact code was writing helpful diff output.)
What is confusing about that error is that it should be allowing you to have extra elements as you've specified $.body.markets to have a minimum length of 1. Perhaps the docker verifier is using version 1 matching instead of version 2?

Resources