Spring-cloud-contract stubrunner test run failing due to 404 - spring-cloud-contract

I am getting the following error when I run my client application using stubrunner
Getting org.springframework.web.client.HttpClientErrorException: 404
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:63)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:700)
wiremock is giving the following information
{
"url" : "/app/crm/customer/40",
"absoluteUrl" : "http://localhost:6565/app/crm/customer/40",
"method" : "GET",
"clientIp" : "127.0.0.1",
"headers" : {
"accept" : "text/plain, text/plain, application/json, application/json, application/*+json, application/*+json, */*, */*",
"domain" : "IND",
"host" : "localhost:6565",
"connection" : "Keep-Alive",
"user-agent" : "Apache-HttpClient/4.5.3 (Java/1.8.0_144)",
"accept-encoding" : "gzip,deflate"
},
"cookies" : { },
"browserProxyRequest" : false,
"loggedDate" : 1508757427831,
"bodyAsBase64" : "",
"body" : "",
"loggedDateString" : "2017-10-23T11:17:07Z"
}
Closest match:
{
"urlPattern" : "/app/crm/customer/[0-9]{2}",
"method" : "GET",
"headers" : {
"Content-Type" : {
"equalTo" : "application/json"
},
"domain" : {
"equalTo" : "IND"
}
}
}
My Contract looks like this
package contracts
import org.springframework.cloud.contract.spec.Contract
Contract.make {
request {
method 'GET'
url value(consumer(regex('/app/crm/customer/[0-9]{2}')), producer('/app/crm/customer/40'))
headers {
contentType(applicationJson())
}
}
response {
status 200
headers {
contentType(applicationJson())
}
}
}
I tried to resolve the issue by changing content types and other details.where am I doing mistake. Thanks.

It's written exactly what the problem is. In other words the answer is there in your question.
To match your request it needs to be looking like this
{
"urlPattern" : "/app/crm/customer/[0-9]{2}",
"method" : "GET",
"headers" : {
"Content-Type" : {
"equalTo" : "application/json"
},
"domain" : {
"equalTo" : "IND"
}
}
}
These are the headers that you are passing
"headers" : {
"accept" : "text/plain, text/plain, application/json, application/json, application/*+json, application/*+json, */*, */*",
"domain" : "IND",
"host" : "localhost:6565",
"connection" : "Keep-Alive",
"user-agent" : "Apache-HttpClient/4.5.3 (Java/1.8.0_144)",
"accept-encoding" : "gzip,deflate"
},
I don't see the Content-Type equal to application/json header. That's why the request is not matched.

Related

Wiremock matchesJSONPath if null or empty

I'm trying to add a Wiremock stub that matches if the JSON in a request body is either non-existent OR an empty string.
The stub I have at the moment is:
{
"id" : "e331007e-3e6d-4660-b575-b04e774e88c6",
"request" : {
"urlPathPattern" : "/premises/([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})/bookings/([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})/non-arrivals",
"method" : "POST",
"bodyPatterns" : [ {
"matchesJsonPath" : "$.[?(#.reason === '' || #.reason == null)]"
} ]
},
"response" : {
"status" : 400,
"jsonBody" : {
"type" : "https://example.net/validation-error",
"title" : "Invalid request parameters",
"code" : 400,
"invalid-params" : [ {
"propertyName" : "reason",
"errorType" : "blank"
} ]
},
"headers" : {
"Content-Type" : "application/problem+json;charset=UTF-8"
}
},
"uuid" : "e331007e-3e6d-4660-b575-b04e774e88c6"
}
It matches is the reason is '', but not if reason is not present. Any ideas?

Can/Should I mix application/x-ndjson with application/hal+json?

Currently trying to get the best of both worlds in the same http response, but an http response can have only one content type.
Should I use one content type for both?
Is this not encouraged/recommended?
Thanks for your help!
EDIT 2021-08-23 14:27Z
This is the type of response I want to return, for which I don't know which Content-Type to use:
{ "firstname" : "Alan", "id" : 21, "_links" : { "self" : { "href" : "http://hos/people/21", "all" : { "href" : "http://hos/people" }, "dogs" : { "href" : "http://hos/people/21/dogs" } }
{ "firstname" : "Dave", "id" : 42, "_links" : { "self" : { "href" : "http://hos/people/42", "all" : { "href" : "http://hos/people" }, "dogs" : { "href" : "http://hos/people/42/dogs" } }
{ "firstname" : "John", "id" : 99, "_links" : { "self" : { "href" : "http://hos/people/99", "all" : { "href" : "http://hos/people" }, "dogs" : { "href" : "http://hos/people/99/dogs" } }
"best of both worlds":
I don't know a lot of both content types, but as I understand application/x-ndjson means each item is delimited with \n, and http clients can use this response an item at a time, or in bulk, but do not need to wait for the whole response to be built/sent to use it.
And application/hal+json as I understand means the returned json has links referencing available actions for the returned entity.

How to create a Body for this Multipart POST request

I ran a API request thorough postman and it ran successfully and this is its HTTP
POST //dummyendpoint/ HTTP/1.1
Host: dummyhost.com
Authorization: token="dummy_token"
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="config_enroll_settings"; filename="/C:/Users/SaurabhKumar/Desktop/config_enroll_settings.xml"
Content-Type: text/xml
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="usagePolicy"; filename="/C:/Users/SaurabhKumar/Desktop/usagePolicy.xml"
Content-Type: text/xml
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW
But then I am trying to achieve the same through JAXRS implementation I am getting 500 response code and when I checked the server side logs it said couldnt determine MIME boundary
This is my JAXRS interface I have explicitly added boundary but still it dosent works
#Consumes("multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW")
#POST
#Path("/dummy_endpoint/{billingId}/")
public Object configureDeviceEnrollSettings(
#PathParam("billingId") String billingId,
#Multipart("multipartBody") MultipartBody multipartBody,
#HeaderParam("Authorization") String authorizationHeaderValue);
This is how I am calling this interface
Attachment configSettingsFile = new Attachment("config_enroll_settings", "multipart/form-data", new File("/C:/Users/SaurabhKumar/Desktop/config_enroll_settings.xml"));
Attachment usagePolicyFile = new Attachment("usagePolicy", "multipart/form-data", new File("/C:/Users/SaurabhKumar/Desktop/usagePolicy.xml"))
/*
I have tried this way also This way it resulted in exception
No serializer found for class java.io.FileDescriptor and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain:
org.apache.cxf.jaxrs.ext.multipart.MultipartBody["childAttachments"]
->java.util.AbstractList$SubList[0]
->org.apache.cxf.jaxrs.ext.multipart.Attachment["dataHandler"]
->javax.activation.DataHandler["dataSource"]
->org.apache.cxf.jaxrs.ext.multipart.InputStreamDataSource["inputStream"]
->java.io.FileInputStream["fd"])
File file1 = new File("/C:/Users/SaurabhKumar/Desktop/config_enroll_settings.xml");
ContentDisposition cd1 = new ContentDisposition("form-data; name=\"config_enroll_settings\"; filename=\"/C:/Users/SaurabhKumar/Desktop/config_enroll_settings.xml\"");
Attachment configSettingsFile = new Attachment("config_enroll_settings", new FileInputStream(file1), cd1);
File file2 =new File("/C:/Users/SaurabhKumar/Desktop/usagePolicy.xml");
ContentDisposition cd2 = new ContentDisposition("form-data; name=\"config_enroll_settings\"; filename=\"/C:/Users/SaurabhKumar/Desktop/usagePolicy.xml\"");
Attachment usagePolicyFile = new Attachment("usagePolicy", new FileInputStream(file2), cd2);
*/
List<Attachment> list=new LinkedList<>();
list.add(configSettingsFile);
list.add(usagePolicyFile);
MultipartBody body = new MultipartBody(list);
Object object = accountResource.configureDeviceEnrollSettings(billingId, body, "token=" + "\"" + authToken + "\"");
here is the stackTrace
com.webservices.utilities.httpclient.http.rest.ApiException: status 500 reading AccountResourceClient#configureDeviceEnrollSettings(String,MultipartBody,String)
at com.webservices.utilities.httpclient.http.rest.ApiErrorDecoder.decode(ApiErrorDecoder.java:42)
at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:149)
at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:78)
at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:103)
at com.sun.proxy.$Proxy16.configureDeviceEnrollSettings(Unknown Source)
at com.webservices.settings.ImportSettingsRoutine.importConfig(ImportSettingsRoutine.java:103)
at com.webservices.abstractclasses.AbstractRoutine.executeRoutine(AbstractRoutine.java:14)
at com.webservices.clients.ImportClient.executeRoutineQueue(ImportClient.java:47)
at com.webservices.Main.main(Main.java:69)
here is the raw request from the jaxrs client
POST https:/dummyendpoint/ HTTP/1.1
Accept: application/xml
Authorization: token="dummy_token"
Content-Length: 1861
Content-Type: multipart/form-data
User-Agent: ApiClient-1.0
X-FL-APPNAME: ApiClient
X-FL-REQ-ID: dumm_id
{
"type" : {
"type" : "multipart",
"subtype" : "related",
"parameters" : { },
"wildcardType" : false,
"wildcardSubtype" : false
},
"rootAttachment" : {
"headers" : {
"Content-ID" : [ "config_enroll_settings" ],
"Content-Type" : [ "text/xml" ]
},
"object" : "C:\\Users\\SaurabhKumar\\Desktop\\config_enroll_settings.xml",
"contentType" : {
"type" : "text",
"subtype" : "xml",
"parameters" : { },
"wildcardType" : false,
"wildcardSubtype" : false
},
"contentId" : "config_enroll_settings"
},
"allAttachments" : [ {
"headers" : {
"Content-ID" : [ "config_enroll_settings" ],
"Content-Type" : [ "text/xml" ]
},
"object" : "C:\\Users\\SaurabhKumar\\Desktop\\config_enroll_settings.xml",
"contentType" : {
"type" : "text",
"subtype" : "xml",
"parameters" : { },
"wildcardType" : false,
"wildcardSubtype" : false
},
"contentId" : "config_enroll_settings"
}, {
"headers" : {
"Content-ID" : [ "usagePolicy" ],
"Content-Type" : [ "text/xml" ]
},
"object" : "C:\\Users\\SaurabhKumar\\Desktop\\usagePolicy.xml",
"contentType" : {
"type" : "text",
"subtype" : "xml",
"parameters" : { },
"wildcardType" : false,
"wildcardSubtype" : false
},
"contentId" : "usagePolicy"
} ],
"childAttachments" : [ {
"headers" : {
"Content-ID" : [ "usagePolicy" ],
"Content-Type" : [ "text/xml" ]
},
"object" : "C:\\Users\\SaurabhKumar\\Desktop\\usagePolicy.xml",
"contentType" : {
"type" : "text",
"subtype" : "xml",
"parameters" : { },
"wildcardType" : false,
"wildcardSubtype" : false
},
"contentId" : "usagePolicy"
} ]
}

Nutch - clone website

I am playing with Apache Nutch and I am crawling a website successfully. I want to make a clone of a website with Nutch so that I can access the crawled webpages offline. Is there a way to do that? I'm looking for something like an endpoint that receives a url and returns the content of the webpage as if I were GETting the url with curl.
I know there are more specialized solutions like HTTrack, but I want to know if it is possible to use Nutch to do this.
I think you don't need this now but anyway I am putting my answer. Of course it's possible using Apache Nutch.After injecting seed urls and generating segments of urls to fetch, when you execute the fetch command -
$ bin/nutch fetch -all
The Hbase(it's very conventional to use Hbase as storage for Nutch) webpage table structure will be like this -
webpage : {
key : "com.exampe.dev:http/",
f : {
bas : {
timestamp : 1293732801833,
value : "http://dev.example.com/"
},
cnt : {
timestamp : 1293732801833,
value : "DOCTYPE html PUBLIC "-//W3C//DTD X...rest of page content"
},
fi : {
timestamp : 1293676557658,
value : "\x00'\x8D\x00"
},
prot : {
timestamp : 1293732801833,
value : "x02\x00\x00"
},
st : {
timestamp : 1293732801833,
value : "x00\x00\x00\x02"
},
ts : {
timestamp : 1293676557658,
value : "\x00\x00\x01-5!\x9D\xE5"
}
typ : {
timestamp : 1293732801833,
value : "application/xhtml+xml"
}
},
h : {
Cache-Control : {
timestamp : 1293732801833,
value : "private"
},
Content-Type : {
timestamp : 1293732801833,
value : "text/html; charset=UTF-8"
},
Date : {
timestamp : 1293732801833,
value : "Thu, 30 Dec 2010 18:13:21 GMT"
},
ETag : {
timestamp : 1293732801833,
value : 40bdf8b9-8c0a-477e-9ee4-b19995601dde"
},
Expires : {
timestamp : 1293732801833,
value : "Thu, 30 Dec 2010 18:13:21 GMT"
},
Last-Modified : {
timestamp : 1293732801833,
value : "Thu, 30 Dec 2010 15:01:20 GMT"
},
Server : {
timestamp : 1293732801833,
value : "GSE"
},
Set-Cookie : {
timestamp : 1293732801833,
value : "blogger_TID=130c0c57a66d0704;HttpOnly"
},
X-Content-Type-Options : {
timestamp : 1293732801833,
value : "nosniff"
},
X-XSS-Protection : {
timestamp : 1293732801833,
value : "1; mode=block"
}
},
mk : {
_injmrk_ : {
timestamp : 1293676557658,
value : "y"
},
_gnmrk_ : {
timestamp=1293732629430,
value : "1293732622-2092819984"
},
_ftcmrk_ : {
timestamp : 1293732801833,
value : "1293732622-2092819984"
}
},
mtdt : {
_csh_ : {
timestamp : 1293676557658,
value : "x80\x00\x00"
}
},
s : {
s : {
timestamp : 1293676557658,
value : "x80\x00\x00"
}
}
}
The column named cnt under f column family will contain the whole html content of your page. So you can use this to clone the page obviously.
Moreover, you can write custom plugins implementing Parser or ParseFilter interfaces to catch the whole content of page. The code will be very straight-forward.

problems on elasticsearch with parent child documents

We work with two types of documents on elastic search (ES): items and slots, where items are parents of slot documents.
We define the index with the following command:
curl -XPOST 'localhost:9200/items' -d #itemsdef.json
where itemsdef.json has the following definition
{
"mappings" : {
"item" : {
"properties" : {
"id" : {"type" : "long" },
"name" : {
"type" : "string",
"_analyzer" : "textIndexAnalyzer"
},
"location" : {"type" : "geo_point" },
}
}
},
"settings" : {
"analysis" : {
"analyzer" : {
"activityIndexAnalyzer" : {
"alias" : ["activityQueryAnalyzer"],
"type" : "custom",
"tokenizer" : "whitespace",
"filter" : ["trim", "lowercase", "asciifolding", "spanish_stop", "spanish_synonym"]
},
"textIndexAnalyzer" : {
"type" : "custom",
"tokenizer" : "whitespace",
"filter" : ["word_delimiter_impl", "trim", "lowercase", "asciifolding", "spanish_stop", "spanish_synonym"]
},
"textQueryAnalyzer" : {
"type" : "custom",
"tokenizer" : "whitespace",
"filter" : ["trim", "lowercase", "asciifolding", "spanish_stop"]
}
},
"filter" : {
"spanish_stop" : {
"type" : "stop",
"ignore_case" : true,
"enable_position_increments" : true,
"stopwords_path" : "analysis/spanish-stopwords.txt"
},
"spanish_synonym" : {
"type" : "synonym",
"synonyms_path" : "analysis/spanish-synonyms.txt"
},
"word_delimiter_impl" : {
"type" : "word_delimiter",
"generate_word_parts" : true,
"generate_number_parts" : true,
"catenate_words" : true,
"catenate_numbers" : true,
"split_on_case_change" : false
}
}
}
}
}
Then we add the child document definition using the following command:
curl -XPOST 'localhost:9200/items/slot/_mapping' -d #slotsdef.json
Where slotsdef.json has the following definition:
{
"slot" : {
"_parent" : {"type" : "item"},
"_routing" : {
"required" : true,
"path" : "parent_id"
},
"properties": {
"id" : { "type" : "long" },
"parent_id" : { "type" : "long" },
"activity" : {
"type" : "string",
"_analyzer" : "activityIndexAnalyzer"
},
"day" : { "type" : "integer" },
"start" : { "type" : "integer" },
"end" : { "type" : "integer" }
}
}
}
Finally we perform a bulk index with the following command:
curl -XPOST 'localhost:9200/items/_bulk' --data-binary #testbulk.json
Where testbulk.json holds the following data:
{"index":{"_type": "item", "_id":35}}
{"location":[40.4,-3.6],"id":35,"name":"A Name"}
{"index":{"_type":"slot","_id":126,"_parent":35}}
{"id":126,"start":1330,"day":1,"end":1730,"activity":"An Activity","parent_id":35}
We see through ES Head plugin that definitions seem to be ok. We test the analyzers to check that they have been loaded and they work. Both documents appear listed in ES Head browser view. But if we try to retrieve the child item using the API, ES responds that it does not exist:
$ curl -XGET 'localhost:9200/items/slot/126'
{"_index":"items","_type":"slot","_id":"126","exists":false}
When we import 50 documents, all parent documents can be retrieved through API, but only SOME of the requests for child elements get a successful response.
My guess is that it may have something to do with how docs are stored across shards and the routing...which certainly is not clear to me how it works.
Any clue on how to be able to retrieve individual child documents? ES Head shows they have been stored but HTTP GETs to localhost:9200/items/slot/XXX respond randomly with "exists":false.
The child documents are using parent's id for routing. So, in order to retrieve child documents you need to specify parent id in the routing parameter on your query:
curl "localhost:9200/items/slot/126?routing=35"
If parent id is not available, you will have to search for the child documents:
curl "localhost:9200/items/slot/_search?q=id:126"
or switch to an index with a single shard.

Resources