Fiddler AutoResponder for many post requests - http

I'm developing aplication which sends thousands of http post requests. I want to record all responses and use them as stubs with help of Fiddler`.
For example (lets assume product price = productid for simplicity):
send request, body<productId>1</productId>
get real response, body <productprice>1</productprice>
save response (headers+body) form previous step in local storage,for
example in some dictionary [1,"HTTP/1.1 200 OK
<productprice>1</productprice>"]. (Since we stored this response,
next requests matching pattern body contains
<productId>1</productId>should be responded from our local storage
)
send request, body <productId>1</productId>
load response from local storage and return HTTP/1.1 200 OK
<productprice>1</productprice>
send request, body <productId>2</productId>
get real response, body <productprice>5</productprice>
save response (headers+body) form previous step in local storage,for
example in some dictionary [1,"HTTP/1.1 200 OK
<productprice>1</productprice>"],[2,"HTTP/1.1 200 OK
<productprice>2</productprice>"]
...
How to configure Fiddler for it?
Details:
I have already captured 1000 real POST requests and i want to debug my application with help of them.
Each request / response is unique and in general looks like:
request
POST https://myurl HTTP/1.1
Authorization: Bearer xxx
Content-Type: application/soap+xml; charset=utf-8; action="GetList"
Host: myurl.net
Content-Length: 358
Expect: 100-continue
<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope">
<s:Body>
<catalogRequest xmlns="https://myurl">
<id xmlns="">1</id>
</catalogRequest>
</s:Body>
</s:Envelope>
response
<?xml version="1.0" encoding="UTF-8"?>
<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:ns1="https://myurl">
<env:Body>
<ns1:catalogResponse>
<result>
<id>1</id>
<name>some text</name>
<price>109.99</price>
... big xml ...
<status>1</status>
</result>
</ns1:catalogResponse>
</env:Body>
</env:Envelope>
I tried Autoresponder, but when i dragged captured sessions to the Autoresponder they were converted to rules like: METHOD:POST EXACT: - this rule doesn't use POST body. I can't manually change 1000 rules to use URLWithBody rule
I think it is possible to create Fiddler script, but i don't know how to store captured requests/responses for this script to use them as mapping.

After small research i found a way to record responses and use them as stubs in future. To acheive that i suggest to use fiddler scripts. Here example of scripts in pseudocode
For BeforeRequest:
var body = session.requestBodyBytes;
var id = GetIdFromBody(body);// code for getting id from request body
session.Reply = id;
For AfterResponse:
var body = session.GetRequestBodyAsString();
var id = GetIdFromRespBody(body);// code for getting id from response body
session.SaveResponse(id);

Related

Configure Firebase Function to accept HAL+JSON as request body

When using HTTP-triggered Functions in Firebase, I'm unable to access a HAL+JSON payload from the HTTP request.
The request looks like:
POST /endpoint
Content-Type: application/hal+json
{ /* some payload */ }
In my function I'm accessing the payload using request.body, like this:
...
var myPayload = request.body;
...
The value of myPayload is:
{}
Instead, when the HTTP request contains Content-Type: application/json, the value of myPayload is the correct one:
{ /* some payload */ }
I believe it's related to the following: express.json()
Is there a way to configure express in Firebase to include application/hal+json? For example:
express.json({
type: [ 'application/json', 'application/hal+json' ]
});
Or to access the raw body myself?
Note: I don't want to create an express app within the function.
Please refer to the documentation for understanding how Cloud Functions automatically processes incoming requests.
Cloud Functions parses request body content types of application/json
and application/x-www-form-urlencoded according to the rules described
above. Plain text content types (text/plain) are passed through as
strings using UTF-8 as a default encoding (or a custom encoding
provided in the content-type header).
For other content types, the rawBody property contains the unparsed
bytes of the request body as a Buffer object.
Your "other content type" here probably means that your raw request body will be available from the Request object in the rawBody property.

Delphi synapse TTCPBlockSocket

EDIT: I need to make a POST connection with TTCPBlocksocket inside a delphi based applications script engine.
How to tell synapse where Header ends and body (post elements) starts? Or should i send them in different packets? Thank you !
begin
Head:= TStringList.Create;
Head.Add('GET / HTTP/1.1');
Head.Add('Accept: */*');
Head.Add('Accept-Encoding: gzip, deflate');
Head.Add('Host: www.google.ru');
Head.Add('Connection: Keep-Alive');
Head.Add(#10#13);
body:= TStringList.Create;
body.Add('username=adr');
body.Add('login=adr');
body.Add('password=adr');
body.Add('r_password=adr');
body.Add('submit=register');
Socket:= TTCPBlockSocket.Create;
Socket.connect('108.167.137.28', '80');
if (Socket.LastError <> 0) then Exit;
Socket.SendString(Head.Text);
Solved by this topic -
How are parameters sent in an HTTP POST request?
The content is put after the HTTP headers. The format of an HTTP POST
is to have the HTTP headers, followed by a blank line, followed by the
request body. The POST variables are stored as key-value pairs in the
body.

How do HTTP conversations flow?

I know that clients and servers most commonly communicate through HTTP forms. I know enough about the GET methods, but the little I do know about POST methods is that they're used to submit data to the server. However, when the server receives a POST method and processes the data, it sends a status code like 200 etc., but how does the server send more data? I know there's a body but what does it look like? Does it have parameters and values just like a POST method?
The format of the body is specified in the Content-Type header.
A commonly used Content-Type for form data submission is application/x-www-form-urlencoded. The body for such a request should look something like this:
key1=value1&key2=value+with+spaces
Where key1 and key2 are input names and value1 and value+with+spaces are the corresponding values. Note that key names and values are url encoded
Another common type is application/json, which means the request body should be interpreted as JSON data, for example:
{
"key": "value",
"another_key": "value"
}
So it's just data, and the Content-Type header tells the server how to interpret it.
Edit: A good way to see what's going on is create a form with method="post", and analyze the request with your browser developer tools when the form is submitted.

Caching in Apigee using proxy.basepath

I have created a new API /v1/StatusCache to point to my end point /v1/Status. My goal is to be able to cache the data :
based on the uri
for a particular header
So if user 1 sends request to /v1/StatusCache/1234 and the same user sends request to /v1/StatusCache/5678, I should hit my server instead of getting previously cached result from the first request.
Also we user header (Authorization: Bearer ) in the request so my second goal is that if user 1 with header Authorization: Bearer token1 sends a request to /v1/StatusCache/1234 vs a user 2 with header Authorization: Bearer token2 sends a request to /v1/StatusCache/1234, I should get different results (non cached results)
I have this code for proxypath but it just caches every request for 10 seconds. What am I missing
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ResponseCache async="false" continueOnError="false" enabled="true" name="responsecache-1">
<DisplayName>ResponseCache-1</DisplayName>
<FaultRules/>
<Properties/>
<CacheKey>
<Prefix/>
<KeyFragment ref="proxy.pathsuffix" type="string">proxy.pathsuffix</KeyFragment>
</CacheKey>
<Scope>Exclusive</Scope>
<ExpirySettings>
<ExpiryDate/>
<TimeOfDay/>
<TimeoutInSec ref="">10</TimeoutInSec>
</ExpirySettings>
<SkipCacheLookup/>
<SkipCachePopulation/>
</ResponseCache>
My suggestion would be create a variable value combining the header and pathsuffix value (header+pathsuffix) - use this as the key for the response cache.
Try the following:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ResponseCache async="false" continueOnError="false" enabled="true" name="responsecache-1">
<DisplayName>ResponseCache-1</DisplayName>
<FaultRules/>
<Properties/>
<CacheKey>
<Prefix/>
<KeyFragment ref="proxy.pathsuffix" type="string"/>
<KeyFragment ref="request.header.Authorization" type="string"/>
</CacheKey>
<ExpirySettings>
<ExpiryDate/>
<TimeOfDay/>
<TimeoutInSec ref="">60</TimeoutInSec>
</ExpirySettings>
<SkipCacheLookup/>
<SkipCachePopulation/>
</ResponseCache>
I used 60 sec timeout for ease of testing.
This shows an example of proxy.pathsuffix + request.header.Authorization values being a unique cache key. Think of the key now looking like /1234__Bearer token1. The same path suffix and Authorization header value combined are needed to return an entry from cache.
Also, when trying to cache the URI, you may want to try the variable request.uri which includes the querystring-- this can sometimes dictate what the response looks like. If using this, be sure that the querystring does not include unique values like a current timestamp (or at least strip that parameter before using it as a cache key fragment).

text/html vs application/x-www-form-urlencoded and FormUrlEncodedMediaTypeFormatter

For a POST request, I got back a response in text/html format and the response body was containing the below info:
oauth_token=XXXXXXXXXXXXXXXXXXXXXXXXX&oauth_token_secret=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&oauth_callback_confirmed=true
I made this request through System.Net.Http.HttpClient and I throught I could read the response with FormUrlEncodedMediaTypeFormatter as FormDataCollection but it turned out that FormUrlEncodedMediaTypeFormatter only supports application/x-www-form-urlencoded format by default. So, I worked around this with the following code:
using (OAuthHttpClient client = new OAuthHttpClient(creds)) {
var response = await client.PostAsync(requestUri, new EmptyContent());
var formatter = new FormUrlEncodedMediaTypeFormatter();
formatter.SupportedMediaTypes.Add(new MediaTypeHeaderValue("text/html"));
var result = await response.Content.ReadAsAsync<FormDataCollection>(new List<MediaTypeFormatter>() { formatter });
}
The question here is:
Is the response provider (in this case it is Twitter) doing it wrong by sending this response as text/html or should FormUrlEncodedMediaTypeFormatter support text/html type by default?
Your question is missing some key info i.e. what is the requestUri supposed to return by default, is it a Web API service or an external one etc. It seems it's not Web API because it's little odd that it returns "text/html".
But the fact that FormUrlEncodedMediaTypeFormatter doesn't support formatting back from text/html is absolutely fine. Because why would it? "application/x-www-form-urlencoded" is effectively a key-value dictionary and text/html is a rich media type.
In Web API, with the way content negotiation works, it looks at
Mediatype mappings (I assume not in place in your case)
Accept headers - looking at your request you don't set them
Request content type - again, looking at your request you don't set it so it's empty
Can the formatter serialize a given type
So if you make the request as you showed to any Web API action, it would return text/xml (if you didn't tweak conneg manually).
I agree with Filip that this is a fine work around to an incorrect content type header.
Henrik

Resources