Decode JSON RPC request to a contract - web-scraping

I am currently using some website to read some useful data. Using the browser's Inspect>Network I can see this data comes from JSON RPC requests to (https://bsc-dataseed1.defibit.io/) the public available BSC explorer API endpoint.
This requests have the following format:
Request params:
{"jsonrpc":"2.0","id":43,"method":"eth_call","params":[{"data":"...LONGBYTESTRING!!!","to":"0x1ee38d535d541c55c9dae27b12edf090c608e6fb"},"latest"]}
Response:
{"jsonrpc":"2.0","id":43,"result":"...OTHERVERYLONGBYTESTRING!!!"}
I know that the to field corresponds to the address of a smart contract 0x1ee38d535d541c55c9dae27b12edf090c608e6fb.
Looks like this requests "queries" the contract for some data (but it costs 0 gas?).
From (the very little) I understand, the encoded data can be decoded with the schema, which I think I could get from the smart contract address. (perhaps this is it? https://api.bscscan.com/api?module=contract&action=getabi&address=0x1ee38d535d541c55c9dae27b12edf090c608e6fb)
My goal is to understand the data being sent in the request and the data given in the response so I can reproduce the data from the website without having to scrape this data from the website.
Thanks.

The zero cost is because of the eth_call method. It's a read-only method which doesn't record any state changes to the blockchain (and is mostly used for getter functions, marked as view or pure in Solidity).
The data field consists of:
0x
4 bytes (8 hex characters) function signature
And the rest is arguments passed to the function.
You can find an example that converts the function name to the signature in this other answer.

Related

What is the meaning of "contain an entity which describes the status of the request and refers to the new resource" in the HTTP/1.1 spec?

Chapter 9.5 POST of the HTTP/1.1 spec includes the sentence:
If a resource has been created on the origin server, the response
SHOULD be 201 (Created) and contain an entity which describes the
status of the request and refers to the new resource, and a Location
header
It is referenced frequently. The itention is clear, but I have issues with the meaning of some of the chosen words.
What does "contain an entity which describes the status of the request and refers to the new resource" exactly mean?
How shall the entity (entity-header fields and entity-body) describe the status of the request? Isn't the status of the request 201 (Created)? Whow shall this status be described? Does "describe the status of the request" mean the result, in other words the current entity status?
Thinking of a Web API with JSON representation does it mean that the entity should be included in a JSON representation after a successful POST that created an entity? Thinking of a created image, should the image data be returned in the response body?
What is meant with refers to the new resource? The uri is already in the location header. Shall it be repeated in the body or does it mean just to add an id?
Is there a good source with examples of different entities and its responses to a creation POST?
I think it varies based on the resource you're creating, suppose your posting to a /profile/ resource maybe a payload containing multiple profile fields to update - your return would indicate it was successful and include a reference to the fields you posted (it can even return the entire profile attributes with fields you've updated including all fields);
Another example in the image sense, suppose you are posting a Base64 encoded image to a service that stores the image, the response should show the status (ie: accepted, rejected, file too larage, MIME type accurate or not, etc.) - and within the returned payload if successful you'd want the response to not be vague but return the path and/or filename of the image uploaded;
The header returns the response code - the body returns information related to the invoked action's entity response (it can be a set of fields, a URL, a useful response that when parsed back it can be actionable or informative);
These are principles of good coding, but also keep note of security and not to expose anything in a return that could potentially be damaging for example; when creating a service you want to be clear and provide concise and useful returns so when the client consumes the API it knows what to do, what to expect, etc.

What exactly is "bulk data" in WADO-RS standard?

[Referring to http://dicom.nema.org/medical/dicom/2016e/output/chtml/part18/sect_6.5.html]
When we are talking about WADO-RS, NEMA mentions that:
Every request (we'll leave out /metadata & /rendered requests for now) can have accept-type of three types:
1. multipart/related; type="application/dicom" [dcm-parameters]
------- (DICOM File format as mentioned in PS3.10)
2. multipart/related; type="application/octet-stream" [dcm-parameters]
------- (Bulk data)
3. multipart/related; type="{media-type}" [dcm-parameters]
------- (Bulk data)
For all these accept types, response is created as multipart with each part corresponding to a particular Instance. Now I understand the first case (application/dicom) in which we'll have fill each response part with each SOP Instance's .dcm counterpart. (for e.g., if the WADO RS is for a Study, then the multipart response will have one part for each SOP Instance's Dicom File Stream)
But when it comes to the bulk data I have few questions:
What exactly is bulk-data in WADO-RS standard? Is it only the 7FE00010 tag, or is it all the binary tags of an SOP Instance combined into one single binary data?
If it is just 7FE00010, then there will be one http response part each for every SOP Instance. Then how will the WADO-RS client come to know which bulk data is of which SOP Instance?
Information about this is limited on the internet. Hence asking here.
If any one has any article about this, that's welcome too.
Ps: I am new to DICOM/DICOMWeb
What I've generally used is to look at what's included (or should it be excluded?) in the Composite Instance Retrieve Without Bulk Data service class:
(7FE0,0010) Pixel Data
(7FE0,0008) Float Pixel Data
(7FE0,0009) Double Float Pixel Data
(0028,7FE0) Pixel Data Provider URL
(5600,0020) Spectroscopy Data
(60xx,3000) Overlay Data
(50xx,3000) Curve Data
(50xx,200C) Audio Sample Data
(0042,0011) Encapsulated Document
(5400,1010) Waveform Data (within a (5400,0100) Waveform Sequence)

How to reuse variables from previous request in the Paw rest client?

I need to reuse value which is generated for my previous request.
For example, at first request, I make a POST to the URL /api/products/{UUID} and get HTTP response with code 201 (Created) with an empty body.
And at second request I want to get that product by request GET /api/products/{UUID}, where UUID should be from the first request.
So, the question is how to store that UUID between requests and reuse it?
You can use the Request Sent Dynamic values https://paw.cloud/extensions?extension_type=dynamic_value&q=request+send these will get the value used last time you sent a requst for a given request.
In your case you will want to combine the URLSentValue with the RegExMatch (https://paw.cloud/extensions/RegExMatch) to first get the url as it was last sent for a request and then extract the UUID from the url.
e.g
REQUEST A)
REQUEST B)
The problem is in your first requests answer. Just dont return "[...] an empty body."
If you are talking about a REST design, you will return the UUID in the first request and the client will use it in his second call: GET /api/products/{UUID}
The basic idea behind REST is, that the server doesn't store any informations about previous requests and is "stateless".
I would also adjust your first query. In general the server should generate the UUID and return it (maybe you have reasons to break that, then please excuse me). Your server has (at least sometimes) a better random generator and you can avoid conflicts. So you would usually design it like this:
CLIENT: POST /api/products/ -> Server returns: 201 {product_id: UUID(1234...)}
Client: GET /api/products/{UUID} -> Server returns: 200 {product_detail1: ..., product_detail2: ...}
If your client "loses" the informations and you want him to be later able to get his products, you would usually implement an API endpoint like this:
Client: GET /api/products/ -> Server returns: 200 [{id:UUID(1234...), title:...}, {id:UUID(5678...),, title:...}]
Given something like this, presuming the {UUID} is your replacement "variable":
It is probably so simple it escaped you. All you need to do is create a text file, say UUID.txt:
(with sample data say "12345678U910" as text in the file)
Then all you need to do is replace the {UUID} in the URL with a dynamic token for a file. Delete the {UUID} portion, then right click in the URL line where it was and select
Add Dynamic Value -> File -> File Content :
You will get a drag-n-drop reception widget:
Either press the "Choose File..." or drop the file into the receiver widget:
Don't worry that the dynamic variable token (blue thing in URL) doesn't change yet... Then click elsewhere to let the drop receiver go away and you will have exactly what you want, a variable you can use across URLs or anywhere else for that matter (header fields, form fields, body, etc):
Paw is a great tool that goes asymptotic to awesome when you explore the dynamic value capability. The most powerful yet I have found is the regular expression parsing that can parse raw reply HTML and capture anything you want for the next request... For example, if you UUID came from some user input and was ingested into the server, then returned in a html reply, you could capture that from the reply HTML and re-inject it to the URL, or any field or even add it to the cookies using the Dynamic Value capabilities of Paw.
#chickahoona's answer touches on the more normal way of doing it, with the first request posting to an endpoint without a UUID and the server returning it. With that in place then you can use the RegExpMatch extension to extract the value from the servers's response and use it in subsequent requests.
Alternately, if you must generate the UUID on the client side, then again the RegExpMatch extension can help, simply choose the create request's url for the source and provide a regexp that will strip the UUID off the end of it, such as /([^/]+)$.
A third option I'll throw out to you, put the UUID in an environment variable and just have all of your requests reference it from there.

How and what do responseattributes return for the Here Batch Geocoder API?

The Here Batch Geocoder API indicates that the responseattributes request parameter can be used. I'm interested in getting geocode quality results back from a batch geocode file.
In the Response Attribute switches documentation it indicates that the default options would be On by default: matchQuality, matchType. But I'm not clear on how this actually gets returned from the batch geocoding API. It sounds like these attributes would be returned in the response, and in the API Explorer for the standard geocoder API it does appear that these are returned in the JSON response.
But the Batch Geocoder API returns an XML response, and then ultimately a collection of files including a final data file. In the Batch Geocoder API explorer, the sample XML response does not include either of those default fields.
At one point the batch geocoder demo credentials listed on the Here documentation worked, and I was able to pull back a small sample results file. I did not see those default responseattributes columns on any of the downloaded files, either.
For the Batch Geocoder API, is there a way to get back information about the quality of the geocoded address, matchQuality, matchType, etc?
The responseattributes parameter enables the additional response attributes to be includable with results, e.g. set it to: responseattributes=all
The outcols parameter attributes then define which ones you get and where.
I use the following request a lot to include scores for results analysis:
http://batch.geocoder.cit.api.here.com/6.2/jobs?action=run&app_code=[your-app-code]&app_id=[your-app-id]&gen=8&header=true&indelim=|&outdelim=|&outcols=displayLatitude,displayLongitude,navigationLatitude,navigationLongitude,mapViewTopLeftLatitude,mapViewTopLeftLongitude,mapViewBottomRightLatitude,mapViewBottomRightLongitude,locationLabel,houseNumber,street,district,city,county,state,postalCode,country,relevance,matchLevel,matchType,matchCode,mapReferenceId,responseAdditionalData,addressAdditionalData&addressattributes=all&locationattributes=all&responseattributes=all&maxresults=5&outputcombined=true&mailto=[yourname#domain.com]
It turns out that when you define the output columns for a batch geocode request, you can also list "Address Match Information" fields which provide the information I'm looking for: https://developer.here.com/rest-apis/documentation/batch-geocoder/topics/data-output.html
These fields are entered in the requset query parameters and then showed up on the resulting output data file that was downloaded.
I'm still not sure what the responseattributes parameter does then on the batch geocoder, or if it's even relevant.

Why is request method send to web server called GET and POST?

I guessed that the name of each of the request method has a relationship with the operations they performed in some manner. But I can't get it!
Detials:
GET means posted argument are showed in the url and POST means they are sent but not shown in the url. But what is that related to POST/GET? What is gotten/posted or what does the posting/getting job? Do you have any glues?
I understand what GET and POST method is. What I wanna know is why do we GET/POST, why don't we call it TYPE1/TYPE2, or another more make-sense name like ON-URL/OFF-URL
Please discuss if you know that.
This should help you:
Methods GET and POST in HTML forms - what's the difference?
http://www.cs.tut.fi/~jkorpela/forms/methods.html
The Definitive Guide to GET vs POST
http://carsonified.com/blog/dev/the-definitive-guide-to-get-vs-post/
get and post
http://catcode.com/formguide/getpost.html
From RFC 2616:
GET
The GET method means retrieve whatever information (in the form of an entity) is identified by the Request-URI.
POST
The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line.
So, GET should be used to read a resource, whereas POST should be used to create, update, or delete a resource.
GET and POST are called HTTP Verbs. See the RFC for details.
GET will get a resource identified by a URL. If using GET as the action for a form the entries will be encoded in the URL (look at a google search for an example).
POST will send the data separately, to the specified URL.
The biggest difference is that if you use GET on a form submit, you can copy the URL of the page you landed at and use it directly to get the same results. All information will also be visible in the URL (don't use this method for passwords). If you POST the data the URL of the landing page will not be enough to reproduce the same results; you will have to go through the form again.
Take a look at the RFC definitions here:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
But essentially, GET is used to retrieve a resource and POST is used to create a new one or make a change to a resource.
Seems to me that #Nam G. VU is asking an English-language question.
"Get" implies that the flow of data is from the server to the client. More specifically, the client is asking the server to send some data.
"Post" implies that the client is pushing data to the server. The word "post" implies that it's a one-way operation.
Of course, neither of these is 100% unidirectional: GETs can send data to the server in the
URL as path and/or query arguments, and POSTS return data to the client.
But, in the simplest sense, the English verbs imply the principal direction of data flow.
From the REST standpoint, GET METHOD signifies that it is used to GET a (list of similar) resource(s). POST is used to create (or POST) a resource.
Apart from this, GET carries all parameters in the URL in the format of ?name=value& pairs, whereas POST carries all of them in the Request Body.

Resources