I'm trying to make a POST action on Bitmex to buy 1 contract called XBTUSD (bitcoin/usd) at price 4009.9, with expiring time 1545674400 (UNIX timestamp, a couple hours from now) and we need to be authenticated. This looks simple.
We are using API and we are writing in R. (api-secret is fake sorry!!!)
We need to transform our request with hmac in a signature to make it a number in base 16.
We try to define our signature
signature=hmac("Kjxd5H5sPnBq6oXmnKrQAbKPIAXuKsInHRmD9CF2Dh3-4I6j", 'POST/api/v1/order1545674400{"symbol":"XBTUSD","price":4009.0,"orderQty":1}', algo = "sha256")
and then to POST
POST("https://www.bitmex.com/api/v1/order",body = 'POST/api/v1/order1545674400{"symbol":"XBTUSD","price":4009.0,"orderQty":1}',add_headers("api-key":"R1IdBlJD0-fCXypR2TTQVCF6", "api-signature":signature))
or similar stuff, and we get
403 or 401
I don't understand what's wrong. I'm able make requests which don't need authentication, but not those with it!
Thanks!
from the first glance, you seem to have forgotten 'api-expires' parameter among headers.
Method that works for me is, that body is a named list (if you are using httr package) with aditional parameter inside 'POST' function call: encode = 'json'.
If you want to see what the errors mean in more detail, do the following:
msg = POST(....)
rawToChar(msg$content)
Related
I want to do something very similar to what's shown in the docs for FSharp.Data:
The URL I'm requesting from though (TFS) requires client authentication. Is there any way I can provide this by propagating my Windows creds? I notice JsonProvider has a few other compile-time parameters, but none seem to be in support of this.
You don't have to provide a live URL as a type parameter to JsonProvider; you can also provide the filename of a sample file that reflects the structure you expect to see. With that feature, you can do the following steps:
First, log in to the service and save a JSON file that reflects the API you're going to use.
Next, do something like the following:
type TfsData = JsonProvider<"/path/to/sample/file.json">
let url = "https://example.com/login/etc"
// Use standard .Net API to log in with your Windows credentials
// Save the results in a variable `jsonResults`
let parsedResults = TfsData.Parse(jsonResults)
printfn "%A" parsedResults.Foo // At this point, Intellisense should work
This is all very generic, of course, since I don't know precisely what you need to do to log in to your service; presumably you already know how to do that. The key is to retrieve the JSON yourself, then use the .Parse() method of your provided type to parse it.
I'm doing some heavy web scraping using Python. In some cases, post data is sent not through a form submit but through some Javascript, which I cannot interact with via this approach. In order to circumvent this, I've been appending names and values for the post requests to the url and then visiting that url.
This method was working fine until I came across a site that used this kind of structure: [sitename].com/?[pagename].do/. I admit total ignorance about this .do extension, though some light searching tells me that it has to do with Struts and a Java-based backend. In this case it seems to be a way of dynamically generating a table; I'm trying to filter the results of that table. What I want to enter is something like [sitename].com/?[pagename].do?[name]=[value]&[name]=[value], but this doesn't work, nor does it even seem like it should work. I attempted it using several variations in syntax. It seems like something I don't quite understand is going on here.
I wish I could direct you to the actual site, but unfortunately I cannot due to the sensitive nature of the project. Let me know, though, if there's any additional information that would be helpful in providing an answer. Thanks in advance.
Edit: This is not really a "my code isn't working" question, as it's the underlying functionality that I would like to emulate in my code which is troubling me, but I'll do my best to get grittier. I'm contractually bound not to share the names of the sites that we're studying, but I will try to model the problem. I am hoping that someone with some familiarity of the back-end activity sending this .do page to the browser will be able to shed light.
import urllib
import urllib2
#
## case 1: a site that i have success in scraping
url = 'http://[sitename]/[pagename]'
values = {'s' : '40', 'pg' : '1'}
data = urllib.urlencode(values)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
the_page = response.read()
print the_page #i get the filtered data that i am looking for
#
## case 2: the site that poses a problem for the encoding of post parameters
url = 'http://[sitename]/?[pagename].do/' # this site uses a .do file to generate
# the content i want to filter. note that the page name is preceded by ?.
values = {'s' : '40', 'pg' : '1'}
data = urllib.urlencode(values)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
the_page = response.read()
print the_page # i am taken back to the root of the site,
# the same result i would get if i entered nonsense
# post parameters that did not correspond with actual control names.
here, also, is an example of some javascript on the page that accomplishes what i'd like to do with my scraper:
function page_next (id) {
$("#loading").fadeIn("normal");
$.post("/?dumps.do/", {s: id, pg: 2},
function( data ) {
var content = $( data ).find( '#dumps' );
}
)
}
I don't know what site you are parsing, but this: [sitename].com/?[pagename].do/ is not something I would call default Struts behaviour, assuming it's indeed a Struts application.
Having a .do extension was indeed something Struts used to use as request mapping, but the URL in that case should be [sitename].com/[pagename].do not [sitename].com/?[pagename].do/
In the second form, the action is in fact a parameter in a query string. This is why this syntax is broken: [sitename].com/?[pagename].do?[name]=[value]&[name]=[value]. You want to send a query string to the action but the action itself is a parameter in the query string.
But that's not the issue. The issue is that the site is doing something with that parameter and expects to receive it's data in a certain way, a way you were not able to reverse engineer.
Assuming again that this is a Struts application, Struts uses a front controller to intercept all action.do URLs and then use the action to invoke a particular class in the application, a class that is mapped to that particular action. The format for this should be [sitename].com/[pagename].do. That would be similar to having, say, [sitename].com/[pagename].php.
But having the action as a parameter makes me think that the site has a different front controller (not that of Struts) that is taking the parameter from the query string and passing it downstream to the Struts framework.
There could be a lot of reasons for having this funky way of handling requests, including making it harder for others to scrape the site, although this seems kinda straight forward:
$.post("/?dumps.do/", {s: id, pg: 2}, ...
Have you tried doing a POST to the root of the application with the action in the query string?
I did create a simple testcase in JMeter.
Open a form and all it's content (css, images etc) :
GET /
GET /css/site.css
GET /favicon.ico
GET /fonts/specific-fonts.woff
GET /images/banner.png
Wait a little...
Post the values
POST /
Receive the "Thank You" page.
- GET /thanks
In the response on the first GET is a hidden input field which contains a token. This token needs to be included in the POST as well.
Now I use the "Regular Expression Extractor" of JMeter to get the token from the response. So far, so good.
Then, after retreiving all the other contents I create the POST message, using the variable name in the RegExp-Extractor in the value field of the token parameter.
But... when executing the testcase it fills in the default value given and not the actual value of the token.
So... first step in debugging this issue was to add a dummy-HTTP-GET request directly after I get the token. In this GET request I also add the token parameter with the token variable as value, but now I can easily check the parameter by looking at the access-log on my webserver.
In this case... the URL looks promising. It contains the actual token value in the GET, but it still uses the default value in the POST.
Second step in debugging was to use the "Debug Sampler" and the "View Results Tree".
By moving the Debug Sampler between the different steps I found out the value of the token-variable is back to the default value after I receive the CSS.
So... now the big question is...
How can I make JMeter to remember my variable value until the end of my test-script ?
JMeter doesn't "forget" variables. However variables scope is limited to the current Thread Group. You can convert JMeter variable to JMeter Property which have "global" scope by i.e. using Beanshell Post Processor with the following code:
props.put("myVar", vars.get("myVar"));
Or by using __setProperty() function. See How to Use Variables in Different Thread Groups guide for details.
As you found it your problem comes from a misunderstanding of scoping rules in jmeter.
https://jmeter.apache.org/usermanual/test_plan.html#scoping_rules
In your case, just put the post processor of the request that will give you the response containing the child node.
Also I think you don't need to share this token with other threads so don't use properties as proposed in the alternate answer.
I need to invoke a process which doesn't require any input from the user, just a trigger. I plan to use POST /uri without a body to trigger the process. I want to know if this is considered bad from both HTTP and REST perspectives?
I asked this question on the IETF HTTP working group a few months ago. The short answer is: NO, it's not a bad practice (but I suggest reading the thread for more details).
Using a POST instead of a GET is perfectly reasonable, since it also instructs the server (and gateways along the way) not to return a cached response.
POST is completely OK. In difference of GET with POST you are changing the state of the system (most likely your trigger is "doing" something and changing data).
I used POST already without payload and it "feels" OK. One thing you should do when using POST without payload: Pass header Content-Length: 0. I remember problems with some proxies when I api-client didn't pass it.
If you use POST /uri without a body it is something like using a function which does not take an argument .e.g int post (void); so it is reasonable to have function to your resource class which can change the state of an object without having an argument. If you consider to implement the Unix touch function for a URI, is not it be good choice?
Yes, it's OK to send a POST request without a body and instead use query string parameters. But be careful if your parameters contain characters that are not HTTP valid you will have to encode them.
For example if you need to POST 'hello world' to and end point you would have to make it look like this: http://api.com?param=hello%20world
Support for the answers that POST is OK in this case is that in Python's case, the OpenAPI framework "FastAPI" generates a Swagger GUI (see image) that doesn't contain a Body section when a method (see example below) doesn't have a parameter to accept a body.
the method "post_disable_db" just accepts a path parameter "db_name" and doesn't have a 2nd parameter which would imply a mandatory body.
#router.post('/{db_name}/disable',
status_code=HTTP_200_OK,
response_model=ResponseSuccess,
summary='',
description=''
)
async def post_disable_db(db_name: str):
try:
response: ResponseSuccess = Handlers.databases_handler.post_change_db_enabled_state(db_name, False)
except HTTPException as e:
raise (e)
except Exception as e:
logger.exception(f'Changing state of DB to enabled=False failed due to: {e.__repr__()}')
raise HTTPException(HTTP_500_INTERNAL_SERVER_ERROR, detail=e.__repr__())
return response
I guessed that the name of each of the request method has a relationship with the operations they performed in some manner. But I can't get it!
Detials:
GET means posted argument are showed in the url and POST means they are sent but not shown in the url. But what is that related to POST/GET? What is gotten/posted or what does the posting/getting job? Do you have any glues?
I understand what GET and POST method is. What I wanna know is why do we GET/POST, why don't we call it TYPE1/TYPE2, or another more make-sense name like ON-URL/OFF-URL
Please discuss if you know that.
This should help you:
Methods GET and POST in HTML forms - what's the difference?
http://www.cs.tut.fi/~jkorpela/forms/methods.html
The Definitive Guide to GET vs POST
http://carsonified.com/blog/dev/the-definitive-guide-to-get-vs-post/
get and post
http://catcode.com/formguide/getpost.html
From RFC 2616:
GET
The GET method means retrieve whatever information (in the form of an entity) is identified by the Request-URI.
POST
The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line.
So, GET should be used to read a resource, whereas POST should be used to create, update, or delete a resource.
GET and POST are called HTTP Verbs. See the RFC for details.
GET will get a resource identified by a URL. If using GET as the action for a form the entries will be encoded in the URL (look at a google search for an example).
POST will send the data separately, to the specified URL.
The biggest difference is that if you use GET on a form submit, you can copy the URL of the page you landed at and use it directly to get the same results. All information will also be visible in the URL (don't use this method for passwords). If you POST the data the URL of the landing page will not be enough to reproduce the same results; you will have to go through the form again.
Take a look at the RFC definitions here:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
But essentially, GET is used to retrieve a resource and POST is used to create a new one or make a change to a resource.
Seems to me that #Nam G. VU is asking an English-language question.
"Get" implies that the flow of data is from the server to the client. More specifically, the client is asking the server to send some data.
"Post" implies that the client is pushing data to the server. The word "post" implies that it's a one-way operation.
Of course, neither of these is 100% unidirectional: GETs can send data to the server in the
URL as path and/or query arguments, and POSTS return data to the client.
But, in the simplest sense, the English verbs imply the principal direction of data flow.
From the REST standpoint, GET METHOD signifies that it is used to GET a (list of similar) resource(s). POST is used to create (or POST) a resource.
Apart from this, GET carries all parameters in the URL in the format of ?name=value& pairs, whereas POST carries all of them in the Request Body.