Oozie server is started without any exceptions. While trying to get the status using below command, it is giving 404 error.
$clusterUriStatus = "https://$clusterName.azurehdinsight.net:443/oozie/v2/admin/status"
$response = Invoke-RestMethod -Method Get -Uri $clusterUriStatus -Credential $creds -OutVariable $OozieServerStatus
Error Message from Server:
Invoke-RestMethod : HTTP Status 404 - /oozie/v2/admin/status
type Status report
message /oozie/v2/admin/status
description The requested resource is not available.
Apache Tomcat/6.0.36
At line:1 char:13
+ $response = Invoke-RestMethod -Method Get -Uri $clusterUriStatus -Credential $cr ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebExc
eption
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
If this is an HDInsight 2.1 cluster try the v1 endpoint:
https://$clusterName.azurehdinsight.net:443/oozie/v1/admin/status
The v2 endpoint should work for HDInsight 3.0, the v1 endpoint should be present for 2.1 and 3.0.
Related
I am trying to get data from an API with a POST request. The request works well with a direct shell command :
system(sprintf('curl POST -k --tlsv1.2 -v "https://api-gateway.inpi.fr/services/apidiffusion/api/marques/search" -H "X-XSRF-TOKEN: %s" -H \'accept: application/xml\' -H "Content-Type: application/json" -H "Cookie: XSRF-TOKEN=%s; access_token=%s; session_token=%s" -d \'%s\' > test.xml',token,token,access_token,refresh_token,json_request))
However, I would like to use httr for many reasons. I use the following code :
test <- httr::POST(
"https://api-gateway.inpi.fr/services/apidiffusion/api/marques/search",
httr::set_config(config(ssl_verifypeer = 0L)),
config = (add_headers(
"X-XSRF-TOKEN" = token,
"accept" = "application/xml",
"Content-Type" = "application/json",
"Cookie" = sprintf("XSRF-TOKEN=%s; access_token=%s; session_token=%s",token,access_token,refresh_token)
))
,set_cookies(`X-XSRF-TOKEN` = token,
`XSRF-TOKEN` = token,
access_token = access_token,
session_token = refresh_token)
,body = json_request
)
But this returns a 403 Forbidden error (my_token being the token I use) :
$error
[1] "access_denied"
$error_description
[1] "Invalid CSRF Token '*my_token*' was found on the request parameter '_csrf' or header 'X-XSRF-TOKEN'.
It seems like httr did not take into account my cookies because the token is different inside the test object I create :
> test2$cookies
domain flag path secure expiration name value
1 api-gateway.inpi.fr FALSE / FALSE <NA> XSRF-TOKEN *another_token*
Any idea ? I am sorry that I can't create a reproducible example for obvious security reasons.
Thank you !
The solution was wierd.
I had to rid off from httr, I used UNIX system commands instead, and it worked with the same request.
system(sprintf('curl POST -k --tlsv1.2 "https://api-gateway.inpi.fr/services/apidiffusion/api/marques/search" -H "X-XSRF-TOKEN: %s" -H \'accept: application/json\' -H "Content-Type: application/json" -H "Cookie: XSRF-TOKEN=%s; access_token=%s; session_token=%s" -d \'%s\' > %s/res.json',tokens$xsrf_token,tokens$xsrf_token,tokens$access_token,tokens$refresh_token,json_request,tempdir()))
It seems like httr tries to handle cookies by its own, so maybe that's what caused my problem.
I am trying to hit a nginx server that is behind an alb. I am triggering a simple HTTP2 POST request using pycurl. My request looks like this'
import io
import pycurl
curl = pycurl.Curl()
curl.setopt(curl.URL, "https://alb-xxxx.us-east-1.elb.amazonaws.com/endpoint")
curl.setopt(pycurl.HTTP_VERSION, pycurl.CURL_HTTP_VERSION_2_PRIOR_KNOWLEDGE)
curl.setopt(pycurl.POST, 1)
curl.setopt(pycurl.VERBOSE, 1)
curl.setopt(pycurl.SSL_VERIFYPEER, 0)
curl.setopt(pycurl.SSL_VERIFYHOST, 0)
with io.BytesIO() as buf:
curl.setopt(curl.WRITEFUNCTION, buf.write)
curl.perform()
print(buf.getvalue())
curl.close()
The error i am getting is
error Traceback (most recent call last)
<ipython-input-91-0756005b6c1b> in <module>()
16 with io.BytesIO() as buf:
17 curl.setopt(curl.WRITEFUNCTION, buf.write)
---> 18 curl.perform()
19 print(buf.getvalue())
20 curl.close()
error: (92, 'HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)')
the error occurrence is random. some times I get correct response and sometimes I get error like this. But when I try to do the same in curl I am getting correct response always. This is my curl command.
curl -v -k --http2-prior-knowledge -X POST https://alb-xxx.us-east-1.elb.amazonaws.com/endpoint
My post request response is a string({"field":"value"}). My NGINX configuration file looks like this
location =/endpoint {
if ($request_method = POST) {
return 200 '{"field":"value"}';
}
}
I don't know why I am getting correct response in curl but getting a different response in pycurl. Am I doing anything wrong in pycurl ?
Pycurl version -> 7.43.0.6
Python version -> 3.8.5
Thanks. Any help is appreciated.
How to resolve submission error:
curl: (6) Couldn't resolve host 'www-origin.coursera.org'
m = 15
error] submission with curl() was not successful
!! Submission failed: Grader sent no response
Function: submitWithConfiguration>validateResponse
FileName: C:\Users\admin\Desktop\ex7\lib\submitWithConfiguration.m
LineNumber: 158
This error is probably caused because your computer is not able to connect to the internet at the moment.
This is the function working behind collecting the data from the online grader.
function response = validateResponse(resp)
% test if the response is json or an HTML page
isJson = length(resp) > 0 && resp(1) == '{';
isHtml = findstr(lower(resp), '<html');
if (isJson)
response = resp;
elseif (isHtml)
% the response is html, so its probably an error message
printHTMLContents(resp);
error('Grader response is an HTML message');
else
error('Grader sent no response');
end
end
Now the statement: "Grader sent no response" is printed when the response is null.
And the response can be null when the computer is not connected.
Hope this is the reason behind your error, if not then let me know.
Matlab?
This steps may helpful:
open file: submitWithConfiguration.m ; and: goto line 131 and 134;
then change:
line131: json_command = sprintf('echo jsonBody=%s | curl -k -X POST -d #- %s', body, url);
line134: json_command = sprintf('echo ''jsonBody=%s'' | curl -k -X POST -d #- %s', body, url);
to:
line131: json_command = sprintf('echo jsonBody=%s | curl -k -X POST -s -d #- %s', body, url);
line134: json_command = sprintf('echo ''jsonBody=%s'' | curl -k -X POST -s -d #- %s', body, url);
(both add -s)
it looks like the code is fine and the problem is that your computer can't connect to the internet.
You can simply solve this by using a VPN.
Good luck!
I have a script which I use to load a webpage, retrieve information from said webpage, then output said information to a file. It had been working perfectly until today, when I have been getting an error which reads:
invoke-webrequest : Response object error 'ASP 0251 : 80004005'
Response Buffer Limit Exceeded
/foo/Reports/SearchLocation.asp, line 0
Execution of the ASP page caused the Response Buffer to exceed its configured limit.
At C:\path.ps1:7 char:12
+ $url = invoke-webrequest "http://url/ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
I do not believe there have been any changes to the site which it is pulling its data from, and the file it is getting the input information from has no formatting errors.
Some googling around leads me to believe that the issue is that the page has more than 4 mb worth of data to load, and the default buffer size is 4 mb, but I can't find any instructions for how to change the buffer size in PowerShell.
I came across the clear-webconfiguration cmdlet, but I'm not certain whether or not that is what I need, or how exactly to implement it within my script. Here is the main portion of my code:
foreach($c in $csv){
[array]$tags = $null
$url = invoke-webrequest "http://url.com" -UseDefaultCredentials
$table = $url.ParsedHTML.getElementsByTagName('table')[7]
$rows = $table.getElementsByTagName('tr')
if($c.'Was this user on the old file?' -eq 'Yes'){
if($table.innerText -like "*N/A*" ){
$man = $c.'Manager Name' -replace ',',';'
$badusers += $c.'User ID' + "," + $c.Name + "," + $man + "," + $c.'CC'
}
else{
foreach($row in $rows){
$bcol = $row.getElementsByTagName('td') | Where-Object{$_.cellIndex -eq 1} | select -First 1
$ccol = $row.getElementsByTagName('td') | Where-Object{$_.cellIndex -eq 7} | select -First 1
$bcol = $bcol.innerText
$ccol = $ccol.innerText
if($ccol -ne $c.'CC'){
$tags += $bcol + ",," + $c.'CC' + "," + $c.'User ID'
}
}
if($tags -ne $null){
$results += $tags
}
}
}
}
Any help on solving this issue is much appreciated.
Below error occurring on my code
${resp}= Post Request sw3 https://sw3- test.test1.com/api/v1/test1/patients
\ ... Content-Type:application/json
\ ... Authorization:authkey02 Accept=application/json
Error as follows
KEYWORD ${resp} = RequestsLibrary . Post Request sw3, https://sw3-test.test1.com/api/v1/test1/patients, Content-Type:application/json, Authorization:authkey02, Accept=application/json
Documentation:
Send a POST request on the session object found using the
Start / End / Elapsed: 20170315 12:08:21.283 / 20170315 12:08:21.285 / 00:00:00.002
12:08:21.283 TRACE Arguments: [ 'sw3' | 'https://sw3-test.test1.com/api/v1/test1/patients' | 'Content-Type:application/json' | 'Authorization:authkey02' | 'Accept=application/json' ]
12:08:21.284 FAIL AttributeError: 'unicode' object has no attribute 'copy'
This works ok if i do not use the third line of code - what gives?
You can give it a try like passing those key:value pairs as a dictionary Header variable.
&{Header}= Content-Type:application/json Authorization:authkey02
Accept=application/json
${resp} = RequestsLibrary.Post Request sw3 https://sw3-test.test1.com/api/v1/test1/patients headers=${Header}