I can access a REST API using curl by sending a system command and using pipe as below
filename fn pipe "curl -k -u &user.:&pass. 'https://blahblah.com/rest/api/content/108428908/child/attachment' ";
data new;
infile fn;
input;
put _infile_;
run;
This works, but I'd like to use PROC HTTP so I get the response back in a .json file rather than in the log, and I can use SAS password encryption. Here's my PROC HTTP code:
filename out "/blah/blahblah/output.json";
proc http
url = 'https://blahblah.com/rest/api/content/108428908/child/attachment'
method = "GET"
webusername = "&user"
webpassword = "&pass"
out = out
;
run;
This works as far as giving me a json file, but the json file I get back says "User not permitted to view attachments on content", which is the response I was getting from curl before adding the -k option (i.e. --insecure).
So how can I do the equivalent of -k in PROC HTTP? (Tell SAS not to check the SSL Cert). Modifying any options outside of those which can be specified within the .SAS file is not an option, as I don't have the access to change those.
Is setting the SSLREQCERT flag an option? According to this it looks like you can use the ALLOW option to tell the server to allow connections that fail the SSL handshake.
The most correct answer here is to NOT ignore the SSL issue and instead add the certificate to the appropriate trust manager/key store on the machine where you are running PROC HTTP. The method by which to do this varies by OS and SAS version. This documentation page will provide some insight into how to do this.
Related
I need to perform a get request from nifi to couchbase. The curl command is:
curl http://HOST:PORT/query/service -d "statement=select item.Date from bucket unnest bucket as item" -u USER:PASSWORD
I tried using InvokeHttp and ExecuteStreamCommand but it keeps returning errors(status code 400). The full error message is:
{ "requestID": "bff62c0b-36fd-401d-bca0-0959e0944323", "errors":
[{"code":1050,"msg":"No statement or prepared value"}], "status":
"fatal", "metrics": {"elapsedTime": "113.31µs","executionTime":
"74.321µs","resultCount": 0,"resultSize": 0,"errorCount": 1
It's important to say that I prefer that the http request will be triggered by an incoming flowfile. I tried using the processors in various of ways but non of them worked.
When I run the command from the nifi server it works fine.
Thanks for the help
the -d parameter of the curl utility forces HTTP POST command
and application/x-www-form-urlencoded mime-type.
so, in the nifi InvokeHTTP select the following parameters
HTTP Method = POST
Remote URL = <your url here>
Basic Authentication Username = <username>
Basic Authentication Password = <password>
Content-Type = application/x-www-form-urlencoded
and the body of the flow file should be
statement=select item.Date from bucket unnest bucket as item
I don't know nifi, but based on the error message, the "statement=" part of the request isn't being included, or you are not sending the request as a POST command.
Artifactory OSS
5.4.6 rev 50406900
Trying to get access token to work.
I created token...
e.g. curl -uadmin:adminpw -X POST "myserver:8081/artifactory/api/security/token" -d "username=moehoward"
Returned msg looks like success...
{
"scope" : "member-of-groups:readers api:*",
"access_token" : <very-long-string>
"expires_in" : 3600,
"token_type" : "Bearer"
}
I can see it in the gui (admin -> Security -> Access Tokens) with "Subject" = to the user ("moehoward" in the example above) and with a "Token ID" that's a lot shorter, something like...
f2eb693a-d4ff-4618-ba52-764dc975c497
To test, I tried to ping using example in the docs...
curl -umoehoward:<very-long-string> myserver:8081/artifactory/api/system/ping
Fails with a 401 (bad credentials).
I replace the token with the "token id" I see in the gui, same result.
I replace again with the hardcoded pw of the "moehoward" user and that works (responds with "OK").
I tried the "-H"Authentication: Bearer " approach using the long string and that worked. So I guess the very long string is the token and not the "Token ID" in the gui.
Q: Any idea why this works for Bearer" and not the user by name ?
So you are right that this is supposed to work for both standard authentication and the Authentication HTTP header.
I did the test on a server with the same version Artifactory OSS 5.4.6 and the following works fine here
Inject the proper variables
export SERVER=server-artifactory
export APIKEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Create and use an access token for user moehoward
curl -u "admin:$APIKEY" -X POST "http://$SERVER/artifactory/api/security/token" -d "username=moehoward" -d "scope=member-of-groups:readers" > token.log
export TOKEN=`cat token.log | jq -r '.access_token'`
curl -u "moehoward:$TOKEN" "http://$SERVER/artifactory/api/system/ping"
curl -H "Authorization: Bearer $TOKEN" "http://$SERVER/artifactory/api/system/ping"
I get "OK" from the last two commands. Can you run exactly these commands and report back?
I have personally experienced the same problem (Bearer header working, standard user credentials not working) with an old version of curl. The obvious workaround is to use Bearer, the more complex workaround is to upgrade curl (or use another tool).
What is the version of curl you use? Any curl from 2015 or more recent should work fine.
I'm trying to setup response headers on my separate webservers that outputs the physical name of the machine that nginx is running on, so that I can tell which servers are serving the responses to our web clients.
Is there a variable that exists to do this already? Or do I just have to hardcode it per-server :(
You're after the $hostname common variable. Common variables are listed in the variable index.
The nginx access log documentation only shows variables that are specific to the access log:
The log format can contain common variables, and variables that exist
only at the time of a log write.
I guess you're looking for $hostname variable.
At first I thought the answer was to use the ENV variable and pull out the hostname from there https://docs.apitools.com/blog/2014/07/02/using-environment-variables-in-nginx-conf.html. But I couldn't get it to work for some reason.
However, this works like a charm:
perl_set $server_int 'sub { use Sys::Hostname; return hostname; }';
And example usage:
add_header 'Server-Int' "$server_int";
Just have to make sure your nginx is compiled with --with-http_perl_module - just run nginx -V to make sure. And that you have Sys::Hostname installed.
Warning: I at first used hostname to return the hostname in the Perl script, but while that did return the name, it for some reason aborted the rest of the output. I don't know if it's a bug with perl_set but you've been warned - using backticks in perl_set may be deadly.
I am trying to add an sslcert using netsh http from within a powershell ps1 file, but it keeps throwing errors:
$guid = [guid]::NewGuid()
netsh http add sslcert ipport=0.0.0.0:443 certhash=5758B8D8248AA8B4E91DAA46F069CC1C39ABA718 appid={$guid}
'JABnAHUAaQBkAA' is not a valid argument for this command.
The syntax supplied for this command is not valid. Check help for the correct syntax.
Usage: add sslcert [ipport=]<IP Address:port>
[certhash=]<string>
[appid=]<GUID>
[[certstorename=]<string>
[verifyclientcertrevocation=]enable|disable
[verifyrevocationwithcachedclientcertonly=]enable|disable
[usagecheck=]enable|disable
[revocationfreshnesstime=]<u-int>
[urlretrievaltimeout=]<u-int>
[sslctlidentifier=]<string>
[sslctlstorename=]<string>
[dsmapperusage=]enable|disable
[clientcertnegotiation=]enable|disable]
Parameters:
Tag Value
ipport - IP address and port for the binding.
certhash - The SHA hash of the certificate. This hash
is 20 bytes long and specified as a hex
string.
appid - GUID to identify the owning application.
certstorename - Store name for the certificate. Defaults
to MY. Certificate must be stored in the
local machine context.
verifyclientcertrevocation - Turns on/off verification of revocation
of client certificates.
verifyrevocationwithcachedclientcertonly - Turns on/off usage of
only cached client
certificate for revocation checking.
usagecheck - Turns on/off usage check. Default is enabled.
revocationfreshnesstime - Time interval to check for an updated
certificate revocation list (CRL). If this
value is 0, then the new CRL is updated
only if the previous one expires. (in
seconds)
urlretrievaltimeout - Timeout on attempt to retrieve certificate
revocation list for the remote URL.
(in milliseconds)
sslctlidentifier - List the certificate issuers that can
be trusted. This list can be a subset of
the certificate issuers that are trusted
by the machine.
sslctlstorename - Store name under LOCAL_MACHINE where
SslCtlIdentifier is stored.
dsmapperusage - Turns on/off DS mappers. Default is
disabled.
clientcertnegotiation - Turns on/off negotiation of certificate.
Default is disabled.
Remarks: adds a new SSL server certificate binding and corresponding client
certificate policies for an IP address and port.
Examples:
add sslcert ipport=1.1.1.1:443 certhash=0102030405060708090A0B0C0D0E0F1011121314 appid={00112233-4455-6677-8899
-AABBCCDDEEFF}
I might be wrong, but I believe it has something to do how I go about specifying the appid GUID in my powershell script file. Could someone please help me solve the error?
It's a problem with the way Powershell parses cmd commands.
This will execute the command successfully:
$guid = [guid]::NewGuid()
$Command = "http add sslcert ipport=0.0.0.0:443 certhash=5758B8D8248AA8B4E91DAA46F069CC1C39ABA718 appid={$guid}"
$Command | netsh
The reason for the error is that the curly braces have to be escaped each with a backtick (`).
The following command will work from the PowerShell commandline:
This will work from the PowerShell commadline:
$AppId = [Guid]::NewGuid().Guid
$Hash = "209966E2BEDA57E3DB74FD4B1E7266F43EB7B56D"
netsh http add sslcert ipport=0.0.0.0:8000 certhash=$Hash appid=`{$Guid`}
The important details are to escape each { } with a backtick (`).
If netsh raises an error 87 try appending certstorename my
There is no need to use variables. Its just for sake of convenience.
Below code will work, & here is used for invoke program with parameters, and "appid={$guid}" make it pass string value.
& netsh http add sslcert ipport=0.0.0.0:443 certhash=5758B8D8248AA8B4E91DAA46F069CC1C39ABA718 "appid={$guid}"
Is there a command line utility where you can simply set up an HTTP request and have the trace simply output back to the console?
Also specifying the method simply would be a great feature instead of the method being a side effect.
I can get all the information I need with cURL but I can't figure out a way to just display it without dumping everything to files.
I'd like the output to show the sent headers the received headers and the body of the message.
There must be something out there but I haven't been able to google for it. Figured I should ask before going off and writing it myself.
I dislike answering my own question but c-smile's answer lead me down the right track:
Short answer shell script over cURL:
curl --dump-header - "$#"
The - [dash] meaning stdout is a convention I was unaware of but also works for wget and a number of other unix utilities. It is apparently not part of the shell but built into each utility. The wget equivalent is:
wget --save-headers -qO - "$#"
Did you try wget:
http://www.gnu.org/software/wget/manual/wget.html#Wgetrc-Commands ?
Like wget --save-headers ...
To include the HTTP headers in the output (as well as the server response), just use curl’s -i/--include option. For example:
curl -i "http://www.google.com/"
Here’s what man curl says about this setting:
-i/--include
(HTTP) Include the HTTP-header in the output. The HTTP-header
includes things like server-name, date of the document, HTTP-
version and more...
If this option is used twice, the second will again disable
header include.
Try http, e.g.
http -v example.org
Further into at https://httpie.org
It even includes a page to try online:
https://httpie.org/run
Telnet has for long been a well-known (though now forgotten, I guess) tool for looking at a web page. The general idea is to telnet to the http port, to type an http 1.1 GET command, and then to see the served page on the screen.
A good detailed explanation is http://support.microsoft.com/kb/279466
A Google search yields a whole bunch more.
Use telnet on port 80
For example:
telnet telehack.com 80
GET / HTTP/1.1
host: telehack.com
<CR>
<CR>
<CR> means Enter