I want to continuously post an infinite binary data stream to a webserver.
So I'm using the following command:
curl -X POST -H "Transfer-Encoding: chunked" -d 'hello' http://127.0.0.1:9000
As a test, I'm piping the output of the 'yes' command:
$ yes | curl -X POST -H "Transfer-Encoding: chunked" -d '#-' http://127.0.0.1:9000
But it doesn't even connect to the webserver, and aborts with an out-of-memory error. It seems curl is trying to read the whole file into memory before starting the transfer.
Does curl supports continuously HTTP posting data coming from a pipe? (I'm using curl version 7.61.0).
Thanks!
Try with
yes | curl -T. http://127.0.0.1:9000
But you will probably get this known bug :
https://github.com/curl/curl/issues/2051
https://github.com/curl/curl/issues/932
Or
yes | curl -T- http://127.0.0.1:9000
Related
I am using a script that gives me some data in json format, I want to send this data to splunk.
I can store the output of the script in a file but how can I send it to HTTP Event Collector?
Couple of things I tried but did not work:
FILE="output.json"
file1="cat answer.txt"
curl -k "https://prd-pxxx.splunkcloud.com:8088/services/collector" -H "Authorization: Splunk XXXXX" -d '{"event": "$file1", "sourcetype": "manual"}'
-----------------------------------------------------------
curl -k "https://prd-pxxx.splunkcloud.com:8088/services/collector" -H "Authorization: Splunk XXXXX" -d '{"event": "#output.json", "sourcetype": "manual"}'
curl -k "https://prd-p-w0gjo.splunkcloud.com:8088/services/collector" -H "Authorization: Splunk d70b305e-01ef-490d-a6d8-b875d98e689b" -d '{"sourcetype":"_json", "event": "#output.json", "source": "output.json}
-----------------------------------------------------------------
After trying this I understand that it literally sends everything specified in the event section. Is there a way I can send the content of the file or use a variable?
Thanks in advance!
(Note - I haven't tried this specifically, but it should get you close)
According to Docs.Splunk on HTTP Event Collector Examples #3, it would seem you can do something very similar to this:
curl -k "https://mysplunkserver.example.com:8088/services/collector/raw?channel=00872DC6-AC83-4EDE-8AFE-8413C3825C4C&sourcetype=splunkd_access&index=main" \
-H "Authorization: Splunk CF179AE4-3C99-45F5-A7CC-3284AA91CF67" \
-d < $FILE
Presuming the content of the file is formatted correctly, it should go straight in.
How is the file being created? Is it in a Deployment App on a managed endpoint? If so, it will likely be simpler to setup a scripted input for the UF to run on whatever schedule you choose.
There is a sample in Jira API doc:
curl -D- -X GET -H "Authorization: Basic ZnJlZDpmcmVk" -H "Content-Type: application/json" "http://kelpie9:8081/rest/api/2/issue/QA-31"
What exactly does the -D- parameter mean? There is no description in curl documentation.
I'm also not sure whether -D- and -D mean the same thing.
It dumps headers to stdout.
The -D flag dumps headers to a file, and the following - instructs it to dump to stdout. From the linked spec:
-D, --dump-header
Write the protocol headers to the specified file.
It doesn't specify for this option, but this works like other options that take a filename:
Use "-" as filename to have the output sent to stdout.
I am trying to post some documents to couchdb by curl and I have succeeded by choosing local file but not http-url... I have trying something like this:
curl -d #http://111.111.11.1/json/myjsonfile -X POST http://127.0.0.1:5984/MyTestDb/_bulk_docs -H "Content-Type: application/json"
I have been trying with many flags and tried many ways but I thing I am missing something. Is there anyone who can help?
The -d option for curl expects a local file only. You'll have to download it first. You could try piping the output of a curl download to a PUT to your CouchDB:
curl http://111.111.11.1/json/myjsonfile | curl -d #- -X PUT http://localhost:5984/MyTestDb....
I am using the url at the developer-api.nest.com site, and my request is re-directed to the firebase-apiserver01...01.dapi.production.nest.com
I get the correct structured data back, using this dos command:
curl -v -k -L -X GET "https://developer-api.nest.com/structures/Za6hCZpmt4g6mBTaaA96yuY87lzLtsucYjbxW_b_thAuJJ7oUOelKA/?auth=c.om2...AeiE"
I get the error 'Invalid content sent' when I send this PATCH
curl -v -k -L -X PATCH "https://developer-api.nest.com/structures/Za6hCZpmt4g6mBTaaA96yuY87lzLtsucYjbxW_b_thAuJJ7oUOelKA/?auth=c.om2...AeiE" -H "Content-Type: application/json" -d '{"away":"home"}'
I have tried adding '.json' before the question mark, but get the same error.
To set the structure to home/away you'll need to send a PUT request for example as follows:
curl -v -L -X PUT "https://developer-api.nest.com/structures/g-9y-2xkHpBh1MGkVaqXOGJiKOB9MkoW1hhYyQk2vAunCK8a731jbg?auth=<AUTH_TOKEN>" -H "Content-Type: application/json" -d '{"away":"home"}'
Hope that helps
--Nagesh
When I try to curl using the -T option, I get an empty reply:
$ curl --digest -u me:pwd -H "Content-Type:application/xml" -T test.xml http://localhost:8085/
curl: (52) Empty reply from server
Anyone know the incantation? The server works fine when connecting to it from the WebDAV client built into MacOSX.
By default curl sends Expect: Continue, but unfortunately java web containers don't play nicely with the Expect header. The simplest answer is to instruct curl not to send that header:
curl --digest -u a2 -H "Content-Type:application/xml" -H "Expect:" -T TestPBE-workspace.xml http://localhost:8080/users/a2/files2/
The better solution is to make expect:continue work, but from the research i've done it appears that depends on what web container you're using.