How do I continuously pipe to curl and post with chunking? - http

Question

I know that curl can post data to a URL:
$ curl -X POST http://httpbin.org/post -d "hello"
{
"args": {},
"data": "",
"files": {},
"form": {
"hello": ""
},
"headers": {
"Accept": "*/*",
"Content-Length": "5",
"Content-Type": "application/x-www-form-urlencoded",
"Host": "httpbin.org",
"User-Agent": "curl/7.50.1"
},
"json": null,
"origin": "64.238.132.14",
"url": "http://httpbin.org/post"
}
And I know I can pipe to curl to achieve the same thing:
$ echo "hello" | curl -X POST http://httpbin.org/post -d #-
{
"args": {},
"data": "",
"files": {},
"form": {
"hello": ""
},
"headers": {
"Accept": "*/*",
"Content-Length": "5",
"Content-Type": "application/x-www-form-urlencoded",
"Host": "httpbin.org",
"User-Agent": "curl/7.50.1"
},
"json": null,
"origin": "64.238.132.14",
"url": "http://httpbin.org/post"
}
Now here is where it gets tricky. I know about http transfer-coding and chunking, for example to send a multiline file:
('silly' is a file that contains several lines of text, as seen here)
$ cat silly | curl -X POST --header "Transfer-Encoding: chunked" "http://httpbin.org/post" --data-binary #-
{
"args": {},
"data": "",
"files": {},
"form": {
"hello\nthere\nthis is a multiple line\nfile\n": ""
},
"headers": {
"Accept": "*/*",
"Content-Length": "41",
"Content-Type": "application/x-www-form-urlencoded",
"Host": "httpbin.org",
"User-Agent": "curl/7.50.1"
},
"json": null,
"origin": "64.238.132.14",
"url": "http://httpbin.org/post"
}
Now what I want to be able to do is to have curl read a line from stdin, send it as a chunk, and then come back and read stdin again (which allows me to keep it going continously). This was my first attempt:
curl -X POST --header "Transfer-Encoding: chunked" "http://httpbin.org/post" -d #-
And it does work ONE TIME ONLY when I hit ctrl-D, but that obviously ends the execution of curl.
Is there any way to tell curl "Send (using chunk encoding) what I've given you so far, and then come back to stdin for more" ?
Thanks so much, I've been scratching my head for a while on this one!