i am new to influxdb and i was trying to add data to influx db . I found on the documentation page that it can be done using curl.
curl -i -XPOST 'http://localhost:8086/write?db=mydb' --data-binary 'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000'
but i want to hit it from the browser and be able to insert data to influxdb.
For select query :
http://localhost:8086/query?db=mydb&q=select * from temperature
this is a url which can be used to get data from influxdb. Is there a similar way to insert data to influxdb.
I tried creating url from curl but wasn't successful.
little bit late but:
you should call REST via POST
url: http://localhost:8086/write?db=mydb
type :POST
body: measurement,value=123 thetime=1502474399000
Related
Helllo,
I'm working on a scraper for this page : https://www.dirk.nl/
I'm trying to get in scrapy shell the 'row-wrapper' div class.
If I enter response.css('row-wrapper'), it gives me some random results, I think an anti scraping system is involved. I need the hrefs from this class.
Any opinions on how can I move forward ?
We would need a little bit more data, like the response you receive and any code if it's already set up.
But from the looks of it, it can be multiple things ( from 429 Response blocking the request because of the rate limit to sites internal API XHR causing data to not be rendered on page load etc. ).
Before fetching any website for scraping reasons, try curl, postman or insomnia software to see what type of the response are you going to receive. Some special servers and website architectures require certain cookies and headers while some don't. You simply have to do this research so you can make your scraping workflow efficient.
I ran curl https://www.dirk.nl/ and it returned data that's generated by Nuxt framework. In this case that data is unusable since Nuxt uses it's own functionality to parse data.
Instead, the best solution would be not to get the HTML based data but API content data.
Something like this:
curl 'https://content-api.dirk.nl/misc/specific/culios.aspx?action=GetRecipe' \
-H 'accept: application/json, text/plain, */*' \
--data-raw '{"id":"11962"}' \
--compressed
Will return:
{"id":11962,"slug":"Muhammara kerstkrans","title":"Muhammara kerstkrans","subtitle":"", ...Rest of the data
I don't understand this language but from my basic understanding this would be an API route for recipes.
I'm trying to deploy a Google Apps Script as a web app, but while I have no problem doing GET requests, I'm having trouble with POST requests.
My code is very simple:
function doGet(request) {
var result = JSON.stringify({ data: 'Thanks, I received the GET request' });
return ContentService.createTextOutput(result).setMimeType(ContentService.MimeType.JSON);
}
function doPost(request) {
var result = JSON.stringify({ data: 'Thanks, I received the POST request' });
return ContentService.createTextOutput(result).setMimeType(ContentService.MimeType.JSON);
}
I deployed the web app with "Execute the app as: Me" and "Who has access to the app: Anyone, even anonymous". Every time I do some change I re-deploy it with a new version ("Project version: New").
After publishing, my curl GET request works perfectly:
> curl -L https://script.google.com/macros/s/$SCRIPT_ID/exec
{"data":"Thanks, I received the GET request"}
However, my POST request (curl -L -XPOST https://script.google.com/macros/s/$SCRIPT_ID/exec) just shows me a generic Google HTML page saying "Sorry, unable to open the file at this time. Please check the address and try again".
I tried sending some data and providing a content type, but nothing changed. I also tried changing the output type to just ContentService.createTextOutput("OK"), but it didn't work either. Curiously, deleting doPost changes the error message to "Script function not found: doPost" as expected. If it makes any difference, this script is attached to a Google spreadsheet.
Are there any special permissions I need to give to the script for POST requests specifically?
It seems that the problem was with my usage of curl, on subtle differences between using -XPOST and not using it. As suggested by Tanaike, changing from:
curl -L -XPOST https://script.google.com/macros/s/$SCRIPT_ID/exec
to
curl -L -d '' https://script.google.com/macros/s/$SCRIPT_ID/exec
Solved the issue. Even though curl helpfully says "Unnecessary use of -X or --request, POST is already inferred" when I do a request with -XPOST and a payload, its behavior is different in the presence of redirects. -XPOST forces all subsequent requests after a redirect to be made using POST as a method. On the other hand, if I don't specify -XPOST, the requests after the first POST are made as GET requests. I don't know if this is curl's intended behavior, but it's certainly unintuitive.
I try to get my linkedin feed using this API :
https://linkedin.api-docs.io/v1.0/feed/42Hm9SaY2p2CGwPzp
I try to use this request : "GET /voyager/api/feed/updates" with this shell code :
curl --request GET \ --url
https://www.linkedin.com/voyager/api/feed/updates \ --data '{}'
But I get this response : "CSRF check failed". I understand why linledin respond this but how to avoid it ?
You missing headers, see API docs here: https://linkedin.api-docs.io/v1.0/feed and explanation how get headers here: https://towardsdatascience.com/using-browser-cookies-and-voyager-api-to-scrape-linkedin-via-python-25e4ae98d2a8
API docs a bit outdated, data output format might be different, this at least true for messaging/conversations, not sure about feed
In regards of headers I suggest to try apify.com and extract them in real time from browser instance (run puppeteer, login to LiN, get headers, save them)
Phantombuster will not allow you to use your own code so not very useful
http://snomedct.t3as.org/ This is a web service that will analyse English clinical text, and report any concepts that can be detected.
For e.g.- I have headache. It will identify headache as a Symptom.
Now what I would like to do is send the sentence to the web service through R, and get the table back from the web page to R for further analysis purpose.
If we take their example curl command-line:
curl -s --request POST \
-H "Content-Type: application/x-www-form-urlencoded" \
--data-urlencode "The patient had a stroke." \
http://snomedct.t3as.org/snomed-coder-web/rest/v1.0/snomedctCodes
that can be translated to httr pretty easily.
The -s means "silent" (no progress meter or error messages) so we don't really have to translate that.
Any -H means to add a header to the request. This particular Content-Type header can be handled better with the encode parameter to httr::POST.
The --data-urlencode parameter says to URL encode that string and put it in the body of the request.
Finally, the URL is the resource to call.
library(httr)
result <- POST("http://snomedct.t3as.org/snomed-coder-web/rest/v1.0/snomedctCodes",
body="The patient had a stroke.",
encode="form")
Since you don't do this regularly, you can wrap the POST call with with_verbose() to see what's going on (look that up in the httr docs).
There are a ton of nuances that one should technically do after this (like check the HTTP status code with stop_for_status(), warn_for_status() or even just status_code(), but for simplicity let's assume the call works (this one is their example so it does work and returns a 200 HTTP status code which is A Good Thing).
By default, that web service is returning JSON, so we need to convert it to an R object. While httr does built-in parsing, I like to use the jsonlite package to process the result:
dat <- jsonlite::fromJSON(content(result, as="text"), flatten=TRUE)
The fromJSON function takes a few parameters that are intended to help shape JSON into a reasonable R data structure (many APIs return horrible JSON and/or XML). This API would fit into the "horrible" category. The data in dat is pretty gnarly and further decoding of it would be a separate SO question.
I would like to stream changes to a Firebase location, but filter the results based on a query to some index, like so:
curl -i -L -H "Accept: text/event-stream" https://mydata.firebaseio.com/path.json?'orderBy="myIndexedField"&equalTo="desiredValue"'
What I observe is that Firebase appears to ignore my query and proceeds to stream all changes to that location whether they match the query or not. Is there any way to do this, other than writing code to perform my own client-side filtering?
EDIT
Frank's answer below shows that Firebase does indeed honor your query parameters. The problem I'm having still persists, and I simply misconstrued what was going on, as the situation turns out to be a little more complicated. I've reproduced my issue in Frank's Firebase, which he was kind enough to supply as a live example for this question.
Here are the steps to reproduce my issue:
Start a streaming query with the constraint type == 1:
$ curl -i -L -H "Accept: text/event-stream" 'https://stackoverflow.firebaseio.com/29265457/.json?print=pretty&orderBy="type"&equalTo=1'
In a separate terminal, post a new item with type==1:
$ curl -X POST -H "Content-Type: application/json" -d '{"type": 1}' https://stackoverflow.firebaseio.com/29265457/.json
{"name":"-JlY1nAmymCKw5lvLvMe"}
This object pops up in my ongoing curl stream as expected, since it matches the query of type==1:
event: patch
data: {"path":"/","data":{"-JlY1nAmymCKw5lvLvMe":{"type":1}}}
Now, here's the part I misinterpreted as Firebase ignoring queries. If I PUT that resource I just POSTed and change it to type==0, it still shows up in my stream! To perform the PUT:
$ curl -X PUT -H "Content-Type: application/json" -d '{"type": 0}' https://stackoverflow.firebaseio.com/29265457/-JlY1nAmymCKw5lvLvMe.json
And here's what I see pop up in my ongoing stream curl:
event: patch
data: {"path":"/","data":{"-JlY1nAmymCKw5lvLvMe":{"type":0}}}
If I PUT "type" to 0 again, it no longer shows up in my curl stream terminal. If I PUT type to 1, then it does pop up in my curl stream (expected). It's the transition from 1->0 where I get the unexpected event.
Also, using PATCH instead of PUT to modify "type" appears to result in the same behavior in the curl stream.
I think it's natural to expect that when I change a value to NOT match my query anymore, I wouldn't see it in my stream. However, it looks like perhaps the query is being matched before the value is edited, or something along those lines... I have no idea how this is implemented on the Firebase end.
So I guess the new question is: How can I avoid seeing changed values that don't match my query in the REST stream?
There are two things that could be wrong here:
you didn't define an index on myIndexedField
your URL is quoted incorrectly
Ad 1) When you access Firebase through the (JavaScript/iOS/Java) libraries, it will fall back to client-side ordering and filtering if you're ordering/filtering data that is not indexed. Given that the REST API doesn't come with a client-library, it cannot perform such a fallback and it will simply return the data unordered/unfiltered.
Ad 2) you need to put your single quotes around the entire URL, not just the query string. An example curl that works for me:
curl -i -L -H "Accept: text/event-stream" 'https://stackoverflow.firebaseio.com/29265457/.json?orderBy="type"&equalTo=0'
I push data to it with this JavaScript snippet:
new Firebase('https://stackoverflow.firebaseio.com/29265457/').push({ type: 0 });
And it shows up in the curl window; or it doesn't if I set type to 1.