I've been using the Bing Web Search v7 API for roughly about a month now and I'm starting to notice some obscure inconsistencies. Sometimes when I run a query, I get a JSON response with no results. I can then run the same exact query shortly after but this time the JSON returned will contain results. I'm not sure if I'm nearing my TPS limit (which is currently 100/s) but I don't believe I am. I also explicitly check for the URL length to ensure Bing doesn't return an error for a string that's too long. I'm doing queries in the same way that's described here. Each query contains a capture group with a set of terms "OR"'d together (which is done following the Bing advanced search guidelines). I had an issue with double spaces causing no results but that has been fixed. Is this possibly a result of the HTTP GET length requirements? Or maybe something wrong with my key? Any help is appreciated and let me know if any more information is required!
Related
I'm working on a simple app to programmatically retrieve ads performance within Linkedin. I have general API experience but this is the first time i get my feet wet with the Linkedin API.
One example from Linkedin API documentation suggest something that would get me started:
GET https://api.linkedin.com/v2/adAnalyticsV2?q=analytics&dateRange.start.month=1&dateRange.start.day=1&dateRange.start.year=2016&timeGranularity=MONTHLY&pivot=CREATIVE&campaigns=urn:li:sponsoredCampaign:112466001
I am encountering two problems:
First this example implies that you already know the campaign ID. However I am unable to find a way to retrieve a list of campaign ID's for a given account.
Second, if I manually pull a campaign ID, I receive an error: "{"serviceErrorCode":2,"message":"Too many fields requested. Maximum possible fields to request: 20","status":400}". Pretty clear error.
A little research tells me that by adding the parameter "&fields=" I will be able to limit my query to less than 20 field (I really need only a dozen anyway) but I can't find and documentation regarding the names of the fields available.
Any help or pointer will be appreciated.
please refer the link below scroll down where you ill see the field names mentioned as metrics , these are the fields.
https://learn.microsoft.com/en-us/linkedin/marketing/integrations/ads-reporting/ads-reporting?tabs=http#analytics-finder
I have been asked by a client to see if Bing Custom Search can order results containing links to PDFs by file modification date.
I know results can be ordered by the date the content was indexed (or re-indexed), but they are concerned with the actual age of PDF files as determined by the filesystem timestamp, and want to order the results by that criteria.
I could not find anything in the Azure documentation, and personally I don't believe it is possible, but I wanted to check in with SO first.
I don't believe this specific scenario is currently supported at the time. But if this is a feature you would like to see supported in the future, you may leave your feedback on Uservoice.
I have a strange behaviour with Bing Web Search.
I have a search query "hawkers" OR "hawkersco" OR "#hawkersco" OR "#hawkers" OR "www.hawkersco.com" with market = 'es-ES', safeSearch = Strict and responseFilter = webPages.
So, I expect, that result will contain at least one of these words and it will be Spanish posts. In fact I get more of posts in English and its not contain these keywords...
If I try search one by one these keywords, without OR operator, I had expected Spanish posts.
Please, explain why it is? How to use search query for get expected results?..
Check the specification for Bing Web Search API. Possibly this might be as simple as changing market to mkt(since you listed all the other parameters as used). And that means you should have a value for setLang as well.
You're not getting Spanish posts at all?
In that case, see here.
Bing results are based on relevance. Regardless of Market or Language.
If the result is deemed relevant. It will rank higher compared to the
selected language, and appear in the results.
Freshness affects the results, in that you need relevant(popular)
sites in your language. For them to attain sufficient relevance in the
selected time period.
You cannot rely on Bing returning a single language exclusively, with
the settings as they are.
I am working on a school project in R where I am attempting to map where the most popular youtube videos are posted around the world. I am able to get the data for the 50 most popular videos, but am having trouble understanding how to use pageToken.
The current get request I am using is with the following:
https://www.googleapis.com/youtube/v3/videospart=snippet%2CrecordingDetails&chart=mostPopular&maxResults=50&key={api_key}
Is it possible to retrieve more than 50 results using "pageToken" (I am unfamiliar with how this works).
Any help would be appreciated thanks!
Videos: list
pageToken string The pageToken parameter identifies a specific
page in the result set that should be returned. In an API response,
the nextPageToken and prevPageToken properties identify other pages
that could be retrieved.
Note: This parameter is supported for use in conjunction with the
myRating parameter, but it is not supported for use in conjunction
with the id parameter.
So when you get the results from the first request you should have an option called page token if you send that to the next request
&pageToken=api_pageToken
it should give you the next bunch of rows.
Note: I am not an R programmer so I cant help with the code for a loop over the results to find out if there are page tokens or not.
I think the question has been answered here before,but i could not find the desired topic.I am a newbie in web scraping.I have to develop a script that will take all the google search result for a specific name.Then it will grab the related data against that name and if there is found more than one,the data will be grouped according to their names.
All I know is that,google has some kind of restriction on scraping.They provide a custom search api.I still did not use that api,but hoping to get all the resulted links corresponding to a query from that api. But, could not understand what will be the ideal process to do the scraping of the information from that links.Any tutorial link or suggestion is very much appreciated.
You should have provided a bit more what you have been doing, it does not sound like you even tried to solve it yourself.
Anyway, if you are still on it:
You can scrape Google through two ways, one is allowed one is not allowed.
a) Use their API, you can get around 2k results a day.
You can up it to around 3k a day for 2000 USD/year. You can up it more by getting in contact with them directly.
You will not be able to get accurate ranking positions from this method, if you only need a lower number of requests and are mainly interested in getting some websites according to a keyword that's the choice.
Starting point would be here: https://code.google.com/apis/console/
b) You can scrape the real search results
That's the only way to get the true ranking positions, for SEO purposes or to track website positions. Also it allows to get a large amount of results, if done right.
You can Google for code, the most advanced free (PHP) code I know is at http://scraping.compunect.com
However, there are other projects and code snippets.
You can start off at 300-500 requests per day and this can be multiplied by multiple IPs. Look at the linked article if you want to go that route, it explains it in more details and is quite accurate.
That said, if you choose route b) you break Googles terms, so either do not accept them or make sure you are not detected. If Google detects you, your script will be banned by IP/captcha. Not getting detected should be a priority.