I was trying to fetch reviews for certain app in iTunes and after some count of requests I started to get 403 response code. I assume that there is some limitation depending on requests count for certain period. If so, does anyone know exact answer? How much time should pass since I'll be able to requests for reviews again?
I have 15 apps and fetching reviews for all countries. I requested JSON format. Is there the same limitation for xml?
I faced same issue. I am parsing rss in JSON format for my n applications for different counties. It's approximately 100-150 requests. And almost all request return 403 error code, but sometimes it's work ok and return json.
So, to solve this problem I migrate from json to xml. Here is my interpolated url (PHP):
$xml_url = "https://itunes.apple.com/{$countryCode}/rss/customerreviews/page={$i}/id={$appId}/sortBy=mostRecent/xml"
Related
Currently, I am using Measurement Protocol to push the data to GA. The problem is I didn't get any response back for Success or Error on Production, If yes Please suggest?
Due to this, I am looking if there is any other options available for the same like can we achieve it using analytics 360?
The google analytics production data collection endpoint does not return a request status back (always 200 OK) by design to ensure ultra-light processing speed.
What I usually recommend to clients using Measurement Protocol server-side is to
To log a reasonable amount (or all of them) of requests somewhere. Storage is extremely cheap nowadays and knowing the data format if an emergency happens you will be able to manually extract the data
Every once in a while (one on thousand or one on a million or even more oftne depending on the importance of the data randomly) validate request on GA debug endpoint and parse the returned json. If there are any warnings or error send a notification for further investigation. This way, if anything went wrong you will be on top of the problem by the time BI & Marketing Team would be affected
We have a REST API implemented as a Cloud Service, that sends telemetry to Application Insights. And we use commands like
POST /api/groups/GRP_75e0b852-ee21-45fb-b943-13aa465c62da/members.
POST /api/groups/GRP_75e0b852-ee21-45fb-b943-13aa465c62da/folders/FLD_080af364-ad37-4351-837e-4fb1d5f02e50/discussions
The sections of the command preceded by GRP_ and FLD_ are parameters.
This makes looking at the breakdown of requests in Application Insights difficult since those requests show up individually.
I’ve implemented an ITelemetryInitializer that “normalizes” the Context.Operation.Name (and the Request URL) in our requests. But I see that those request are showing up bucketed as “Other Values”.
Requests with Other Values
Is there any way to reset the "bucketing" of the top-level list, or do I need to get a new AppInsights instance?
Standard dimentions like request name should be reset after a week. So if you stopped collecting parameters in the names it should clear up after a week. Current limit is 1000.
I'm currently polling the youtube rss feed servers about 3 times per second (multiple channels getting the feed every 3 seconds each). Is this too much and will getting it this often result in getting updates slower since it's going to start caching it for me? I'm trying to get new videos updated as quickly as possible but I can't find any gudielines on this sort of stuff.
I used to get them using the /v3/search/ at the same rate but the response from my server was always late compared to what I got when I did it on my local machine(where I didn't poll this often).
Also, is there any good alternatives? (I tried to push notifications but they were highly unreliable)
You should be using caching in your app to reduce bandwidth and latency. When caching, store the eTag so that you can include it when getting a resource. If the resource has not changed, you will get a 304 response code (NOT MODIFIED), which means you can use your cached version. Otherwise, you will get the updated version of the resource.
More info:
https://developers.google.com/youtube/v3/getting-started#etags
I'm writing a little application for my own use which will consume a publicly published RSS feed.
As far as I can tell, there's no subscribe/post mechanism in the protocol; I need to have my application HTTP-GET the RSS feed periodically.
If that's the case, I'd like to grab it every ten minutes or so, but I'm worried about being seen as an abuser. I'd certainly be concerned if I saw someone poking my server every ten minutes for weeks on end.
Is this a valid concern? Is there any general advice on what a "reasonable" refresh rate is? Do I even have my facts straight?
Since RSS is built on the HTTP protocol, in general, most sites should respect the If-Modified-Since HTTP header. This is fairly lightweight and most servers should be able to return this information quickly.
So for the client-side, you'll need to keep track of the last time you've sent the request and pass it to the server. If the server returns a 304 code, then you'll know that nothing has changed. But even more importantly, the server doesn't need to return the feed info, saving bytes of traffic. If the server returns a 200, then you'll need to process the results and save the response date.
Ultimately, the answer to this question depends on what type of information is at the other end of the RSS feed. If it is a blog, then probably once every 4-8 hours is sufficient. But if RSS feed is a feed of stock quote (not likely, just an example), then every 10 minutes is not sufficient.
So, say I'm a journalist, who wants some way of easily posting links to stories I've written that are published to my newspaper's website. Alas, my newspaper's website doesn't offer user-level RSS feeds (user-level anything for journalists, really).
Running a search (I.e., http://www.calgaryherald.com/search/search.html?q=Rininsland) brings up everything I've done in reverse chronological order (albeit with some duplicates; ignore for now, will deal with later). Is there any way I can parse this into an RSS feed?
It seems like Yahoo! Pipes might be an easy way to do this, but I'm open to whatever.
Thanks!
Normally this would be a great use of Yahoo Pipes, but it appears that the search page you cited has a robots.txt file, which Pipes respects. This means that Pipes will not pull data from the page.
For more info: "How do I keep Pipes from accessing my web pages?"
http://pipes.yahoo.com/pipes/docs?doc=troubleshooting#q14
You would have to write a scraper yourself that makes an HTTP request to that URL, parses the response, and writes RSS as output. This could be done in many server-side environments such as PHP, Python, etc.
EDIT: Feedity provides a service to scrape web pages into feeds. Here is a Feedity feed of your search url:
http://feedity.com/rss.aspx/calgaryherald-com/UFJWUVZQ
However, unless you sign up for a subscription ($3.25/mo), this feed will be subject to the following constraints:
Free feeds created
without an account are limited to 5
items and 10 hours update interval.
Free feeds created without an account
are automatically purged from our
system after 30 days of inactivity.
Provided it's just links and a timestamp you want for each article then the Yahoo Pipes Search module will return the latest 10 in it's search index of the Herlad site.