I'm currently trying to build a news reader that gets rss feeds from different google news topics and I found this website that breaks down the url parameters pretty well, but under the topic parameter, the link that it has for a list of topics brings you to this page which says nothing about topic keywords. The only topic key that I've been able to find is tc being the key for the tech topic.
Does anybody know where I could find a list of these topics or if the topic=[topic] parameter is deprecated what I could use instead?
It will depend of country.
Example (for US):
WORLD
NATION
BUSINESS
TECHNOLOGY
ENTERTAINMENT
SPORTS
SCIENCE
HEALTH
RSS business (US)
https://news.google.com/news/headlines/section/topic/BUSINESS?ned=us&hl=en
But, if you query other country, the topic will change.
Example (Colombia):
WORLD.es_co
NATION.es_co
BUSINESS.es_co
TECHNOLOGY.es_co
ENTERTAINMENT.es_co
SPORTS.es_co
SCIENCE.es_co
HEALTH.es_co
Here is the RSS about economy/business in Colombia:
https://news.google.com/news/rss/headlines/section/topic/BUSINESS.es_co/Econom%C3%ADa?ned=es_co&hl=es
Check too the ned and hls parameters added.
Related
I can't seem to find anything in the documentation (link), but the previous version of the Places API (link) said:
Certain types of categories, such as restaurants or hotels, are ranked via a "recommendations-style" algorithm where measures of popularity or quality, such as number of stars or reviews, are taken into account.
Does that mean that I would be able to retrieve info on number of starts for a hotel or ratings for restaurants?
Rating is Rich content. Rich content is not covered within the base or extended content and is generally provided by third-party data
suppliers.
See more on references from Geocoding and Search API 7 Discover API here https://developer.here.com/documentation/geocoding-search-api/dev_guide/topics/endpoint-discover-brief.html
Is there a way to get Google and Bing news searches as RSS?
I'd like to have the most recent at the top of the results (ordered by time) if possible.
And show 100 results if that is possible.
I found this for Google, but it only shows 5 items or so, not that great.
https://news.google.com/news/feeds?output=rss&q=politics
I just want to search their news categories and get results as RSS.
You are right that Google no longer publicly shows RSS feeds for News searches. But there's a basic URL format that should continue to work.
If my search topic is "education", my search URL should look like:
https://news.google.com/news/section?cf=all&ned=us&q=education
Replace:
value of ned (probably stands for 'news edition'), i.e. us with the appropriate country code that's relevant to you. E.g. uk (not gb for UK), in (India), etc.
value of q (probably stands for 'search query'), i.e. education with your search term. Combine multiple words with a plus sign, e.g. education+policy (if your search is for 'Education Policy').
Now the RSS feed button in your browser should've become active because an RSS feed is available for the page. It shows 10 latest news items in the feed, not 5. And the RSS feed URL would look like this:
https://news.google.com/news/feeds?cf=all&ned=us&hl=en&q=education&output=rss
Conclusion: RSS feeds aren't available for actual searches, i.e. typing in the search box on Google. You'd have to do it like this.
(PS: Credit goes to the question itself. It would not have occurred to me otherwise.)
Try adding
&output=rss
https://news.google.com/news/section?cf=all&ned=us&q=education**&output=rss**
The custom RSS feed is in this format:
https://news.google.com/news/feeds?q={yourquery}&output=rss.
For example, https://news.google.com/news/feeds?q=developer&output=rss returns news about developer.
Your best bet is somthing like Google Alerts feeds. You can type a query select the type of data in there and get it delivered via RSS. Since they support PubSubHubbub for that, you will also get results in realtime.
If you want another dataset feel free to also check Superfeedr's track feeds which will help you get notified in realtime when keywords are matched across any RSS feed.
I think Google disabled the RSS search output in the past few weeks, and I have not found an alternative. The XML output requires a Google paid account. I now have to scrape the HTML for what I want.
Here's the new way to access Google News RSS feeds:
https://news.google.com/news/rss/search/section/q/{yourquery}
I am writing an application to read the updates happening in khan academy youtube playlists, by parsing their feeds, but the feed is not upto date
According to this feed url, the last update was on Aug,2012, how to get the latest feed?
http://gdata.youtube.com/feeds/api/users/khanacademy/playlists?v=2
Please tell me if there is anyway to get a new feed
Reg,
Karthi
The youtube method or organizing videos seems like something Khan academy has abandoned at some point in favor or organizing the lists separately. That is why newer videos don't appear in previous lists you would think they would. They now have a JSON returning RESTful API. For example to see the playlists visit this. There is a wiki entry explaining more on how to consume these lists and example client code is available on their github channel for a deeper direct use or code dive to emulate in your code.
http://gdata.youtube.com/feeds/api/users/khanacademy/playlists?v=2 returns a list of the most recent 25 playlists that are in the "khanacademy" channel, and that's what I see being returned. It matches what you see in the web interface at http://www.youtube.com/user/khanacademy/videos?shelf_index=5&sort=dd&tag_id=&view=1
Playing with the idea of making a video blog of locally trending videos that pulls from Youtube's API and posts trending videos related to Houston.
If I simply query for recent videos with the word Houston, I get a mix of car videos no one has seen, some Whitney Houston videos and some interesting local uploads.
Any suggestions for better paramaters? Could I get more specific with Youtube Trends? The Youtube Trends blog lets me search for trending videos in Houston, but there doesn't seem to be an API...
I'm not sure that there's currently a way to get videos via the API recently trending in Houston or the Houston area, but one idea for getting recent videos about Houston would be to use the topicId field in v3 of the API -- using Freebase topics yields much more semantically relevant results than a simple keyword search. For example, this query (you'll have to be oAuth authenticated to make this call):
GET https://www.googleapis.com/youtube/v3/search?part=snippet&topicId=%2Fm%2F03l2n
(/m/0312n is the Freebase Topic ID for the city of Houston) will return videos of live concerts in Houston, the Houston coffee & cars series, etc. If you were to collect a series of Freebase IDs that you think your viewers would be interested in (sports teams, local cuisine, etc. etc.), you could then aggregate your results and sort them by date or by view count.
When creating an RSS reader, you download the XML formatted document pointed to by the RSS feed link, and you can parse it manually or using the functionality in the SyndicationFeed namespace.
So if we take Scott Guthrie's blog as an example, you download the RSS feed document here, and parse it. My problem is that this document only holds 15 items, yet he has been blogging for a number of years.
Is there a standard or established way of getting the older posts not included in the RSS feed document? Or do you have to find the base address for the blog posts and then parse the pages of the site from there to get them? How do you avoid missing posts on high volume blogs?
With RSS/Atom you can't query older articles.
I built a RSS archival service (https://app.pub.center). All of our data is free to use via REST. We charge money for push notifications.
PubCenter daily polls it's catalog of RSS feeds, and caches the articles. Then, you can get these articles back in a chronological order. For example:
Page 1 of The Atlantic https://pub.center/feed/02702624d8a4c825dde21af94e9169773454e0c3/articles?limit=10&page=1
Page 2 of The Atlantic https://pub.center/feed/02702624d8a4c825dde21af94e9169773454e0c3/articles?limit=10&page=2
As the replies to How Do I Fetch All Old Items on an RSS Feed? already mentioned, a feed may not provide archival data but historical items may be available from another source.
Archive.org’s Wayback Machine has an API to access historical content, including RSS feeds (if their bots have downloaded it). I’ve created the web tool Backfeed that uses this API to regenerate a feed containing concatenated historical items. If you'd like to discuss the implementation in detail please get in touch.