What is the correct response an RSS client should have when it encounters a feed that has multiple items with the same guid/identifier?
Currently in my application, any items that use an existing guid won't be cached or displayed because it believes it already has that item.
In this example feed a lot of items share this id:
tag:blizzard.com,2010-10-22:diablo3:feed:en-us:1
According to w3 when there are duplicate entries in an RSS feed:
Atom Processors MAY choose to display all of them or some subset of them. One typical behavior would be to display only the entry with the latest atom: updated timestamp.
I would go with the spec and display only the entry with the latest updated timestamp. Don't forget to send an email to Blizzard support and have them get their RSS validated - just don't threaten to keep them out of the next raid.
Take care.
I think your app is doing it right. Don't get fancy. If you've already seen an item with that guid you don't present it a second time. You should contact whe webmaster for the feed if possible and alert them to the problem.
Does each item have a unique URL? If so, fall back to using the URL.
Related
I'm generating an RSS feed from my blog. I'm using node-rss. When I make a minor edit to one the posts listed in the feed, Google Reader lists the item as unread, even though I marked it as read a week ago.
My RSS feed contains title, description, link, guid and pubDate elements for each item. For guid, I'm just using the canonical URL to the item. The pubDate element is the date/time that the entry was first published, rather than the time of the last edit.
The feed itself contains lastBuildDate, which is set to the time that the RSS feed was generated (i.e. when it was requested).
As far as I can tell, there's nothing in the RSS feed that flags the item as being changed. So why does Google Reader think that the item has been updated, and why does it show it as unread again?
Does it look at the content (which has changed)? If so, can I do something in the RSS feed to mark this as a minor update, thus preventing Google Reader from showing it as unread?
If, in Google Reader, you mouse over the date in the top-right of each post, you'll see that it has "Received" and "Published" dates.
"Received" appears to be when the Google Reader server saw the new content, whereas "Published" comes from the feed itself.
Google Reader appears to use the "Received" date to decide whether something is new.
So, to get the correct behaviour
Don't put anything in the feed that's older than (say) 6 months.
Limit the feed XML to the most-recent 10 or so items.
Of course, the second could imply the first...
I am building RSS feed for the first time and I have some simple, direct questions that I was unable to find on the web, well at list in a sense that would be clear to me. Can you help me understand following
Which items should I include in RSS generation? should I always put in all the articles or what is the criteria when I query my articles for the feed?
What value should I set for pubDate? The specification says "The publication date for the content in the channel. For example, the New York Times publishes on a daily basis, the publication date flips once every 24 hours. That's when the pubDate of the channel changes.". I do not quite understand how to apply this to my feed. I have new articles daily, should I set the pubDate to let say 06:00 AM today and update it every day?
lastBuildDate: if I understand this right is the date of the latest updated item?
Which items should I include in RSS generation?
You should have one generic feed with all the new articles you post (for example: news). Additionally if you got your webpage split into categories, or you have some specific feeds (eg. calendar of the events) then it's good to create additional separate RSS for each one of them
What value should I set for pubDate? I do not quite understand how to apply this to my feed. I have new articles daily, should I set the pubDate to let say 06:00 AM today and update it every day?
Always set pubDate to the time when your news/articles went online. So if you have new articles daily pubDate should be a date when they were released to the public. Not random hour in the morning. Not the moment when you started writing them.
lastBuildDate: if I understand this right is the date of the latest updated item?
lastBuildDate is the most recent date when any of the results was posted or modified. Usually you should skip it - especially if your lastBuildDate will be simply a most recent pubDate. It's an optional parameter.
I use lastBuildDate only for calendar RSS feeds to show when the calendar was updated (as in calendars you not only add new entries but also often edit existing).
You should put every article, but the best is to provide different feeds for different categories, even search keywords. You can build it like any dynamic page, with a querystring.
that's not super important, you can put whatever. I don't think may feed readers use it.
theoretically it's the date the content changed. So the date of the latest updated item should work.
Something super important, since people are going to do polling on this page (meaning a lot of requests on the page)
- Cache it on your server
- Serve and Etag header and/or a LastModifiedDate. That way your server can respond with just a "not modified" if the client has it in cache already.
I'm using feedparser for working with RSS.
I'm getting regularly (e.g. every 15 minutes) RSS channel with items and store it. In the channels there aren't often any new items. So, it's unefficient.
Is there a way to detect quickly if there are some new items in the channel and if not, do nothing with this channel?
thank you
For RSS 2.0, the channel element has an optional lastBuildDate eleement. For atom, there's a similar "atom:updated" element, but the standard does state that this is when "an entry or feed was modified in a way the publisher considers significant. Therefore, not all modifications necessarily result in a changed atom:updated value".
There's also a PubDate element in RSS 2.0, also optional, but lastBuildDate should be the one to use, assuming it's there and the publisher is using it correctly.
You can store the previous one and compare the newly retrieved value with the old one.
Added material on feedparser:
For feedparser, see feed-updated_parsed and feed-updated.
I'm writing an RSS feed. Let's say it's for a list of entries as in a blog.
How do I handle updating the feed? I mean, let's assume that The feed always displays the last 10 entries.
If someone subscribes now, he'll get the last 10 entries (1..10)... what if there are for example 2 new articles, and then what will his feed reader do? Because I will return the articles (2..12).
Do I have to do any special handling to start from a certain article in the feed, or do I just always put the last 10 and this will be fine
Returning the last n articles will be fine. Because you assign a unique identifier to each article (you do, right?) the feed reader can easily keep track of what it has already seen or not.
The feed reader will probably watch to see how often new articles appear, to help determine how often it checks for new articles.
How would I get the next page or more results for a feed?
For example, when I go to Security Now feed page, there is no "next" link of any kind and the url parameter of "page=100" does nothing:
http://leoville.tv/podcasts/sn.xml
I get only 1 page of results of about 20 episodes. However my Google Reader can successfully retrieve episodes that are earlier than that.
Indeed it is true that Google Reader caches the items and it is NOT possible to paginate on RSS2, RSS or Atom feeds (unless they have rel=next link, which none of them seem to have).
However, we can leverage the existing Google Reader infrastructure, with some work, to retrieve a list of, say 200 items!
Given the above podcast url we retrieve the latest 200 episodes by:
Using the ...google.ca/reader/atom/feed prefix instead of the usual view/feed as can be seen in your google reader.
Appending n=200 as the query parameter.
So we have:
http://www.google.ca/reader/atom/feed/http://leoville.tv/podcasts/sn.xml?hl=en&n=200
There is a very insightful reverse-engineered google-reader API project located at http://code.google.com/p/pyrfeed/wiki/GoogleReaderAPI
Google reader caches RSS entries. You can't get any more from the actual feed if they don't allow for it.