I've been working with pipes for a while now, I am trying to output more than the basic structure of:
Item
title
link
description
guid
pubDate
I want to publish more data in the RSS feed under different fields but cannot figure out if this is even possible. Any ideas?
This post at the Yahoo Pipes blog goes through the basics of building a complex RSS feed with a couple examples.
http://blog.pipes.yahoo.net/2009/06/10/new-create-rss-and-rss-item-builder-modules/
I know this is not related to yahoo pipes, but if you are looking for etl tools, i found yahoo pipes very limiting. I have had the best luck with Open Kapow. Just in case you have not heard about/used it.
Related
how do I read RSS feeds in apple script? I want it to pick and speakout latest news along with reading weather info
The easiest way would be to script an rss reader like NetNewsWire or even Safari. The second easiest way is to write it in Ruby or Python and call Applescript stuff using Appscript. Python has robust rss reading and parsing routines. Dealing with a lot of text processing in Applescript is painful. The third choice is to use an rss OSAX. There was one at http://osaxen.com but it appears down right now.
Satimage has a free osax (scripting addition) with xml capability. That would be a way to go.
I'd like an rss feed from this google scholar search: Scholar Fish Oil Search
I've looked a little bit at yahoo pipes, and I thought I had found a solution when I found this pipe: Old Pipe But it doesn't work (it's a couple years old now). If someone can either tell me what's wrong with that pipe, or tell me how to retrieve a feed from that search through another means, I'd be very appreciative.
Thanks for your time,
-Landon
You could try a 3rd party website that creates feeds from other websites. See 7 Tools To Make An RSS Feed Of Any Website. (Disclaimer: I have no idea if they work or are any good, but they may be worth investigating).
[Edit: Google disallows indexing of this content via their robots.txt file, apparently. Check out http://scholar.google.com/robots.txt. Yahoo Pipes respects the robots.txt file—perhaps one of the other tools doesn't suffer from this snag?]
It appears that markup may have been altered slightly since the publication of this Pipe.
When I use the URL builder module in Pipes and populate the sample query with "fish oil", I get the following search string:
http://scholar.google.com/scholar?scoring=r&q=%22fish+oil%22&lr=&hl=en&as_ylo=2007
(Which, when entered into a browser window, does generate results.)
I am currently parsing through their regular expressions to make sure the proper elements are captured.
Did you have any luck with the tools Dan mentioned? Would also be quite interested if any were simple, effective, and (ideally) non-proprietary or self-hostable.
I admit to being a bit behind the times when it comes to understanding RSS/Atom feed issues.
All I know is, I want to have an Atom feed converted to an RSS feed inline. Meaning that I want to specify an option somewhere that says use this xsl or whatever to convert the Atom feed to an RSS feed.
Is there some way to accomplish this that doesn't require running a separate listener on my local box to do the translation? I.e. Instead of directly asking for the Atom feed, I get my 'service' to get the feed, transform it and give it to the real subscriber?
Sorry for the (probably) dumb question. Somehow I never got the memo on this stuff.
Thanks - Jon
Similar to Paul's answer... we use Feedburner for feeds - it's trivial to have it convert one type of feed to another.
Atom to RSS on the fly
Sounds like a Yahoo Pipes job to me..
http://pipes.yahoo.com/pipes/pipe.info?_id=Aj_YHfi62xGuBj4KkAtvUw
If you want to write your own, using C#. You would download the Atom and save it as RSS using the SyndicationFeed class. It's about 4 lines of code.
I want to merge multiple rss feeds into a single feed, removing any duplicates. Specifically, I'm interested in merging the feeds for the tags I'm interested in.
[A quick search turned up some promising links, which I don't have time to visit at the moment]
Broadly speaking, the ideal would be a reader that would list all the available tags on the site and toggle them on and off, allowing me to explore what's available, keep track of questions I've visited, new answers on interesting feeds, etc, etc . . . though I don't suppose such a things exists right now.
As I randomly explore the site and see questions I think are interesting, I inevitably find "oh yes, that one looked interesting a couple days ago when I read it the first time, and hasn't been updated since". It would be much nicer if my machine would keep track of such deails for me :)
Update: You can now use "and", "or", and "not" to combine multiple tags into a single feed: Tags AND Tags OR Tags
Update: You can now use Filters to watch tags across one or multiple sites: Improved Tag Stes
Have you heard of Yahoo's Pipes.
Its an interactive feed aggregator and
manipulator. List of 'hot pipes' to
subscribe to, and ability to create
your own (yahoo account required).
I played with it during beta back in the day, however I had a blast. Its really fun and easy to aggregate different feeds and you can add logic or filters to the "pipes". You can even do more then just RSS like import images from flickr.
I create a the stackoverflow tag feeds pipe. You can list your tags of choice into the text box and it will combine them into a single feed with all the unique posts. It escapes '#' and '+' characters for you.
Alternatively, you can use the pipe's rss feed by appending your html-encoded tags separated by '+'s:
http://pipes.yahoo.com/pipes/pipe.run?_id=uP22vN923RG_c71O1ZzWFw&_render=rss&tags=.net+c%23+powershell
Unfortunatley, though, this seems to strip out the content of the posts. The content is visible in the debug view, but the output only contains the post title.
[Thanks to everyone for suggesting Yahoo Pipes! Had heard of it before, but never tried it until now :-]
SimplePie is a PHP library that supports merging RSS feeds into one combined feed. I don't believe it does dupe checking out-of-the-box, but I found it trivial to write a little function to eliminate duplicate content via their GUIDs.
Here is an article on Merge Multiple RSS Feeds Into One with Yahoo! Pipes + FeedBurner.
Another option is Feed Rinse, but they have a paid version as well as the free version.
Additionally:
I have heard good things about AideRss
Yahoo Pipes?
23 minutes later:
Aww, I got answer-sniped by #Bernie Perez. Oh well :)
In the latest Podcast, Jeff and Joel talked about the RSS feeds for tags, and Joel noted that there is only the current ability to do AND on tags, not OR.
Jeff suggested that this would be included at some stage in the future.
I think that you should request this on uservoice, or vote for it if it is already there.
It seems like such a simple thing, but I can't find any obvious solutions...
I want to be able to take two or three feeds, and then merge then in to a single rss feed, to be published internally on our network.
Is there a simple tool out there that will do this? Free or commercial..
update: Should have mentioned, looking for a windows application that will run as a scheduled service on a server.
There are a whole pile of options here: http://allrss.com/rssremixers.html.
Maybe http://www.planetplanet.org/
will do what you want.
It's for creating blog aggregations like planet lisp.
Google reader, create a group, add your feeds into the folder and then share that as an RSS feed.
:-)
Works while you're asleep!
Yahoo Pipes could be nice. Depends on how much "private" you want the resulting feed to be.
For 100% offline solution investigate Atomisator. It's a Python framework basically for doing offline what Yahoo Pipes does online.
If you're using PHP, the SimplePie library will do this. Here's a tutorial.