Rss seems only have the latest n posts, I just wonder is there anyway to get all the posts including the history post. Thanks
Jeff Zhang
This isn't generally possible since a RSS reader only shows what is currently in the feed. You can only pull as much as is published at that time. Finding the dataset which backs the RSS feed items and downloading directly from there is something else entirely though and is sometimes possible.
using c# you can do it but it will return the Json not in Rss format.
you have to do like
string AccessToken="Your Access Token e.g KIMJSLIFJEILMFSLJFSDIIIIFLDFJSLFJLSFSLFJSLJF";
var client= new facebookclient(AccessToken);
dynamic allFeeds=client.Get("me/feed");//OR "me/feeds
foreach (var uniquefeed in (JsonArray)allFeeds["data"])
{
string feedids = (string)(((JsonObject)uniquefeed )["id"]);
//Write more stuff here what you want.
}
Related
How can I make certain parameters of the WordPress Rest API accessible to anyone without first being authenticated – for example, the page parameter doesn't work (where blog is a custom post type) in this query:
mysite.com/wp-json/wp/v2/blog?page=2&per_page=20
I've seen that in the past it's been possible to make these params available, for instance :
add_filter( 'json_query_vars', function( $valid_vars ) {
$valid_vars[] = 'offset';
return $valid_vars;
});
Is there any way to do something similar with today's version of the API?
For anyone who has the same problem, I've solved it. The page parameter is actually publicly available, offset is the one you need authentication for.
The reason the API didn't paginate was because the request url didn't have the paged query string set. Every time I tried to add it with the params option of the WordPress Node API, it didn't work:
wpapi.getNews().params('paged', 'paged').perPage( perPage ).page( pageNumber ).then(data=>
It didn't work because the request url created by the API seemed to always put the page parameter before the paged one, which resulted in paged being ignored when the query actually runs.
So in the end, I created a custom query (bit of a hacked way to do it, but it worked) like so:
Register the route:
wpapi.getNews = wpapi.registerRoute('wp/v2', '/news/(?P<customQuery>)');
Usage:
wpapi.getNews().customQuery('?paged&per_page=20&page='+pageNumber).then(data =>
Using the above, you can build any query, in any order you want. This helped me get the correctly paginated result. Also, we see 'getNews' here because I registered a route for accessing my custom post type called news.
I use this code to get RSS from stackoverflow.com
SyndicationFeed feed = SyndicationFeed.Load(XmlReader.Create("http://stackoverflow.com/feeds"));
foreach (SyndicationItem item in feed.Items)
{
Console.WriteLine(item.Title.Text);
Console.WriteLine(item.Title.Type);
Console.WriteLine(feed.Items.Count());
Debug.Print(item.Title.Text);
}
I get just 30 items but when I check in Google Reader I get more than this count.
Is there a limitation here?
30 is what stackoverflow returns, it is not a limitation of the SyndicationFeed class.
Google Reader stores old articles from RSS feeds. So we are limited to what the RSS feed contains, but Google has an archive that'll let you keep scrolling.
Two related questions:
Is there any good documentation on the Fusion Tables Javascript API? I've found a list of methods, but with little info on return values, semantics, or usage idioms.
Is there any guidance (or suggested plugins or idioms) for integrating the FT Javascript API into a locally hosted Wordpress site?
There is some documentation here:
https://developers.google.com/fusiontables/docs/v1/getting_started#JS
but I didn't find it very useful.
But this example, in the context of the Google Maps API I found very useful for the new API 1.0
https://googledrive.com/host/0B5KVZ6J1ohN_Q3ZqVkFGSGZ2cEE/custom%20markers%20code/customicons_viaApi.html
You'll need to view and save the source. Also if you search the FT tag for JSONP you will find many examples using the old pre 1.0 API but the concepts are the same, just the AJAX end point has changed and the need for an apiKey.
The basic idea is that any FT query will return a JSON object with both columns and rows members, very much like a CSV response.
As the example above shows:
function onDataFetched(data) {
var rows = data.rows;
var cols = data.cols;
...
}
I would like to write a scraping script to retrieve comments from cnn articles. For example, this article: http://www.cnn.com/2012/01/19/politics/gop-debate/index.html?hpt=hp_t1
I realize that cnn uses disqus for their comment discussion. As the comment loading is not webpage-based (ie, prev page, next page) and is dynamic (ie, need to click "load next 25"), I have no idea how to retrieve all the 5000+ comments for this article.
Any idea or suggestion?
Thanks so much!
I needed to get comments via scraping a page that had disqus comments via ajax. Because they were not rendered on the server, I had to call the disqus api. In the source code, you will need the identifier code:
var identifier = "456643" // take note of this from the page source
// this is the ident url query param in the following js request
also,look in the js source code to get the pages public key, and forum name. Place these in the url where appropriate.
I used javascript nodejs to test this, ie :
var request = require("request");
var publicKey = "pILMw27bsbJsdfsdQDh9Eh0MzAgFL6xx0hYdsdsdfaIfBHRvLGqFFQ09st";
var disqusUri = "https://disqus.com/api/3.0/threads/listPosts.json?&api_key=" + publicKey + "&thread:ident=456643&forum=nameOfForumFromSource";
request(disqusUri, function(res,status,err){
console.log(res.body);
if(err){
console.log("ERR: " + err);
}
});
The option for scraping (other then getting the page), which might be less robust (depends on you're needs) but will offer a solution for the problem you have, is to use some kind of wrapper around a full fledged web browser and literally code the usage pattern and extract the relevant data. Since you didn't mention which programming language you know, I'll give 3 examples: 1) Watir - ruby, 2) Watin - IE & Firefox via .net, 3) Selenium - IE via C#/Java/Perl/PHP/Ruby/Python
I'll provide a little example using Watin & C#:
IE browser = new IE();
browser.GoTo(YOUR CNN URL);
List visibleComments = Browser.List(Find.ById("dsq-comments"));
//do your scraping thing
Link moreComments = Browser.Link(Find.ByClass("dsq-paginate-append-text");
moreComments.click();
//wait util ajax ended by searching for some indicator
Browser.WaitUntilContainsText(SOME TEXT);
//do your scraping thing
Notice:
I'm not familiar with disqus, but it might be a better option to force all the comments to show by looping the Link & click parts of the code I posted until all the comments are visible and the scrape the List element dsq-comments
How to bulk edit tag of the google reader item ?
Now I'm using /reader/api/0/edit-tag to edit tags, but it's very slow to update tags for all items (in loop).
Dow you know any way to send tags for many items at once?
Possible solution looks like using some threads to send these requests to Google Reader Server.
You can include multiple i= and s= in the same post. just make sure that as you add a new i= you add the corresponding s= for that item even if you've already included the s= for that exact same stream previously (this is really important, or you'll get a 400 error when making the call). I was doing batches of 10 with my code, I'm sure you can do more but I don't know the limit.
Could a working URL for marking all items as read be posted?
I have tried:
<?php echo 'http://www.google.com/reader/api/0/edit-tag?'.'s=feed%2F'.urlencode('http://feeds.feedburner.com/filehippo').'&i='.urlencode('tag:google.com,2005:reader/item/c7701cf414f3539e').'&a=user%2F-%2Flabel%2Fread'.'&T=bbi44C5CQzjzM43yKUPwnA'; ?>
but I just keep getting a 400 error back.
Many thanks.