Let's take for example the following rss-link:
http://yourdailygerman.wordpress.com/feed/
As you can see, the RSS publishs only 1 item at a time.
However, RSS Readers services like Feedly (and more), success to retrive more items:
http://feedly.com/index.html#subscription%2Ffeed%2Fhttp%3A%2F%2Fyourdailygerman.wordpress.com%2Ffeed%2F
I opened fiddler and I saw they have their own API (following "count" paramter as the items count to get), and I wonder - How they do that???
Thanks!
You can't get more item from a feed if it's the first time you fetch it... however (and this is what Feedly does!), you can store the content of today, so that when new items are added tomorrow, you have the items from today and tomorrow... etc. Eventually you'll have enough data!
Related
I am trying to retrieve all categories using Woocommerce rest API.
here is my URL
https://wapi.sriyagcommerce.com//wp-json/wc/v2/products/categories?consumer_key=ck&consumer_secret=cs&per_page=100
while using above url I am getting Error.
code "rest_invalid_param"
message "Invalid parameter(s): per_page"
data
status 400
params
per_page "per_page must be between 1 (inclusive) and 100 (inclusive)"
I want to increase per page limit. I already have done with products by using below code and want to do the same with categories
function maximum_api_filter($query_params) {
$query_params['per_page']["maximum"]=100000;
return $query_params;
}
add_filter('rest_product_collection_params', 'maximum_api_filter');
I must've commented very early in the morning
There is NO Way of increasing the size per page past a 100 Its a limit set by the API team you cannot exceed a 100.
Now I do not know what Your using to develop, But i'd consider using a Collection and just pull the first 100, Index the Page pull a 100 Combine Collections and so on.
But as for Getting all 300 in one instance from one API call that cannot be done sadly.
I also face the same problem.
You couldn't increase the list size. Maximum limit in Woocommerce is 100.
You can follow bellow URL pattern to fetch 100 records,
https://www.yourcompany.com/wp-json/wc/v2/products?per_page=100
You can also specify pages with the ?page parameter
https://www.yourcompany.com/wp-json/wc/v2/products?per_page=100&page=2
You can find total number of pages and records from
the X-WP-TotalPages and X-WP-Total HTTP headers.
After getting total page from X-WP-TotalPages you can execute a loop with per_page=100&page=${pageNumber} and increase pageNumber dynamically.
By this approach you can find all records.
We all know microscope from discovery meteor. The app is fine, it operates only with the number of upvotes. In the Best page it sorts posts by upvotes in descending order, upvote number is stored for each post and gets updated each time some user upvotes a post.
Imagine now we want to implement something like hacker news have - not only a click-based rating, but also a time-based rating. Lets now define that I will use word 'click' to describe an user action of clicking on post in the post list. This 'click' increases total number of clicks of this post by 1.
For thouse who do not knowe how hacker news algorithm work I will briefly explain. In common the total number of clicks of certain link (post) is divided by:
(T+2)^g
where T - total number of hours passed since post publishing time and now, and g is a "sensitivity" thing, lets call it that, which is just a number, 1.6, or 1.8, doesn't matter. This decrease influence of clicks as the time goes by. You can read more info (http://amix.dk/blog/post/19574)[here], for example.
Now, we want to have top-50 click&time-rated posts, so we need to query mongo to find all posts, sorted by score, calculated with formula from above.
I can see two major approaches to do so, and I find all of them quite bad.
First one, (the way I do now) subscribe to all posts, in template hepler prepare data for rendering by
rankedPosts: function() {
rawPosts = posts.find().map( function(item) { item.score = clicks/(T+2)^g; } ); // to add score for each post
rawPosts = _.sortBy( rawPosts, function(item) { return item.score*(-1); }) // to sort them by calculated score
rawPosts = _.first( rawPosts, 50 ); // to get only first 50
}
and then use rankedPosts for rendering. The bottleneck here is that each time I have to run through all posts.
Second one - somehow (I do not know how, or if it even possible) to subscribe for already scored/sorted/filtered collection, assuming meteor/mongodb can apply their magic to score/sort/filter (and recalculate score each new hour or new click) for me.
Now, obvious question, what will you recommend?
Thanks in advance.
Think about numbers. In a working page, you can have thousands of mosts, millions if the page is successful. Fetching all of them just to find the top 50 doesn't make sense.
I'd recommend storing the final calculated rating in a field. Then in subscription you apply sort by that field and desired limit. When post gain a new click, you simply recalculate the value and save it to db. Finally, in a cron job or meteor interval you update the rating of all items in the database.
Front end application will use firebase.com as database.
Application should work like blog:
Admin add / remove posts.
There are "home" page which show several articles per page with "Next" and "Prev" buttons and page which show single article.
Also I need deep linking.
I.e. when user enter URL http://mysite.com/page/2/ I need that application show last articles from 10 to 20, when user enter URL http://mysite.com/page/20/ application show last articles from 200 to 210. Usual pagination.
Main question - is it possible to achieve this if use firebase.com.
I read docs at firebase.com, I read posts here. "offset()" and "count()" will be in the future, use priority, in order to know count of items and not load all of them use additional counter.
Based on this I suppose this:
ADD POST action:
get value of posts counter
set priority for new post data equals to value of posts counter
increase value of posts counter
DELETE POST action
get value of priority for post data
query posts data which have value of priority more than priority for post data which will be deleted and decrease their value
decrease value for posts counter
GET POSTS FROM x TO y
postsRef.startAt(null, x).endAt(null, y).on(... and so on)
Thing which I not like in this way are actions for DELETE POST. Because index of posts will be from 0 to infinity and post with less priority is considered as post which was created earlier so if I have 100000000 posts and if I need to delete 1st post then I need to load data for 99999999 next posts and update their priority (index).
Is there any another way?
Thanks in advance, Konstantin
As you mentioned, offset() will be coming, which makes your job easier.
In the meantime, rather than using startAt and endAt in combination, you could use startAt and limit in combination to ensure you always get N results. That way, deleted items will be skipped. You then need to check the last id of each page before moving to the next to find the proper id to start on.
I would like to have a simple linklist of blogs with their latest entry, author, blog name, and date be listed in order of latest update. So, kinda like what Blogger has on their sidebars -- whenever they have linklists, it auto-reorders based on which blog has been updated recently (though it usually doesn't have summaries, just the title of the site).
What I'd like to do is plug in multiple RSS feeds into one place that just checks the latest entry in each feed and then reorders them
The magpie plugin or RSS plugin in EE currently only allows you plug in ONE feed. It will show the latest entry per blog if I repeat the magpie tags several times (one per blog), but that doesn't help with re-ordering, and also it's a bit messy.
Help? Thoughts?
UPDATE: I want to make sure that each blog is still featured even if the last post they've done is a year ago. A blog like that will be the last in the link list for example, while an entry/site updated this hour will be at the top. So for example, if I limit the linklist to 5, it's not going to show the 5 most recent entries, period, from just one site if that one site updated 5 times in one hour. It will show 1 recent entry per site.
Sample behaviour: like the Flickr Friends Page sorted by date and 1 photo per person. http://www.flickr.com/photos/friends/ Except this time, just titles of blog posts and the site they belong to, author, etc.
There is an RSS Aggregator EE Add-on available which combines feeds for you
http://devot-ee.com/add-ons/rss-aggregator
{exp:rss_aggregator:items feed_ids="6|7" limit="10" order_by="pub_date" order="desc" pub_date_format="m-d-Y g:ia" cache_timeout="60"}
<li>
{title}<br />
<small>{pub_date} - From {channel_title}</small><br />
</li>
{/exp:rss_aggregator:items}
You could do this using Yahoo! Pipes. In fact, I created one a bit ago: http://pipes.yahoo.com/pipes/pipe.info?_id=82e8b23b84bf2d6c3014f50c0f6b2f1d
Just call it with a comma-delimited list of RSS feeds and voila! It gives you the latest entry for all the RSS feeds, sorted in reverse chronological order.
Example:
http://pipes.yahoo.com/pipes/pipe.run?_id=82e8b23b84bf2d6c3014f50c0f6b2f1d&_render=json&feedurls=appleinsider.com.feedsportal.com/c/33975/f/616168/index.rss,feeds.foxnews.com/foxnews/latest?format=xml
So now that you have the feed's URL, you can pull out the individual entries using the REST module from Phil Sturgeon.
For those interested, here's how the pipe is setup:
Fairly sure Stash could handle this via set_list and get_list. I haven't tested this but perhaps something like:
{exp:channel:entries channel="feed_urls"}
{exp:magpie url="{url}" limit="1"}
{items}
{exp:stash:append_list name="feed_items"}
{stash:title}{title}{/stash:title}
{stash:pubdate}{pubdate}{/stash:pubdate}
{/exp:stash:append_list}
{/items}
{/exp:magpie}
{/exp:channel:entries}
{exp:stash:get_list name="feed_items" orderby="pubdate"}
<h3>{title}</h3>
<p>{pubdate}</p>
{/exp:stash:get_list}
https://github.com/croxton/Stash
What about using an entries loop with multiple channels, pulling lastest from that and stashing it. Then get the stash in the RSS feed? I think that would work.
You could probably do it with Datagrab because it lets you import XML feeds into your own site and then you'd be able to order them as EE channel entries.
I'm using YQL to retrieve several RSS feeds (channels) at once, using the following query:
SELECT * FROM feednormalizer
WHERE output="rss_2.0"
AND url in ("http://rss.cnn.com/rss/edition.rss",
"http://bad.url.com/nothing",
"http://rss.cnn.com/rss/edition_space.rss")
so i actually have 2 questions:
How can i limit each feed to a certain amount of items? (for example, I want to get top 3 items from each channel)
Notice that the 2nd URL is invalid (not a URL of a real RSS). in that case, the YQL result returns 2 valid channels plus an error, but there is no indication of which URLs are valid and which one failed.
in other words - for each result feed, there is no indication from which URL it arrived.
any ideas as to how identify each channel?
thanks
1st question:
the first thought that comes to mind is using query.multi:
SELECT rss.channel.item FROM query.multi WHERE queries="
SELECT channel.item FROM feednormalizer WHERE output='rss_2.0' AND url = 'http://rss.cnn.com/rss/edition.rss' LIMIT 3;
SELECT channel.item FROM feednormalizer WHERE output='rss_2.0' AND url = 'http://rss.cnn.com/rss/edition_space.rss' LIMIT 3;"
Not the most elegant way but it works.
2nd question:
I don't think it is possible. As YQL's WHERE x IN ()... syntax is not really an SQL join, it is not possible to select parts of the inner subquery in the projection, what you would need for your case.
Would be happy if anybody would proove me wrong on this one, as I needed that a couple of times myself and always had to work around it programatically :)