How do I generate Eleventy and display tags from a localized version of a collection file? - collections

I'm new to Eleventy and working on a personal project - most of it is working fine, but I need help with generating and filtering tags based on posts stored in separate folders (as part of internationalizing/localizing my site).
I have two folders: en and ff. Inside each folder, I have a folder called posts - this contains a handful of posts.
I'm trying to work out a way to create a set of tags from just the en/posts folder or the fr/posts folder - each would be in a separate collection, not as one global collection.
I currently have this to generate an array of tags across all pages:
eleventyConfig.addFilter("filterTagList", filterTagList);
// Create an array of all tags
eleventyConfig.addCollection("tagList", function (collection) {
let tagSet = new Set();
collection.getAll().forEach(item => {
(item.data.tags || []).forEach(tag => tagSet.add(tag));
});
return filterTagList([...tagSet]);
});
(filterTagList is a separate filter that excludes certain redundant tags)
I can create a collection based on posts within the en/posts folder called posts_en, using this:
eleventyConfig.addCollection("posts_en", function (collection) {
return collection.getFilteredByGlob("./en/posts/*.md");
});
...which works fine to display English posts. I've also been able to create a collection that picks up posts in the en/posts folder, if I add posts_en as a tag to the post:
eleventyConfig.addCollection("tagGroup", function (collection) {
console.log(">>>> TAGGROUP:", collection.getFilteredByTags("posts_en"));
return collection.getFilteredByTags("posts_en");
});
(I've checked the console.log statement in this code extract - it renders all of the posts I tag with posts_en, but I'm not sure if this will keep the other tags as well? The collection needs to only contain English posts with their appropriate tags, but no French content (and vice-versa, for the FR content).
My problem is that I've been unable to adapt the original tagList method only to contain posts in the en folder (or the fr folder).
I've tried changing tagList to tagList_en, and adapting collection.getAll() to only reference the posts_en collection - this compiled, but gave zero results on my Tags page. I've scoured across SO and various posts on the Web, but so far, no joy - can anyone help advise, please? I'm happy to duplicate the solution from English to a French equivalent - as long as I can get something working for English content!

If your posts_en collection works as expected, then it should be possible to adapt the tagList collection to only show tags from English posts by replacing collection.getAll() with collection.getFilteredByGlob("./en/posts/*.md").
// Create an array of all tags
eleventyConfig.addCollection("tagList", function (collection) {
let tagSet = new Set();
collection.getFilteredByGlob("./en/posts/*.md").forEach(item => {
(item.data.tags || []).forEach(tag => tagSet.add(tag));
});
return filterTagList([...tagSet]);
});
This will set collections.tagList to an array of strings (the tags from English posts).

Related

Specific filter for URL popover in post editor

I'm using WP 5.3 with the default (Gutenberg) editor along with the Polylang 2.7.2 plugin for making the site multilingual. Using Polylang, each post consists of one translation post per language (Polylang groups translation posts together).
Now I have the following problem: When a user is writing a post and tries to link to another already existing post, the search drop down for linking text (Ctrl+K) shows posts in all available languages. If the post title isn't language-specific (e.g., "Smart Home") but the "Smart Home" post exists in two languages, it's trial and error for the user to select the one matching the language of the currently edited post:
I wanted this URL popover drop down to either only list posts of the same language as the current post, or modifying the results in the drop down to show the specific language (by flag, or "[en]" before the title).
First, I tried using the admin menu bar "Filter content by language" drop down to limit it to English posts - didn't affect the drop down at all.
Next, I tried using a filter to tamper with the query results for this drop down. Using pre_get_posts and setting the language hard-coded to English worked:
add_filter('pre_get_posts', [self::class, 'filterQueryLanguage']);
}
public static function filterQueryLanguage($query) {
$query->set('lang', 'en'); // this limits the drop down results to English posts
return $query;
}
The search drop down then only listed English posts. Unfortunately, this limits all queries and it's impossible in the filter function to know for sure if this query originated from an AJAX request by this popover drop down.. also, I didn't manage to detect the language of the currently edited post (pll_current_language() returned false in this case).
Therefore, I need a way to post-filter the results only for this popover drop down and prepend the language to the post name in some way. But I have no idea if there even is a filter for this. get_posts doesn't seem to fire and even if it did, I don't want to affect all queries.
Are there any more specific filters for this purpose?
Okay, found a solution myself. The problem is: this URL popover does a REST API query for search=phrase. This search query also has a lang= argument that is already set to the appropriate post language - however, when assembling the REST response this lang argument is ignored.
I post-filtered the REST result like so:
add_filter('rest_pre_echo_response', [self::class, 'filterRESTResponse'], 10, 3);
}
public static function filterRESTResponse($result, $server, $request) {
$params = $request->get_params();
if (!empty($params['search']) && !empty($params['lang'])) {
$filtered = [];
$lang = $params['lang'];
foreach ($result as $post) {
$post_lang = pll_get_post_language($post['id']);
if ($post_lang === $lang) {
$filtered []= $post;
}
}
$result = $filtered;
}
return $result;
}
This solution is a bit awkward as it might've been possible to make the REST query respect the language from the start. Not sure how to hook into that one, tho.
Well, whatever works.

How can I use the Envato Market API to export a list of WordPress plugins?

I'd like to use the Envato API to download a list of plugins for WordPress available on Code Canyon. However I have been unable to find a way to do so.
For example, I tried to use a get /search/item per their documentation using the parameter category and setting it to "wordpress" however this returned a number of results but nowhere near the 6,040 the site says it has.
The document also mentions the "category code" is what I should be entering as the parameter for "category" but it never defines the category code - unfortunately, this seems common throughout the documentation - there isn't any definition. Another example of this is calling get /catalog/collection. The parameter required is "id" which it describes as "The numeric ID of the collection to return" - but what numeric ID? This one wasn't hard to figure out, if you open a collection the url looks like:
https://codecanyon.net/collections/4945814-about
And the numeric portion is the ID...but I sure could wish for more definitions or examples of what the parameters should look like. :-)
I looked around, but didn't find anything helpful on the web nor does there appear to be a forum hosted by Envato for discussing the API.
Any help is appreciated!
Please find the code which I did to fetch data from envato API :
var themeforest_api="http://marketplace.envato.com/api/v2/new-files:themeforest,wordpress.json";
$.getJSON( themeforest_api, {
format: "json"
}).done(function( data ) {
var html='';
$.each( data['new-files'], function( i, item ) {
html=html+'<li><img src="'+item.thumbnail+'"></li>';
if ( i === 8 ) {
return false;
}
});
$("#all_items").append( html );
});
Hope it will help you for your one too.

Drupal: How to restrict apachesolr search results by user/article facets

I have a wiki built with drupal, with a taxonomy category Workgroup, assigned to both the users and the articles. I am using apache solr search module with facet api and my end goal is to set up the search so that by default when users search for the articles, only articles from their workgroup are shown.
That is, when they launch the search from a search box, they should get the same results as for /search/site/hello?f[0]=im_field_kb_workgroups%3A4529 (where 4529 is one workgroup id) instead of just /search/site/hello (current behavior) Users should still be allowed to search in other workgroup facets when they want, by removing the checkbox in the facet block.
I have this working almost by hacking the apachesolr module (not recommended I know but really want this to work). In the function apachesolr_search_custom_page_search_form_submit, I have:
// Get the workgroup id
global $user;
$account = user_load($user->uid);
$user_kb_wg_fieldinfo = field_get_items('user', $account, 'field_kb_workgroups');
$user_kb_wg_tid= '';
if ($user_kb_wg_fieldinfo) {
$user_kb_wg_tid = $user_kb_wg_fieldinfo[0]['tid'];
}
// Add the solr filter for this workgroup facet so that by default, search results are
// fetched only from that workgroup.
if ($user_kb_wg_tid === '4529') {
$get['f[0]'] = 'im_field_kb_workgroups:4529';
}
This does the job but the problem is that this relies on the apachesolr search form. I have users coming to the wiki by searching from sites external to the wiki, where the wiki search form is just a simple POST form pointing to the KB domain and the path /search. So this will work only when people are searching from inside the wiki, where I present them the apachesolr search form.
I have investigated some other options:
In my custom module, I implement this hook (without the user workgroup checks for now, for testing):
function kb_hacks_apachesolr_query_prepare($query) {
$query->addFilter('im_field_kb_workgroups', '4529');
}
This filters the results from searches everywhere, but the filter is applied all the time, and users don't get to deselect this or other filters. (in fact, other filters appear only when passing the filter as a GET param like above with f[0])
I also played with the url_inbound_alter hook, but could not figure out how to pass the solr query param as GET. The following did not work.
function kb_hacks_url_inbound_alter(&$path, $original_path, $path_language) {
if ($path == 'search/site/hello') {
$_GET['f[0]'] = "im_field_kb_workgroups:4529";
//$_GET['f[0]'] = "im_field_kb_workgroups%3A4529";
//$path = 'search/site/hello?f[0]=im_field_kb_workgroups%3A4529;
}
}
Is there a way to set GET params from this hook? But even if this had worked, I would still have to figure out how to do this only by default (when the search form is submitted), and not when the filter itself is deselected. Is there a way to detect checkbox changes in the facet block?
Maybe there's another way to do this? I have been stuck here for the last two days and would really appreciate any help I can get. Thanks!
You can add a relationship to the taxonomy term "Workgroup" and use a contextual filter for the current user. In the contextual filters section, you can change the behavior when the filter is not present.

Scraping product information for comic book website

I'm making a comic book website built upon a WordPress platform for an old friend for his business. I would like to be able to have a script that goes to various publisher sites and pulls in the data. I'm new to programming and I've read of many different alternatives and just don't know where to begin. Firstly, would this be legal to pull this content from these websites? Secondly, here's an example for what I would like to do.
Page displays what's coming out for the month. Copy all links from
that page within the appropriate div that leads to the comic book
details. Save each hyperlink as $comiclink or whatever. The script will
execute each hyperlink at a time.
Go to the hyperlink for $comiclink and scrape content out of the page based
upon what's in certain DIV's on that page. Example:
Copy & save comic title within a defined div into $title
Copy & save previous and future title hyperlinks within a defined div into $othertitles
Note: $othertitles will loop off and start the same process itself from 1.
Save & download all images within a defined div to $images
Copy & save all content within a defined div to $content. $content is then broken down
and pulled apart based upon the content that is within it. Example:
In stores: $date
format: $format
UPC: $upc
Price: $price
The Story: $story
Copy & save defined div hyperlink and save into $seriesinfo
Copy & save defined div $relatedinfo and then break it down.
images within $relatedinfo to $relatedimages
content within $relatedinfo to $relatedcontent
links within $relatedinfo to $relatedlink. $relatedlink will loop off and restart this process itself from 1.
Now that everything is broken apart and saved into it's own little pieces. I want WordPress to automatically create a post and then start assigning all this info into the post. Working something like this.
Check for existing post with same $title if does not exist place $title in title for post and page-name. If it exists abort script and move on to the next.
Remove numbers and alpha characters from $title and check for existence of category if it does not exist; create it and assign to post. If it exists assigns category to the post.
Check for existing category with value $format if exists assign to post, if not create & assign category to post.
upload images that were downloaded from $image into this post.
Check for images that contain the word "cover" and assign as featured image.
Also how this whole thing executes also. I don't want this running 24/7 - just once a week I would like this to execute by itself and automatically go to the websites in question and scrape the content and create the pages.
I'm not asking you guys to write out the whole darn thing for me; though I definitely won't object to it! Just help point me in the right directions to get this going. Over the past day I've read probably 30+ articles on pulling content and there's so many options from what I can tell that I just don't know where to begin or how to get the ball moving in the right direction with this.
Updated Code
Notes: So I've managed to successfully copy the content and paths for each block and instead of downloading the images just echoing them from their present location. Next up is actually automating the process to create a post in wordpress to dump the data into.
function scraping_comic()
{
// create HTML DOM
$html = file_get_html('http://page-on-site-to-scrape.com');
// get block to scrape
foreach($html->find('li.browse_result') as $article)
{
// get title from block
$item['title'] = trim($article->find('h4', 0)->find('span',0)->plaintext);
// get title url from block
$item['title_url'] = trim($article->find('h4', 0)->find('a.grid-hidden',0)->href);
// get image from block
$item['image_url'] = trim($article->find('img.main_thumb',0)->src);
// get details from block
$item['details'] = trim($article->find('p.browse_result_description_release', 0)->plaintext);
// get sale info from block
$item['on_sale'] = trim($article->find('.browse_comics_release_dates', 0)->plaintext);
$ret[] = $item;
}
// clean up memory
$html->clear();
unset($html);
return $ret;
}
// ===== The Code ====
$ret = scraping_comic();
if ( ! empty($ret))
{
// place main url for instance when hyperlinks and image srcs don't use the full path.
$scrape = 'http://site-to-scrape.com';
foreach($ret as $v)
{
echo '<p>'.$v['title'].'</p>';
echo '<p><img src="'.$v['image_url'].'"></p>';
echo '<p>'.$v['details'].'</p>';
echo '<p> '.$v['on_sale'].'</p>';
}
}
else { echo 'Could not scrape page!'; }
?>
Typically, no this wouldn't be legal. Companies that share their data these days will implement an API you can call and use in your application (subject to their Terms of Use and Copyright Policy). They don't like you making automated requests that bog down their server and kill their bandwidth.
That being said, often times product information is available from other sources such as Amazon which does have an API.
This project you are describing has a lot of work to be done essentially customizing the WordPress CMS and would be less than trivial for someone without any programming experience. You might want to consider hiring a freelancer at oDesk or one of the many other freelance job boards.

Re-processing attached images in drupal 7

I'm trying to import nodes from my forum to drupal 7. Not in bulk, but one by one so that news posts can be created and referenced back to the forum. The kicker is that I'm wanting to bring image attachments across as well...
So far, using the code example here http://drupal.org/node/889058#comment-3709802 things mostly work: Nodes are created, but the images don't go through any validation or processing.
I'd like the attached images to be validated against the rules defined in the content type. in particular the style associated with my image field which resizes them to 600x600.
So, instead of simply creating the nodes programatically with my own form, i decided to modify a "new" node using hook_node_prepare and using the existing form to create new content (based on passed in url args). This works really well and a create form is presented pre-filled with all my data. including the image! very cute.
I expected that i could then hit preview or save and all the validation and resizing would happen to my image, but instead i get the error:
"The file used in the Image field may not be referenced."
The reason for this is that my file doesn't have an entry in the file_usage table.. *le sigh*
so, how do i get to all the nice validation and processing which happens when i manually choose a file to upload? like resizing, an entry in the file_usage table.
The ajax upload function does it, but i can't find the code which is called to do this anywhere in the api.
What file upload / validation functions does Drupal call which i'm not doing?
Anybody have any experience with the file/image api for Drupal 7 who can help me out?
For getting the usage entry (in essence, checking out a file to a specific module so that it doesn't get deleted while its in use) look up the Drupal function 'file_usage_add()'
For validating incoming images, I got this example from user.module (if you're comfortable with PHP, you can always look at the core to see how something is done the 'Drupal way'):
function user_validate_picture(&$form, &$form_state) {
// If required, validate the uploaded picture.
$validators = array(
'file_validate_is_image' => array(),
'file_validate_image_resolution' => array(variable_get('user_picture_dimensions', '85x85')),
'file_validate_size' => array(variable_get('user_picture_file_size', '30') * 1024),
);
// Save the file as a temporary file.
$file = file_save_upload('picture_upload', $validators);
if ($file === FALSE) {
form_set_error('picture_upload', t("Failed to upload the picture image; the %directory directory doesn't exist or is not writable.", array('%directory' => variable_get('user_picture_path', 'pictures'))));
}
elseif ($file !== NULL) {
$form_state['values']['picture_upload'] = $file;
}
}
That function is added to the $form['#validate'] array like so:
$form['#validate'][] = 'user_validate_picture'

Resources