How do I navigate inside a web page when web scraping? - web-scraping

I need to scrape data from this webpage - https://www.goodreads.com/book/show/693771.Ignited_Minds.
I am using beautifulsoup, how do I get the comments from the second comments section inside the comment box?

Related

Wordpress blog snippet on HTML web page

Trying to figure out if there is a way to take a snippet of a wordpress blog, same site, same hosting, and plug that into a pure HTML page.
Example... I want to show the title of the blog, and first 150 characters. With a link to ..."read more" which would take the visitor to the actual blog.
Widget? or custom type of coding?
One way that WordPress exposes blog output to other applications is through the RSS feed.
It is often available by URL at
"http:// your-Wordpress-Root-URL.com/ feed".
You can use that URL in a 3rd-party widget that you use via HTML/JS on any HTML page.
They'll give you the code snippet to drop in your page.
And they'll have different options to customize things like the number of blog entries to show, and the length to preview - as you mentioned.
I'll paste a couple links to start with, and you can easily search for more online with something like "RSS feed widget" in google.
https://www.rssinclude.com
http://www.webrss.com/
http://www.rssfeedwidget.com/

ASP.Net webform/mvc3 - add link to each page of a multi-page pdf document

My upcoming mobile web application project would include linking from a web page to each page of a multi-page pdf document.
There will be a menu on the left with all the page titles, and when the user clicks on one, it will take them to that page inside the pdf document.
Is it possible to link the individual pages this way?
The pdf will be dynamically chosen from a list of pdfs.
Thanks in advance and any help is appreciated.
You can open PDF at a specific page using URLs like the following
http://www.example.com/my.pdf#page=3
You should somehow find out titles for pages, though.

Like button on each page

I have a like button on my main page (master page in asp.net).
I added a like button on each dynamic page.
When I try to enter the link:
http://www.friend.co.il/details.aspx?id=2321
to one of the dynamic page in face book developer page:
http://developers.facebook.com/docs/reference/plugins/like/
I get the same number of like as the total site number and not just a number for the specific page.
Any body know whats I'm doing wrong?
Using Facebook's URL Linter on your dynamic page's URL shows that it has Open Graph tags that refer to your website's main page. As a result, Facebook thinks that the canonical URL for http://www.friend.co.il/details.aspx?id=2321 is http://www.friend.co.il/.
You should either remove the meta tags from your subpages, or construct them so that they properly represent the content on each page.
Remove your og:url meta tag and also be sure to not include the Facebook javascript file multiple times like you are currently doing.

Changing open graph image tag based on blog post item

I am maintaining a wordpress blog and am familiar with using the og:image meta tag to set the default image that is included when a like button is pressed. What I want to know is if there is a way to make this image dynamic based on the item liked.
I am trying to build it so that the first image in a particular blog post that is liked is what is pushed to the facebook feed post, with the blog's default image only used if there is no image in the post. Does facebook provide any way to do this with the graph api?
I tried writing javascript to change the meta tag on article pages (rather than the homepage) but found out later the crawler does not run javascript, so am now considering implementing the same but server side with PHP. This still would not solve the problem of Likes from the blog homepage, where there are multiple blog items on the page though. Any advice would be greatly appreciated.
Thanks
The only is changing how the meta tags get rendered to the client, in server code or in your layout template. There is no graph api method for this and you can't modify the tags with javascript as their scraper doesn't execute javascript.
Also, the scraper won't keep re-visiting your site so once it access it, it will be cached until you go to the url linter tool manually. so for the home page, you would probably just want the default image.

Edit Sharepoint shared web part CSS for extra actions

Currently I have a web part with some custom CSS. This is a Document Library web part which also contains the link to "Add a new document" below.
Once I click Add new document I am taken to the upload document page. Is there any way to have the upload document section displayed on the same page or is there an easy way to custom style the upload document page for just this library only?
Well, the upload page is also a web part page which hosts a list form web part. You can add the web part to any page and add custom CSS to that page, even using Content Editor WP. The problem is that the Add new document link always points to the page that is designated as the New form page of the list, you can change this setting using SharePoint Designer in 2007 (it's even simpler in 2010).

Resources