How to solve PageErrors when using pyppeteer for web scraping? - web-scraping

I'm trying to scrape data from this site https://b2b.10086.cn/b2b/main/listVendorNotice.html?noticeType=2 using pyppeteer. I wrote a for-loop running through each page and used "await page.click" to change pages. However, the script could only work for a few pages and then encountered "pyppeteer.errors.PageError: No node found for selector". Does anyone have any ideas on how to solve this problem? I have tried adding "await asyncio.sleep(1)" and user agent information, which did not work.
Thank you for your help in advance!

Related

fail to crawl a dynamic loading web page using scrapy-splash

Greetings. I am new to scrapy-splash.
i try to extract car information from following webpage by using scrapy-splash
https://www.lexusserramonte.com/new-inventory/index.htm
i tried a lot, just does not work. Make no difference compared to just using scrapy without splash plugin.
Just wondering anyone can extract information from that URL? if you can, please share your code.
much much appreciate
You don't need to use splash for this task.. all you need is to reverse engineer API calls.
All the needed info on cars can be found by this link https://www.lexusserramonte.com/apis/widget/INVENTORY_LISTING_DEFAULT_AUTO_NEW:inventory-data-bus1/getInventory?compositeType=new

How to Remove Malicious Code from WordPresss Website?

I have created my website by myself. Recently, I found malicious code is appearing in my website as
" script type="text/javascript" src="http://12js.org/jquery.js">
Sometimes, this code is appeared in source code. Another time, it is disappeared. It is like a re-infection.
I cannot find this code & I need to solve this issue. Anyone here to help me??
I am suffering with this problem.
You need to search your theme and plugin code files.
Look for the below code WP function malicious code

Drupal 7 - Server Error 500 when adding a Filter to views

For some reason when I try adding a filter to any view that I have, I receive the Server Error 500.
In Google Chrome I get
this.
In Firefox I get this this.
To be detailed, I have a view that currently displays published articles. All Articles are translated into Spanish and English. I am trying to add the filter: Content:language and set it to user current language.
When I save, the page refreshes and the temporary changes notice is shown. I try saving a second time and the Server Error comes up. This happens for both web browsers.
Could it be a that I have to enable a read/write for a certain file? If so, I don't know which file to look for.
Can anyone shed some light on this for me ? I have been at it for about 13 hours...
After taking the advice from #Rainfall in the comment, I found another post on Drupal's forum with the solution/ patch for this issue.
Here is the post with the solution
Comment #9 gave a patch that worked for others and it worked for me too. More in depth info on the issue is there as well.

Drupal Computed Field with Views

I have some Computed Fields and in these fields, I get some HTML codes with simple_html_dom.php class from some sites. Codes works perfectly on pages, but when I try to get these fields to use in a View, Drupal gives the following error.
http://i.stack.imgur.com/W2Rhc.jpg
Anybody can help?
Even if it is not the best solution, after installing Views PHP module of Drupal, problem is slightly solved. I still get the error message on every time I try to change something, also I am having difficulties on making a change on the view, like saving and adding the same thing 5 times and got successfull, still it is a solution.
This message usually appears when you try to save the form before the ajax process has finished.
Try checking the 'network' tab on developer tools installed within Google Chrome or by using a tool similar to firebug. This will show you the request being made via ajax and should give you more information to debug.

How do I view wordpress themes without logging in?

Many times I want to show a client how their site is progressing or show someone my current work on a theme to get feedback. I would like a way to send a link possibly with ?theme=.... in the URI that doesn't require a log in.
Currently I have to log in to show the client or others to show progress and get feedback.
Any known ways of doing this?
This could solve you problem.
Works with a sidebar which you could set temperarly
http://wordpress.org/extend/plugins/theme-switcher/
this one works with ?preview_theme=my-theme
http://wordpress.org/extend/plugins/theme-preview/

Resources