Im trying to make a webscraper, currently im able to pull all the data like price and ram from a laptop. but everything I try to get the main large photo on the website pulls up with nothing, currently I am able to pull the low res photos, but i was hoping for the better quality one
https://www.bestbuy.com/site/asus-11-6-chromebook-intel-celeron-4gb-memory-32gb-emmc-flash-memory-gray-grey/6449513.p?skuId=6449513
I was using this one for example. And i only need the way to pull the url from each scrape not the photo itself. thank you!
try:
=INDEX(IMAGE(REGEXEXTRACT(QUERY(FLATTEN(IMPORTDATA(A1)),
"where Col1 starts with 'thumbnailUrl:'
and Col1 ends with '}'"), """(.+)"""), 3))
change 3 to 1 if you don't need such a big image (just to fit your cell)
add limit 1 to query if you need only the main image
Related
I have a situation where I need to extract tables from 13 different links, which have the same structure, and then append them into only one table with all the data. This way, at first I extracted the links from a home page by copying the link from the respective hyperlink, and then import the data through the Web connector on Power BI. However, 3 months later, I realized that those links changed every quarter but the link from the homepage where they are listed remain the same.
This way, I did some research and I found out this video on YouTube (https://www.youtube.com/watch?v=oxglJL0VWOI), which explained how I can scrape the links from a website, by building a table with the header of the link as a column and the respective link as another column. This way, I can have the links automatically updated, whenever I refresh the data.
The thing is that I am having issues to figure out how can I use this links to extract the data automatically without having to copy them one by one and then import the data using the Power BI Web connector (Web.BrowserContents). Does anyone can give me a hint of how can I implement this?
Thanks in advance!
I'd like to scrape data from the following website: http://maps2.roktech.net/durhamnc_gomaps4/
In a separate spreadsheet on my computer, I have a list of parcel IDs, corresponding to various properties in the county.
Here's what needs to happen:
1. First, copy and paste parcel ID (from a separate spreadsheet) into the search box, to search by parcel.
2. Then, copy and paste all the columns of data that show up associated with that parcel ID, and paste it into the spreadsheet.
And that's it! it sounds pretty simple, but I can't seem to figure it out. I've tried using UI path but I'm not experienced with the software.
How could I go about doing this? How difficult is this to do?
Thanks so much for any help or assistance.
Ryan
Please watch the following training video on how to scrape data using UIPATH.
https://www.tutorialspoint.com/uipath/uipath_studio_data_scraping_and_screen_scraping.htm
It is highly recommended to go through the free UiPath RPA Academy training videos that will quickly put you in the know
https://academy.uipath.com/learn/course/internal/view/elearning/372/level-1-foundation-training-revamped-20182
You don't need uipath for this job
Go to the site
zoom out
click "select options"
select the whole area
5.A table with "1000 results" will appear on the same line where "1000 results" is written there are 4 buttons the last one is "export to EXCEL" - click it and you will have the whole data in one table and then you can filter this table
I would like to make a form that users would fill over time. For example one day they would answer question 1, next day question 2, perhaps week later questions 3 and 4 or someone maybe wants to fill the whole form at once. In the form would be fields for text and images.
I hoped that I would accomplish this by using pdf form, but this solution lets Acrobat Reader users upload only pdf files. I need the process to be as easy ad possible for the users, so they should be able to upload jpg images. Otherwise this solution would be optimal.
I really hope that the solution would lie in a file, which user would download and send back when finished. I'm using Wordpress site, so I'm open for online suggestions also.
I have a list from approx. 36,000 URLs, ranging from https://www.fff.fr/la-vie-des-clubs/1/infos-cles to https://www.fff.fr/la-vie-des-clubs/36179/infos-cles (a few of those pages return 404 erros).
Each of those pages contains a number (the number of teams the soccer club contains). In the HTML file, the number appears as <p class="number">5</p>.
Is there a reasonably simple way to compile an excel or csv file with the URL and the associated number of teams as a field ?
I've tried looking into phantomJS but my method took 10 seconds to open a single webpage and I don't really want to spend 100 hours doing this. I was not able to figure out how (or whether it was at all possible) to use scraping tools such as import.io to do this.
Thanks !
For the goal you want to achieve, I can see two solutions:
Code it in Java: Jsoup + any CSV library
In a few minutes, the 36000+ urls can be downloaded easily.
Use a tool like Portia from scrapinghub.com
Portia is a WYSIWYG tool quickly helping you create your project and run it. They offer a free plan which can take in charge the 36000+ links.
We have an ASP.NET 2 application and at the moment for the reporting, we are utilising DataSet/DataTable taking some storedprocedure/query and display on the screen (grids). For the graphics, we are using Dundas Chart component and again we are utilising DataSet/DataTable to display it. These thing has been running well ... no issue.
Now, we are having a challenge that every single report that we produce above (grid or graph), the user wants to consolidate into 1 single click solution which means if they click the "Consolidated Report" it will produce the page as well (grid/graph) into 1 page. But the issue is that the user wants use the display (grid/graph) and download into 1 file. It could Ms Words/Ms Excell.
How we are going to achieve this?
I am thinking to do as follow:
1. Convert everything to use the Reporting Services (client) .. or
2. Is it possible to do like convert the page/grapf into Word/Excel into a single page? Is there any other solution?
The issue is that the we have simple stored procedures to get the data source out and then we are using that data and do the massive data manipulation in DataSet/DataTable (loop through the records, create a new dataset and manipulate etc etc) and then to construct the final report and display on the screen.
I am appreciated your comment.
Thank you
In my opinion the best solution will be integrating the HTML into a PDF.
You can convert the HTML generated into PDF easily with ABCPdf from WebSuperGoo (I suspect that other solutions can work, but I have only used this product)
See this live demo, and select "Add URL" and point to your URL or check out this example