I am not able to perform web scraping by using Power Automate. It did work on some websites but some are not workable.
The one that I would try to scrap data from is: https://apps.sfc.hk/edistributionWeb/gateway/EN/circular/products/product-authorization/
I have no coding experience. If Power Automate cannot work on this website, I would be much obliged if anyone can advise for any alternative. Thank you!
Related
I'm creating a web scraper with golang and I just wanted to ask some questions about how most of them work. For example, how does Googlebot not use a lot of bandwidth when scraping because you have to go to each URL to get data and that can be thousands of URLs so not only will that take bandwidth, it will also take a lot of time. I'm building a web scraper and I'm experiencing these issues and I wanted to ask what the best way of fixing these issues is. https://github.com/hackermondev/cosmic
I have a hobby of reading news. The problem is, there are quite a lot of websites I often go to, and this gives me an idea: building my own database of news. The idea is similar to the newspaper clippings. For example, I read something interesting about Germany economics news, therefore, I can use this software to save all the text and images from the said site(into my computer), and I can add tags such as "Germany", "econ" so I can find it and read it later. I shared this idea with my friend and he said web scraping is not easy because not every site allows you to do that. So my question is, how should I begin this with? I study computer engineering so I got some programming understanding but obviously not enough. Any clues or experience(for the web scraping and tagging) sharing will be helpful, thank you!
Python has a couple of good web-scraping tools that work well. Beautiful Soup 4, Scrapy, Selenium, requests to name a few.
Before web scraping I would recommend to learn the basics of python and how the web works.
Note that most websites disregard it if you scrape them. It is hard for them to track you doing it and if you only download a few specific sites it shouldnt be something they complain about - as it is not much more than pressing CTRL+C and download the whole site as an HTML.
Don't share it and don't spam requests - be a fair player. If you want to be on the safe side check out TOS of the website.
Well, I am developing a virtual classroom i.e. Online classroom, It is a website which provides faculties and students an interactive way of learning. I am almost done just I am stuck with this video conferencing module. I am using ASP.NET C#. I have searched a lot for this. Some information I found is:
1) I got to know about ConferenceXP module. Which is developed by microsoft and enables developers to use it to build their own video-audio conference. I also searched a lot about it, but I am confused on how to implement it in my website. Any suggestion with steps on how to use conferenceXP in website.
2) Though i searched a lot, but yet if some links were left, I am asking this again that Is there any better and easier way( Easier and fast beacuse i need to implement this into my website in 7 days) to implement video-audio chat application in website using C#(Or any other technology working with visual studio).
3) While searching I found something like I can use Skype with my online classroom, Any suggestions related this?
Thanks in advance.
You can use Openmeetings from regardless of your technology. The whole interface is SOAP/REST based.
I planned to develop the job portal as similar to naukri/monster.
I'm going to do through ASP.NET. For this, what is the best environment.
At early, I planned to use "Dot Net Nuke" and then I came to know that its for CMS,
Shall I go with MVC framework for this or anything is there which is good than MVC. I don't have any experience on web based application, but I'm good enough in C#.
A CMS isn't going to be flexible enough to allow you to accomplish the types of things you'll need to do for a job board type website. I would stick with MVC or even WebForms if you're completely new to the web.
Unless this is a pet project to help you learn how to program for the web, you may want to start smaller and get some experience before trying something like this. Developing for the web is a completely different animal than just knowing some c#. There are a ton of other technologies you will need to know, including HTML, JavaScript or JavaScript library such as jQuery, stateless design, etc.
We had a similar question while we were developing a job site for our client. After weighing all the pros and cons we finally decided to go with PHP. We found that the performance of the site in terms of speed was much better and needed far less resources. In the end the client was happy with the decision we helped them make.
I usually only work on system admin projects and software packaging and have little knowledge of web technologies, but I've been given the task of building a simple website that searches an SQL 2005 database from the users input and lists the results. That's it.
Can anybody help me out with some sample code or some such? Since we're a pure Microsoft house, it will be running on SQL 2005 and IIS so I'm assuming I'll need to use ASP.NET, but perhaps my inexperience is leading me down the wrong path.
Any help or suggestions would be much appreciated!
You might want to look at ASP.NET Dynamic Data. This is exactly the scenario it was designed for.