Alternative to Selenium scripts to download web data - web-scraping

I have to extract web data every day,to automate it I am using selenium recording script if i run recording script it will automatically download data based on my recording.Now i want alternative to do this task which has to do changes the dates and a few other changes in the website, then clicks the download button.
Thanks in advance.

#raju your question is very generic.Please make it more precise next time.
As per my understanding search for parameterization in selenium ide should help you. These are very basic case. Try with this at least something. Still you face some issue post the query with complete error log. We will be more than happy to help you.

Related

Crystal Report Hangs/Freezed when Loading

​Hi Everyone,
Currently i'm having trouble to solve an issue which caused Crystal Reports hang when loading process. Below are the scenario:
1. Basically, i've built an windows service to export Crystal report to PDF simultaneously
2. But some times when the service export the reports until certain limit, the service will suddenly Hang, meaning Crystal Report suddenly stuck in the middle of ReportDocument.Load() function, which caused i need to restart the service in order to let the service be able to load the report again.
I believed that i've put all the necessary codes to release/free memory resources which attached to the report object when the report is exported, such as:
1. reportDocument.Close();
reportDocument.Dispose();
GC.Collect();
But still, the issue has not been solved yet. I've did a lot of research and found that actually this problem is somehow related to the PrintJobLimit configuration, but before i change the configuration, i would like to know:
Is there any way except than increasing the print job limit can solve this issue ?
If increasing the print job limit is the only option, can anyone explain to me how does the print job limit works ? Is one page of report considered as 1 print job ? If that's the case, then why i can still be able to export a report that contains 100 more pages without any issue while my print job limit is 75?
Any advice from you will be very appreciated. Thanks.
If anyone faced the issue i mentioned above, can refer to below links, i've asked the SAP people, and i think they have a good answer about this.
https://answers.sap.com/questions/702435/crystal-report-hang-when-load-report.html

phantomjs intercept response headers

I am currently creating a test script using codeception webdriver, I decided to run this script automatically every after a specified interval to know whether or not my application is still working. With its capability on generating error logs and screenshots this automated testing tool has been helpful. But I personally think that getting response headers would be more helpful. Is there anyway I could do this without changing the frameworks I have been using? If so, could you guide me on how to implement this?

How to select link during ROAuth step in RStudio?

Sorry for asking a newbie question. I'm trying to do R sentiment analysis in RStudio. I could get the ROAuth script to work, but during this phase:
To enable the connection, please direct your web browser to:
http://api.twitter.com/oauth/authorize?oauth_token=HZ6dfylyLYN6yk1WArBsM7fjkDHEByZqhFKiSEYPzg
When complete, record the PIN given to you and provide it here:
I'm unable to click the link or select it with my mouse (and don't feel like manually typing in that URL in a web browser). I therefore am not able to use this script in RStudio, I have to go back to regular R. Is there some specific thing I can do to be able to click this link?
Thanks for your time!

Is there any video recorder for Selenium WebDriver?

i am working in Selennium Webdriver2 with IE specific application. i know we can take the screenshots of the execution. similarly, Is there any option to record selenium execution as a video?
There is nothing in WebDriver.
Sauce uses http://www.ffmpeg.org/ to record video's. There are other options like https://code.google.com/p/java-remote-control/.
Generally the answer is that if you want to do it, you are going to have to implement it yourself.
This API helped me in solving the above problem, this is for your reference guys,
Selenium provides us with the feature to run the automation code/suite and record the output as a video and save it on your system. This video can then be shared with anyone who can see what the automation is doing. First of all, to enable video recording, we need to download some important jars. They are freely available to download from Google. Below are the jars to be downloaded. These are the latest version available, any other version can also be downloaded:
ATUReporter_Selenium_testNG_5.1.1 and ATUTestRecorder_2.1
Complete code can be found here:
https://mytechdevice.com/how-to-record-video-of-automation-output-in-selenium/

SCORM - Report failure

I'm trying to create a very simple SCORM course, but if I exit before the last page it still say that I have completed the course. Why is that?
Source files for the simple course I'm building
The
SCORM ENGINE Debug Log
I'm grateful for any help I get.
Ah, I needed to use set a cmi.core.lesson_status.
Now it works just fine!

Resources