Scraping invisible data from a website - web-scraping

I want to parse data using jsoup from a website which contains data in tables which are in a "div" tag, when I inspect table data I can see the data in inspect element window, but if I see the page source the data is not present in the div tag.(They have taken a class called "invisible" for the entire div tag)
How to get that invisible data from the div tag

Related

Load link into target frame on hover

I have a page with 2 iframes, 1 nav frame, 1 content frame. I want to load a link from the nav from into the content on mouse over. Essentially the same behavior as clicking a link with target="f" but without having to click.
This onmouseover="window.location=event.target.href" works to autoload a link in general but I want it to do it in the iframe I specify.
Code for link in the nav frame.
<a target="f" href="pfh.html" onmouseover="window.location=event.target.href">pfh</a>
on click it goes to target frame f but the onmouseover event it loads in the current frame.
I had searched along time here before posting but I did some experimenting & got it to work with help from this page Target a Window or Frame Using JavaScript or HTML
onmouseover="top.frames['f'].location=event.target.href"

Accessing elements inside iFrame documents in Selenium WebDriver

I am trying to automate a scenario where I would need to access elements inside an iFrame and continue my testing.
My current problem is that, I am able to switch to the iframe successfully, but I am unable to access contents inside the frame.
The contents inside the iframe are Shadow DOM contents and are inside a document.
The HTML code looks like this::(Please see attachedHTML)
I have tried the below and get
org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element
By accessing the shadow root
driver.switchTo().frame("iframe_id");
WebElement root1 = driver.findElement(By.className("class_abs"));
WebElement shadowRoot1 = expandRootElement(root1);
Here webdriver is unable to find element by classname.
2.By using Javascript executor
driver.switchTo().frame("iframe_id");
JavascriptExecutor js=(JavascriptExecutor) driver;
js.executeScript("arguments[0].click();", driver.findElement(By.xpath("button_xpath")));
Here webdriver is unable to access the Xpath.
3.
Using normal Xpath after switching to IFrame.
I would like to know if I need to access the contents of the document inside iFrame first to access its elements. If that is the case, how can I do it.
P.S- Tried this on both chrome and Firefox browser.
All,
I have finally managed to resolve the issue.
So the issue resolutions goes like this.
-The frame designed on the page were actually hidden.
-I had to switch to the hidden frame after switching to the main frame.
-Then the elements were then identified easily.
Problems what I faced:
1. In Firefox browser-The hidden frame was not detected in the 'Firefinder' when I checked with filter 'frame' which was detecting all the frames except the hidden frame. Same was the case with 'Firepath'
2. In Chrome browser- Though the document inside the main frame was displayed, traversing inside did not show the hidden iframe.
Solution:
In Firefox browser, inspecting the element using the HTML tab helped me to identify the hidden frame inside the main frame.
Carefully traversing the iframe hidden document helped me.
But, I am still not sure why the Chrome browser 'Inspect' doesn't show this frame.

How to take out both html and css of the whole webpage using snappysnippet?

In snappysnippet we can only take out css and html of the selected element. But how to extract the whole webpage's css and html
Click on sources option in snappysnippet, whole foulder structure will be displayed on leftpanel, right click on desired file and save as in your workspace.
For html code you can right click to the particular webpage and select View Source Code option.

Dynamically creating a TreeViewin HTML with indent lines in ASP.NET

i'm quite new to HTML, and am trying to create a professional looking TreeView.
I can not use the in built TreeView in ASP.NET as i need to point the target of the selection to another frame (I have tried, and this doesn't seem possible).
My TreeView is built up as follows:
Folder1
ChildOne
ChildTwo
ChildThree
Folder2
ChildOne
ChildTwo
ChildThree
I have the collapsing of the folders working, but would like to know how to format this TreeView so it has dotted lines down to the child nodes (as most TreeViews tend to have).
How would i go about this?
Cheers.
If your only concern is to point the target of the selection to another frame, then you just need to use the Target propery,
The TreeView.Target Property gets or sets the target window or frame in which to display the Web page content associated with a node.
Moreover, Target value must begin with a letter in the range of A through Z (case-insensitive), except for certain special values that begin with an underscore, as shown below :
_blank : Renders the content in a new window without frames.
_parent : Renders the content in the immediate frameset parent.
_search : Renders the content in the search pane.
_self : Renders the content in the frame with focus.
_top : Renders the content in the full window without frames.
The default value is an empty string (""), which targets the window or frame with focus.

webusercontrol problem

I have one webusercontrol for Datepicker. When I am using this control in content pages it works fine. I have many controls in content page (textbox, label, etc.). All controls are inside a table. I have placed usercontrol inside <td>. When I click on an image, the calendar control is showing. But that time this calender control pushes all other controls (textbox, label, etc.). I have set the z-index at a higher value, but still it's not working.
alt text http://www.shareimages.com/images/thumbs/0/0/1/39586-qJyWopyZlqGlk6CVmw-datepicker.jpg
Z-index will not work for you becuase it is made for absolute,fixed and relative positioned elements.
in your case what happened is when the calendar was not loaded the cell was empty but when you opened the calenday it filled the cell and expanded the row which it expanded the table. which causes the elements who are inside the tabe to change their position.
try not to put the calendar over the other rows. create new column. set rowspan and put the calendar inside that cell which it will expand in its cell with out affecting the other rows since it is in different column (this will change the desiegn but it is the easy way)

Resources