How do you select this text above the HTML table using XPath? - web-scraping

Below is the HTML I'm working with. I've removed some lines that aren't relevant to this question, such as the content within the table.
My objective is to capture the names, and the corresponding information found within the table. Each Name/Table combination would be one row.
<div class="row">
<div class="col-sm-4 col-md-4">
<span class="LIST_TITLE">
Contact Person
</span>
</div>
<div class="col-sm-6-left" style="display: table-cell;">
Name A
<table>
</table>
Name B
<table>
</table>
</div>
</div>
I currently have this XPath '//div[#class="row"]/div/span[#class="LIST_TITLE"][contains(text(),"Contact Person")]/ancestor::div/div[#class="col-sm-6-left"]/table', which I am able to loop over to extract out the information in the table.
My issue is how to capture the name for each table, which I am finding difficult as they're both contained within the same tag.
I have tried using './ancestor::div[1]/text()', though this will capture both names.
Any help is greatly appreciated

preceding-sibling::text()[1] will return the text node prior to the context node. If the table elements are used as the context node, that will return you the following text nodes:
Name A
and
Name B
NB I don't know what web scraping tool you are using, but I know that some of them have XPath APIs that won't return text nodes; only elements. If that's the case for you, you might need to switch to a different XPath API that is capable of returning text nodes, e.g. lxml https://lxml.de/xpathxslt.html#xpath

Related

add button is missing for Content:Toolbar

The add button that appears over the 2sxc items is missing all of a sudden. It was there a couple days agao but now when I log into any portal in my DNN instance the "+" or add button is missing
here is a screen shot:
As you can see, the change layout and edit buttons are there. Not sure why the add button disappeared.
This is true for apps that I import from the 2sxc.org website as well. So I know its not just my template becasue it also happens on all the apps I have created which use different templates.
But to be thorough, here is my template code, its token based:
<div class="kr-gallery animation">
<p>Hover or touch image and click brush icon for more details</p>
<div class="isotope_grid isotope_grid2">
<div class="isotope_main animation" data-min-width="230">
<repeat repeat="Content in Data:Default">
<div class="isotope_item kr-gallery-item sc-element">[Content:Toolbar]
<div class="photo"><a href="[Tab:FullUrl]/details/[Content:EntityId]"> <img alt="" src="[Content:Image]?h=500" />
<span class="fa fa-paint-brush"></span></a>
</div>
</div>
</repeat>
</div>
</div>
</div>
Any idea why this is?
UPDATE:
Here is my visual query:
SOLUTION:
Based on answer, I switched to razor because I am using a custom query. Here is my simple template code now:
#* this will show an "add" button if the current user is an editor *#
#Edit.Toolbar(actions: "new", contentType: "Image")
#{
// get all images as delived from the standard query
var images = AsDynamic(Data["Default"]);
}
<div class="kr-gallery animation">
<p>Hover or touch image and click brush icon for more details</p>
<div class="isotope_grid isotope_grid2">
<div class="isotope_main animation" data-min-width="230">
#foreach(var img in images)
{
<div class="isotope_item kr-gallery-item sc-element">#img.Toolbar
<div class="photo"><a href="#Link.To(parameters: "details=" + img.EntityId)"> <img alt="#img.Title" src="#img.Image?h=500" />
<span class="fa fa-paint-brush"></span></a>
</div>
</div>
}
</div>
</div>
</div>
The missing + is by design, because editors are used to the + adding an item right after the previous one. This behavior cannot be guaranteed with a query, as the order of things is determined by the query. It is even possible, that adding an item will not show up, if a query-parameter hides that item.
So the design pattern is to provide a separate + button. The easiest way is in razor, I believe the code is something like
#Edit.Toolbar(actions: "new", contentType: "your-content-type-name")
In Tokens it's a bit more messy, and you cannot conditionally check if a user has edit-permissions.
So I recommend you go the edit.toolbar way
You can also find an example of this in the blog app: http://2sxc.org/en/apps/app/dnn-blog-app-for-dnn-dotnetnuke
I could be wrong but did you recently experiment with the visual query designer? Because this could be the cause.
The most common reason is when you use a pipeline (visual query) to deliver data to a template, which is not assigned to this instance. Reason is that "add" in a instance-list of items add it to a specific position (like right after the first one). This isn't the same when you use data like a data base - as there is no sorting in that scenario. So if this is the cause, I'll help you more.

ASP.NET: Can an individual SQL record spread its data over a fully user designed template?

I've been trying to get data from a SQL file to populate a set template for a web page. When the user navigates through the database records, the page updates accordingly. The data will include image URLs so that a image will update and things like the alt tag caption and descriptive text should also update.
Its basically a customised gallery based on SQL (text) data.
The .ASP data controls seem very restrictive and even details and form view with templates make it difficult to get full customisation of the layout of data. It there a way to spread data from a database record around a page template (ie HTML/CSS styling) in a consistent manner while maintaining the flexibility of the database functionality?
I basically want to write a HTML layout and put tags in that are replaced by the current field of the database, eg in pseudo code:
<div class="style1"> <asp: SQL_Datafragment field="Title"> </div>
<div class="picframe">
<img src ="
<asp: SQL_Datafragment field="Image_URL">
"/> />
</div>
<div style = "caption"> <asp: SQL_Datafragment field="Caption"> </div>
<div style = "spacer"><div>
<div style = "pictxt"> <asp: SQL_Datafragment field="Description"> </div>
And have a navigation control at the bottom like a prev and next field.
Is this possible?
Many thanks.
Kw

How to get verification code using css or xpath in Selenium Webdriver

<body>
<div class="row-fluid">
<div class="span12">
<div class="mailview" style="margin-right:18px;">
<p>Dear MohanNimmala First,</p>
<p>Thank you for registering with MediAngels!</p>
<p>
<p>
Verification Code:
<b> 95527</b>
Can someone help me to get verification code as 95527 from above
using xpath or css?
I am using following xpath=html/body/div[1]/div/div/p[4]/b
Based only on the HTML snippet posted in question, you may want to try the following XPath :
//p[contains(text(), 'Verification Code')]/b
I think you are setting yourself up for problems later if you nest the 'verification code' so deep in the HTML. When someone changes the layout/html, the test is very likely to fail.
I suggest giving the b tag an ID and to use that ID in your selector. Your test will be more resilient.
Add the HTML attribute to the b-tag:
Verification Code:
<b id="verification-code"> 95527</b>
....
And the xpath
//*[#id='verification-code']
The issue with your xpath is probably that you're missing a slash at the beginning. One slash starts from the root, two slashes starts anywhere.
/html/body/div/div/div/p[4]/b
Thanks Deef& Har, for your answers below code works to extract value from innterhtml tags
driver.switchTo().frame("rendermail");
WebElement OTP=driver.findElement(By.tagName("b"));
System.out.println(OTP.getText());

Handlebarsjs Templates & using html content

So, I've got a fairly straightforward handlebars template which an element that looks like this:
<div>
{{include_text}}
</div>
I am trying to insert some html like:
<table>
....lots of table stuff
</table>
When I use the table with the template, what I get looks like:
<div>
<table>
... lots of table stuff
</table>
</div>
and I want:
<div>
<table>
....lots of table stuff
</table>
</div>
Is this possible? If so, how?
Handlebars (and Mustache) escape the double Mustaches.
Use triple ones
{{{include_text}}}
From the official GitHub:
By default, the {{expression}} syntax will escape its contents. This helps to protect you against accidental XSS problems caused by malicious data passed from the server as JSON.
To explicitly not escape the contents, use the triple-mustache ({{{}}}).
Note, the escaping isn't "just there", if your output contains user-entered data, not-escaping might enable them to perform XSS.

How to use Excel VBA to click a web CSS button?

I am creating a macro with Excel VBA that will submit an entry into an online database using information from an Excel spreadsheet. During this entry process, the macro needs to click on a CSS button. It isn't a form button, does not have an input type, no name, no id, and no source image except for a background image. I think my only hopes are either to click on the button based on the div class. Can anyone help?
The button is here :
<div class="v-captiontext">
By Ankit
</div>
<td class="v-tabsheet-tabitemcell v-tabsheet-tabitemcell-selected" style="">
<div class="v- tabsheet-tabitem v-tabsheet-tabitem-selected">
<div class="v-caption" style="width: 39px;">
<div class="v-captiontext">
By LOT</div>
<div class="v-caption-clearelem">
</div>
</div>
</div>
</td>
Thanks to Remou's answer on this thread: Use VBA code to click on a button on webpage
Here is a first stab in your issue, you could try this:
Set tags = wb.Document.GetElementsByTagname("div")
For Each tagx In tags
If tagx.class = "v-caption-clearelem" Then
tagx.Click
End If
Next
Yet, I've never tried to use the Click method on a div.

Resources