How to extract cell data for a specific column with xpath? - web-scraping

So I've been looking for a solution to this, and while I've found some answers none of them seem to work. The table I'm attempting to extract data from looks like so:
<table class="table table-responsive table-striped" style="color:#fff; ">
<thead>
<tr>
<th scope="col"><div style="width:100%;">#</div></th>
<th scope="col" sort="outfit_name"><div style="width:100%;min-width:150px" class="btn btn-primary" sort="name" onclick="reorder('1')">1</div></th>
<th scope="col" sort="price"><div style="width:100%;" class="btn btn-primary" sort="rarity" onclick="reorder('rarity')">2</div></th>
<th scope="col" sort="text"><div style="width:100%;min-width:50px" class="btn btn-primary" sort="outfitType" onclick="reorder('2')">3</div></th>
<th scope="col" sort="text"><div style="width:100%;min-width:50px" class="btn btn-primary" sort="craftable" onclick="reorder('3')">4</div></th>
<th scope="col" sort="text"><div style="width:100%;min-width:50px" class="btn btn-primary" sort="locationText" onclick="reorder('4')">5</div></th>
<th scope="col" sort="instructions"><div style="width:100%;" class="btn btn-primary" sort="vendorText" onclick="reorder('5')">6</div></th>
<th scope="col" sort="outfit_class"><div style="width:100%;" class="btn btn-primary" sort="themeText" onclick="reorder('6')">7</div></th>
</tr>
</thead>
<tbody>
<tr>
<th scope="row" style="font-size: 14px; vertical-align: middle;">1</th>
<td style="font-size: 13px; vertical-align: middle;">info here</td>
<td style="font-size: 13px; text-align:center; vertical-align: middle;">info here</td>
<td style="font-size: 13px; text-align:center; vertical-align: middle;">info here</td>
<td style="font-size: 13px; text-align:center; vertical-align: middle;"> info here </td>
<td style="vertical-align: middle;"></td>
<td style="vertical-align: middle;"><div style="width:85px;" onclick="load_vendor(this)" context="<p>data here</p>" outfit="Name Here" class="btn btn-primary"> 🛒️</div></td>
<td style="vertical-align: middle;"><div style="width:93px;" onclick="load_theme(this)" context="<p>data here</p>" outfit="Name Here" class="btn btn-primary"> ℹ️ </div></td>
</tr>
</tbody>
</table>
In this example, how would I select cells for the 6th column? Another caveat to this is that not all cells have info in them. Ive tried using
//td[position()<=(count(//tr/th[.='6']/preceding-sibling::*)+1)]
however this doesn't seem to work. Lastly, I'm attempting to use this xpath for data extraction using Octoparse. Literally nothing I try works, anyone have any ideas?

position()<=<NUMBER> means that you want to select all the cells with index less or equal to <NUMBER>. Did you mean
//td[position()=(count(//tr/th[.='6']/preceding-sibling::*)+1)]
?

Related

What <form:xxx> tag should be used when outputting data?

When displaying information about my student from the database, I use the Spring MVC <form:xxx> tags, but I don't know how to display it simply in a cell, I don't use input tags, textarea tags, etc. For some reason it doesn't work otherwise. It looks like this now:
and I need this information to be just in the cell, I also attach my output code:
<table border="1" cellspacing="0" cellpadding="3"
style="font-size: small; line-height: 25px; border-color: #D1EEEE">
<tr bgcolor="#DCDCDC" align="left" style="border-color: #838B83">
<th width="100px">Name</th>
<th width="230px">Surname</th>
<th width="270px">Group</th>
<th width="220px">Date of enrollment</th>
</tr>
<form:form action="studentProgress" modelAttribute="student">
<form:hidden path="id"/>
<tr valign="bottom">
<td style="padding-bottom: 20px"><form:form path="name"/></td>
<td style="padding-bottom: 20px"><form:input path="surname"/></td>
<td style="padding-bottom: 20px"><form:select path="group"/></td>
<td style="padding-bottom: 20px"><form:textarea path="date"/></td>
</td>
</tr>
</form:form>
</table>
The solution was found, it was necessary to place my data between the tags and the information began to be displayed correctly :
<table border="1" cellspacing="0" cellpadding="3"
style="font-size: small; line-height: 25px; border-color: #D1EEEE">
<tr bgcolor="#DCDCDC" align="left" style="border-color: #838B83">
<th width="100px">Name</th>
<th width="230px">Surname</th>
<th width="270px">Group</th>
<th width="220px">Date of enrollment</th>
</tr>
<form:form action="studentProgress" modelAttribute="student">
<form:hidden path="id"/>
<tr valign="bottom">
<td style="padding-bottom: 20px"><p>${student.name}</p></td>
<td style="padding-bottom: 20px"><p>${student.surname}</p></td>
<td style="padding-bottom: 20px"><p>${student.group}</p></td>
<td style="padding-bottom: 20px"><p>${student.date}</p></td>
</td>
</tr>
</form:form>
</table>

Columns in Table not aligned properly

I have this html styled with bootstrap. My problem is that the row inside thead is not aligned with the row inside tbody (see the jsfiddle link to see an example of what's happening):
<table class="table">
<thead>
<tr id="table-header">
<th scope="col">#</th>
<th class="search" scope="col">nome</th>
<th scope="col">
</th>
</tr>
</thead>
<tbody id="table-body" style="display: block;" data-json="/categoria/list.json">
<tr>
<th scope="row">1</th>
<td>hum</td>
<td>
<div class="btn-group" id="buttons" role="group" aria-label="comandos">
<button sec:authorize="hasPermission(#user, 'atualiza_categoria')" type="button" class="btn btn-secondary" id="update" th:attr="data-url=#{/categoria/update}" onclick="open_tab(this)">
edit
</button>
<button sec:authorize="hasPermission(#user, 'remove_categoria')" type="button" class="btn btn-secondary" id="delete" th:attr="data-url=#{/categoria/delete}" onclick="open_tab(this)">
del
</button>
</div>
</td>
</tr>
</tbody>
<tbody id="table-search" style="display: none;" data-json="/categoria/search.json"></tbody>
</table>
this code is based on the first example available here: https://getbootstrap.com/docs/4.4/content/tables/. I tried change the line:
<th scope="row">1</th>
inside tbody to:
<td scope="row">1</td>
but the same issue occurs.
jsfiddle: https://jsfiddle.net/klebermo/0gdtf7qy/
Anyone can tell how to fix that?
You can't use tbody display: block;. tbody display property will be table-row-group.
So removed style="display: block;" from tbody tag. Check here working example
Remove display: block; css property from tbody tag
<table class="table">
<thead>
<tr id="table-header">
<th scope="col">#</th>
<th class="search" scope="col">nome</th>
<th scope="col">
</th>
</tr>
</thead>
<tbody id="table-body" data-json="/categoria/list.json">
<tr>
<th scope="row">1</th>
<td>hum</td>
<td>
<div class="btn-group" id="buttons" role="group" aria-label="comandos">
<button sec:authorize="hasPermission(#user, 'atualiza_categoria')" type="button" class="btn btn-secondary" id="update" th:attr="data-url=#{/categoria/update}" onclick="open_tab(this)">
edit
</button>
<button sec:authorize="hasPermission(#user, 'remove_categoria')" type="button" class="btn btn-secondary" id="delete" th:attr="data-url=#{/categoria/delete}" onclick="open_tab(this)">
del
</button>
</div>
</td>
</tr>
</tbody>
<tbody id="table-search" style="display: none;" data-json="/categoria/search.json"></tbody>
</table>

HTML - Table head width not working

I have tried with this.
<th style="width:500px;overflow:hidden;">
If I have a large description It is going outside the table & I want to fix it.
<div class="Tablecontent">
<div class="table-responsive">
<div style="min-height:300px">
<table class="table table-hover" >
<thead class="tableHeadDesign">
<tr>
<th style="width:40px; overflow:hidden;">S.No.</th>
<th>Name</th>
<th style="max-width:500px;overflow:hidden;">
Description
</th>
<th>Size </th>
<th>Status </th>
<th></th>
</tr>
</thead>
<tbody class="tableBodyDesign">
<tr ng-repeat="x in media |filter:searchText">
<td>$index </td>
<td>{{x.Name}}</td>
<td>{{ x.Description }}</td>
<td > </td>
<td>{{x.Status}}</td>
<td>
<input type="button" class="btn btn-info btn-lg popbtn ViewBtn" value="View Detail"></a></input>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
Here I want to fix Description Column.
You can use text ellipsis
see this example
td{
max-width:20px;
text-overflow: ellipsis;
white-space: nowrap;
overflow: hidden;
}
table{
width:100%
}
<div class="Tablecontent">
<div class="table-responsive">
<div style="min-height:300px">
<table class="table table-hover" >
<thead class="tableHeadDesign">
<tr>
<th style="width:40px; overflow:hidden;">S.No.</th>
<th>Name</th>
<th style="max-width:500px;overflow:hidden;">
Description
</th>
<th>Size </th>
<th>Status </th>
<th></th>
</tr>
</thead>
<tbody class="tableBodyDesign">
<tr ng-repeat="x in media |filter:searchText">
<td>1 </td>
<td>name</td>
<td>aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa</td>
<td >dsdsdsdsds </td>
<td>Status</td>
<td>
<input type="button" class="btn btn-info btn-lg popbtn ViewBtn" value="View Detail"></a></input>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
Are you looking for text-overflow?
<td style="max-width:500px; display:inline-block; overflow:hidden; text-overflow:clip; white-space:nowrap;">
{{ x.Description }}
</td>
Here is an example:
https://jsfiddle.net/59166290/aan0s31t/15/embedded/result/
Problem solve...
Actually I had fixed the and forgot to set the fixed size of :p

Remove default style from boostrap anchor link

I want to remove the default styling from the a tag.
What I am trying to achieve is, to place the a tag and other things in one line only. But, the default a tag styling is not allowing me.
Here's the code:
<table style="background: #DEE3E0" class="table table-responsive table-condensed">
<thead>
<tr style="padding: 0">
<th class="text-center" style="font-size: 12px; padding: 0">Name & Qty</th>
<th class="text-center" style="font-size: 12px; width: 50px;padding: 0">Rate</th>
<th class="text-center" style="font-size: 12px; width: 50px;padding: 0">Amount</th>
</tr>
</thead>
<tbody>
<tr>
<td style="font-size: 12px">
<img src="//placehold.it/45" alt="Product Image" />
1x <a href="" style="margin: 0; padding: 0">
Product Name
</a>
<a href="javascript:void(0)" title="Remove">
<span class="pull-right text-danger">
<i class="fa fa-trash fa-lg"></i>
</span>
</a>
</td>
<td class="text-right" style="padding-top: 7.5px; font-size: 12px">150.00</td>
<td class="text-right" style="padding-top: 7.5px; font-size: 12px">150.00</td>
</tr>
<!-- and other rows... -->
</tbody>
<tfoot>
<tr>
<td colspan="2" style="font-size: 12px; font-weight: bold" class="text-right">Sub Total</td>
<td class="text-right" style="font-size: 12px; font-weight: bold">425.00</td>
</tr>
<tr>
<td colspan="2" style="font-size: 12px" class="text-right">Taxes</td>
<td style="font-size: 12px" class="text-right">50.00</td>
</tr>
<tr>
<td style="font-size: 12px" colspan="2" class="text-right">Shipping</td>
<td style="font-size: 12px" class="text-right">100.00</td>
</tr>
<tr>
<td style="font-size: 12px; font-weight: bold" colspan="2" class="text-right">Grand Total</td>
<td style="font-size: 12px; font-weight: bold" class="text-right">575.00</td>
</tr>
<tr>
<td colspan="3"> View Cart
Checkout
</td>
</tr>
</tfoot>
</table>
Here's the complete fiddle: http://jsfiddle.net/j4o8tff5/
1x is the quantity. I want that Product Name should be placed next to the quantity itself. And hence I am trying to remove the default styling.
I have searched a lot, but could not find the solution.
Kindly help me out. Thanks.
Wrap your quantity text in span tag, Demo
<span>1x</span>
Instead of trying to find out the default styling, better you use some styling to achieve your target. simply use -
a{ position: relative; top: -12px; }
this will solve your problem.

Bootstrap table disappearing when less than 1200 pixels

I have a Bootstrap issue where my table disappears. When my page displays more than 1200 pixels wide my table shows perfectly. But when my page is shrunk to less than 1200 pixels wide my table disappears. I am not super advanced with bootstrap(not a design guy) so i am unsure what is causing the problem. I think it has something to do do with responsive tables. Also I have not edited my bootstrap.
<div class="row">
<div class="col-md-12">
<div class="panel panel-cascade">
<!-- heading-->
<div class="panel-heading text-primary">
<h3 class="panel-title">
Friends List
<span class="pull-right">
<i class="fa fa-chevron-up"></i>
<i class="fa fa-times"></i>
</span>
</h3>
</div>
<!-- body -->
<div class="panel-body">
<table class="table table-condensed table-hover">
<thead>
<tr>
<th class="visible-lg">First Name</th>
<th class="visible-lg">Last Name</th>
<th class="visible-lg">Phone Number</th>
<th class="visible-lg">Carrier</th>
<th class="visible-lg">Send Text</th>
</tr>
</thead>
<tbody>
#foreach (var row in Model)
{
<tr>
<td class="visible-lg">
#row.FirstName
</td>
<td class="visible-lg">
#row.LastName
</td>
<td class="visible-lg">
#row.PhoneNumber###row.Carriers[0].CarrierEmail
</td>
<td class="visible-lg">
#row.Carriers[0].CarrierName
</td>
<td class="visible-lg">
#using (Html.BeginForm("SendEmail", "Admin"))
{
#Html.Hidden("Id", row.Id)
<button type="submit" class="btn btn-success"><i class="fa fa-envelope"></i></button>
}
</td>
</tr>
}
</tbody>
</table>
</div>
</div>
</div>
</div>
I think it is because of class you are using "visible-lg", wich means visible when large, bootstrap 3 uses 4 different sizes xs, sm, md, lg. take a read at this article http://getbootstrap.com/css/.
I use this classes in my tables and they work perfect.
<table id="proviers-table" class="table table-striped table-bordered">
<thead>
<tr>
<th>Nombre</th>
<th>Teléfono</th>
<th>Correo Electrónico</th>
<th>País</th>
<th>Entidad Federativa</th>
<th>Dirección</th>
<th></th>
</tr>
</thead>
<tr>
<td>coty</td>
<td>1234567890</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>
<div class="btn-group">
<button class="btn btn-default btn-xs edit-provider" id="1" >
<span class="glyphicon glyphicon-pencil"></span>
</button>
<button class="btn btn-default btn-xs edit-provider" id="1">
<span class="glyphicon glyphicon-trash"></span>
</button>
</div>
</td>
</tr>
<tr>
</table>
here is my view:
when small:

Resources