Get value of "data-..." attribute with .css selector with Scrapy - css

I am trying to get the value of a data-attribute with scrapy:
response.css('.product-header-top div::attr("data-background-image")').get()
But I do not get the value of data-background-image and Python throws an error:
raise SelectorSyntaxError(cssselect.parser.SelectorSyntaxError: Got
pseudo-element
::FunctionalPseudoElement[::attr(['data-background-image'])] not at
the end of a selector
Here is the relevant HTML Code of the webpage:
<div data-background-image="/images/image.jpg" style="background-image: url("/images/image.jpg");"></div>
Thanks
UPDATE
F.Hoque is right and it works fine. The website is dynamic and renders the data-background-image with JS. So the ::attr("data-...") is working. Thanks for your help #F.Hoque!

Your CSS selection is working fine. There is a typo ); just remove it.
response.css('.product-header-top div::attr("data-background-image")').get()
Proven by Scrapy shell:
In [26]: sel.css('div::attr("data-background-image")').get()
Out[26]: '/images/image.jpg'

Related

Tiny cme html editor with angular js

I am using TinyCME html editor with the angular directive
and I render the output of the editor -which is data-bidden to property "description" in the scope- into a div using ng-html-bind.
<div ng-bind-html="description" ></div>
everything is working fine but I didn't get in the div what I see in the editor
especially when it comes to styling like the background color and text color
here is what I get in the editor
and here is what I get in the div
it sounds like all the styles applied in the editor will eventually be overwritten by the styles in the div context
I don't have any experience in CSS so please excuse my lack of knowledge
What I really want to do is to render the editor output in a div in a way exactly the way it looks in the editor any help?
I have solved the problem the issue comes from that the ng-bind-html strips out all the styling info comes from the editor that's why there is no styling info
to solve the problem we should use angularjs service $sec which tells the ng-bind-html not to strip out anything from the html string
so to use it in the angular expression we should make it as a filter
app.filter('trustAsHtml', ["$sce", function ($sce) { return $sce.trustAsHtml; } ] );
then you can use this filter in the binding expression like the following:
<div ng-bind-html="currentModel.description | trustAsHtml" ></div>

Data Scraping: How to select width from a style tag by using Scrapy CSS?

I'm new to learn scrapythese days. I'm trying to select width from a div by using CSS Selector, but its not being possible for me. I tried a lot to find out solution, but every time i fond solution with xpath instead of css selector.
The HTML code is:
<div class="stars-container">
<div class="stars" style="width: 60.606%"> Rating</div>
</div>
After getting response by scrapy shell URL, my attempts to select width from the above html is:
response.css('.stars-container .stars ::attr(width)')
response.css('.stars-container ::attr(width)')
It'll be great pleasure for me to learn it easily if someone help me out in this problem. Thank You
If you want to get just the width value and insist on pure CSS solution, here you go:
response.css('.stars-container .stars::attr(style)').re_first('width:\s*(\d+\.\d+)\s*%')
Here is how you can get width.
response.css('.stars-container .stars ::attr(style)').re_first(r'width:\s+(\d+.\d+)')
Also get comfortable with python regx liberary you can find documentation here Python re
It can be help for your.
response.css(".stars-container .stars").xpath('#style').extract()

RSpec Issue with have_css

I'm using Rails 5.1.1, RSpec 3.5.0 & Capybara 2.7.1.
I want to have a test that checks for a navbar on the home page, according to some documents that I've found I should be using have_css for this. The example given is:
have_css("input#movie_title")
My understanding is that this would look for an input tag with an id of movie_title. Is that correct?
I'm trying this in my code:
have_css("div.navbar-default")
I get this error, however:
Failure/Error: expect(page).to have_css("div.navbar-default") expected to find css "div.navbar-default" but there were no matches
Why is this not working? I have a div with the class 'navbar-default', so this should work as far as I can work out.
Edit / Solved:
I realised my mistake. I have to expect statements, the navbar-default class is in a nav tag, not a div tag. The 2nd statement is in a div tag but wasn't running due to the error on the first one.
I've fixed it now, all working
The have_css matcher is applied to the parent container instead of the actual element. Is there a parent container?
Try something like this:
# Find the parent
parent = page.find('div#nav')
# Check for the nested div
expect(parent).to have_css(".navbar-default")

Bind Angular 2 variable to css pseudo-class content: attr() expression

I wanted to compare performance of rendering Angular 2 variable binding [innerText]/{{}} with binding variable as content of pseudo-class(because methods above forces element re rendering)
However, I struggle trying to make angular markup work with css.
That works
CSS:
.my-el:after {
content: attr(my-attr);
}
HTML
<div class="my-el" my-attr="text"></div>
But after change it to my-attr="{{myVar}}" Angular throws error:
browser_adapter.js:77 EXCEPTION: Template parse errors(...)
So I red that I should use attr.my-attr="{{myVar}}"
But after changing CSS to
.my-el:after {
content: attr(attr.my-attr);
}
it doesn't work (I guess dot isn't valid symbol here?).
I know that all above may have not much sense, however I'm finding it as interesting problem which I can't solve so far.
Any ideas how to make these two work together? Thanks in advance!
You will have to bind your value with the following way
<div class="my-el" [attr.my-attr]="myVar"></div>
This way angular will attach the myVar contents to the my-attr attribute
If you need to prepend it with data- use
<div class="my-el" [attr.data-my-attr]="myVar"></div>
Then you can access the value from your css with
attr(my-attr) or attr(data-my-attr)

AngularJS style issue IE

Im trying to figure out how to make this work on IE:
<div ng-if="result.is_mesurable==true" style="left:{{ (result.user_score * 20)-10}}%" class="test-box">
The code basically generates a dynamic table, and the left position of the object is taken from the user_score value.
I know that IE doesn't read this declaration properly, i had a similar bug in the past:
AngularJS weird render issue
"Because {{xxx.xxx}} is invalid css it is trucated by IE and when the angular compiler scans all attributes, the style attribute is empty."
I know there must be a similar solution, but so far i've been unable to figure it out.
Thx in advance.
also, just to note, the result on IE is an empty style attr.
The issue is the same, solvable with ng-style or ng-attr-style:
ng-style:
<div ng-style="{left: ((result.user_score * 20) - 10) + '%'}" class="test-box" ng-if="result.is_mesurable==true">
ng-attr-*
<div ng-attr-style="left:{{ (result.user_score * 20)-10}}%" class="test-box" ng-if="result.is_mesurable==true" >

Resources