Nokogiri and isolating select elements from an array full of Nokogiri nodes - css

Question

I'm trying to scrape http://www.ign.com/games/reviews using Nokogiri and I'd like to instantiate new review objects that correspond to each game review on the page. Naturally, I'd also like to grab each numeric Score from each review and assign that score value as a class attribute to my review objects.
The problem is, the best I can do is return an entire string of scores mushed together instead of a list consisting of each score.
class VideoGameReviews::Review
attr_accessor :name, :score, :url
def self.scrape_titles
#doc = Nokogiri::HTML(open("http://www.ign.com/games/reviews?"))
#doc.search("#item-list div.itemList div.itemList-item").each do |review|
new_review = VideoGameReviews::Review.new
new_review.score = review.search("span.scoreBox-score").text
=> "99996.37.17.17.17778.58.58.586.36.47.187.57.88.95.587.6" #Not what I want
end
end
end
Any advice on how to extract a list of scores with each score separate and unique from other scores? Maybe use a more specific CSS selector?

Answer 1

You are using nokogiri properly but need to revise your logic to store the scores properly. For instance, we can get the score for an individual game pretty easily:
new_review.score = fourth_item.search("span.scoreBox-score").text
=> "6.3"
Instead of having to do everything in a single method, you can start by breaking your code into smaller methods and cacheing values as needed. I would change this class name as well since your Review class both represents a Review item and also scrapes (violation of Single Responsibility Principle). Maybe something like the following would be better?
require ‘nokogiri’
class VideoGameReviews::ReviewScraper
def reviews
#reviews ||= Nokogiri::HTML(open("http://www.ign.com/games/reviews?"))
end
def review_items
#review_items ||= reviews.search("#item-list div.itemList div.itemList-item")
end
def store_reviews
review_items.each do |review|
new_review = VideoGameReviews::Review.new #Review class still used to save review
new_review.score = review.search("span.scoreBox-score").text
#get other data
new_review.save! #or however you plan on persisting the data
end
end
end
The question will be: how will you save the reviews (in local memory, in a db, etc)? For something quick, ActiveRecord is pretty simple (and you use it independently from Rails).
Note that the :each method in Ruby will always return the original collection on which it's called. so for instance the following will return [1,2]:
[1,2].each do |n|
n * 4
end