I am going to scrape NFL depth charts from web pages, e.g. https://www.ourlads.com/nfldepthcharts/archive/220/BUF . Now I want to get all the links to this kind of pages, but the source code of the dropdown menu "Archive Dates" does not include any links:
<option value="">-- Archive Dates --</option>
<option value="220">05/01/2019</option>
<option value="219">04/01/2019</option>
<option value="218">03/01/2019</option>
<option value="217">02/01/2019</option>
<option value="216">01/01/2019</option>
<option value="215">12/01/2018</option>
<option value="214">11/01/2018</option>
<option value="213">10/01/2018</option>
<option value="212">09/01/2018</option>
<option value="211">08/01/2018</option>
I read a post Web scrape get drop-down menu data python , which I think is helpful since he indicated that the web page uses JavaScript.
But that answer uses selenium. I wonder if I can solve the problem using scrapy or beautifulsoup.
The following is the structure of my scraper.
class depth_chart_archive_spider(scrapy.Spider):
name = "depth_chart_archive"
start_urls = ('https://www.ourlads.com/nfldepthcharts/',)
def parse(self, response):
dchome = BeautifulSoup(response.body, 'html.parser')
# get the links somehow
for link in links:
yield scrapy.Request(link, callback = self.parse_team)
def parse_team(self, response):
# parse the page
You can build the URL using the value parameter found on each option tag.
For example, the menu that refer to 05/01/2019 has value=220 in the option tag
<option value="220">05/01/2019</option>
The url opened when you click on this menu is:
https://www.ourlads.com/nfldepthcharts/archive/220/BUF
So it follows a pattern, you can request all items using something like:
site_url = 'https://www.ourlads.com/nfldepthcharts/archive/{code}/BUF'
for code in response.xpath('//option/#value').re(r'\d+'):
yield Request(site_url.format(code=code))
The regex is only to avoid requesting the first item <option value="">-- Archive Dates --</option>
Related
I would like to ask how I can focus on a specific item in the long list via a search bar.
example: when a user enters something on the search bar, the list is filtered to find the elements that contain that search key, I would like that if at least one result is found, the focus is placed on the first element found.
when user visit the page:
after the user search a something:
thanks!
i don't know if this solves your problem or not u can use datalist tag in html which work kind of similar it will filter everything except what u have typed in search box instead of focusing at that element.
<label for="browser">Choose your browser from the list:</label>
<input list="browsers" name="browser" id="browser">
<datalist id="browsers">
<option value="Edge">
<option value="Firefox">
<option value="Chrome">
<option value="Opera">
<option value="Safari">
</datalist>
Ok, so i have a simple thing on my website where users can enter a phone number and text them, so when they enter the phone number they need to select an item in a dropdown list with all country dial codes, the problem is that the dropdown list is as long as my textbox which makes everything look quite awkward, I've done some research and have messed around with it myself without any luck, just for an example here is what i am talking about (btw i use the bootstrap framework so any answer would need to support bootstrap)
As you can see the text box is actually smaller than the dropdown list which makes it look kinda weird, i would like to somehow scale down the dropdown so it looks better along side my text field.
Thanks for reading, any help will be appreciated!
EDIT:
My current code for this <select> dropdown looks like this
<select class='form-control'>
<option value='213'>Algeria (+213)</option>
<option value='376'>Andorra (+376)</option>
<option value='244'>Angola (+244)</option>
<option value='1264'>Anguilla (+1264)</option>
<option value='1268'>Antigua & Barbuda (+1268)</option>
<option value='54'>Argentina (+54)</option>
<option value='374'>Armenia (+374)</option>
<option value='297'>Aruba (+297)</option>
<option value='61'>Australia (+61)</option>
//There is more countries..... I just cut it here so it won't be too long
</select>
I have the following code:
<form:select path="roles" items="${roleList}" itemLabel="roleType" itemValue="id" />
It generates html as below:
<select id="roles" name="roles" multiple="multiple">
<option value="1">ROLE_ADMIN</option>
<option value="2">ROLE_HQ</option>
<option value="3">ROLE_MASTER</option>
<option value="4">ROLE_STATE</option>
<option value="5">ROLE_CENTRE</option>
</select>
Also I do not use the multiple optional attribute. Any idea why does the generated HTML contain "multiple="multiple" ?
Just a guess, but since you mapped it on roles, and since roles is probably a collection, it makes sense to make the select box multiple. If it was not multiple, you would only be able to store a single item in the collection. And the tag would not be able to display the selected roles.
EDIT: after reading the source code of the tag, it appears my guess was right. See the forceMultiple() method in SelectTag.
Question - Re: R brew package with HTML (using Rook)
When using forms to retrieve queries, usually when the screen refreshes, the user's preselected option remains selected in the form. For eg., if there was a name field and I enter John,... enter other form data, hit Submit, when the page refreshes, the form will still show "John" in the name field instead of an empty box.
I can pre-fill an entry in a text type input field using something like --
...
<input class="someclass" type="text" name="Name" value="<%=Name%>" />
... (other form data)
When the user hits submit and the page refreshes, R prefills the name field with the value from <%=Name%>
I'm having some difficult in replicating the same when using drop-down menus. The idea is that when the page refreshes, the option that the user had selected in the previous query will be active.
<select class="someclass" name="group3" style="width:200px;">
<option value="," selected>None</option>
<option value="Name,">Name)</option>
<option value="Phone">Phone</option>
</select>
I think one way could be to enter something like say,
<%if (group3=="Name"){ print("selected") } ... etc for each of those options in the respective lines, but that is a bit cumbersome. Could you please share your thoughts on the same,
Got this working using jQuery finally. The inline <%= if(..) print ("selected") ... within the tags wasn't working.
Add the id=something tag to select --
<select class="someclass" name="group3" id="myselect" style="width:200px;">
<option value=",">None</option>
<option value="Name">Name)</option>
<option value="Phone">Phone</option>
</select>
Add jquery.js to the html file in the header--
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.5/jquery.min.js"></script>
In my Rook brew file, I have something like,
forminput <- req$GET()
userselection <- forminput$group3 #group3 is the class name for the select statement
and in the HTML document, before the closing form tag, enter the following --
<script>
$("#myselect").val("<%=userselection%>").attr("selected", "selected");
</script>
How to design a dropdown on XSLT? I need a dropdown with four values
option 1
option 2
option 3
option 4
I have the following on the XSLT
<div>
<select>
<option value="option1">Option 1</option>
<option value="option2">Option 2</option>
<option value="option3">Option 3</option>
<option value="option4">Option 4</option>
</select>
</div>
I wanted to pass the selected value from the dropdown to the asp.net
XSLT has nothing to do with "designing a dropdown". It is an XML-based language to transform an input XML tree into some other form of output. XSLT has no concept of a "dropdown", that would be in whatever web or UI system you're using.
First design the output, in other words, decide specifically what HTML you want to produce.
Then determine the input XML and figure out what parts of it need to be reflected in the output dropdown.
If you can show us those two things, we can help you figure out what sort of XSLT to use.
FYI there is an example here of an XSLT template for generating an OPTION/SELECT. You could try that and let us know if you need help figuring out how to use it.