In Python, I'm trying to read the values on http://utahcritseries.com/RawResults.aspx. How can I read years other than the default of 2002?
So far, using mechanize, I've been able to reference the SELECT and list all of its available options/values but am unsure how to change its value and resubmit the form.
I'm sure this is a common issue and is frequently asked, but I'm not sure what I should even be searching for.
So how about this:
from mechanize import Browser
year="2005"
br=Browser()
br.open("http://utahcritseries.com/RawResults.aspx")
br.select_form(name="aspnetForm")
control=br.form.find_control("ctl00$ContentPlaceHolder1$ddlSeries")
control.set_value_by_label((year,))
response2=br.submit()
print response2.read()
With problems relating to AJAX-loading of pages, use Firebug!
Install and open Firebug (it's a Firefox plugin), go to the Net page, and make sure "All" is selected. Open the URL and change the select box, and see what is sent to the server, and what is received.
It seems the catchily-named field ctl00$ContentPlaceHolder1$ddlSeries is what is responsible.. Does the following work..?
import urllib
postdata = {'ctl00$ContentPlaceHolder1$ddlSeries': 9}
src = urllib.urlopen(
"http://utahcritseries.com/RawResults.aspx",
data = urllib.urlencode(postdata)
).read()
print src
Related
I am trying to scrape this website: https://www.casablanca-bourse.com/bourseweb/Societe-Cote.aspx?codeValeur=12200 the problem is i only want extract data from the tab "indicateurs clès" and i can't find a way to access to it in code source without clicking on it.
Indeed, i can't figure out the URL of this specific tab... i checked the code source and i found that there's a generated code that changed whenver i clicked on that tab
Any suggestions?
Thanks in advance
The problem is that this website uses AJAX to get the table in the "Indicateurs Clès", so it is requested from the server only when you click on the tab. To scrape the data, you should send the same request to the server. In other words, try to mimic the browser's behavior.
You can do it this way (for Chromium; for other browsers with DevTools it's pretty much similar):
Press F12 to open the DevTools.
Switch to the "Network" tab.
Select Fetch/XHR filter.
Click on the "Indicateurs Clès" tab on the page.
Inspect the new request(s) you see in the DevTools.
Once you find the request that returns the information you need ("Preview" and "Response"), right-click the request and select "Copy as cURL".
Go to https://curl.trillworks.com/
Select the programming language you're using for scraping
Paste the cURL to the left (into the "curl command" textarea).
Copy the code that appeared on the right and work with it. In some cases, you might need to inspect the request further and modify it.
In this particular case, the request data contains `__VIEWSTATE` and other info, which is used by the server to send only the data necessary to update the already existing table.
At the same time, you can omit everything but the __EVENTTARGET (the tab ID) and codeValeur. In such a case the server will return page XHTML, which includes the whole table. After that, you can parse that table and get all you need.
I don't know what tech stack you were initially going to use for scraping the website, but here is how you can get the tables with Python requests and BeautifulSoup4:
import requests
from bs4 import BeautifulSoup
params = (
('codeValeur', '12200'),
)
data = {
'__EVENTTARGET': 'SocieteCotee1$LBFicheTech',
}
response = requests.post('https://www.casablanca-bourse.com/bourseweb/Societe-Cote.aspx', params=params, data=data)
soup = BeautifulSoup(response.content)
# Parse XHTML to get the data you exactly need
Tokenize2 is a javacsript lib to select multiple options.
It provides a very neat UI to start writing and then get a list of options to select from. Selected options will show up as "tags" that can be removed with "x" link.
So far all is fine. But Right now you need to know what your looking for and start write at least one character to see matching alternatives.
In my scenario there are very few alternatives and they are not known to the user. I would like to show ALL options when the user clicks the input box. There is a configuration option named searchMinLength but it is already set to 0.
Is there a workaround that can be used? Maybe like triggering load and dropdown manually?
I know there are a lot of similar alternatives but I picked Tokenize2 because:
It looks clean and nice
It works in mobile browsers
I don't know if there is an "official" approach, but after some investigation I have found an acceptable workaround.
After downloading the Tokenizer2 sourceode I found the following line that triggered my attention:
if(this.input.val().length > 0){
this.trigger('tokenize:search', [this.input.val()]);
}
My interpretation is that the internal search command is not triggered unless the user input has at least one character. This line in sourcecode could easily be modified. I have filed a suggestion for this here: https://github.com/zellerda/Tokenize2/issues/26
My current workaround is to add an event listener for the select event and there trigger the internal search command. That works fine for my scenario and does not force a source code rewrite.
$("#my-dropdown").on("tokenize:select", function (e: Event, routedEvent: boolean) {
$("#my-dropdown").trigger('tokenize:search', "");
});
Tokenize2
This link worked for me GitHub
$('.tokenize-sample-demo1').on('tokenize:select', function(container){
$(this).tokenize2().trigger('tokenize:search', [$(this).tokenize2().input.val()]);
});
I am using MediaWiki (Sometimes I think that it could be better to use Drupal) to create a wiki.
I have tried to find out a correct api or something similar to import a table (csv, xml, or another format) with text fields.
The idea is to bring a document with "name-pages" and "tags" to create automatically empty pages.
Finally, the users will see that there are new empty pages to fill!.
And every day pass a scheduler (something like Feed Import in Drupal) to bring new pages. I mean, if the text exists, don't do anything; however, it the text is new, create a new wikimedia page!
I don't find the correct api to do this. Somebody knows any way to do this?
Thank you
Regards!
Is there a way to open a specified document, eg "production order 123" or form, eg "purchase orders" in Ax2012 from an external application directly?
In detail, I'm looking for something similiar like AXPath, but this doesn't work with versions greater then 2009.
Is there any ( maybe included ) way to achieve this?
There is! It's using AX's drilldown functionality which uses AxHLink.exe to handle dynamics:// URLs, which are passed to the Classes\SysStartupCmd function. You could also create some custom code there if you wanted to launch the AX client executable directly.
My question I asked some while back should have a great deal of useful information in it here:
What handles dynamics:// URLs?
Some more can be found: http://technet.microsoft.com/en-us/library/aa834337.aspx
EDIT:
It sounds like you are confused or the posts weren't clear enough. I think you have 3 basic options.
Dynamics:// URLs are handled by AxHLink.exe and they only seem to handle drilldown, viewalert, and viewalertrule. So if you want to use Dynamics:// URLs, you will need to hi-jack those somehow. There is a pastbin from Jan in that other stack post.
Create a custom URI handler and event poller (lot of work) see http://axcoder.blogspot.dk/2010/10/how-to-open-form-in-running-ax-from.html
Extend SysStartupCmd and then instead of using Dynamics:// URLs, just call Ax32.exe -startupCmd directly and a parameter can be passed to your custom class.
An issue has been noticed on one of our old sites running 2.4 where when the user creates a link in the CMS content, selecting an existing page to link to, the link is not being converted to the actual URL on the front end and all links are coming through in the format of <a href="[sitetree_link_id=12]">
What would be causing this and how do I fix it?
The tag looks like it's being set incorrectly. It should be [sitetree_link id=12], not [sitetree_link_id=12].
We later added support to the parser for [sitetree_link,id=12] so that links didn't need to contain spaces, but I can't recall if that's in 2.4 or only 3.0+.
Can you confirm that your WYSIWYG insertion is putting in that errant _? If so, you might want to checkout the handleaction_insert function in tiny_mce_imporvements.js to confirm that it has a line like so:
case 'internal':
href = '[sitetree_link id=' + this.elements.internal.value + ']';
If the inserted links don't actually have the errant _ but they aren't being parsed, then try checking your sapphire/_config.php file for this:
ShortcodeParser::get('default')->register('sitetree_link', array('SiteTree', 'link_shortcode_handler'));
If your site makes changes to the ShortcodeParser at all you might have inadvertently turned off sitetree_link support.
If all of that looks in order, perhaps the ShortcodeParser isn't being called for some reason. In HTMLText::forTemplate(), put a debug statement (I like die("I got here!");) to confirm that HTMLText::forTemplate() is actually getting called. If it's not, you might need to manually call it in some pre-processing of your Content variable. Instead of this:
$content = $this->Content;
Do this:
$content = $this->obj('Content')->forTemplate();
I hope that one of those answers help. Either way, it would be great if you could post back, so we could isolate what caused this. It might help us make the API easier to use in SilverStripe 3.1.