Web automation with R code - r

Is it possible to get information from a url by making search on it, under R with any package?
I would like to make a loop for searching different code on the url and click on a precise place to write the code and to download document. A software does that, Actionaz, but I don't manage to make a loop with different code executing one after one.
Thank's,
Xavier

I found on forum "RSelenium" package like you suggest but it is not avalable on the CRAN ? I will see the over packages. I would add some informations of what i would do to know if it's realy possible on not ^^ :
1/open the url : http://hydro.eaufrance.fr/selection.php?consulte=rechercher
2/clic on the search region (precise point on screen)
3/type a name of station the first : "K0120020"
4/Search it on the website by clic on buton search
5/Select the station found with a clic
6/an over clic to get access to the data
7/Select the year on a dawn drop menu
8/Export data by clic on buton export

Related

How can I automate data extraction in this website? (can't see query options, if any)

I want to extract election results from this website (currently all data is at zero because voting is ongoing). In the menu you can select the data you want (I need "En Chile" -> "División Geográfica" -> "Comunas"). This is the voting results for each municipality ("comuna"). When you select the desired "comuna", then you can click the excel file and results are downloaded.
The "problem" is that no url is shown along the process. I cannot see the url link to the particular "comuna" I need. I was hoping to get a specific link so then I can automate using wget with the name of each municipality. Instead, the data seems to be masked. I know you can extract JSON data, perhaps using the properties shown in the website's HTML code:
<select class="form-control" id="selComunas" ng-model="comuSelected"
ng-options="item.d for item in comunas" name="comunas"
ng-show="(vistaVertical==='G' || vistaVertical==='E') && subMenu === null"
ng-change="updateComuna()">
<option value="">Comunas...</option>
</select>
But where to make the call? No idea. Also, I see no link to the excel file in the HTML. Stuff seems to be masked inside "ng" elements, which seems to be AngularJS, as far as I can read online. No idea how to proceed. As you can see, I'm noob on this. Any help is more than welcome.
Open chrome dev tools and look for something useful there while clicking on the UI
http://www.servelelecciones.cl/data/elecciones_constitucion/filters/comunas/all.json
This link returns all communas with their ids
[{"c":2564,"d":"ALGARROBO"},{"c":2801,"d":"ALHUE"},{"c":2674,"d":"ALTO BIOBIO"},...
Next thing, when we select value from dropdown, we get another link in network tab. Number in the end matches, isn't it?
http://www.servelelecciones.cl/data/elecciones_constitucion/computo/comunas/2570.json
"d":"Integer",
"e":"Integer",
"f":"Integer",
"sd":null
},
"data":[
{
"a":"Apruebo",
"b":null,
"c":"0",
"d":"0,00%",
"e":null,
"f":"",
"sd":null
},
{
"a":"Rechazo",
"b":null,
"c":"0",
"d":"0,00%",
"e":null,
"f":"",
"sd":null
}
From this point you can process JSON directly by whatever programming language you use

Show all Available Bookable Product in "WooCommerce Bookings" extension

User select date, time and then search
After click "Search Availability", it is needs to show all the available product within the provided date and time slot.
Using "WooCommerce Bookings" extension, how to do it?
I have customized both WooCommerce & Booking Extension:
To save bookable product :
woocommerce-bookings/includes/class-wc-bookings-admin.php
Method executes saving :
save_product_data( $post_id ) at line 434
File for searching listing page:
woocommerce/templates/archive-product.php
Skip loading boooking:
/wp-content/plugins/woocommerce-bookings/includes/booking-form/class-wc-booking-form.php
skip loading date time:
/wp-content/plugins/woocommerce-bookings/assets/js/booking-form.js
/templates/single-product/add-to-cart/booking.php
I tried to solve it as database driven solution first but data seems like unstructured. Also without identifying the Property and Method at the Code, you won't be able to make any customization. Doing customization the plugin and extension is easy but he hard part is understand the code structure and flow. Once you get it, you may do any level of customization.
There are lots of files (.PHP,JS) you needs to check but I have made it narrow for you. You only check the mentioned file above and can do your modification most of the case.

Retain Project Find Result Tab in Atom Editor

I'm trying to use Atom 1.8.0 for grokking code, at most a few thousand Java and XML files. When I perform a global search with Ctrl+Shift+F , the results appear in a tab named "Project Find Results". However, if I do another global search the new results appear in the same tab, replacing older results. I'd like Atom to open show results in a new tab every time. Is it possible?
Note: There seems to be a similar feature request here but I cannot access GitHub at work.

xls form for conducting survey using android phone

Am working on an xls survey form which when I loaded onto ODK to look for errors it produced this error"You must have a sheet named(case-sensitive):survey"which I dont quite understand.
It's referring to the worksheet in your excel file. Instead of the normal "Sheet 1" etc it needs to be named "survey".
I recommend you use http://ona.io to load and manage your forms. You can sign up and create an account there.. then your profile URL becomes the server URL that you have to enter in the ODK Collect app on the phone. They have given a basic example xlsform that should help you get started.. https://ona.io/examples/
Further documentation on making XLSForms : http://xlsform.org

Why is ImportXML not working for a specific field while trying to scrape kickstarter.com?

I am trying to screen scrape funding status of a specific Kickstarter project.
I am using following formula in my Google spreadsheet, what I am trying here is to get the $ amount of project's funding status:
=ImportXML("http://www.kickstarter.com/projects/1904431672/trsst-a-distributed-secure-blog-platform-for-the-o","//data[#class='Project942741362']")
It returns #N/A in the cell, with comment:
error: The xPath query did not return any data.
When I try using ImportXML on other parts of the same webpage it seems to work perfectly well. Could someone please point out what I am doing wrong here?
It seems that the tag "data" is not correctly parsed.
A choice of workaround may be:
=REGEXEXTRACT(IMPORTXML("http://...", "//div[#id='pledged']"), "^\S*")

Resources