I'm trying to write some programs to download a lot of economic data (on the order of hundreds of distinct tables from different websites, that'd need to be updated frequently). Take this website:
http://www.oecd-ilibrary.org/economics/country-statistical-profiles-key-tables-from-oecd_20752288
I want an R program to be able to click on one of those little green buttons that will download an xls file, so I don't have to click it by hand. Is there a package / function in R for this type of thing? (And if not, is there another simple-ish way to do it?)
Thanks!
The buttons just link to .xls files. So, you could use the URL of the page that the button points to, and use that URL as an input to a script / function that does the scraping. There are plenty of packages like rcurl that you could use to manage the download.
Related
I am now working with 500 pdf recipes files, which I want to display in my website. How can I batch extract them and display information on PDF to my website? PDF has all the information for recipes. For each recipe, I need to display its description, image, ingredients, instructions, nutrition label and so on. Is there any way so that I don't need to work on it manually?
Do these all have the same basic template for how the information is structured? This isn't really specifically a WordPress issue. One thing you can do is use Go to loop through and process all the files. I played with Go and it's incredibly fast to parse large amounts of information. Maybe you can fiddle with it in this library here https://github.com/unidoc/unidoc.
There are a lot of library options to try in PHP also. Here's just one example https://www.pdfparser.org/. There's documentation here and you can install it via composer. https://www.pdfparser.org/documentation
If every recipe follows the same sort of template, and you want to extract specific details in specific sections of the PDF, it should be easy enough. If you don't mind extracting all the text from a PDF and just display that on your website, it should be easy enough using one of the libraries. If you go the Golang route, you could just parse all the text for each PDF, save them to a file, and just upload them using PHP and have the PHP code insert everything into custom post types or something.
I would like to access the history of what have been typed in the source panel in RStudio.
I'm interested in the way we learn and type code. Three things I would like to analyse are: i) the way a single person type code, ii) how different persons type code, iii) the way a beginner improve typing.
Grabbing the history of commands is quite satisfying as first attempt in this way but I would like to reach a finer granularity and thus access the successive changes, within a single line in a way.
So, to be clear, I'm neither looking for the history of commands or for a diff between different versions of and .R file.
What I would like to access is really the successive alterations to the source panel that are visible when you recursively press Ctrl+Z. I do not know if there is a more accurate word for what I describe, but again what I'm interested in is how bits of code are added/moved/deleted/corrected/improved in the source panel but not necessary passed to the Console and thus absent from the history of command.
This must be somewhere/somehow saved by RStudio as it is accessible by the later. This may be saved in a quite hidden/private/memory/process/... way and I have a very vague idea of how a GUI works. I do not know it if would be easily accessible, then programmaticaly analyzed, typically if we could save a file from it. Timestamps would be the cherry on top but I would be happy without.
Do you have idea how to access this history?
RStudio's source panel is essentially a view to an Ace Editor. As such you'd need to access the editor session's editSession and use getDocument or getWordRange along with the undo of the editSession's undoManager instance.
I don't think you'll be doing that from within RStudio without hacking on the RStudio code unless the RStudio Addin api is made to pass-thru editor events in the future.
It might be easier to write a session recorder as changes are made rather than try to mess with the undo history. I imagine you could write an Addin that calls a javascript to communicate over the existing RStudio port using the Ace Editor's events (ie. onChange).
As #GegznaV said, RStudio saves code history to a ".RHistory" file. It's in the "Documents" folder on my computer. It's probably the same folder if you're using Windows. If you do not know the location, you can find the file by searching.
It also allows saving RStudio history to a file manually. There is a "Save" button in the History panel. So you can also get a timestamp. You can ask different users to save their code history after they have finished writing code. It may be indirectly useful to your research.
I have a COM Add In, in excel from RIMES. It basically allows me to get data, there is a specific button refresh all data. I am loading the data in R and do calculus. I would like to avoid having to open and refresh the excel file before loading the data. In other words I would like something to "click on refresh all". I figured I could do a VBA script that would do that, however I cannot figure what are the functions embedded in the excel add-in. How can one "explore" the com addin ?
Thanks
I was finally able to find a proper solution, turns out there are templates that are created form the COM ADD IN that uses the function refresh which is then available in VBA. One can then use a VB.script to create a way to compute everything without opening excel.
How To Create a VB.Script (StackOverflow)
Cheers
R
I am trying to do some work for my thesis which has me using the R programming language (and the R app on Mac OS Lion) for the first time. I had to download the files from here to work with but none of the files comes in .rda or .r they all come in different file extensions. Can someone please tell me how to either convert or use these files?
This page might be relevant to your interests. Also note that if you click on the link on that page that says "Analyze with GEO2R" you can get an R script created for you that will do some of the legwork for you. I would highly suggest reading the instructions on that page in order to get things prepared the way you want them to be.
I'm trying to automate some excel processing. With autoit I open several Excel with an add-in. This add-in is used to update data in this excel.
In autoit I managed to open the excel files with the add-in but now I need to select the menuitem to update the excel automatically. I don't have any clue howto do this in autoit3.
I can't find any tutorials or manual howto do this.
It sounds like you would be better off with an Excel macro instead. If you click "tools>macros>record new macro" and set the name of that auto_open then select the cells you want to update and update them and then stop recording, you will have a macro that refreshes the cells every time you open them. If you want a separate file that refreshes all the files you can make a new spreadsheet and start recording and use file open to have opening (or closing) a spread sheet included in the code. Don't forget to have that macro save too.
PS- If you need to open the file without the macro running you need to open excel first then hold shift when you open the file using the "file>open" path.
If you know the keystroke(s) for the menu command, you can use the AutoIt Send function.
For example, the Save function in Excel is CTRL+s.
In AutoIt, that would be:
Send("^s")
I see from your other question on StackOverflow that you are using the ExcelCOM library. You must realize that the MenuItem you are trying to click does some operation on the underlying code in the same way as the ExcelCOM library.
The best solution for your problem is to find out which methods the MenuItem calls, and then reproduce the same behavior with the ExcelCOM library. What you mean by "Update the excel automatically" I don't understand, otherwise I would have found out how to reproduce the behavior by ExcelCOM for you.
If you're using excel 2007 (I am not sure about earlier versions) and higher there is a combination of letters you can use in queue to call (probably) every menu option you want. When you have the excel open - press and release Alt button and you will see letters below main menu options, like in the picture, http://img.photobucket.com/albums/v479/vaeryn/Excel1_zps98c2f450.png Then letter by letter you may dig down the menu option you wish.