Ok this question is not exactly technical but very pertinent and current.
If you may have not heard Eureqa (http://creativemachines.cornell.edu/eureqa) is a machine learning (?) based tool that helps you find hidden equations and mathematical relationships within the data. It does sound futuristic and experimental and to a great degree it seems it is.
This is the relevant talk by Eureqa inventor Hod Lipson
http://www.youtube.com/watch?v=Xja6sLl6dVg entitled Mining experimental data.
So i believe this can become popular amongst many R users .
On the official site one can obtain Eureqa clients for MATLAB, Mathematica, Python etc but none so far for R.
So this question is just what it is , is anyone of you working on creating one...or if you know this project do you know what it will take to make one ?
I asked the same question on the Eureqa group a few months ago. Here's the link:
http://groups.google.com/group/eureqa-group/browse_thread/thread/cb251327b50dbd4f
The last entry has the following links:
http://r.eureqa.ivi.eu.com/
http://groups.google.com/group/eureqa-r
I haven't tried this, so I don't know if it works. If you try it, please post your results.
Related
R offers a breadth and depth in statistical computing beyond what is available in commercial
closed source products. Yet R remains, primarily, a programming language for the highly
skilled statistician, and out of the reach of many. --- The R Journal Vol. ½, December 2009
Note: Name changed from Interactive R Language Online Learning Platform: CloudStat School
As stated, R is the best tool and is the lingua franca of statistics. But many people, especially my students found difficulties to use R.
I wish to make an interactive R Learning Platform, called CloudStat School.
The best way to learn R programming is doing while learning.
In CloudStat School, you will see a console box at your top left hand side, while the lesson notes at your top right. Bottom is the output box. Anything you “Run” in console box will be shown as a result in the output box.
So, while learning the notes, you can “run” the R examples immediately without open another windows, software or tabs. You can do it in a page.
I did make a simple working prototype:
Lesson 1: Overview of R Language & CloudStat School
The prototype is simply integrating R Web that hosted in Pôle Bioinformatique Lyonnais in iframe.
If many of you think that this idea great, I would start making a better version.
This is my current simple idea, hope to get some feedback from you.
Thanks a lot.
It would probably be much more resource intensive and require more effort to create, but check this out: I found Code Academy to be a fun way to tinker with JavaScript. Unfortunately the site is (so far) only for a single language and a closed-source, venture-backed startup.
The main problem is that Rweb (as I am learning right now) does execute everything in batch, so this interpreted line-by-line approach used in CA probably cannot be done with it. If you were to create a similar app to CA for R, you'd have to open an R session for every user, hence the resource intensiveness disadvantage stated above. Hope this can be overcome, maybe someone will have an idea.
Hope you find this useful, at least as an inspiration for your endeavors. I wish you the best of luck.
A couple of pointers that might help: Eloquent Javascript and CodingBat.
Eloquent Javascript is an "interactive Hyperbook" where the students can edit and try out the examples right there in HTML as they are learning Javascript. Might be worth a look to get ideas for CloudStat.
In CodingBat, Stanford professor Nick Parlante's has been doing (for Java and Python) exactly what you are attempting to do for R learners. Especially relevant is the Authoring Page.
The success of your CloudStat School will be in getting crowdsourced contributions. To that end, my suggestion is for you to create 4-5 really good exercises with levels and hints, and then to focus on the 'meta' aspect of directing others to create the R exercises for you. Provide instructions for creating hints, tests, code, and tags. You could even consider assigning the task of 'creating new exercises' as a midterm/endterm projects to your R students for extra credit.
Hope that helps and good luck.
Commonly, there are two method of R Learning, one is step-by-step, like what CloudStat School is doing, as well as other R Language books and websites. This is good especially for those (newbie) learn R without specific purpose.
Another one is learn through problems. When you face specific problems, need specific functions, you are forced to learn it. Instead of "start from zero", the better way is learning through examples. This work even for experienced R users.
Since we want to make the best Interactive R Language Online Learning Platform, we need to add as many analysis examples/study cases here. If you need to get some ideas to make a statistical analysis with R, there is a place you will visit first. :)
I have the responsibility of ensuring that a colleague who is just learning R knows the basics before a course where that is a requirement. The colleague has gone through a couple of tutorials so hopefully she is ok, but I would like to give her a test to gauge it.
I was therefore wondering if anyone knew if there were any materials on the web that would be suitable? and is possible had both questions and answers.
PS Cross-posted to r-help#stat.math.ethz.ch
There is a whole set of exercises with solutions from the book Data Analysis and Graphics Using R. (Maindonald & Braun, 2nd edn, CUP 2007) available online : http://maths.anu.edu.au/~johnm/r-book/2edn/exercises/
Next to that, a quick search using the obscure randomized pagecollector Google brought to me a set of exercises where you can pick out whatever you want. Try the magic phrase "R exercises". ;)
Some I found interesting :
http://www2.imperial.ac.uk/~das01/RCourse/Exercises.pdf (very nice)
http://dial.liacs.nl/Courses/MicroArrayDataAnalysis/Exercises/Introduction_to_R_Exercises_Nov_2004.pdf (rather basic)
http://www.shlrc.mq.edu.au/masters/students/raltwarg/altwargslp802.html (rather basic)
If this is just a one time single person evaluation then a oral style exam is probably going to tell you a lot more than a set of fixed problems. Get a data set and have her read it into R, do some basic data manipulation, a couple of plots, and a standard analysis or two. Based on what she does well or has a challenge with you can modify the direction that you have things go and what additional questions you ask.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I've been using R for a little over a year now and it's been a successful venture. But all too often, I find that there is something that I can't figure out for lack of knowing how to find it or an example of it.
Stackoverflow,
Could you recommend a pathway for learning R in a manner that provides one with a toolset at their disposal to solve problems of a statistical nature?
There's a wealth of knowledge on the internet, between the r-project website and the mailings lists but it seems to be "everywhere" and nowhere when you're actually looking for it.
For example, when I first started using R, I went through "Intro to R". Then I read the language definition (which obviously hasn't sunk in). But every time I ask a question on Stackoverflow I'm presented with some new badass function that is the solution to all my problems in the short term. My question is, how did you know these functions existed in the first place? And how does one go about finding them? Presumably, you read something or found some resources that detoured your learning to the exponential part of the curve. What was it?
Obviously, R's functionality as a statistical tool is broad. For my own purposes I work mostly with economic or financial data. Hence, answers with this in mind would be most helpful.
Completely biased response: learn plyr, reshape2 and ggplot2. They will cover 90% of your data manipulation and visualisation needs. All three packages have a consistent philosophy of data (which the ggplot2 book touches upon), and are designed to be consistent and easier to
learn.
Rather than learning many specialised functions, I really encourage you to learn about simple functions that can be flexibly composed to solve a wide range of problems. This is what plyr strives to do for data manipulation, and what ggplot2 strives to do for visualisation. It does mean you need to invest more time up front to learn a little about the underlying theory, but it's my belief that it will pay off handsomely in the long run.
My way how I learned R.
R resources:
To learn R, the most important resource is google. search for: “TOPIC r-project”, “TOPIC filetype:r”, or “TOPIC site:nabble.com”.
Second, look at the example code provided with most packages. go to “http://bm2.genes.nig.ac.jp/”, search for a topic and look at the example code. run it and adapt it, this way you can often solve part of your problem.
Third: the r-help mailing list. Read the posts, the basic questions get asked over and over again. If you have a problem and you are completely stuck, ask a question on the mailing list.
Finally, look at the source code of the R-packages. that’s the hardest part. if you can alter the code to your needs, you have mastered R ;-)
Some Tips:
R has a steep learing curve. that’s a feature ;-) , it is designed to solve advanced problems and in the end you are fast than when using an alternative to R.
Know every single R package and function that is relevant to your problem. the strength of R is that there are so many packages availiable (around 2000, I think). Usually there is always a package that’s more suited or that already solves your problem. (some help pages are badly written and hard to understand - I got used to it)
R books are not helpful in learning R. yes, that’s true. If you are an expert programmer and expert statistician, you don’t need any book on R. (only exception is Hadley Wickham’s ggplot2 book). If your are not, learn programming in general and/or advanced statistics.
Some R package have known bugs, which nobody will fix (package owner left university, etc.). just a warning, this can be tricky if you are looking for a bug in your code and the bug is in a R package.
I'll start with this:
My question is, how did you know these functions existed in the first place?
Simple - we tried to solve a similar problem and came across that function. It either suited or didn't suit our needs but we now know it's there. I haven't used R much personally but what you're describing is the learning curve for every programming language ever. Firstly, you learn the "grammar" i.e. what you can do. Then you try to do something. You find you can't.
At that stage a programmer has a number of options. What do I do personally? Depends. I'll try and look up that package/header/library/whatever's member functions to see if something suits my needs. I might Google it, because unless you're really pushing the boundaries someone somewhere has probably tried and failed to do it before and had their question answered. If you are pushing the boundaries, someone somewhere has probably tried and failed before, but got no answer. I might try a forum or two to see what happens. I personally don't use IRC much, but that's another option, as are mailing lists depending on how specialised the problem is.
I also have a folder on my computer full of books which I search through depending on the problem and a small library of books I look through/learnt from, which often contain practical, not-quite-there-but-adaptable examples.
My only comment would be attempting to read the language specification is unlikely to be massively useful to you as a beginner. You won't fully understand what it means because you haven't pushed the bounds and tried things yet. For example, a novice in C might try this:
char c = '7';
int x = (int) c;
to convert the character '7' into an integer form. It's not a bad thought process until you understand how characters and ASCII work, then you see why the above doesn't give you what you want.
In short, I think this is going to be part of the learning process and I don't think you can cut it any shorter. The consolation is like any research, the more you do it the more you'll know where to look and what questions to ask on various communities.
One of the things I do is follow the RSS feed of R questions on SO (https://stackoverflow.com/feeds/tag/r). Then I can browse what other people have asked/answered.
Often I will favourite a particular question/answer if I think I'll use it, or jot down the salient points into my notebook software (OneNote), occaisonaly I'll even try the question/answer out myself.
EDIT:
I'd also recomend Patrick Burn's book R-Inferno. It's not so much of a training book as a description of all the gotchas and oooh moments Patrick has found (so far).
There's a free book you might be interested in: Introduction to Probability and Statistics Using R
Here is a good list of resources for learning R:
https://stats.stackexchange.com/questions/138/resources-for-learning-r
Also, that website in general is a good resource.
In general I would say that following a mailing list, or a help list is the best way I have found for learning new things. (That and the "R magazine": http://www.r-bloggers.com )
Learning the RODBC package to interact directly with Oracle data made a big impact at my job. My boss was amazed when I pulled Oracle data directly into R and cranking out a plot in only a few lines of code. Try doing that in Excel!
Moral of the story, learn how to pull in data and manipulate it within R. Then move to some of the cooler stuff like ggplot.
I can recommend Penn University's Introductory Course on R.
The ggplot chapter alone is worth reading - I found ggplot very confusing but this is a great explanation.
The book that helped my learning the most was The Art of R Programming. A lot of programming books can be dry. Since R is commonly an entry point to programming it's important for the voice of materials to resonante with the student. That book did just that with me. The voice felt very casual and I liked that.
Some interesting links:
Intro, links and examples: http://manuals.bioinformatics.ucr.edu/home/programming-in-r
A lot of documentation: https://en.wikibooks.org/wiki/R_Programming
R forum: http://r.789695.n4.nabble.com/
The [R] tag FAQ, right here on Stackoverflow, https://stackoverflow.com/questions/tagged/r?sort=frequent provides numerous reproducible examples that one can use to "learn by doing".
Most of the problems are very common and will eventually be something that you will have to look up as a beginner. The FAQ also provides highly literate (and experienced) examples of usage for a diverse range of functions and useful packages.
If you're new to R, and you prefer a more hands on approach to learning, the FAQ should not be overlooked as a potential resource for learning. Many of the questions also provide useful discussion surrounding paradigms of the language itself (vectorization, workflow, debugging are just a few examples).
Nearly every question in the FAQ is worth studying as a new user as it touches on elements that, speaking for myself, I wish I had been pointed to when I asked this question originally.
Just a few examples:
How to make a great R reproducible example
Grouping functions (tapply, by, aggregate) and the *apply family
Workflow for statistical analysis and report writing
How to sort a dataframe by multiple column(s)?
What is your favorite R debugging trick?
I’m trying to find some objective comparisons of Captaris and K2 workflow products. I haven’t been able to find anything through the usual channels so if anyone out there has an understanding of both products I’d really like to hear from you. I’m primarily interested in the pros and cons of each for use with ASP.NET and also with SharePoint. Thanks.
You're in for a good fight Troy :)
A couple years ago I was trying to answer the same question but we had Ultimus as the main option. At the time I found an even older PDF file with a few info (http://www.scribd.com/doc/6910318/DOt-Net-FAQ24) but we ended up getting their sales reps to give formal presentations.
We ended up with Ultimus for that project, but a few months later I saw a presentation from Nintex and it was interesting too, even tho the project related to it did not came thru.
For the sake of actually completing this question and providing an answer, there are no comparisons! At best there are several mentions of the two in the same sentence but these are many years old and certainly don’t rate as objective comparisons. Captaris is also now OpenText Workflow Server.
I'm interested in learning more about pattern recognition. I know that's somewhat of a broad field, so I'll list some specific types of problems I would like to learn to deal with:
Finding patterns in a seemingly random set of bytes.
Recognizing known shapes (such as circles and squares) in images.
Noticing movement patterns given a stream of positions (Vector3)
This is a new area of experimentation for me personally, and to be honest, I simply don't know where to start :-) I'm obviously not looking for the answers to be provided to me on a silver platter, but some search terms and/or online resources where I can start to acquaint myself with the concepts of the above problem domains would be awesome.
Thanks!
ps: For extra credit, if said resources provide code examples/discussion in C# would be grand :-) but doesn't need to be
Hidden Markov Models are a great place to look, as well as Artificial Neural Networks.
Edit: You could take a look at NeuronDotNet, it's open source and you could poke around the code.
Edit 2: You can also take a look at ITK, it's also open source and implements a lot of these types of algorithms.
Edit 3: Here's a pretty good intro to neural nets. It covers a lot of the basics and includes source code (albeit in C++). He implemented an unsupervised learning algorithm, I think you may be looking for a supervised backpropagation algorithm to train your network.
Edit 4: Another good intro, avoids really heavy math, but provides references to a lot of that detail at the bottom, if you want to dig into it. Includes pseudo-code, good diagrams, and a lengthy description of backpropagation.
This is kind of like saying "I'd like to learn more about electronics.. anyone tell me where to start?" Pattern Recognition is a whole field - there are hundreds, if not thousands of books out there, and any university has at least several (probably 10 or more) courses at the grad level on this. There are numerous journals dedicated to this as well, that have been publishing for decades ... conferences ..
You might start with the wikipedia.
http://en.wikipedia.org/wiki/Pattern_recognition
This is kind of an old question, but it's relevant so I figured I'd post it here :-) Stanford began offering an online Machine Learning class here - http://www.ml-class.org
OpenCV has some functions for pattern recognition in images.
You might want to look at this :http://opencv.willowgarage.com/documentation/pattern_recognition.html. (broken link: closest thing in the new doc is http://opencv.willowgarage.com/documentation/cpp/ml__machine_learning.html, although it is no longer what I'd call helpful documentation for a beginner - see other answers)
However, I also recommend starting with Matlab because openCV is not intuitive to use.
Lot of useful links on this page on computer vision related pattern recognition. Some of the links seem to be broken now but you may find it useful.
I am not an expert on this, but reading about Hidden Markov Models is a good way to start.
Beware false patterns! For any decently large data set you will find subsets that appear to have pattern, even if it is a data set of coin flips. No good process for pattern recognition should be without statistical techniques to assess confidence that the detected patterns are real. When possible, run your algorithms on random data to see what patterns they detect. These experiments will give you a baseline for the strength of a pattern that can be found in random (a.k.a "null") data. This kind of technique can help you assess the "false discovery rate" for your findings.
learning pattern-recoginition is easier in matlab..
there are several examples and there are functions to use.
it is good for the understanding concepts and experiments...
I would recommend starting with some MATLAB toolbox. MATLAB is an especially convenient place to start playing around with stuff like this due to its interactive console. A nice toolbox I personally used and really liked is PRTools (http://prtools.org); they have an implementation of pretty much every pattern recognition tool and also some other machine learning tools (Neural Networks, etc.). But the nice thing about MATLAB is that there are many other toolboxes as well you can try out (there is even a proprietary toolbox from Mathworks)
Whenever you feel comfortable enough with the different tools (and found out which classifier is perfomring best for you problem), you can start thinking about implementing the machine learning in a different application.