I am having some trouble being able to pick up data on the following website: https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/Dashboard/Medicare-Drug-Spending/Drug_Spending_Dashboard.html
It's interactive nature makes it difficult for me to do, but I really just want the first table you see with the different drug names. I am tried inspecting different elements in Chrome to try to find the data source, but I cannot find any raw files. Any ideas how I could approach this problem?
Related
how are you. For a while I've been working for a Gynecologist building her a data base. For the project I am using Firebase and JavaScript. The database is for her to keep track of their patients and she keeps reports on each one of them. I am almost done with the job, the UI is almost finished, the core functionalities of the database (save data, delete, retreive, and update) are up and running but I am stuck in one little thing. She asked me for a way to turn those reports she keeps in the database into a format like PDF so she can print them and give them in case needed to her patients. The thing is that Ive tried with html2pdf, a git repository that works kind of clunky, and tried looking for others but I still cant find one that works correctly. So I wanted to ask you guys if you know of some alternatives. I started thinking about using EXCEl or Word document. But either way it seems quite complicated. Thank you for your time.
Best to all.
I am trying to harvest the world football elo ratings with rvest but I keep getting an empty list
Using the inspect element in google chrome I get the xpath //*[(#id = "maintable_2014_World_Cup_start")]/div[6]
library(rvest)
library(dplyr)
page<-"http://www.eloratings.net/2014_World_Cup_start"
elo_rating<-read_html(page)%>%
html_nodes(xpath='//[#id="maintable_World"]/div[6]')%>%
html_table()
I get an empty list
Searching online and within SE, I came across this and perhaps it has to do something with javascript (which I know nothing about..:). Also, when at the page source (with google chrome) I see a lot of calls to javascript
Lastly, I came across this R publication, with an example of extracting data from the same website, but when I try to replicate the R code, I still get empty lists and empty character objects
I went through many threads here in SE (this, this, this but I can't find a solution
If the obstacle is javascript, is there anything I can do to extract the data?
The obstacle does seem to be javascript as the tables are generated by it. I think you need to use PhantomJS to render the tables and grab them. See this page for help.
I am wondering if there is a way to click items in a data set or click items on the Menu, to create code that plots charts and graphs in R. In Excel, the Macro Recorder does this pretty well, although it tends to spit out a lot more code than you actually need, but it is easy to go through the auto-generated code, clean it up, and in no time at all, you have what you need to create all kinds of visuals with VBA code. Is there something similar in R?
I am reading a book about charts in R, and there are many great samples, with all kinds of code, but I don't see how it is easy to memorize all of this stuff!
Maybe there is some tool or package can be added in to do this work... I'm just wondering... Thanks.
I would really like to take the data that would normally be seen by hovering over each column at once. The graph is an interactive one so its hard to extract all the data at once. I would really like it.
I suggest that you pick a programming language that you know fairly well.
Then load the web pages, use a selector to select the desired elements, and output the data in the format you like.
Please begin writing the code, and update your question when you have something working at least partially, so you can ask precisely where you need help
I am trying to automate web scraping for different Physician Names. The process is I am having a list of Physician names in .csv file
The first process is, the names of the Physician should be entered in the search bar of this site.
Then the search button is to be hit.
Then the first link is to be selected.
Then I want to perform web scraping to collect required details of the Physician.
These are the things to be performed.
The same thing is to be applied for every Physician.
Can anyone help me with this process using R?
Google searching 'web scraping with R' brought me this tutorial and this tutorial. Both of these seem simple enough that you should be able to accomplish what you need. Also, heed hrbrmstr's warning, and see if you can acquire the data you need with abusing metacrawler's website.