I am trying to learn R, and use the corrplot library to draw Y:City and X: Population graph. I wrote the below code:
When you look at the picture above, there are 2 columns City and population. When I run the code I get this error message:
Error in cor(Illere_Gore_Nufus) : 'x' must be numeric.
My excel data:
In general, correlation plot (Scattered plot) can be plotted only when you have two continuous variable. Correlation is a value that tells you how two continuous variables are linearly related. The Correlation value will always fall between -1 and 1, where correlation value of -1 depicts weak linear relationship and correlation value of 1 depicts strong linear relationship between the two variables. Correlation value of 0 says that there is no linear relationship between the two variables, however, there could be curvi-linear relationship between the two variables
For example
Area of the land Vs Price of the land
Here is the Data
The correlation value for this data is 0.896, which means that there is a strong linear correlation between Area of the land and Price of the land (Obviously!).
Scatter plot in R would look like this
Scatter plot
The R code would be
area<-c(650,785,880,990,1100,1250,1350,1800,2200,2800)
price<-c(250,275,280,290,350,340,400,335,420,460)
cor(area,price)
plot(area,price)
In Excel, for the same example, you can select the two columns, go to Insert > Scatter plot (under charts section)
Scatter plot
In your case, the information can be plotted in bar graph with city in y axis and population in x axis or vice versa!
Hope I have answered you query!
Some assumptions
You are asking how to do this in Excel, but your question is tagged R and Power BI (also RStudio, but that has been edited away), so I'm going to show you how to do this with R and Power BI. I'm also going to show you why you got that error message, and also why you would get an error message either way because your dataset is just not sufficient to make a correlation plot.
My answer
I'm assuming you would like to make a correlation plot of the population between the cities in your table. In that table you'd need more information than only one year for each city. I would check your data sources and see if you could come up with population numbers for, let's say, the last 10 years. In lack of the exact numbers for the cities in your table, I'm going to use some semi-made up numbers for the population in the 10 most populous countries (following your datastrutcture):
Country 2017 2016 2015 2014 2013
China 1415045928 1412626453 1414944844 1411445597 1409517397
India 1354051854 1340371473 1339431384 1343418009 1339180127
United States 326766748 324472802 325279622 324521777 324459463
Indonesia 266794980 266244787 266591965 265394107 263991379
Brazil 210867954 210335253 209297939 209860881 209288278
Pakistan 200813818 199761249 200253292 197655630 197015955
Nigeria 195875237 192568158 195757661 191728478 190886311
Bangladesh 166368149 165630262 165936711 166124290 164669751
Russia 143964709 143658415 143146914 143341653 142989754
Mexcio 137590740 137486490 136768870 137177870 136590740
Writing and debugging R code in Power BI is a real pain, so I would recommend installing R studio, write your little R snippets there, and then paste it into Power B.
The reason for your error message is that the function cor() onlyt takes numerical data as arguments. In your code sample the city names are given as arguments. And there are more potential traps in your code sample. You have to make sure that your dataset is numeric. And you have to make sure that your dataset has a shape that the cor() will accept.
Below is an R script that will do just that. Copy the data above, and store it in a file called data.xlsx on your C drive.
The Code
library(corrplot)
library(readxl)
# Read data
setwd("C:/")
data <- read_excel("data.xlsx")
# Set Country names as row index
rownames(data) <- data$Country
# Remove Country from dataframe
data$Country <- NULL
# Transpose data into a readable format for cor()
data <- data.frame(t(data))
# Plot data
corrplot(cor(data))
The plot
Power BI
In Power BI, you need to import the data before you use it in an R visual:
Copy this:
Country,2017,2016,2015,2014,2013
China,1415045928,1412626453,1414944844,1411445597,1409517397
India,1354051854,1340371473,1339431384,1343418009,1339180127
United States,326766748,324472802,325279622,324521777,324459463
Indonesia,266794980,266244787,266591965,265394107,263991379
Brazil,210867954,210335253,209297939,209860881,209288278
Pakistan,200813818,199761249,200253292,197655630,197015955
Nigeria,195875237,192568158,195757661,191728478,190886311
Bangladesh,166368149,165630262,165936711,166124290,164669751
Russia,143964709,143658415,143146914,143341653,142989754
Mexcio,137590740,137486490,136768870,137177870,136590740
Save it as countries.csv in a folder of your choosing, and pick it up in Power BI using
Get Data | Text/CSV, click Edit in the dialog box, and in the Power Query Editor, click Use First Row as headers so that you have this table in your Power Query Editor:
Click Close & Apply and make sure that you've got the data available under VISUALIZATIONS | FIELDS:
Click R under VISUALIZATIONS:
Select all columns under FIELDS | countries so that you get this setup:
Take parts of your R snippet that we prepared above
library(corrplot)
# Set Country names as row index
data <- dataset
rownames(data) <- data$Country
# Remove Country from dataframe
data$Country <- NULL
# Transpose data into a readable format for cor()
data <- data.frame(t(data))
# Plot data
corrplot(cor(data))
And paste it into the Power BI R script Editor:
Click Run R Script:
And you're gonna get this:
That's it!
If you change the procedure to importing data from an Excel file instead of a textfile (using Get Data | Excel , you've successfully combined the powers of Excel, Power BI and R to produce a scatterplot!
I hope this is what you were looking for!
Is there a way to remove unused levels in a SpatialPolygonsDataFrame object in R?
I have large shapefile of geological data Geology that I am clipping with the raster::crop tool. This seems to work fine.
But when I try to work using my new, cropped shapefile Geo, polygon types present in Geology but absent in area covered by Geo still appear as levels in Geo. This interferes with my later analysis.
I have tried to remove these "ghost" levels/attributes using droplevels, but this function is not valid for SpatialPolygons or SpatialPolygonsDataFrame objects.
For reference, I am using the wygeol_dd_polygon.shp shapefile (downloadable here - 41.4 MB) as a starting point. The salient parts of my code are below:
library(maptools)
Geology <- readShapePoly("~/wygeol_dd_polygon.shp")
library(raster)
Geo <- crop(Geology, extent(-111.05, -110.25, 44.2667, 44.7667))
After cropping, I have ten unique rock types, but still 46 levels:
unique(Geo$ROCKTYPE1)
[1] alluvium rhyolite mixed clastic/volcanic intermediate volcanic rock
[5] basalt water trachyandesite sandstone
[9] conglomerate shale
46 Levels: alkalic intrusive rock alkalic volcanic rock alluvium andesite anorthosite basalt carbonate clastic ... water
How do I get rid of these?
Try this:
Geo#data<-droplevels(Geo#data)
The above will handle all the factor columns in one call.
The column you are having issues with as a factor variable. When you crop factors in R, it often retains the cropped factors even though you no longer have any remaining in your dataset. Gladly it's an easy fix as follows:
Geo$ROCKTYPE1 <- factor(Geo$ROCKTYPE1)
This redefines the factor so now you should only have 10 levels, as you want.
I have a problem for building barplot.
I am working on air traffic in different countries. I would like to get barplots for each countries with the different airport names in the X axis. The Y axis will show the quantity of airlines using the airport.
My plan is to make the script for 1 country and to replicate it manually for the others.
in my data, I have in the different columns:
Country / aiport / destination.
So each rows is actually one airline that is using the airport.
Do you have an idea about how to do this?
For now I have this idea:
UK<-traffic[traffic$Country=="UK",]
UK$airport <- as.factor(UK$airport)
countUK<-table(UK$airport)
barplot(countUK)
This is not working, I have a bunch of airports that are not in UK in the X axis...
Thanks for your help
Answer found:
You could try to drop unused factor levels, i.e.
UK <- droplevels(UK) after the line UK$airport <- as.factor(UK$airport).
I have a list of boroughs and a list of localities (like this one). Each locality lies in exactly one borough. What's the best way to store this kind of hierarchical structure in R, considerung that I'd like to have a convenient and readable way of accessing these, and using this list to accumulate data on the locality-level to the borough level.
I've come up with the following:
localities <- list("Mitte" = c("Mitte", "Moabit", "Hansaviertel", "Tiergarten", "Wedding", "Gesundbrunnen",
"Friedrichshain-Kreuzberg" = c("Friedrichshain", "Kreuzberg")
)
But I am not sure if this is the most elegant and accessible way.
If I wanted to assign additional information on the localitiy-level, I could do that by replacing the c(...) by some other call, like rbind(c('0201', '0202'), c("Friedrichshain", "Kreuzberg")) if I wanted to add additional information to the borough-level (like an abbreviated name and a full name for each list), how would I do this?
Edit: For example, I'd like to condense a table like this into a borough-wise version.
Hard to know without having a better view on how you intend to use this, but I would strongly recommend moving away from a nested list structure to a data frame structure:
library(reshape2)
loc.df <- melt(localities)
This is what the molten data looks like:
value L1
1 Mitte Mitte
2 Moabit Mitte
3 Hansaviertel Mitte
4 Tiergarten Mitte
5 Wedding Mitte
6 Gesundbrunnen Mitte
7 Friedrichshain Friedrichshain-Kreuzberg
8 Kreuzberg Friedrichshain-Kreuzberg
You can then use all the standard data frame and other computations:
loc.df$population <- sample(100:500, nrow(loc.df)) # make up population
tapply(loc.df$population, loc.df$L1, mean) # population by borough
gives mean population by Borough:
Friedrichshain-Kreuzberg Mitte
278.5000 383.8333
For more complex calculations you can use data.table and dplyr
You can extract all of this data directly into a data.frame using the XML library.
library(XML)
theurl <- "http://en.wikipedia.org/wiki/Boroughs_and_localities_of_Berlin#List_of_localities"
tables<-readHTMLTable(theurl)
boroughs<-tables[[1]]$Borough
localities<-tables[c(3:14)]
names(localities) <- as.character(boroughs)
all<-do.call("rbind", localities)
#Roland, I think you will find data frames superior to lists for the reasons cited earlier, but also because there is other data on the web page you reference. Loading to a data frame will make it easy to go further if you wish. For example, making comparisons based on population density or other items provided "for free" on the page will be a snap from a data frame.
I am not sure how to start this, as my GIS playing in R has been to plot things using ggplot2 and other packages using latlong coordinates. What I need to do now, is to use a visualization component in Microstrategy that uses a shapefile in the form of an HTML file containing x-y coordinates for the plot (ie. top left is 0,0). An example of a state level file is:
<HTML><HEAD><TITLE>untitled</TITLE></HEAD><BODY>
<IMG SRC="" USEMAP="#myMap" WIDTH="812" HEIGHT="713" BORDER="0" />
<MAP NAME="myMap">
<AREA SHAPE="POLY" HREF="#" ALT="Texas" COORDS="299,363,299,360,....." />
</MAP></BODY></HTML>
The points listed in 'coords' are the X and Y points with respect to a 812 by 713 'image' that is plotted and colored on the fly.
I have shp, shx, dbf files for Zip3 and Zip5 from http://www.vdstech.com/usa-data.aspx but am unsure of where to even start the conversion! I don't mind doing the grunt work of formatting the HTML file by hand, it is the X-Y conversion that I am stuck at (rusty, not touched GIS for quite a while):
The following code imports the shapefile into R
library(rgdal)
zip3 <- readOGR(dsn = '/Users/adempsey/Downloads/zip3'), layer = 'zip3')
After which I am stuck and currently hunting for tutorial of how to extract zip3 + x-y coordinates into a dataframe that I can then use to create my final file with
update 2
using the following, I ca convert to a data frame, but I am unable to pull across the associated zip3 code, which appeared to be stored in the associated dbf file
Row long lat order hole piece group id
1 -151.0604 70.41873 1 FALSE 1 0.1 0
2 -150.7620 70.49722 2 FALSE 1 0.1 0
Yes, this is beyond my current rusty R
update3
This code dumps the zip codes into a data frame
zip3.codes <- as.data.frame(zip3)
Which should be combinable with something like
zip3.df <- fortify(zip3#polygons[[1000]])
Where the 1000 would be replaced with all the rows zip3.codes associated with a particular zip3
You can use fastshp package to load the data:
install.packages("fastshp",,"http://rforge.net")
library(fastshp)
s <- read.shp("zip5.shp", format="polygon")
s is now a list of all ZIP shapes. You're interested in the x and y components
- for example to plot the first ZIP simply use something like
plot(s[[1]]$x, s[[1]]$y, asp=1.25)
polygon(s[[1]]$x, s[[1]]$y, col="#eeeeee")
To match the names, use read.dbf from foreign:
library(foreign)
d <- read.dbf("zip5.dbf", as.is=TRUE)
names(s) <- d$ZIP5
See ?read.shp for more details on the available formats. The "polygon" one uses NA to separate individual polygons, "list" uses indexing to give you the parts.
BTW the dataset is somewhat dubious, you may want to look into TIGER/Line census ZCTA5 data (most recent is 2010).