I am trying to download a particularly messy .xlsx file from a URL to a local directory and then read this file using read_xlsx.
# Download file into directory
my_url <- 'https://docs.google.com/spreadsheets/d/0Bw4a10rhk2QqaTZkUmQwaXU4aEE/edit?resourcekey=0-RQa9gRpFX0x3z5bSJGn0Dg#gid=1944035140'
download.file(url=my_url, destfile='./dat/df.xlsx')
# Load file
df <- read_xlsx('./dat/df.xlsx')
This last line throws the following error:
Error: Evaluation error: zip file '/Users/... some path .../dat/df.xlsx' cannot be opened.
I believe this is happening because download.file() is messing up the format somehow. A few other similar issues have been solved, but the solution (mode='wb') did not help.
Could you help me download the file without messing up the format so I can later read this file using read_xlsx?
As an additional request, I would like to use as few external dependencias as possible (that's the reason I tried this with download.file()).
Indeed, the link takes you to Google Docs and is for non-downloadable editing. You cannot download this file this way. Just save it to your hard drive. However, I did a function that reads data from a file downloaded to disk. Maybe it will be useful to you.
library(tidyverse)
library(readxl)
urlFile = "https://docs.google.com/spreadsheets/d/1SF0PkBz9BR4yqiQ27Bt5OsD33Y8Rt5lh/edit?usp=sharing&ouid=107152468748636733235&rtpof=true&sd=true"
xlsFile = "refugios_nayarit.xlsx"
download.file(url=urlFile, destfile=xlsFile, mode="wb")
fReadXls = function(xlsFile, sheet) {
data = read_excel(
xlsFile, sheet = sheet, skip = 6,
col_names = c("No.", "REFUGIO", "MUNICIPIO", "DIRECCIÓN", "USO DEL INMUEBLE",
"SERVICIOS", "CAPACIDAD DE PERSONAS", "COORD. LATITUD L",
"COORD. LATITUD W", "COORD. ALTITUD MSNM", "RESPONSABLE",
"TELÉFONO"))
data %>% slice_head(n=nrow(.)-1)
}
df = tibble(sheet = excel_sheets(xlsFile)) %>%
mutate(data = map(sheet, ~fReadXls(xlsFile, .x)))
df$data[[1]]
output
# A tibble: 20 x 12
No. REFUGIO MUNICIPIO DIRECCIÓN `USO DEL INMUEB~ SERVICIOS `CAPACIDAD DE PE~ `COORD. LATITUD~ `COORD. LATITUD~
<dbl> <chr> <chr> <chr> <chr> <chr> <dbl> <chr> <chr>
1 1 PRIMARIA LAB~ ACAPONETA LOPEZ RAYON EDUCACION AGUA, SANITA~ 200 "22°29'56.06\"" "105°21'37.27\""
2 2 JARDIN DE NI~ ACAPONETA ALDAMA ESQ C~ EDUCACION AGUA, SANITA~ 100 "22°29'53.14\"" "105°21'29.48\""
3 3 PRIMARIA CAR~ ACAPONETA E. CARRANZA EDUCACION AGUA, SANITA~ 200 "22o30'00.43\"" "105°21'37.46\""
4 4 PRIMARIA LAZ~ ACAPONETA AMADO NERVO EDUCACION AGUA, SANITA~ 100 "22°29'27.17\"" "105°21'39.68"
5 5 PRIMARIA H. ~ ACAPONETA VERACRUZ No.~ EDUCACION AGUA, SANITA~ 150 "22o29'40.21\"" "105a21'40.23\""
6 6 PRIMARIA MIG~ ACAPONETA MORELOS Y OA~ EDUCACION AGUA, SANITA~ 200 "22o29'23.26\"" "105a21'41.99\""
7 7 PRIMARIA CEN~ ACAPONETA MATAMOROS Y ~ EDUCACION AGUA, SANITA~ 250 "22o29'37.31\"" "105a21'33.33\""
8 8 SINDICATO CTM ACAPONETA QUERETARO Y ~ GREMIO SINDICAL AGUA, SANITA~ 100 "22°29'39.32\"" "105°21'46.60"
9 9 ESTADIO MUNI~ ACAPONETA JUAN ESCUTIA DEPORTE AGUA, SANITA~ 300 "22a29'55.10\"" "105a21'52.29\""
10 10 CASA DE LA C~ ACAPONETA MORELOS CULTURAL AGUA, SANITA~ 300 "22a29'20.78\"" "105a21'46.46\""
11 11 CENTRO RECRE~ ACAPONETA México ENTRE~ RECREATIVO AGUA, SANITA~ 400 "22a29'39.76\"" "105a21'37.87\""
12 12 CENTRO RECRE~ ACAPONETA VERACRUZ No15 RECREATIVO AGUA, SANITA~ 400 "22a29'30.03\"" "105a21'39.47\""
13 13 IGLESIA CRIS~ ACAPONETA VERACRUZ No ~ RELIGIOSO AGUA, SANITA~ 50 "22a29'41.60\"" "105a21'40.35\""
14 14 ESCUELA FRAY~ AHUACATLAN 20 DE NOVIEM~ EDUCACION AGUA, SANITA~ 80 "21a03'06.07\"" "104a29'03.50\""
15 15 SECUNDARIA F~ AHUACATLAN 20 DE NOVIEM~ EDUCACION AGUA, SANITA~ 250 "21a03'18.33\"" "104a28'56.26\""
16 16 ESCUELA JOSE~ AHUACATLAN MORELOS Y MA~ EDUCACION AGUA, SANITA~ 200 "21a03'04.55\"" "104a29'12.67\""
17 17 ESCUELA PREP~ AHUACATLAN CALLE EL SAL~ EDUCACION AGUA, SANITA~ 200 "21a02'57.01\"" "104a29'16.71\""
18 18 ESCUELA PLAN~ AHUACATLAN OAXACA E HID~ EDUCACION AGUA, SANITA~ 200 "21a03'02.43\"" "104a28'58.82\""
19 19 UNIDAD ACADE~ AHUACATLAN CARR A GUADA~ EDUCACION AGUA, SANITA~ 200 "21a03'28.20\"" "104a29'06.67\""
20 20 CLUB SOCIAL ~ AHUACATLAN 20 DE NOVIEM~ DEPORTE AGUA, SANITA~ 400 "21a03'07.37\"" "104a29'01\"57\~
# ... with 3 more variables: COORD. ALTITUD MSNM <dbl>, RESPONSABLE <chr>, TELÉFONO <chr>
Related
I am trying to estimate home range size using telemetry data using adehabitatHR. Every time I create a SpatialPointsDataFrame, it rounds the y-coordinate for my UTM locations. I have tried having the UTMs saved as integers, as numbers, removing NAs before importing the .csv, removing NAs after importing the .csvs, importing as a .txt file (in case there's a bug in excel), etc.
Here is the console output:
> data<-read.csv("C:/Workspace/URAM/Data/HomeRange/NEW_TelemetryLocs_AllBears_asof_9Sept19_noNAs.csv", header=T)
>
> summary(data)
Alias Order Sex AnimalID X Y
Brandy : 2839 Min. : 1 F:15546 102-16 : 2839 Min. :397286 Min. :5236180
Bernie : 2674 1st Qu.: 306 M:11650 06-16 : 2674 1st Qu.:406966 1st Qu.:5251887
Eve : 2646 Median : 635 01-18 : 2646 Median :413166 Median :5258742
Deedee : 2606 Mean :1239 17-17 : 2606 Mean :412579 Mean :5257164
Buddha : 2346 3rd Qu.:2018 04-17 : 2346 3rd Qu.:418419 3rd Qu.:5262669
Bailey : 1192 Max. :5583 12-17 : 1192 Max. :432690 Max. :5291985
(Other):12893 (Other):12893
> head(data)
Alias Order Sex AnimalID X Y
1 Calliope 128 F 19-22 432690.3 5262636
2 Calliope 191 F 19-22 432522.3 5262409
3 Calliope 127 F 19-22 432491.0 5263274
4 Calliope 189 F 19-22 432466.3 5262413
5 Calliope 190 F 19-22 432376.1 5262121
6 Calliope 202 F 19-22 432262.3 5264390
> dim(data)
[1] 27196 6
> str(data)
'data.frame': 27196 obs. of 6 variables:
$ Alias : Factor w/ 26 levels "Artemis","Bailey",..: 9 9 9 9 9 9 9 9 9 9 ...
$ Order : int 128 191 127 189 190 202 201 188 129 422 ...
$ Sex : Factor w/ 2 levels "F","M": 1 1 1 1 1 1 1 1 1 1 ...
$ AnimalID: Factor w/ 26 levels "01-18","01-19",..: 26 26 26 26 26 26 26 26 26 26 ...
$ X : num 432690 432522 432491 432466 432376 ...
$ Y : num 5262636 5262409 5263274 5262413 5262121 ...
> test.sp<-SpatialPointsDataFrame(data[,5:6],data = data, coords.nrs = 5:6,
+ match.ID = TRUE,
+ proj4string=CRS("+proj=utm +zone=10+datum=NAD83+ellps=GRS80"))
> head(test.sp)
coordinates Alias Order Sex AnimalID X Y
1 (432690, 5262640) Calliope 128 F 19-22 432690.3 5262636
2 (432522, 5262410) Calliope 191 F 19-22 432522.3 5262409
3 (432491, 5263270) Calliope 127 F 19-22 432491.0 5263274
4 (432466, 5262410) Calliope 189 F 19-22 432466.3 5262413
5 (432376, 5262120) Calliope 190 F 19-22 432376.1 5262121
6 (432262, 5264390) Calliope 202 F 19-22 432262.3 5264390
Coordinate Reference System (CRS) arguments: +proj=utm +zone=10+datum=NAD83+ellps=GRS80
>
If I enter 6 lines of data manually in R, it seems to work:
> Alias<-c("Brandy","Brandy","Brandy","Brandy","Brandy","Brandy")
> Sex<-c("F","F","F","F","F","F")
> Order<-c(5,6,7,8,9,10)
> X<-c(409483,409481,409442,409438,409443,409576)
> Y<-c(5263356,5263356,5263335,5263340,5263342,5263685)
> test2<-data.frame(Alias,Sex,Order,X,Y)
> head(test2)
Alias Sex Order X Y
1 Brandy F 5 409483 5263356
2 Brandy F 6 409481 5263356
3 Brandy F 7 409442 5263335
4 Brandy F 8 409438 5263340
5 Brandy F 9 409443 5263342
6 Brandy F 10 409576 5263685
> test2.sp<-SpatialPointsDataFrame(test2[,4:5],data = test2, coords.nrs = 4:5,
+ match.ID = TRUE,
+ proj4string=CRS("+proj=utm +zone=10+datum=NAD83+ellps=GRS80"))
>
> test2.sp
coordinates Alias Sex Order X Y
1 (409483, 5263356) Brandy F 5 409483 5263356
2 (409481, 5263356) Brandy F 6 409481 5263356
3 (409442, 5263335) Brandy F 7 409442 5263335
4 (409438, 5263340) Brandy F 8 409438 5263340
5 (409443, 5263342) Brandy F 9 409443 5263342
6 (409576, 5263685) Brandy F 10 409576 5263685
However, if i use the coordinates() command to create the SpatialPointsDataFrame, using the manually entered data, the result is a rounded Y coordinate:
> coordinates(test2)<-c("X","Y")
> head(test2)
coordinates Alias Sex Order
1 (409483, 5263360) Brandy F 5
2 (409481, 5263360) Brandy F 6
3 (409442, 5263340) Brandy F 7
4 (409438, 5263340) Brandy F 8
5 (409443, 5263340) Brandy F 9
6 (409576, 5263680) Brandy F 10
Coordinate Reference System (CRS) arguments: NA
Any help or suggestions would be greatly appreciated.
The rounding takes place in the printing of the object, not in the internal representation or subsequent computations.
I am trying to scrape the first table of multiple PDF's that look quite similar. So far I have isolated the page of the table, converted the table to a string and loaded it into R. Additionally, I also managed to remove the parts of the table I am not interested in as well as the header since it seemed like it will cause trouble because of the awkward spacing.
x <- pdf_text("2010 Table.pdf") # x is the string that I have attached at below
x <- unlist(strsplit(x, "Männer\r\n", fixed = T))[1]
x <- unlist(regmatches(x, regexpr("Insgesamt", x), invert = TRUE))[2]
cat(x)
0 b. unter 2 564.855 356.279 13.019 191.169 128.236 38 14.135 9.682 208.190 386
2 b. unter 4 300.245 205.375 31.056 96.882 68.185 18 7.032 2.202 94.062 808
4 b. unter 6 279.717 167.463 10.312 78.783 69.751 33 6.886 1.698 111.252 1.002
6 b. unter 8 247.614 140.412 22.926 62.535 47.390 88 6.554 919 105.818 1.384
8 b. unter 10 268.805 144.298 21.682 66.518 48.945 140 6.278 735 123.181 1.326
10 b. unter 12 393.303 144.576 18.387 65.387 51.705 177 8.500 420 245.470 3.257
12 b. unter 15 433.705 216.278 13.858 97.828 88.199 432 15.485 476 210.658 6.769
15 b. unter 18 423.441 224.532 4.804 106.780 94.846 1.116 16.552 434 188.577 10.332
18 b. unter 20 273.151 156.661 1.426 74.867 67.740 1.260 11.177 191 107.232 9.258
20 b. unter 25 653.650 389.246 1.330 190.154 156.275 5.086 36.186 215 234.087 30.317
25 b. unter 30 607.956 408.773 - 189.718 162.837 9.942 46.086 190 166.255 32.928
30 b. unter 35 501.137 357.262 - 141.724 150.686 19.403 45.196 253 110.461 33.414
35 b. unter 40 356.800 269.775 - 84.928 127.510 24.572 32.501 264 61.994 25.031
40 b. unter 50 422.582 348.116 - 70.498 187.365 54.388 35.555 310 41.506 32.960
50 b. unter 70 368.803 318.168 - 19.958 201.735 70.950 25.001 524 23.373 27.262
70 b. unter 100 160.051 140.411 - 850 105.534 25.262 8.311 454 9.441 10.199
100 b. unter 150 55.966 50.910 - - 42.024 5.961 2.541 384 3.336 1.720
150 b. unter 200 11.776 10.977 - - 10.028 587 252 110 674 125
200 und mehr 9.654 9.146 - - 8.828 182 104 32 475 33
Insgesamt ... 6,333.211 4,058.658 138.800 1.538.579 1.817.819 219.635 324.332 19.493 2.046.042 228.511
At this point, I thought that the best way to get this into a data.frame() was to use read.table(), unfortunately, since the spacing in between the columns is so inconsistent I cannot get it to work.
I appreciate any ideas, hints or solutions.
Thanks!
Data
"Steuerpflichtige 2010 nach Geschlecht, sozialer Stellung und Bruttobezugsstufen\r\n Tabelle 2\r\n Davon\r\n Unselb- Pensionisten u.\r\n Stufen der Steuer- Pers. m. Pensionistinnen Beamte und\r\n ständig Arbeiter und\r\n Bruttobezüge pflichtige Beamte und sonst. o. Beamten und Beamtinnen\r\n Erwerbs- Lehrlinge Arbeite- Angestellte VB\r\n in 1.000 EUR insgesamt Beamtinnen Aktiv- Beamtinnen i. R. i.R.\r\n tätige rinnen\r\n bezügen\r\n Insgesamt\r\n 0 b. unter 2 564.855 356.279 13.019 191.169 128.236 38 14.135 9.682 208.190 386\r\n 2 b. unter 4 300.245 205.375 31.056 96.882 68.185 18 7.032 2.202 94.062 808\r\n 4 b. unter 6 279.717 167.463 10.312 78.783 69.751 33 6.886 1.698 111.252 1.002\r\n 6 b. unter 8 247.614 140.412 22.926 62.535 47.390 88 6.554 919 105.818 1.384\r\n 8 b. unter 10 268.805 144.298 21.682 66.518 48.945 140 6.278 735 123.181 1.326\r\n 10 b. unter 12 393.303 144.576 18.387 65.387 51.705 177 8.500 420 245.470 3.257\r\n 12 b. unter 15 433.705 216.278 13.858 97.828 88.199 432 15.485 476 210.658 6.769\r\n 15 b. unter 18 423.441 224.532 4.804 106.780 94.846 1.116 16.552 434 188.577 10.332\r\n 18 b. unter 20 273.151 156.661 1.426 74.867 67.740 1.260 11.177 191 107.232 9.258\r\n 20 b. unter 25 653.650 389.246 1.330 190.154 156.275 5.086 36.186 215 234.087 30.317\r\n 25 b. unter 30 607.956 408.773 - 189.718 162.837 9.942 46.086 190 166.255 32.928\r\n 30 b. unter 35 501.137 357.262 - 141.724 150.686 19.403 45.196 253 110.461 33.414\r\n 35 b. unter 40 356.800 269.775 - 84.928 127.510 24.572 32.501 264 61.994 25.031\r\n 40 b. unter 50 422.582 348.116 - 70.498 187.365 54.388 35.555 310 41.506 32.960\r\n 50 b. unter 70 368.803 318.168 - 19.958 201.735 70.950 25.001 524 23.373 27.262\r\n 70 b. unter 100 160.051 140.411 - 850 105.534 25.262 8.311 454 9.441 10.199\r\n100 b. unter 150 55.966 50.910 - - 42.024 5.961 2.541 384 3.336 1.720\r\n150 b. unter 200 11.776 10.977 - - 10.028 587 252 110 674 125\r\n200 und mehr 9.654 9.146 - - 8.828 182 104 32 475 33\r\n Insgesamt ... 6,333.211 4,058.658 138.800 1.538.579 1.817.819 219.635 324.332 19.493 2.046.042 228.511\r\n Männer\r\n 0 b. unter 2 248.906 160.190 6.586 93.371 48.212 29 5.114 6.878 88.630 86\r\n 2 b. unter 4 125.032 89.536 19.507 44.637 21.775 14 2.271 1.332 35.221 275\r\n 4 b. unter 6 89.016 63.413 5.071 35.777 19.838 22 1.784 921 25.212 391\r\n 6 b. unter 8 82.674 58.262 13.407 29.985 12.776 54 1.506 534 23.882 530\r\n 8 b. unter 10 85.708 56.336 12.730 29.898 11.816 106 1.303 483 28.905 467\r\n 10 b. unter 12 113.358 56.129 13.686 29.542 11.193 69 1.414 225 56.581 648\r\n 12 b. unter 15 151.134 76.371 11.879 45.603 16.424 64 2.189 212 73.581 1.182\r\n 15 b. unter 18 165.399 74.808 4.262 51.314 16.662 77 2.331 162 87.632 2.959\r\n 18 b. unter 20 114.691 56.582 1.327 40.448 12.967 120 1.631 89 54.817 3.292\r\n 20 b. unter 25 333.825 180.786 1.253 132.966 37.908 1.335 7.212 112 137.384 15.655\r\n 25 b. unter 30 368.614 239.655 - 162.373 56.488 4.739 15.925 130 110.753 18.206\r\n 30 b. unter 35 329.970 230.335 - 131.377 68.331 12.250 18.175 202 79.989 19.646\r\n 35 b. unter 40 236.845 178.551 - 80.873 68.707 14.633 14.118 220 43.882 14.412\r\n 40 b. unter 50 280.849 234.429 - 68.109 119.325 31.658 15.089 248 27.331 19.089\r\n 50 b. unter 70 260.231 225.846 - 19.120 152.331 41.375 12.658 362 16.225 18.160\r\n 70 b. unter 100 128.140 112.891 - 802 88.423 18.037 5.277 352 7.340 7.909\r\n100 b. unter 150 47.894 43.663 - - 36.447 5.003 1.924 289 2.759 1.472\r\n150 b. unter 200 10.380 9.685 - - 8.888 507 205 85 584 111\r\n200 und mehr 8.813 8.353 - - 8.081 153 94 25 431 29\r\n Insgesamt ... 3,181.479 2,155.821 89.708 996.195 816.592 130.245 110.220 12.861 901.139 124.519\r\n Frauen\r\n 0 b. unter 2 315.949 196.089 6.433 97.798 80.024 9 9.021 2.804 119.560 300\r\n 2 b. unter 4 175.213 115.839 11.549 52.245 46.410 4 4.761 870 58.841 533\r\n 4 b. unter 6 190.701 104.050 5.241 43.006 49.913 11 5.102 777 86.040 611\r\n 6 b. unter 8 164.940 82.150 9.519 32.550 34.614 34 5.048 385 81.936 854\r\n 8 b. unter 10 183.097 87.962 8.952 36.620 37.129 34 4.975 252 94.276 859\r\n 10 b. unter 12 279.945 88.447 4.701 35.845 40.512 108 7.086 195 188.889 2.609\r\n 12 b. unter 15 282.571 139.907 1.979 52.225 71.775 368 13.296 264 137.077 5.587\r\n 15 b. unter 18 258.042 149.724 542 55.466 78.184 1.039 14.221 272 100.945 7.373\r\n 18 b. unter 20 158.460 100.079 99 34.419 54.773 1.140 9.546 102 52.415 5.966\r\n 20 b. unter 25 319.825 208.460 77 57.188 118.367 3.751 28.974 103 96.703 14.662\r\n 25 b. unter 30 239.342 169.118 - 27.345 106.349 5.203 30.161 60 55.502 14.722\r\n 30 b. unter 35 171.167 126.927 - 10.347 82.355 7.153 27.021 51 30.472 13.768\r\n 35 b. unter 40 119.955 91.224 - 4.055 58.803 9.939 18.383 44 18.112 10.619\r\n 40 b. unter 50 141.733 113.687 - 2.389 68.040 22.730 20.466 62 14.175 13.871\r\n 50 b. unter 70 108.572 92.322 - 838 49.404 29.575 12.343 162 7.148 9.102\r\n 70 b. unter 100 31.911 27.520 - 48 17.111 7.225 3.034 102 2.101 2.290\r\n100 b. unter 150 8.072 7.247 - - 5.577 958 617 95 577 248\r\n150 b. unter 200 1.396 1.292 - - 1.140 80 47 25 90 14\r\n200 und mehr 841 793 - - 747 29 10 7 44 4\r\n Insgesamt ... 3,151.732 1,902.837 49.092 542.384 1.001.227 89.390 214.112 6.632 1.144.903 103.992\r\n 38\r\n"
You could use tabulizer::extract_tables(). strsplit "weird-spaced" columns and cbind the snippets.
link <- "my.pdf"
library(tabulizer)
ext <- el(extract_tables(link, encoding="UTF-8"))
res <- cbind(ext[, 1:5], do.call(rbind, strsplit(ext[, 6], " ")),
ext[, 7:12])[, -c(2, 8)]
# store information for dim. names
dim.nm <- list(res[1:20, 1],
c("insg", "uns", "lehrl", "arb",
"ang", "beam", "VB", "sonst",
"pens", "beam.ir"))
# I would divide in a lists here
res <- list(insg=res[1:20, -1],
mann=res[22:41, -1],
frau=res[43:62, -1])
# convert to numbers (using gsub() to get rid of separators)
res <- Map(function(x) apply(x, 2, function(i) as.numeric(gsub("\\D", "", i))), res)
res <- lapply(res, `dimnames<-`, dim.nm)
head(res$insg)
# insg uns lehrl arb ang beam VB sonst pens beam.ir
# 0 b. unter 2 564855 356279 13019 191169 128236 38 14135 9682 208190 386
# 2 b. unter 4 300245 205375 31056 96882 68185 18 7032 2202 94062 808
# 4 b. unter 6 279717 167463 10312 78783 69751 33 6886 1698 111252 1002
# 6 b. unter 8 247614 140412 22926 62535 47390 88 6554 919 105818 1384
# 8 b. unter 10 268805 144298 21682 66518 48945 140 6278 735 123181 1326
# 10 b. unter 12 393303 144576 18387 65387 51705 177 8500 420 245470 3257
I will appreciate any help I can get. It says we have to read in the attached file and use it as an edge list to create a directed graph with weighted edges.
Then there are about 20 other things I have to do from there. This is part of the .txt file to import:
Columns are inbound locations, outbound locations, and travel time in minutes.
Inbound Outbound Minutes
ACY ATL 102
ACY FLL 136
ACY MCO 122
ACY MYR 90
ACY RSW 137
ACY TPA 129
ATL ACY 102
ATL BOS 132
ATL BWI 106
ATL CLE 104
.... and so on, there are probably 50+ locations in total, with around 400 lines
I tried using
read.graph(file.choose(), format="edgelist")
and when I select the .txt file I get the error:
"Error in read.graph.edgelist(file, ...) :
At foreign.c:101 : parsing edgelist file failed, Parse error"
-----EDIT-----
I just used the following code:
inbound <- c(data[, 1])
outbound <- c(data[, 2])
testing <- data.frame(inbound, outbound)
gd <- graph_from_data_frame(testing, directed=TRUE,vertices=NULL)
Which gave this output:
edges from d654854 (vertex names):
[1] 1 ->2 1 ->18 1 ->30 1 ->35 1 ->46 1 ->58 2 ->1 2 ->7 2 ->9 2 ->11 2 ->15 2 ->16 2 ->18 2 ->21 2 ->23 2 ->24 2 ->30
[18] 2 ->33 2 ->34 2 ->36 2 ->37 2 ->41 2 ->58 3 ->18 4 ->18 5 ->18 5 ->30 5 ->35 5 ->46 5 ->58 6 ->18 7 ->2 7 ->9 7 ->11
[35] 7 ->15 7 ->16 7 ->18 7 ->23 7 ->30 7 ->33 7 ->34 7 ->35 7 ->37 7 ->58 8 ->18 9 ->2 9 ->7 9 ->13 9 ->15 9 ->16 9 ->18
[52] 9 ->21 9 ->23 9 ->24 9 ->30 9 ->33 9 ->34 9 ->35 9 ->36 9 ->37 9 ->48 9 ->51 9 ->58 10->18 10->23 10->30 10->35 11->2
[69] 11->7 11->15 11->18 11->23 11->24 11->30 11->34 11->35 11->51 12->18 13->9 13->15 13->18 13->21 13->37 14->15 14->16
[86] 14->18 14->21 14->23 14->24 14->26 14->30 14->33 14->37 15->2 15->7 15->9 15->11 15->13 15->14 15->16 15->18 15->23
[103] 15->24 15->26 15->30 15->33 15->34 15->35 15->36 15->37 15->41 15->42 15->43 15->48 15->52 15->58 16->2 16->7 16->9
[120] 16->14 16->15 16->18 16->21 16->23 16->24 16->26 16->29 16->30 16->33 16->34 16->35 16->36 16->41 16->46 16->54 16->58
+ ... omitted several edges
Is that what I am supposed to get? Or am I still way off?
Using is.igraph(gd) returns true, and using V(gd) and E(gd) both return information.
So I guess my question is how do I properly import the "table" so that the pairs of inbound/outbound flight names are used as edges (I think) for this? I have to make a directed graph with weighted edges to finalize the set up.
Any information on where I should start? I looked through the igraph documentation but I can't find anything about importing from a table and using pairs of characters as edges.
You can import the data as a data.frame and coerce it to a graph. Once you have the graph, you can assign weights.
library(igraph)
xy <- read.table(text = "
ACY ATL 102
ACY FLL 136
ACY MCO 122
ACY MYR 90
ACY RSW 137
ACY TPA 129
ATL ACY 102
ATL BOS 132
ATL BWI 106
ATL CLE 104", header = FALSE, sep = " ")
colnames(xy) <- c("node1", "node2", "weight")
g <- graph_from_data_frame(xy[, c("node1", "node2")])
E(g)$weight <- xy$weight
plot(g, edge.width = E(g)$weight/50, edge.arrow.size = 0.1)
I have 963 lists, all containing the same type of information per list instance. The amount of data in any list at a given instance can vary, however. Instead of creating many lists, is there an efficient way to group the lists? Examples follow.
list001 <- c(originApt = 'ATL', destinApt = 'BOS', flightIndxs = c( 1 : 7 ) )
list002 <- c(originApt = 'ATL', destinApt = 'DEN', flightIndxs = c( 9 :19 ) )
:
list963 <- c(originApt = 'DCA', destinApt = 'TPA', flightIndxs = c( 8582, 8583, 8584, 8585, 8586, 8587 ) )
and so forth. Note that the length of integers in the third entry of each list varies in length. In matlab, I'd just construct a structure called 'flight' with an index for each list instance. Is there a way to organize my lists in R short of having many individual instances?
You can create a list of lists:
list001 <- list(originApt = 'ATL', destinApt = 'BOS', flightIndxs = c( 1 : 7 ) )
list002 <- list(originApt = 'ATL', destinApt = 'DEN', flightIndxs = c( 9 :19 ) )
large_list = list(list001, list002)
> large_list
[[1]]
[[1]]$originApt
[1] "ATL"
[[1]]$destinApt
[1] "BOS"
[[1]]$flightIndxs
[1] 1 2 3 4 5 6 7
[[2]]
[[2]]$originApt
[1] "ATL"
[[2]]$destinApt
[1] "DEN"
[[2]]$flightIndxs
[1] 9 10 11 12 13 14 15 16 17 18 19
A list can contain any other R object as a member. Do note to not construct the sublists as c(), but also use list.
You can also create a long formatted data.frame:
do.call('rbind', lapply(large_list, function(x) as.data.frame(do.call('cbind', x))))
originApt destinApt flightIndxs
1 ATL BOS 1
2 ATL BOS 2
3 ATL BOS 3
4 ATL BOS 4
5 ATL BOS 5
6 ATL BOS 6
7 ATL BOS 7
8 ATL DEN 9
9 ATL DEN 10
10 ATL DEN 11
11 ATL DEN 12
12 ATL DEN 13
13 ATL DEN 14
14 ATL DEN 15
15 ATL DEN 16
16 ATL DEN 17
17 ATL DEN 18
18 ATL DEN 19
Do note that this only works because flightIndxs is the only entry to have multiple values, and there is a clear interpretation that each flight index only has one origin and destination. It can also work with multiple variables having multiple values, as long as they all contain the same number of multiple values.
Given the following example:
library(metafor)
dat <- escalc(measure = "RR", ai = tpos, bi = tneg, ci = cpos, di = cneg, data = dat.bcg, append = TRUE)
dat
rma(yi, vi, data = dat, mods = ~dat[[8]], subset = (alloc=="systematic"), knha = TRUE)
trial author year tpos tneg cpos cneg ablat alloc yi vi
1 1 Aronson 1948 4 119 11 128 44 random -0.8893 0.3256
2 2 Ferguson & Simes 1949 6 300 29 274 55 random -1.5854 0.1946
3 3 Rosenthal et al 1960 3 228 11 209 42 random -1.3481 0.4154
4 4 Hart & Sutherland 1977 62 13536 248 12619 52 random -1.4416 0.0200
5 5 Frimodt-Moller et al 1973 33 5036 47 5761 13 alternate -0.2175 0.0512
6 6 Stein & Aronson 1953 NA NA NA NA 44 alternate NA NA
7 7 Vandiviere et al 1973 8 2537 10 619 19 random -1.6209 0.2230
8 8 TPT Madras 1980 505 87886 499 87892 NA random 0.0120 0.0040
9 9 Coetzee & Berjak 1968 29 7470 45 7232 27 random -0.4694 0.0564
10 10 Rosenthal et al 1961 17 1699 65 1600 42 systematic -1.3713 0.0730
11 11 Comstock et al 1974 186 50448 141 27197 18 systematic -0.3394 0.0124
12 12 Comstock & Webster 1969 5 2493 3 2338 33 systematic 0.4459 0.5325
13 13 Comstock et al 1976 27 16886 29 17825 33 systematic -0.0173 0.0714
Now what i basically want is to iterate with the rma() command (only for mods argument) from - let's say - [7:8] and to store this result in a variable equal to the columnname.
Two problems:
1) When i enter the command:
rma(yi, vi, data = dat, mods = ~dat[[8]], subset = (alloc=="systematic"), knha = TRUE)
The modname is named as dat[[8]]. But I want the modname to be the columname (i.e. colnames(dat[i]))
Model Results:
estimate se tval pval ci.lb ci.ub
intrcpt 0.5543 1.4045 0.3947 0.7312 -5.4888 6.5975
dat[[8]] -0.0312 0.0435 -0.7172 0.5477 -0.2185 0.1560
2) Now imagine that I have a lot of columns more and I want to iterate from [8:53], such that each result gets stored in a variable named equal to the columnname.
Problem 2) has been solved:
for(i in 7:8){
assign(paste(colnames(dat[i]), i, sep=""), rma(yi, vi, data = dat, mods = ~dat[[i]], subset = (alloc=="systematic"), knha = TRUE))}
To answers 1st part of your question, you can change the names by accessing the attributes of the model object.
In this case
# inspect the attributes
attr(model$vb, which = "dimnames")
# assign the name
attr(model$vb, which = "dimnames")[[1]][2] <- paste(colnames(dat)[8])