i have a list with indexes like this:
> mid_cp
[1] 3065 4871 13153 15587 18100 24010 26324 25648 38195 38196 39384 42237 45686 54217 55032 63684 62800 9134 35261 36449 36866 53968 16969
[24] 43529 46995 52351 4174 7011 18962 18151 18889 24036 32916 34061 34815 36866 51973 55802 53593 55421 56615 88 150 161 192 781
[47] 830 1300 1573 2396 2784 2547 3214 3135 3297 3301 4053 4249 4919 5856 6297 7328 7621 7708 8063 8219 8864 8887 9201
[70] 9214 9533 10334 10301 11235 10529 11356 10566 10872 12228 12250 12507 12048 12643 12913 13224 14297 16772 15363 18759 18979 16264 17363
[93] 20732 17971 22194 22422 19417 22903 22929 23087 19627 19961 23954 24297 25422 25423 25704 25765 25780 22769 22796 26871 27095 23789 24066
[116] 24069 27423 24366 24600 24871 25110 28374 26280 27873 29722 28839 29063 31031 31150 31546 32491 30356 33045 30863 33555 34201 34404 34684
[139] 35498 32912 33207 35874 33488 33716 36761 34543 36807 37000 35157 38195 38196 38458 36438 36619 39484 40109 37532 40143 40160 40458 41257
[162] 38434 38653 41866 41899 39429 42818 40001 43398 43441 40282 40566 43979 43996 40793 40806 40992 41065 41102 41330 41964 46322 43351 46670
and I have a table like this:
> head(movie.cp)
name id
252 $ (Dollars) (The Heist) 252
253 $5 a Day (Five Dollars a Day) 253
1 $9.99 1
254 $windle (Swindle) 254
255 "BBC2 Playhouse" Caught on a Train 255
256 "Independent Lens" Race to Execution 256
How do i get the mid_cp list to be a name list using the movie.cp table?
P.S.: I am completely newbie regarding R
are the numbers in mid_cp equivalent to movie.cp$id? if so try mid_cp <- movie.cp$name[match(mid_cp,movie.cp$id)]
Related
See a snippet of the following data set:
$ HOMEMTOT : int 4278 2389 1264 3249 6048 1705 5304 11091 1289 13347 ...
$ MULHERTOT : int 4213 2298 1278 3170 5667 1541 4931 11388 1254 11777 ...
$ pesoRUR : int 4464 1649 1588 1369 9269 885 2886 12910 221 10895 ...
$ pesotot : int 8491 4687 2542 6419 11715 3246 10235 22479 2543 25124 ...
$ pesourb : int 4027 3038 954 5050 2446 2361 7349 9569 2322 14229 ...
$ Estados : Factor w/ 26 levels "AC","AL","AM",..: 2 10 22 25 10 25 12 6 17 12 ...
I am intending to conduct a PCA and LASSO regression, but the issue is that I am supposed to set aside a sample size of 10 municipalities, coined "Estados," as seen above. How would I say make a sample set of, say, 300 observations based on 10 random municipalities "Estados." ?
You could use:
library(tidyverse)
df %>%
filter(Estados %in% sample(Estados, 10, replace = FALSE)) %>%
group_by(Estados) %>%
slice_sample(n = 30) %>%
ungroup()
I am trying to scrape the first table of multiple PDF's that look quite similar. So far I have isolated the page of the table, converted the table to a string and loaded it into R. Additionally, I also managed to remove the parts of the table I am not interested in as well as the header since it seemed like it will cause trouble because of the awkward spacing.
x <- pdf_text("2010 Table.pdf") # x is the string that I have attached at below
x <- unlist(strsplit(x, "Männer\r\n", fixed = T))[1]
x <- unlist(regmatches(x, regexpr("Insgesamt", x), invert = TRUE))[2]
cat(x)
0 b. unter 2 564.855 356.279 13.019 191.169 128.236 38 14.135 9.682 208.190 386
2 b. unter 4 300.245 205.375 31.056 96.882 68.185 18 7.032 2.202 94.062 808
4 b. unter 6 279.717 167.463 10.312 78.783 69.751 33 6.886 1.698 111.252 1.002
6 b. unter 8 247.614 140.412 22.926 62.535 47.390 88 6.554 919 105.818 1.384
8 b. unter 10 268.805 144.298 21.682 66.518 48.945 140 6.278 735 123.181 1.326
10 b. unter 12 393.303 144.576 18.387 65.387 51.705 177 8.500 420 245.470 3.257
12 b. unter 15 433.705 216.278 13.858 97.828 88.199 432 15.485 476 210.658 6.769
15 b. unter 18 423.441 224.532 4.804 106.780 94.846 1.116 16.552 434 188.577 10.332
18 b. unter 20 273.151 156.661 1.426 74.867 67.740 1.260 11.177 191 107.232 9.258
20 b. unter 25 653.650 389.246 1.330 190.154 156.275 5.086 36.186 215 234.087 30.317
25 b. unter 30 607.956 408.773 - 189.718 162.837 9.942 46.086 190 166.255 32.928
30 b. unter 35 501.137 357.262 - 141.724 150.686 19.403 45.196 253 110.461 33.414
35 b. unter 40 356.800 269.775 - 84.928 127.510 24.572 32.501 264 61.994 25.031
40 b. unter 50 422.582 348.116 - 70.498 187.365 54.388 35.555 310 41.506 32.960
50 b. unter 70 368.803 318.168 - 19.958 201.735 70.950 25.001 524 23.373 27.262
70 b. unter 100 160.051 140.411 - 850 105.534 25.262 8.311 454 9.441 10.199
100 b. unter 150 55.966 50.910 - - 42.024 5.961 2.541 384 3.336 1.720
150 b. unter 200 11.776 10.977 - - 10.028 587 252 110 674 125
200 und mehr 9.654 9.146 - - 8.828 182 104 32 475 33
Insgesamt ... 6,333.211 4,058.658 138.800 1.538.579 1.817.819 219.635 324.332 19.493 2.046.042 228.511
At this point, I thought that the best way to get this into a data.frame() was to use read.table(), unfortunately, since the spacing in between the columns is so inconsistent I cannot get it to work.
I appreciate any ideas, hints or solutions.
Thanks!
Data
"Steuerpflichtige 2010 nach Geschlecht, sozialer Stellung und Bruttobezugsstufen\r\n Tabelle 2\r\n Davon\r\n Unselb- Pensionisten u.\r\n Stufen der Steuer- Pers. m. Pensionistinnen Beamte und\r\n ständig Arbeiter und\r\n Bruttobezüge pflichtige Beamte und sonst. o. Beamten und Beamtinnen\r\n Erwerbs- Lehrlinge Arbeite- Angestellte VB\r\n in 1.000 EUR insgesamt Beamtinnen Aktiv- Beamtinnen i. R. i.R.\r\n tätige rinnen\r\n bezügen\r\n Insgesamt\r\n 0 b. unter 2 564.855 356.279 13.019 191.169 128.236 38 14.135 9.682 208.190 386\r\n 2 b. unter 4 300.245 205.375 31.056 96.882 68.185 18 7.032 2.202 94.062 808\r\n 4 b. unter 6 279.717 167.463 10.312 78.783 69.751 33 6.886 1.698 111.252 1.002\r\n 6 b. unter 8 247.614 140.412 22.926 62.535 47.390 88 6.554 919 105.818 1.384\r\n 8 b. unter 10 268.805 144.298 21.682 66.518 48.945 140 6.278 735 123.181 1.326\r\n 10 b. unter 12 393.303 144.576 18.387 65.387 51.705 177 8.500 420 245.470 3.257\r\n 12 b. unter 15 433.705 216.278 13.858 97.828 88.199 432 15.485 476 210.658 6.769\r\n 15 b. unter 18 423.441 224.532 4.804 106.780 94.846 1.116 16.552 434 188.577 10.332\r\n 18 b. unter 20 273.151 156.661 1.426 74.867 67.740 1.260 11.177 191 107.232 9.258\r\n 20 b. unter 25 653.650 389.246 1.330 190.154 156.275 5.086 36.186 215 234.087 30.317\r\n 25 b. unter 30 607.956 408.773 - 189.718 162.837 9.942 46.086 190 166.255 32.928\r\n 30 b. unter 35 501.137 357.262 - 141.724 150.686 19.403 45.196 253 110.461 33.414\r\n 35 b. unter 40 356.800 269.775 - 84.928 127.510 24.572 32.501 264 61.994 25.031\r\n 40 b. unter 50 422.582 348.116 - 70.498 187.365 54.388 35.555 310 41.506 32.960\r\n 50 b. unter 70 368.803 318.168 - 19.958 201.735 70.950 25.001 524 23.373 27.262\r\n 70 b. unter 100 160.051 140.411 - 850 105.534 25.262 8.311 454 9.441 10.199\r\n100 b. unter 150 55.966 50.910 - - 42.024 5.961 2.541 384 3.336 1.720\r\n150 b. unter 200 11.776 10.977 - - 10.028 587 252 110 674 125\r\n200 und mehr 9.654 9.146 - - 8.828 182 104 32 475 33\r\n Insgesamt ... 6,333.211 4,058.658 138.800 1.538.579 1.817.819 219.635 324.332 19.493 2.046.042 228.511\r\n Männer\r\n 0 b. unter 2 248.906 160.190 6.586 93.371 48.212 29 5.114 6.878 88.630 86\r\n 2 b. unter 4 125.032 89.536 19.507 44.637 21.775 14 2.271 1.332 35.221 275\r\n 4 b. unter 6 89.016 63.413 5.071 35.777 19.838 22 1.784 921 25.212 391\r\n 6 b. unter 8 82.674 58.262 13.407 29.985 12.776 54 1.506 534 23.882 530\r\n 8 b. unter 10 85.708 56.336 12.730 29.898 11.816 106 1.303 483 28.905 467\r\n 10 b. unter 12 113.358 56.129 13.686 29.542 11.193 69 1.414 225 56.581 648\r\n 12 b. unter 15 151.134 76.371 11.879 45.603 16.424 64 2.189 212 73.581 1.182\r\n 15 b. unter 18 165.399 74.808 4.262 51.314 16.662 77 2.331 162 87.632 2.959\r\n 18 b. unter 20 114.691 56.582 1.327 40.448 12.967 120 1.631 89 54.817 3.292\r\n 20 b. unter 25 333.825 180.786 1.253 132.966 37.908 1.335 7.212 112 137.384 15.655\r\n 25 b. unter 30 368.614 239.655 - 162.373 56.488 4.739 15.925 130 110.753 18.206\r\n 30 b. unter 35 329.970 230.335 - 131.377 68.331 12.250 18.175 202 79.989 19.646\r\n 35 b. unter 40 236.845 178.551 - 80.873 68.707 14.633 14.118 220 43.882 14.412\r\n 40 b. unter 50 280.849 234.429 - 68.109 119.325 31.658 15.089 248 27.331 19.089\r\n 50 b. unter 70 260.231 225.846 - 19.120 152.331 41.375 12.658 362 16.225 18.160\r\n 70 b. unter 100 128.140 112.891 - 802 88.423 18.037 5.277 352 7.340 7.909\r\n100 b. unter 150 47.894 43.663 - - 36.447 5.003 1.924 289 2.759 1.472\r\n150 b. unter 200 10.380 9.685 - - 8.888 507 205 85 584 111\r\n200 und mehr 8.813 8.353 - - 8.081 153 94 25 431 29\r\n Insgesamt ... 3,181.479 2,155.821 89.708 996.195 816.592 130.245 110.220 12.861 901.139 124.519\r\n Frauen\r\n 0 b. unter 2 315.949 196.089 6.433 97.798 80.024 9 9.021 2.804 119.560 300\r\n 2 b. unter 4 175.213 115.839 11.549 52.245 46.410 4 4.761 870 58.841 533\r\n 4 b. unter 6 190.701 104.050 5.241 43.006 49.913 11 5.102 777 86.040 611\r\n 6 b. unter 8 164.940 82.150 9.519 32.550 34.614 34 5.048 385 81.936 854\r\n 8 b. unter 10 183.097 87.962 8.952 36.620 37.129 34 4.975 252 94.276 859\r\n 10 b. unter 12 279.945 88.447 4.701 35.845 40.512 108 7.086 195 188.889 2.609\r\n 12 b. unter 15 282.571 139.907 1.979 52.225 71.775 368 13.296 264 137.077 5.587\r\n 15 b. unter 18 258.042 149.724 542 55.466 78.184 1.039 14.221 272 100.945 7.373\r\n 18 b. unter 20 158.460 100.079 99 34.419 54.773 1.140 9.546 102 52.415 5.966\r\n 20 b. unter 25 319.825 208.460 77 57.188 118.367 3.751 28.974 103 96.703 14.662\r\n 25 b. unter 30 239.342 169.118 - 27.345 106.349 5.203 30.161 60 55.502 14.722\r\n 30 b. unter 35 171.167 126.927 - 10.347 82.355 7.153 27.021 51 30.472 13.768\r\n 35 b. unter 40 119.955 91.224 - 4.055 58.803 9.939 18.383 44 18.112 10.619\r\n 40 b. unter 50 141.733 113.687 - 2.389 68.040 22.730 20.466 62 14.175 13.871\r\n 50 b. unter 70 108.572 92.322 - 838 49.404 29.575 12.343 162 7.148 9.102\r\n 70 b. unter 100 31.911 27.520 - 48 17.111 7.225 3.034 102 2.101 2.290\r\n100 b. unter 150 8.072 7.247 - - 5.577 958 617 95 577 248\r\n150 b. unter 200 1.396 1.292 - - 1.140 80 47 25 90 14\r\n200 und mehr 841 793 - - 747 29 10 7 44 4\r\n Insgesamt ... 3,151.732 1,902.837 49.092 542.384 1.001.227 89.390 214.112 6.632 1.144.903 103.992\r\n 38\r\n"
You could use tabulizer::extract_tables(). strsplit "weird-spaced" columns and cbind the snippets.
link <- "my.pdf"
library(tabulizer)
ext <- el(extract_tables(link, encoding="UTF-8"))
res <- cbind(ext[, 1:5], do.call(rbind, strsplit(ext[, 6], " ")),
ext[, 7:12])[, -c(2, 8)]
# store information for dim. names
dim.nm <- list(res[1:20, 1],
c("insg", "uns", "lehrl", "arb",
"ang", "beam", "VB", "sonst",
"pens", "beam.ir"))
# I would divide in a lists here
res <- list(insg=res[1:20, -1],
mann=res[22:41, -1],
frau=res[43:62, -1])
# convert to numbers (using gsub() to get rid of separators)
res <- Map(function(x) apply(x, 2, function(i) as.numeric(gsub("\\D", "", i))), res)
res <- lapply(res, `dimnames<-`, dim.nm)
head(res$insg)
# insg uns lehrl arb ang beam VB sonst pens beam.ir
# 0 b. unter 2 564855 356279 13019 191169 128236 38 14135 9682 208190 386
# 2 b. unter 4 300245 205375 31056 96882 68185 18 7032 2202 94062 808
# 4 b. unter 6 279717 167463 10312 78783 69751 33 6886 1698 111252 1002
# 6 b. unter 8 247614 140412 22926 62535 47390 88 6554 919 105818 1384
# 8 b. unter 10 268805 144298 21682 66518 48945 140 6278 735 123181 1326
# 10 b. unter 12 393303 144576 18387 65387 51705 177 8500 420 245470 3257
I am having a problem with the calculating of accuracy in ETS of test set.
train_ts<- ts(head(t$value,141), frequency=7) # this is train set (first 141 rows)
fit=auto.arima(train_ts)
forecasts = forecast(fit,h=12)
vector = ts(tail(t$value,12),frequency=7) # this is test set (last 12 rows)
accuracy(forecasts, vector, test=NULL, d=NULL, D=NULL) # I try to calculate accuracy
And I have this error:
Error in window.default(x, ...) : 'start' cannot be after 'end'
In addition: Warning message:
In window.default(x, ...) : 'start' value not changed
Result of forecasting:
Point Forecast Lo 80 Hi 80 Lo 95 Hi 95
191 4742.038402 3781.130910 5702.945894 3272.457210 6211.619593
192 5068.467231 4105.169285 6031.765177 3595.230155 6541.704307
193 5233.951079 4270.487205 6197.414954 3760.460238 6707.441921
194 4883.850503 3910.172814 5857.528191 3394.738981 6372.962025
195 4857.666612 3883.140593 5832.192631 3367.257681 6348.075543
196 5180.408585 4203.616284 6157.200886 3686.533674 6674.283496
197 5091.348011 4112.687519 6070.008503 3594.615948 6588.080074
198 4833.290365 3848.222297 5818.358433 3326.758761 6339.821969
199 5003.034291 4017.771775 5988.296807 3496.205304 6509.863278
200 5175.020752 4189.555595 6160.485908 3667.881854 6682.159650
201 4963.008654 3972.665298 5953.352010 3448.409193 6477.608114
202 4882.858876 3890.856391 5874.861360 3365.721997 6399.995754
vector:
Time Series:
Start = 1
End = 12
Frequency = 1
[1] 5243 5010 5374 4952 6911 4260 6063 5597 4536 5522 4254 5048
How can I fix my error or how can I calculate accuracy correctly?
Example data (t$value):
[1] 5564 6657 7184 6456 5597 5951 6771 5990 6289 6885 6171 4739 5737 5950 6721
[16] 6579 6763 6829 5779 5346 5652 6319 6407 7232 6600 6244 5631 5198 6360 7922
[31] 6035 4221 4361 4475 5585 4845 5958 6833 3617 5036 4560 3820 5724 6352 5773
[46] 6200 4378 5614 5165 6345 5769 6228 6378 4827 4402 5829 4880 6333 6406 434
[61] 4754 4303 5498 5048 6042 6664 5492 5684 6194 5349 5846 5916 5069 5071 4367
[76] 5381 5694 5731 6029 5639 5539 4490 5223 5436 5819 941 6576 5235 3574 6319
[91] 5063 5765 5919 6006 5479 3653 4281 5433 4851 5543 5995 5049 4728 5449 5728
[106] 6009 5378 5730 5206 4764 5458 5970 5254 5653 5539 1907 4438 5421 5529 5225
[121] 6158 5572 4777 4575 5275 4742 5648 5198 5624 4781 3959 4368 5478 4681 5288
[136] 5758 4540 3899 5760 4797 5580 5433 4898 4473 3566 4779 4897 5099 5866 6231
[151] 4982 4375 5976
Firstly, something seems off in the forecast output you posted; it starts at point 191 which means the fitted series ended at 190, but that doesn't seem right given the code you posted.
Regardless, DatamineR is correct in his comment. You are providing two time series with different ranges of time. The forecast function will pick up where the fitted time series left off, but when you use ts(tail(t$value,12),frequency=7) you are creating a new time series that starts at 1.
One option is to convert one (or both) into numeric vectors, as DatamineR suggested. Otherwise you can set the start time for your test set to the correct value doing something like:
vector = ts(tail(t,12),start=end(train_ts)+c(0, 1), frequency=7)
where end(train_ts) gives you the last time point of the training series, and then I added one more time step (in the same cycle) by adding c(0,1) to set the start time of the test series.
I have received some encoded data from Arduino via PySerial. I have access to an application which decodes the data, but I need to know what it is doing and I do not have access to the source code.
Data file contents:
%N|nkNsnrNlnzNqnEOknJOlM
%VA#_##hpZzbdIvzegvxefvkeavdeXvXeXvPeMvReDvlM
%PaA#gH#lnMO#QaLN#mbzM#cbmM#^beM#Pb_M#Fb]M#xaUM#balM
%Ma##HI#FzJP#auPO#~uPO#{uPO#}uMO#vuN#wuyN#uuqN#xulM
%knOOinSOXnMOAnFOcmxNYmlNBm_NslSNqlHNclnM^N
%PezuReouLeluDeju~diuFe`uBeXuAeUu~dJuxdAu^N
%MM#NaJM#`MM#t`VM#h`aM#f`fM#Y`jM#O`mM#G`uM#{_BN#u_^N
%rN#tuhN#nu[N#kuRN#huEN#au{M#[uqM#Nu^M#CuFM#ttuL#at^N
%XlPMMlvLMlWLPlBLVllKMlWKDlCKKlrJNl[J`lHJPO
%pd|trdrttdjtudbtmd_tkd[tkdWtmdOtldGtvdHtPO
Output from application:
86 31 -48 97 -51 33 -1109 -3121
-984 -358 551 -1108 584 -378 -1111 -3117
-1758 -631 973 -1967 1034 -671 -1128 -3123
-1670 -601 908 -1875 976 -642 -1151 -3130
-1672 -602 890 -1885 976 -645 -1181 -3144
-1685 -607 877 -1890 976 -643 -1191 -3156
-1692 -616 869 -1904 973 -650 -1214 -3169
-1704 -616 863 -1914 959 -649 -1229 -3181
-1712 -627 861 -1928 953 -651 -1231 -3192
-1710 -636 853 -1950 945 -648 -1245 -3218
-1712 -646 845 -1970 946 -652 -1256 -3248
-1710 -657 842 -1985 936 -658 -1267 -3274
-1716 -660 845 -1996 923 -661 -1267 -3305
-1724 -662 854 -2008 914 -664 -1264 -3326
-1730 -663 865 -2010 901 -671 -1258 -3348
-1722 -672 870 -2023 891 -677 -1267 -3369
-1726 -680 874 -2033 881 -690 -1276 -3389
-1727 -683 877 -2041 862 -701 -1269 -3406
-1730 -694 885 -2053 838 -716 -1266 -3429
-1736 -703 898 -2059 821 -735 -1248 -3448
I have tried several encodings like ASCII, UTF-8, and UUEncoding but none have given me any tangible results.
Does anyone have an idea as to what this could be?
Thanks in advance,
Cheers
I have a numeric matrix from which I want to retrieve the index given specific values.
I am trying the which() function to find values in the matrix.
The problem is that some values are found and some are not.
My matrix is as follows:
x_lat <- as.double(seq(48.0 ,60.0, by=0.1))
y_long <- as.double(seq(-10.0 ,2.0, by=0.1))
xv <- as.double(rep(x_lat,each = 121))
yv <- as.double(rep(y_long, 121))
vMatrix <- as.matrix(cbind(xv,yv))
If I want to retrieve the indices where the value -2.3 is TRUE the function returns correctly a vector with the indices where -2.3 appears.
xx<- which(vMatrix==-2.3,arr.ind=TRUE)
> xx
[1] 78 199 320 441 562 683 804 925 1046 1167 1288 1409 1530 1651 1772 1893 2014 2135 2256 2377 2498
[22] 2619 2740 2861 2982 3103 3224 3345 3466 3587 3708 3829 3950 4071 4192 4313 4434 4555 4676 4797 4918 5039
[43] 5160 5281 5402 5523 5644 5765 5886 6007 6128 6249 6370 6491 6612 6733 6854 6975 7096 7217 7338 7459 7580
[64] 7701 7822 7943 8064 8185 8306 8427 8548 8669 8790 8911 9032 9153 9274 9395 9516 9637 9758 9879 10000 10121
[85] 10242 10363 10484 10605 10726 10847 10968 11089 11210 11331 11452 11573 11694 11815 11936 12057 12178 12299 12420 12541 12662
[106] 12783 12904 13025 13146 13267 13388 13509 13630 13751 13872 13993 14114 14235 14356 14477 14598
But for some numbers (that appear in the matrix) the function does not work, e.g.,
xx<- which(vMatrix==-2.2,arr.ind=TRUE)
> xx
integer(0)
Floating point numbers can be misleading. Two such numbers are usually not "equal", even though the console may display the same output. The machine only has a certain accuracy with which it can represent the numbers.
Here's a simple example:
a <- 0.15 - 1/8
b <- 0.025
> a
[1] 0.025
> b
[1] 0.025
However, if we compare these numbers with "==", we obtain:
> a==b
[1] FALSE
That is because there are differences resulting from the floating point arithmetic which are beyond the machine's accuracy:
> a-b
[1] -6.938894e-18
Probably you can resolve the issue by simply rounding the numbers in the matrix to the necessary amount of relevant digits, like, e.g.,
xx<- which(round(vMatrix,3)==-2.2,arr.ind=TRUE)