R: The 61928th question about the "singular gradient matrix at initial parameter estimates" error message - r
I have the following data:
479117.562500000 -100.366333008
479117.625000000 -100.292800903
479117.687500000 -100.772460937
479117.750000000 -101.344261169
479117.812500000 -102.828948975
479117.875000000 -103.842330933
479117.937500000 -102.289733887
479118.000000000 -101.856155396
479118.062500000 -101.972282410
479118.125000000 -101.272254944
479118.187500000 -101.042846680
479118.250000000 -101.957427979
479118.312500000 -101.363922119
479118.375000000 -101.065864563
479118.437500000 -99.710098267
479118.500000000 -98.789115906
479118.562500000 -99.854644775
479118.625000000 -100.956558228
479118.687500000 -100.456512451
479118.750000000 -100.779090881
479118.812500000 -101.598800659
479118.875000000 -100.329147339
479118.937500000 -100.486946106
479119.000000000 -102.275772095
479119.062500000 -103.128715515
479119.125000000 -103.075996399
479119.187500000 -103.266349792
479119.250000000 -102.390190125
479119.312500000 -101.386428833
479119.375000000 -102.008850098
479119.437500000 -103.579475403
479119.500000000 -103.382720947
479119.562500000 -100.842361450
479119.625000000 -98.478569031
479119.687500000 -98.745864868
479119.750000000 -99.653961182
479119.812500000 -100.032035828
479119.875000000 -99.955345154
479119.937500000 -99.842536926
479120.000000000 -100.187896729
479120.062500000 -100.456726074
479120.125000000 -101.258850098
479120.187500000 -102.649017334
479120.250000000 -104.833518982
479120.312500000 -102.760551453
479120.375000000 -101.653732300
479120.437500000 -102.729179382
479120.500000000 -102.752014160
479120.562500000 -103.103675842
479120.625000000 -102.842521667
479120.687500000 -102.692077637
479120.750000000 -102.499221802
479120.812500000 -101.806587219
479120.875000000 -102.124893188
479120.937500000 -101.700584412
479121.000000000 -101.385307312
479121.062500000 -101.242889404
479121.125000000 -100.172935486
479121.187500000 -100.230110168
479121.250000000 -100.861007690
479121.312500000 -101.013366699
479121.375000000 -100.585502625
479121.437500000 -100.897743225
479121.500000000 -101.453987122
479121.562500000 -102.233383179
479121.625000000 -102.231163025
479121.687500000 -99.512817383
479121.750000000 -97.662391663
479121.812500000 -97.647987366
479121.875000000 -100.217674255
479121.937500000 -102.411224365
479122.000000000 -101.892311096
479122.062500000 -102.475875854
479122.125000000 -103.164466858
479122.187500000 -103.406997681
479122.250000000 -104.319549561
479122.312500000 -102.138801575
479122.375000000 -99.946632385
479122.437500000 -100.355888367
479122.500000000 -101.683120728
479122.562500000 -101.582458496
479122.625000000 -99.907981873
479122.687500000 -100.329666138
479122.750000000 -100.243255615
479122.812500000 -100.713218689
479122.875000000 -102.436210632
479122.937500000 -103.173072815
479123.000000000 -103.720008850
479123.062500000 -105.225852966
479123.125000000 -104.841903687
479123.187500000 -103.589698792
479123.250000000 -101.543907166
479123.312500000 -101.051879883
479123.375000000 -103.181671143
479123.437500000 -104.825492859
479123.500000000 -103.848281860
479123.562500000 -102.969032288
479123.625000000 -101.002128601
479123.687500000 -100.698005676
479123.750000000 -102.078453064
479123.812500000 -103.582519531
479123.875000000 -105.085006714
479123.937500000 -103.349472046
479124.000000000 -100.479156494
479124.062500000 -100.558197021
479124.125000000 -101.563316345
479124.187500000 -101.261054993
479124.250000000 -102.108535767
479124.312500000 -104.861206055
479124.375000000 -105.044944763
479124.437500000 -105.712318420
479124.500000000 -105.045219421
479124.562500000 -104.131736755
479124.625000000 -104.060478210
479124.687500000 -103.435829163
479124.750000000 -103.167121887
479124.812500000 -102.186767578
479124.875000000 -101.180900574
479124.937500000 -101.686195374
479125.000000000 -102.167709351
479125.062500000 -102.771011353
479125.125000000 -103.367576599
479125.187500000 -103.127212524
479125.250000000 -103.924591064
479125.312500000 -103.187667847
479125.375000000 -102.220222473
479125.437500000 -102.674034119
479125.500000000 -101.717445374
479125.562500000 -100.879615784
479125.625000000 -100.964996338
479125.687500000 -102.864616394
479125.750000000 -102.009140015
479125.812500000 -99.761398315
479125.875000000 -99.798591614
479125.937500000 -101.713653564
479126.000000000 -103.273422241
479126.062500000 -102.664245605
479126.125000000 -101.682983398
479126.187500000 -101.853103638
479126.250000000 -103.193588257
479126.312500000 -104.359184265
479126.375000000 -105.037651062
479126.437500000 -104.446434021
479126.500000000 -103.674736023
479126.562500000 -103.374031067
479126.625000000 -102.921363831
479126.687500000 -103.374008179
479126.750000000 -104.299362183
479126.812500000 -104.015937805
479126.875000000 -103.758834839
479126.937500000 -103.698440552
479127.000000000 -103.501396179
479127.062500000 -101.677307129
479127.125000000 -101.010841370
479127.187500000 -103.159111023
479127.250000000 -105.232284546
479127.312500000 -105.949432373
479127.375000000 -104.999694824
479127.437500000 -104.207763672
479127.500000000 -103.822082520
479127.562500000 -103.189147949
479127.625000000 -102.943603516
479127.687500000 -102.586914062
479127.750000000 -102.973297119
479127.812500000 -104.049942017
479127.875000000 -106.436325073
479127.937500000 -105.395500183
479128.000000000 -106.032653809
479128.062500000 -106.538482666
479128.125000000 -105.961471558
479128.187500000 -106.049240112
479128.250000000 -104.937507629
479128.312500000 -104.842300415
479128.375000000 -104.720268250
479128.437500000 -105.791313171
479128.500000000 -106.022468567
479128.562500000 -103.848289490
479128.625000000 -103.887428284
479128.687500000 -104.258583069
479128.750000000 -105.152420044
479128.812500000 -107.673591614
479128.875000000 -107.705734253
479128.937500000 -105.925376892
479129.000000000 -105.528671265
479129.062500000 -106.021476746
479129.125000000 -107.750610352
479129.187500000 -108.693489075
479129.250000000 -108.675323486
479129.312500000 -109.919746399
479129.375000000 -110.940391541
479129.437500000 -109.279312134
479129.500000000 -108.321495056
479129.562500000 -107.995155334
479129.625000000 -109.164222717
479129.687500000 -111.977653503
479129.750000000 -113.194961548
479129.812500000 -114.239585876
479129.875000000 -115.780212402
479129.937500000 -116.979713440
479130.000000000 -117.042602539
479130.062500000 -116.658126831
479130.125000000 -116.624031067
479130.187500000 -116.923446655
479130.250000000 -118.727882385
479130.312500000 -120.354904175
479130.375000000 -121.513587952
479130.437500000 -121.322601318
479130.500000000 -121.338325500
479130.562500000 -120.500923157
479130.625000000 -116.656593323
479130.687500000 -113.295486450
479130.750000000 -111.713729858
479130.812500000 -111.394592285
479130.875000000 -109.731071472
479130.937500000 -108.571876526
479131.000000000 -109.059860229
479131.062500000 -106.810707092
479131.125000000 -106.095306396
479131.187500000 -106.258293152
479131.250000000 -106.243156433
479131.312500000 -106.613525391
479131.375000000 -105.910820007
479131.437500000 -104.405731201
479131.500000000 -102.325592041
479131.562500000 -101.502128601
479131.625000000 -103.445144653
479131.687500000 -105.970573425
479131.750000000 -105.379684448
479131.812500000 -102.992294312
479131.875000000 -100.679176331
479131.937500000 -99.553001404
479132.000000000 -100.532035828
479132.062500000 -102.480346680
479132.125000000 -104.630592346
479132.187500000 -103.669296265
479132.250000000 -101.364990234
479132.312500000 -100.193199158
479132.375000000 -98.483375549
479132.437500000 -98.084083557
479132.500000000 -100.955741882
479132.562500000 -102.788536072
479132.625000000 -102.540054321
479132.687500000 -102.550140381
479132.750000000 -101.182907104
479132.812500000 -100.926239014
479132.875000000 -100.933807373
479132.937500000 -101.358642578
479133.000000000 -100.544723511
479133.062500000 -99.536102295
479133.125000000 -99.533355713
479133.187500000 -100.520698547
479133.250000000 -99.944213867
479133.312500000 -100.118461609
479133.375000000 -101.425323486
479133.437500000 -102.523521423
479133.500000000 -102.540794373
479133.562500000 -101.491882324
479133.625000000 -100.919067383
479133.687500000 -100.623329163
479133.750000000 -99.431541443
479133.812500000 -99.252487183
479133.875000000 -101.166763306
479133.937500000 -102.311378479
479134.000000000 -101.306701660
479134.062500000 -100.665534973
479134.125000000 -100.248069763
479134.187500000 -99.179161072
479134.250000000 -100.506088257
479134.312500000 -101.349990845
479134.375000000 -101.028564453
479134.437500000 -101.089591980
479134.500000000 -100.819961548
479134.562500000 -100.899681091
479134.625000000 -102.236335754
479134.687500000 -101.911392212
479134.750000000 -101.253051758
479134.812500000 -102.417984009
479134.875000000 -101.647750854
479134.937500000 -100.494926453
479135.000000000 -99.920089722
479135.062500000 -101.046142578
479135.125000000 -102.893470764
479135.187500000 -102.895072937
479135.250000000 -103.607261658
479135.312500000 -104.568321228
479135.375000000 -104.253341675
479135.437500000 -102.952842712
479135.500000000 -101.928634644
479135.562500000 -101.746994019
479135.625000000 -102.218338013
479135.687500000 -102.627662659
479135.750000000 -102.185234070
479135.812500000 -103.266464233
479135.875000000 -104.480552673
479135.937500000 -102.991035461
479136.000000000 -101.333572388
479136.062500000 -102.019165039
479136.125000000 -100.434211731
479136.187500000 -99.072113037
479136.250000000 -100.616592407
479136.312500000 -101.648803711
479136.375000000 -102.449073792
479136.437500000 -103.141990662
479136.500000000 -101.611976624
479136.562500000 -101.742187500
479136.625000000 -102.974266052
479136.687500000 -101.894943237
479136.750000000 -101.637077332
479136.812500000 -101.545288086
479136.875000000 -101.042068481
479136.937500000 -101.836784363
479137.000000000 -103.539382935
479137.062500000 -105.681159973
479137.125000000 -102.126953125
479137.187500000 -98.450904846
479137.250000000 -98.859046936
479137.312500000 -102.353157043
479137.375000000 -105.606437683
479137.437500000 -104.589866638
479137.500000000 -103.607994080
479137.562500000 -102.202362061
479137.625000000 -101.861511230
479137.687500000 -101.010215759
479137.750000000 -100.456481934
479137.812500000 -101.639465332
479137.875000000 -102.876907349
479137.937500000 -103.880729675
479138.000000000 -105.811225891
479138.062500000 -106.915206909
479138.125000000 -108.233901978
479138.187500000 -106.625434875
479138.250000000 -103.686866760
479138.312500000 -102.977874756
479138.375000000 -105.153343201
479138.437500000 -106.966751099
479138.500000000 -104.752532959
479138.562500000 -104.894256592
479138.625000000 -105.125381470
479138.687500000 -102.721633911
479138.750000000 -102.299125671
479138.812500000 -102.762176514
479138.875000000 -101.316398621
479138.937500000 -100.695121765
479139.000000000 -101.257949829
479139.062500000 -102.382209778
479139.125000000 -104.331405640
479139.187500000 -106.033081055
479139.250000000 -105.467399597
479139.312500000 -104.492301941
479139.375000000 -104.413681030
479139.437500000 -103.263702393
479139.500000000 -103.199569702
479139.562500000 -104.447860718
479139.625000000 -104.169952393
479139.687500000 -105.357246399
479139.750000000 -105.624694824
479139.812500000 -104.329673767
479139.875000000 -104.890480042
479139.937500000 -103.739471436
479140.000000000 -102.343170166
479140.062500000 -102.630371094
479140.125000000 -103.861930847
479140.187500000 -102.614120483
479140.250000000 -102.544586182
479140.312500000 -103.947563171
479140.375000000 -104.194770813
479140.437500000 -103.187141418
479140.500000000 -102.442695618
479140.562500000 -103.064849854
479140.625000000 -104.047111511
479140.687500000 -103.641082764
479140.750000000 -104.192665100
479140.812500000 -105.001426697
479140.875000000 -106.180221558
479140.937500000 -106.504646301
479141.000000000 -104.772674561
479141.062500000 -104.167114258
479141.125000000 -102.925132751
479141.187500000 -102.731872559
479141.250000000 -104.101806641
479141.312500000 -104.532470703
479141.375000000 -103.677726746
479141.437500000 -103.467483521
479141.500000000 -104.314605713
479141.562500000 -106.088348389
479141.625000000 -105.849678040
479141.687500000 -104.784294128
479141.750000000 -104.685859680
479141.812500000 -102.816184998
479141.875000000 -103.009178162
479141.937500000 -105.581695557
479142.000000000 -104.964607239
479142.062500000 -103.978279114
479142.125000000 -104.709609985
479142.187500000 -105.373786926
479142.250000000 -105.477348328
479142.312500000 -107.076698303
479142.375000000 -108.599830627
479142.437500000 -107.518699646
To which I want to fit the function
While the formula is kind of a beast, it has physical meaning, so I would like to not change it.
I have the following code:
index_min <- which(mydf[,2] == min(mydf[,2]))[1]
n0start <- -119
n1start <- 16
df0start <- 120
df1start <- 1
f0start <- mydf[index_min,1]
f1start <- mydf[index_min,1]
plot(x=mydf[,1],y=mydf[,2])
eq = function(f,n0, n1, f0, f1, df0, df1){ n0+n1*4*(f-f1)^2/(4*(f-f1)^2+(4*((f-f0)/df0)*(f-f1)-df1)^2)}
lines(mydf[,1], eq(mydf[,1],n0start, n1start, f0start, f1start, df0start, df1start), col="red" )
res <- try(nlsLM( y ~ n0+n1*4*(f-f1)^2/(4*(f-f1)^2+(4*((f-f0)/df0)*(f-f1)-df1)^2),
start=c(n0=n0start, n1=n1start,f0=f0start,df0=df0start,f1=f1start,df1=df1start) , data = mydf))
coef(res)
As you can see, the starting values look rather decent, but I get the "singular gradient matrix at initial parameter estimates" error. I have looked through all the other posts, however, I don't see why my formula is overdetermined or why the starting values should be bad.
Okay, I figured out the mistake. nlsLM requires data to be a data-frame and not just a bare matrix. The error message is simply misleading.
Related
Convert set of coordinates in specyfic format into sf object (polygon) in R
I have data frame that contains variable with set of coordinates, which supposed to be a polygon. I’d like to try to convert it into sf geometry polygon, but I have little idea how to achieve this goal.
Data looks like this:
a <- c("[30.523311, 50.40919], [30.523111, 50.409093], [30.522475, 50.408781], [30.522484, 50.408771], [30.523591, 50.407804], [30.524049, 50.407403], [30.526558, 50.406062], [30.526791, 50.405939], [30.527487, 50.4057], [30.527787, 50.405564], [30.528793, 50.405209], [30.528718, 50.404554], [30.530223, 50.404552], [30.530133, 50.404363], [30.529104, 50.404185], [30.529018, 50.403965], [30.528933, 50.403337], [30.529986, 50.403227], [30.531422, 50.403077], [30.531336, 50.402585], [30.531743, 50.402489], [30.531612, 50.401577], [30.531505, 50.401447], [30.531483, 50.401211], [30.531489, 50.40103], [30.531873, 50.400999], [30.531932, 50.400429], [30.531961, 50.400257], [30.531965, 50.400233], [30.532439, 50.400174], [30.533329, 50.400062], [30.533369, 50.399868], [30.533435, 50.399542], [30.533478, 50.39954], [30.534231, 50.399498], [30.534229, 50.399863], [30.5354583263, 50.4005476511], [30.5361664295, 50.4004929412], [30.5361825227, 50.4008211999], [30.5366814136, 50.4008656514], [30.537257, 50.400902], [30.537703, 50.400882], [30.538053, 50.400829], [30.538949, 50.40062], [30.539305, 50.400619], [30.539605, 50.40051], [30.540023, 50.40049], [30.54056, 50.400838], [30.540828, 50.400443], [30.541166, 50.400788], [30.541501, 50.401137], [30.542203, 50.40115], [30.54236, 50.40171], [30.542792, 50.401676], [30.543223, 50.401641], [30.548297, 50.400637], [30.558722, 50.398574], [30.558969, 50.398794], [30.559252, 50.399041], [30.559338, 50.399931], [30.55938, 50.400163], [30.559447, 50.400539], [30.559532, 50.400916], [30.559051, 50.400983], [30.555229, 50.401518], [30.555258, 50.402649], [30.554082, 50.402791], [30.553204, 50.402945], [30.552119, 50.40318], [30.551907, 50.403224], [30.551414, 50.403334], [30.55056, 50.403525], [30.550566, 50.403544], [30.547881, 50.404161], [30.547819, 50.404175], [30.546416, 50.404497], [30.544978, 50.404827], [30.54398, 50.404963], [30.543771, 50.404992], [30.54248, 50.405173], [30.540812, 50.405404], [30.538729, 50.405488], [30.537509, 50.405536], [30.536298, 50.405576], [30.534761, 50.405593], [30.53247, 50.405613], [30.530065, 50.40586], [30.528, 50.406381], [30.527158, 50.406595], [30.525696, 50.407249], [30.52562, 50.407313], [30.525481, 50.40743], [30.525423, 50.407478], [30.525274, 50.4076], [30.524885, 50.407919], [30.524221, 50.408445], [30.523308, 50.409165], [30.52333, 50.409175], [30.523311, 50.40919]")
b <- c("[30.517824, 50.405568], [30.517689, 50.40543], [30.517727, 50.405406], [30.517798, 50.405361], [30.517947, 50.405265], [30.518736, 50.405337], [30.519452, 50.405434], [30.520105, 50.405521], [30.52047, 50.405706], [30.520844, 50.405896], [30.521225, 50.405894], [30.521758, 50.405744], [30.524134, 50.404085], [30.524483, 50.403844], [30.524582, 50.403776], [30.524901, 50.403571], [30.524915, 50.403562], [30.524967, 50.403529], [30.525058, 50.40347], [30.525317, 50.403308], [30.525573, 50.403147], [30.526089, 50.402824], [30.526607, 50.4025], [30.527259, 50.402092], [30.527528, 50.401796], [30.528232, 50.40102], [30.528482, 50.400748], [30.528687, 50.400524], [30.528731, 50.400476], [30.528953, 50.400242], [30.529032, 50.400158], [30.529059, 50.400054], [30.529836, 50.399809], [30.530864, 50.399677], [30.530971, 50.39987], [30.53074, 50.400041], [30.53095, 50.400096], [30.53118, 50.401053], [30.531489, 50.40103], [30.531483, 50.401211], [30.531505, 50.401447], [30.531612, 50.401577], [30.531743, 50.402489], [30.531336, 50.402585], [30.531422, 50.403077], [30.529986, 50.403227], [30.528933, 50.403337], [30.529018, 50.403965], [30.529104, 50.404185], [30.530133, 50.404363], [30.530223, 50.404552], [30.528718, 50.404554], [30.528793, 50.405209], [30.527787, 50.405564], [30.527487, 50.4057], [30.526791, 50.405939], [30.526558, 50.406062], [30.524049, 50.407403], [30.523591, 50.407804], [30.522484, 50.408771], [30.519427, 50.407208], [30.519404, 50.407185], [30.519397, 50.407177], [30.518728, 50.406494], [30.518557, 50.406319], [30.517824, 50.405568]")
polygons<- as.data.frame(c(a,b), ncol=1, nrow=2)%>%
rename(polygon=1)
You need to parse these text strings into numbers, then convert into polygons. This function creates an sfc object that contains the polygons these vectors describe:
polygonise <- function(strings) {
do.call(c, lapply(strings, function(x) {
cutstring <- unlist(strsplit(x, "\\[|\\]"))
cutstring <- cutstring[nchar(cutstring) > 3]
sf::st_sfc(sf::st_polygon(list(do.call(rbind,
lapply(strsplit(cutstring, ", "), as.numeric)))))
}))
}
So, for example, we can do:
polygons$geometry <- polygonise(polygons$polygon)
ggplot(polygons) + geom_sf(aes(geometry = geometry), fill = "forestgreen")
Your solution works perfectly on the example I provided. However, when I try to use your function on real data I get an error ("'MtrxSet(x, dim, type = "POLYGON", needClosed = TRUE)': polygons not (all) closed ". Perhaps my coordinates are flawed? Sorry for that, but I’m pretty new to R.
a <- read_csv("https://raw.githubusercontent.com/slawomirmatuszak/Covid.UA/master/dzielnice.csv")
b<- polygonise(a)
Fitting distributions with R
Good afternoon. I have a vector 'a' containing 16000 values. I get the descriptive statistics with the help of the following: library(pastecs) library(timeDate) stat.desc(a) skewness(a) kurtosis(a) Especially skewness=-0.5012, kurtosis=420.8073 (1) Then I build a histogram of my empirical data: hist(a, col="lightblue", breaks = 140, border="white", main="", xlab="Value",xlim=c(-0.001,0.001)) After this I try to fit a theoretical distribution to my empirical data. I choose Variance-Gamma distribution and try to get its parameter estimates on my data: library(VarianceGammma) a_VG<-vgFit(a) The parameter estimates are the following: vgC=-11.7485, sigma=0.4446, theta=11.7193, nu=0.1186 (2) Further, I create a sample from the Variance-Gamma distribution with the parameters from (2) and build a histogram of created theoretical values: VG<-rvg(length(a),vgC=-11.7485,sigma=0.4446,theta=11.7193,nu=0.1186) hist(VG,breaks=140,col="orange",main="",xlab="Value") Bu the second histogram differs absolutely from the first (empirical) histogram. Moreover, it is built on the basis of the parameters (2), which I got on the empirical data. What's wrong with my code? How can I fix it? P.S. When I type dput(a[abs(a) > 5e-4]) I get: c(0.000801110480004752, 0.000588162271316861, 0.000555169128569233, 0.000502563410256229, 0.000854633994686438, 0.00593622112246628, -0.000506168123513007, -0.000502909585836875, 0.000720924373137422, 0.00119141739181039, 0.000548159382141478, -0.000516511318695123, -0.000744590777740584, 0.000595213912401249, 0.000514055190913965, -0.000589061375421807, -0.00175392114572581, 0.000745548313668465, -0.00075910234096277, -0.00059987613053103, 0.000583568488865538, 0.00426484136013094, 0.000610760059768012, 0.000575522836335551, 0.000823785810599276, 0.00181936036509178, -0.00073316272551871, -0.00184238143420679, -0.000519146793923397, -0.00120324664043103, -0.000882469414168696, -0.00148118339830283, 0.000929612782487155, 0.000565364610238817, 0.000578158613453894, 0.00060479145432879, -0.00520576206828594, 0.000708404040882016, 0.00105224485893451, 0.000636486872540587, -0.00359655507585543, 0.000769164650506582, 0.000635701125126786, 0.000570489501935612, -0.000641260260277221, 0.000735092947873994, 0.000757195823062773, 0.000556002742616357, -0.00207489740356159, -0.000553386431560554, 0.000511326871983186, 0.000504591469525195, -0.000749886905655472, -0.0013939718643865, -0.000513742626250036, -0.00105021597423516, -0.00156667292147716, 0.000864563166150134, 0.00433724128055069, 0.00053855648931922, -0.00150732363190365, 0.00052621785349416, 0.000987781100809215, 0.000560725818171903, 0.00176012436713435, -0.000594895431092368, -0.000686229580335151, 0.00138682284509528, -0.000531964338888358, -0.00179959148771403, 0.000574543871314503, -0.000686996216439084, -0.000559043343629995, 0.00055881173674166, -0.000636332688477736, -0.000623778186703561, -0.00173834148094443, -0.000567224129968125, -0.00122578683434504, 0.00130960156515414, -0.000548203197176633, -0.000522749285863711, -0.000820371086264871, 0.000756014225812507, -0.000714081490558627, -0.000617600335221624, 0.000523639760748651, -0.000578502663833191, 0.00107478825239227, 0.000612725356974764, -0.00065509337422931, 0.000505887803587513, -0.000566716376848575, 0.000511727090058756, 0.000572807738912218, -0.000756026937699161, 0.000547948751494332, 0.000628323894238392, -0.000541350489317693, -0.00133529454372372, -0.000590618859845904, -0.000700581963648972, 0.000735987224462775, 0.000528958898682319, 0.000838250041022448, -0.000519084424130511, -0.00052258402856431, -0.000538130765869838, -0.000631819887885854, 0.00054800880764283, 0.00266115500510899, -0.000839092093771754, 0.000559253571783103, -0.000801028189803432, -0.000608879021022801, -0.000538018076854385, -0.000689859734395171, 0.00329650346269972, 0.000765494493951024, -0.000689450477848297, -0.000560199139975737, 0.00159082699266122, -0.00208548663121455, -0.000598493596793759, 0.000563544422691464, 0.000626996183768824, -0.000653166846808162, -0.000851350174739807, -0.00140687473245116, -0.000887003220306326, -0.000765614651347946, -0.00100676206277761, 0.000724714394852555, 0.00108872127644233, -0.000678558537305918, -0.000705087556212902, 0.000544828152248655, -0.000791700964308362, 0.000606125736727137, -0.00119335967326073, 0.00075413211796338, 0.000526038939010931, 0.00086543737231537, -0.000817788712950573, -0.000584070926663571, 0.000619657281937691, 0.000680783312420274, -0.000513831718574664, -0.00050972403875349, -0.00114542220685365, -0.00070564389723593, -0.01057964950882, -0.000610357922434801, 0.000818264221596365, 0.000940825400308043, -0.000726555639413817, -0.000591089505560305, 0.000564738888193972, -0.00068515060569041, 0.000668920238348747, -0.00110103375121717, -0.0015480433031172, 0.000663030855223568, 0.000500097431997304, -0.000600730311271391, -0.000672397772962796, -0.000607852365856587, 0.000536711920570809, 0.000595055206488837, 0.000523123873687581, 0.000977280737528119, 0.000616410821629998, 0.000788593666889881, -0.000671642905915704, 0.000717328711735021, -0.000551853104219902, -0.000565153434708421, -0.000802585212152707, 0.000536342062561701, 0.000682048510343591, -0.000541902545439399, 0.000779676683974273, 0.000698841439971787, 0.000559313965908359, -0.00064986819016255, 0.000795421518319017, 0.00364973919549527, 0.000669658692276087, 0.00109045476974678, 0.000514411572742901, 0.000503832507211754, -0.000507376233564116, 0.001232871590787, 0.000561820312542594, -0.000501190337518054, -0.000769036505996468, -0.000695537959007453, -0.000572065848166048, -0.00167929926328192, 0.000597078186826749, 0.00710238430870014, 0.000745192112519888, -0.00116091022028009, -0.000791139281769659, -0.00148898466632552, 0.000565144038962018, -0.000514019821833855, -0.00148427996685285, -0.000822717245339888, -0.00062922111212238, -0.000636011367371125, 0.00119640327632808, 0.000548455410294579, 0.000652678152560426, 0.000509244387833618, 0.000961872348987924, 0.000662064072514568, -0.00068116858054168, -0.000569930302445343, 0.00188358126928101, 0.00130560555273895, 0.000593470885775105, 0.00160093110088155, 0.000785262438315115, -0.000912313442922752, 0.000609996052359563, 0.000720137994393966, 0.000568163899000496, 0.00128685533068307, -0.000756787473447318, 0.000765932134255465, 0.00064884753100003, 0.000687571386270847, -0.000582094290400903, -0.000693177295971736, -0.000601776208094762, 0.000503616387996786, -0.000615095866544735, -0.000799593899689199, 0.000773750859128342, -0.000522576090260074, 0.000503578107212022, -0.00104492224837571, 0.000547928732299141, 0.00310304337507183, 0.000893382870797765, -0.000577792878910799, -0.000647710366578735, -0.00061992948706191, 0.000825702487162516, 0.000606579510524341, 0.000552792484727505, 0.000688600840895504, 0.000505093563534231, -0.000728420573667066, -0.00157924525963438, -0.000603846616019865, -0.000521941317177976, 0.00150498158245682, -0.000584572670337735, 0.000713757870583365, 0.000524287801789924, 0.00107217649464886, 0.00213147531822244, 0.000566012832157625, -0.00069828890607937, 0.000641567963736378, -0.000509531713644762, -0.000547564140049417, -0.00115275240244728, 0.000560465768010943, -0.000651807371497171, -0.00096487058986483, 0.000753687665266511, -0.000665599418910645, -0.000691278087025182, -0.000578010050725553, -0.000685833148198256, 0.000698470819832764, 0.00102943368139208, -0.000725840586788706, 0.00125882415960632, -0.000630791474954151, -0.000764813558678412, -0.000638539347184164, 0.000654486496518558, 0.000547453642294471, 0.000572020020495501, -0.000605791001705214, 0.00660211658324172, 0.00114928683282756, 0.000985676480677711, -0.000694668292547718, -0.000528955637964401, 0.000647975568638159, 0.00116454536417443, 0.000506748841724303, -0.000500925156604382, -0.000567015088082101, 0.00128711230206946, 0.000533633762033858, 0.00505991432758357, 0.000518058378462527, -0.000592822519784875, 0.00177414999018666, 0.00059845426944527, -0.000511614433724716, 0.0016614697907098, 0.000852196464322219, 0.00241689725305427, -0.000614317948913978, -0.000729717143318709, -0.000612900648802039, -0.000727983564232204, -0.000694965869158182, -0.000527752006066251, -0.000584233784708843, 0.000522097476268968, 0.000543092880677776, 0.000947121210698398, -0.00241810275096377, 0.00181893137435019, 0.000931873879297385, 0.000512116215015013, 0.000724985702444059, -0.000566713495050664, 0.000603953591362227 ) After fitting the data look like the following (empirical histogram-blue, theoretical histogram-orange): The same when include freq=FALSE in hist
This will all be due to anomalous values in a not represented by the histogram you've shown. This could be the cause of both the very high kurtotsis, and the vgFit() algorithm failing to find a good fit. Type dput(a[abs(a) > 5e-4]) in the console and copy the output into your question. People then may be able to recreate aomething like the vector a without having to get all 16000 values and debug the vgFit issue. Thanks for the extra data. There are some extreme values in there, but I don;t think those are what is causing the problem in vgFit. Fitting 4 parameters which can be almost any value is difficult, but you can help it along by rescaling your data to something typical. Try this: b <- (a-mean(a))/sd(a) vgf <- vgFit(b) vgf$param VG <- rvg(16000, param = vgf$param) VG_rescaled <- VG*sd(a)+mean(a) hist(VG_rescaled, breaks=140, col="orange", main="", xlab="Value") and see if the two histograms are close enough now.
How to solve Warning in log(z) : NaNs produced in R?
I try to run K-S test simulation with different data and with GEV distribution but I get 50 warnings of Warning in log(z): NaNs produced
This is my data t1 = 2.1466558 2.9447386 2.1410642 1.8847492 2.0233282 2.1907725 3.1755095 2.1142972 2.1889601 2.8422979 1.8555857 1.0429501 2.1383811 3.9976282 14.0612719 6.0016379 4.0426939 3.9845386 5.1821300 3.8703266 2.9009807 3.8450287 6.1019829 4.1811626 8.0737452 2.8416879 3.1656657 2.1342049 1.9984793 2.8037649 6.9629563 2.8223349 10.9695854 1.8985456 0.9444765 6.0065642 2.0394709 9.1677515 5.0589429 4.1036932 4.9599679 3.0425898 1.9477278 3.0447457 8.1563085 4.9423730 3.1336760 1.8389239 3.1262185 1.1628846 3.8445247 2.1454052 1.9209593 0.9197765 2.8171347 8.0249643 13.1267940 6.8506226 2.0811591 2.8517716 2.8864796 1.8227987 8.1442224 2.8798242 5.1112049 2.8529055 6.8265215 1.0436781 3.8380311 2.9659720 3.0153516 2.8497134 24.0545609 3.1952943 3.9564030 7.1348925 3.0067497 2.8581224 13.0294469 1.8576194 2.8081190 6.0940443 1.9729950 4.1334539 9.9874363 2.0489537 1.9479052 2.8050009 3.1097060 8.9115900 23.8376271 3.9219177 2.9991323 3.8501608 4.1910852 4.9508869 11.9260378 5.1473547 6.1864583 2.8165587 3.8589393 5.1742220 2.8812650 4.1469513 2.9816058 6.9291070 4.0086371 2.8950365 3.1368533 2.9716707 4.0936148 4.0987735 8.8792285 2.9914305 15.9832293 11.1705646 4.1691180 2.0268396 9.1313510 2.8457873 5.8162405 5.1019303 2.9493099 3.1892744 6.1027555 5.9852653 6.0070368 5.0606722 3.8827039 2.8579010 3.1809342 2.8639117 4.0446142 8.1086074 6.9708477 3.9406243 3.9113551 2.8471808 3.9408469 1.8318536 4.8696027 6.1638158 10.0075047 4.0620721 2.1995222 2.9713600 0.9827086 11.8048057 3.1639570 4.1820899 2.8913417 5.1807095 1.8735194 3.8650210 2.9308563 6.9203276 7.0470336 2.1721080 1.9304191 2.9782089 4.9717892 1.8260324 4.0094237 6.0354774 4.1934337 3.8605304 6.9868062 9.0001938 19.9510362 10.0213967 1.9980948 1.9564188 10.0595901 5.9441410 5.9212171 1.9805753 2.8141160 9.8859371 2.1912938 5.0260191 7.0394183 3.1071499 4.8651357 4.8464983 3.1653826 4.0813080 0.9293124 2.0533324 3.1302422 5.0649879 1.9045972 3.0304574 6.1638933 1.8765108 2.1042605 5.0677281 7.9328270 5.0485400 11.8101217 2.8496955 3.9641349 2.0423748 3.9535697 10.1833001 1.9963743 3.9404075 1.0794579 5.1952880 2.1310139 3.1615550 4.1934939 2.1528778 1.8080386 7.8411243 9.8299614 6.0534968 4.0174467 2.0321006 6.8884815 3.1990381 3.9448174 4.1087308 2.8989261 3.1667614 3.0734750 4.9591400 4.0537864 5.1886589 2.0860818 3.9166460 3.8028030 2.8333645 2.0257119 3.9347423 2.1057551 2.9608942 5.8546608 3.1449161 1.8630542 5.0733393 1.8331204 3.1629142 4.0929211 6.9565034 3.8253997 2.8041233 5.1246350 3.8994802 2.0389505 5.0663955 3.8854816 1.8575128 1.9655496 3.0056002 4.9812668 4.8201262
I fit the GEV to the data:
fit5 = fevd(t1, type="GEV") using the function fevd in extRemes package. I wanna do K-S test simulation but get 50 warnings at this part:
stats <- replicate(n.sims, {
r <- rgevd(n = length(t1),location=fit5$results$par["location"],
shape= fit5$results$par["shape"],
scale = fit5$results$par["scale"])
estfit.gev <- fevd(r, type = "GEV") # added to account for the estimated parameters
as.numeric(ks.test(r, "pgevd",location=estfit.gev$results$par["location"],
shape= estfit.gev$results$par["shape"],
scale = estfit.gev$results$par["scale"])$statistic)
how do I solve this Warning in log(z): NaNs produced? I tried to use absolute r value but I still get the same warnings
Bootstrap p value contradicts p value for likelihood ratio test
I have the same problem as the one posted by #soapsuds here. I did not want to ask a duplicate question but when I tried to edit the original question to provide the reproducible example that was missing in the original post my edits got rejected. Since the reproducible example has a lot of elements, I could not write it as a comment to the original question either, so I provide my code and my reproducible data here, as a separate question.
I am trying to compare two models using the likelihood ratio test. From bootstrapping I get a set of 1000 p-values. Here are the numbers I get:
chi2 <- c(41.83803376, 69.23970174, 42.5479637, 50.90208302, 39.18366824, 78.88589665, 28.88469406, 34.99980796, 85.80860848, 66.01750186, 29.06286, 46.43221576, 46.50523792, 59.87362884, 46.17274808, 77.97429928, 48.04404216, 12.88592623, 43.1883816, 33.24251471, 53.27310465, 56.92595147, 47.99838583, 46.0718587, 49.0760042, 29.70866297, 66.80696553, 66.61091741, 37.82375112, 50.19760846, 30.99961864, 27.17687828, 37.46944206, 66.36226432, 48.30737714, 43.64410333, 23.78480451, 42.52842793, 60.49309556, 46.29154, 26.96744296, 32.21561396, 48.20316788, 38.73153704, 67.80328765, 55.00664931, 36.74645735, 23.3647159, 56.35290442, 38.11055268, 58.3316501, 36.00500638, 41.36949956, 49.09067881, 64.42712507, 23.97787069, 54.5394799, 87.02114296, 26.01402166, 50.47426712, 38.58006084, 48.47626864, 22.28809699, 58.87590487, 17.59264288, 33.32650413, 67.77868338, 60.95427815, 37.19931376, 36.23280256, 53.54379697, 70.06479334, 41.3482703, 34.54099647, 55.99585144, 30.60500406, 32.02745276, 37.92670127, 44.23450124, 40.38607671, 44.02263294, 40.89874789, 62.74174279, 50.95137406, 47.12851204, 26.03848394, 36.6202765, 61.06296311, 50.17094183, 35.93242228, 41.8913277, 35.19089913, 38.88574534, 66.075866, 26.34296242, 49.99887059, 42.97123036, 34.89006324, 66.5460019, 67.61855859, 48.52166614, 41.41324193, 46.76294302, 14.87650733, 24.11661382, 62.28747719, 43.94865019, 44.20328393, 41.17756328, 43.74055584, 49.46236395, 38.59558107, 42.85073398, 49.81046036, 36.60331917, 39.85328124, 59.31376822, 61.36038822, 52.56707689, 29.19196892, 46.473958, 39.12904163, 38.75057931, 36.32493909, 49.61088785, 33.42904297, 34.73661836, 33.97736002, 37.44094284, 57.73605417, 43.14773064, 42.78707831, 26.84112684, 48.47832871, 45.94043053, 71.13563773, 46.28614795, 42.33386157, 59.31216832, 46.72946806, 47.76027545, 52.45174304, 49.99459367, 59.00971014, 24.03299408, 17.09453132, 37.44112252, 46.6352525, 60.42442286, 39.35194465, 46.57121135, 56.28622077, 59.20354176, 57.72511864, 41.97053375, 27.97077407, 29.70497125, 46.63976021, 40.24305901, 24.84335714, 36.08600444, 61.619572, 69.31377401, 86.91496878, 44.47955842, 44.1230351, 46.12514671, 43.97381958, 71.99269072, 47.01277643, 50.08167664, 27.01076954, 31.32586466, 40.96782215, 19.07024825, 53.00009679, 43.15397869, 42.49652848, 53.47325607, 43.45891027, 42.57719313, 39.40459925, 42.15077856, 52.23784844, 33.07947933, 45.02462309, 59.187763, 51.9198527, 48.3179841, 76.10501177, 34.95091433, 40.75545034, 31.27034043, 39.83209227, 47.87278051, 46.25057806, 62.84591205, 41.24656655, 68.14749236, 53.11576938, 39.20515676, 61.96116013, 35.64665684, 72.52689101, 54.64239536, 34.14169048, 34.32282338, 49.60786171, 50.32976034, 43.83560386, 57.49367366, 81.65759842, 61.59398941, 37.77960776, 30.74484476, 34.72859511, 32.46631033, 37.41725027, 34.04569722, 54.11932007, 34.62264522, 28.36753913, 30.95379445, 84.06354755, 29.32445434, 56.7720931, 33.23951864, 48.61860157, 39.3563214, 32.44713462, 61.25078174, 32.49661836, 40.38508488, 26.73565294, 58.16191656, 61.12461262, 23.701462, 22.14004554, 57.80213129, 57.15936762, 31.51238062, 44.60223083, 30.60135802, 46.96637333, 42.79517081, 56.85541543, 48.79421654, 29.72862307, 41.61735121, 43.37983393, 41.16802781, 61.69637392, 37.29991153, 39.0936012, 57.39158494, 57.55033901, 50.72878897, 34.82491685, 42.66486539, 34.54565803, 55.04161695, 44.56687339, 53.46745359, 57.22210412, 34.8578696, 28.81098073, 51.4033337, 51.9568532, 60.98717632, 62.98817996, 44.1335128, 33.38418814, 59.71059054, 45.82016411, 29.47178401, 30.64995791, 28.52106318, 53.98066153, 64.22209517, 58.29438562, 39.18280924, 38.1302144, 41.90062316, 28.68650929, 69.42769639, 33.79539164, 26.08549507, 55.29167497, 97.25975259, 63.07957724, 56.59002373, 51.40088678, 71.33491023, 46.24955174, 33.90101761, 38.0669817, 52.50993176, 51.84637529, 39.93642798, 61.9268346, 30.25561485, 49.57396856, 44.70170977, 57.00286149, 40.39009586, 63.23642634, 59.23643766, 55.80521902, 68.58421775, 24.04456631, 51.64338572, 61.14103174, 59.29371792, 46.51493959, 43.48297587, 39.99164284, 44.62589755, 58.89385062, 60.96824416, 54.02310453, 43.54420281, 44.24628098, 47.0991445, 58.9015349, 60.54157696, 34.86277089, 33.79969585, 34.57183642, 47.21383117, 55.3529805, 36.49813553, 44.94388291, 29.43134497, 43.41469037, 43.033338, 63.37329389, 38.22029171, 43.2894392, 23.42769168, 55.18117532, 19.39227876, 28.29656641, 28.56075122, 39.57260362, 65.48606054, 31.05339648, 24.87488959, 61.6027878, 59.56983406, 37.53918879, 28.67095839, 36.51499868, 44.43350204, 53.35842664, 48.30182354, 31.03494822, 45.68689659, 46.11113306, 53.89204524, 29.75548276, 35.60906482, 53.35195594, 56.28657675, 44.77245145, 60.20671942, 41.62253735, 40.34528594, 38.48551456, 27.39317425, 51.05414332, 38.41986986, 75.05074423, 34.16773046, 52.18497954, 49.63059496, 28.7365636, 10.59466471, 38.1033901, 52.20531405, 47.031987, 47.45955635, 44.64312012, 50.32229588, 62.40798968, 37.7455721, 31.97746406, 51.17250147, 45.91231295, 66.58450378, 32.68956686, 34.35845347, 70.34703042, 41.47493453, 53.67684859, 35.66735299, 19.76630329, 35.69026569, 76.57475236, 62.11269107, 37.06632602, 57.91686258, 33.95869501, 55.18034702, 66.09725866, 46.80608564, 46.75623531, 55.49605214, 45.7813294, 22.37612777, 62.40414132, 50.51745906, 46.86535062, 54.4172637, 35.44713601, 45.40918234, 43.83215257, 57.14754799, 24.20941074, 44.8145542, 50.79673435, 42.14561269, 32.73720673, 28.51047028, 32.14753623, 28.43006627, 39.50188334, 58.51806717, 37.96898151, 73.14656287, 48.23605238, 75.31273481, 29.57608972, 43.62952257, 30.47534709, 43.24927262, 43.61475563, 53.48883918, 53.85263136, 41.91477406, 56.16405384, 46.21202327, 55.52602904, 49.88481191, 46.31478116, 72.29722834, 40.48187205, 35.31368051, 40.57713079, 34.15725967, 65.85738596, 32.16093944, 32.07117679, 46.44579516, 53.3243447, 69.35531671, 21.70205174, 44.30678622, 40.13349937, 51.7431728, 43.03690121, 26.53566586, 18.74773427, 25.97768442, 66.68668827, 42.97352559, 31.61567696, 61.57362103, 55.07104736, 25.05950764, 53.04884067, 30.47176616, 43.33249885, 44.48360752, 40.59006165, 44.29759954, 69.71063388, 47.70186943, 51.12166943, 40.15048072, 44.96459746, 56.31842906, 57.79593771, 49.19795057, 33.58506451, 42.67650993, 47.96512915, 57.98722437, 42.08107371, 66.85903821, 45.30286487, 38.39187118, 48.02442004, 35.97047743, 56.71378254, 40.51082047, 43.78022461, 60.33208664, 35.78159098, 40.98937317, 36.20547787, 45.2382906, 47.81497885, 20.44519563, 16.68817267, 38.31035896, 38.60590267, 70.75756511, 31.73001452, 45.85476281, 47.11473565, 31.40248172, 42.94971714, 39.34376633, 21.09018956, 31.45915941, 53.82696054, 73.59824534, 31.5694168, 39.02189966, 46.91790827, 60.66603832, 59.81148782, 20.46813743, 54.95108785, 66.71844123, 49.48461319, 25.10459028, 60.26169536, 21.90344297, 63.56310687, 38.70295559, 58.19794152, 25.68981924, 61.4804908, 41.97067608, 22.77156359, 48.51789441, 50.31845297, 42.36456456, 43.35814281, 41.32891651, 35.17106573, 48.45296117, 30.55292595, 55.26758567, 71.25929921, 34.62580089, 43.89804598, 46.06384675, 30.74209253, 47.99143497, 34.02715801, 37.95367551, 45.14366438, 40.73655716, 45.32116105, 48.17651965, 63.54774876, 16.32237452, 54.22730144, 46.02331286, 45.44633826, 53.56976595, 53.96781286, 19.79116777, 42.05820938, 45.48852278, 37.34932167, 45.134461, 49.60637239, 29.99017683, 35.2785614, 71.54855053, 61.55744768, 55.7627296, 37.72455372, 62.51288842, 48.17063649, 65.26648616, 48.4831201, 33.49833137, 32.10986243, 15.42586026, 41.95660905, 30.07072484, 42.33604863, 53.20660203, 48.27036556, 32.92677161, 33.59521848, 44.04333058, 59.30038922, 48.84064622, 63.31815488, 36.01169023, 44.42967033, 23.14247159, 53.6314237, 42.43225997, 28.18151375, 44.0733306, 55.93530003, 30.86515779, 34.10702034, 59.38495522, 57.79906004, 64.86160093, 56.70670687, 43.24880707, 40.00049219, 44.08430336, 17.50391283, 72.81320114, 41.55481964, 63.461066, 50.81938548, 58.7427594, 35.27822458, 33.5188344, 46.13196979, 56.94022883, 66.96258461, 39.19601268, 21.95750575, 51.67252792, 46.51047909, 30.42289547, 46.47496475, 41.6440483, 42.36900563, 68.29398345, 30.14059255, 38.90124252, 40.87014585, 51.33635945, 51.72908337, 50.8177621, 31.65411733, 56.75197699, 47.76885318, 34.18305356, 52.52137441, 48.39806899, 18.34609209, 32.5461584, 60.15104883, 36.29250847, 39.02418361, 34.68801402, 48.02453889, 31.36738248, 42.44522981, 71.79176852, 34.25588794, 38.46866138, 45.01393624, 63.38509325, 32.44823195, 64.59346474, 53.80793998, 41.2889141, 28.86534461, 34.85039051, 37.04622686, 31.83207726, 36.65410743, 27.66293315, 23.11203257, 41.61059067, 19.97321534, 59.879676, 39.84187157, 47.324581, 38.24903991, 41.0234849, 62.30809429, 48.47191326, 23.26696808, 29.91547934, 78.39181209, 41.86240014, 33.53717515, 39.63756903, 74.86377649, 56.30173648, 40.29403413, 59.12602764, 47.23561802, 51.32370456, 45.44426051, 55.54666292, 58.85362888, 38.30516953, 46.11300177, 37.96931091, 41.01315149, 63.09345867, 26.74145771, 31.37447907, 39.26896396, 65.35880308, 60.0670218, 45.48057201, 29.76683425, 51.39638136, 46.12180705, 60.72093818, 45.01613513, 37.04611291, 31.32979098, 57.82548455, 29.89919764, 38.77980495, 55.71511912, 66.9872235, 48.74616069, 32.87503301, 56.10335632, 28.72445387, 41.00675821, 55.22238115, 38.56391412, 21.82487917, 51.87394855, 41.62740713, 72.32943223, 49.85456187, 41.76869194, 55.686196, 46.18471338, 52.57455653, 23.03383172, 51.460223, 45.88045256, 47.91709836, 53.09464847, 65.17159616, 48.0076358, 42.50038253, 50.57143193, 22.05776575, 25.5770314, 57.41889173, 37.07408252, 69.83286794, 53.31690771, 36.14562381, 35.3626014, 70.74448842, 30.01870438, 41.95755074, 64.41141845, 48.12704663, 29.33183678, 47.45391445, 35.76760392, 17.57864013, 42.66918162, 27.84884911, 37.83419437, 56.38203205, 32.93395446, 19.45549279, 48.49557175, 63.74692618, 48.36501421, 38.45370018, 63.77499738, 43.40984685, 61.28735474, 47.00513455, 31.82012086, 40.85624032, 32.79590137, 43.79441893, 47.93350586, 26.44410209, 22.71480768, 41.74097624, 29.7828174, 35.24077319, 37.1436077, 63.62150539, 35.27952907, 30.9258966, 35.22384343, 45.0069715, 47.38652625, 60.86474384, 53.19528479, 37.61239521, 64.78497877, 39.50008676, 43.11733875, 34.67761458, 55.21401193, 57.22836509, 30.10411603, 30.03903287, 53.62027996, 40.63516283, 50.229386, 39.59707517, 55.53993024, 62.31160356, 48.65142538, 59.51279601, 51.46268896, 36.70086545, 45.73324953, 39.82026282, 51.51657943, 39.9507342, 26.65847555, 18.11032673, 41.57393548, 37.24804734, 59.78878572, 42.18870686, 57.73556775, 29.83442692, 24.27687775, 44.54663257, 48.40426261, 34.13830576, 64.47843419, 53.82888778, 45.77073351, 41.95910655, 56.25654343, 42.44938602, 18.92651056, 62.89841562, 42.28210051, 60.01632343, 56.38799965, 53.56842386, 71.059581, 59.21196097, 72.29678294, 40.0820475, 74.53163756, 46.35508897, 48.65592196, 36.69711286, 54.84914739, 57.62299813, 63.0750109, 25.53592874, 19.43203054, 63.18532427, 54.79806194, 28.75123602, 47.68037559, 36.06887062, 48.53619627, 42.05208952, 14.47366507, 26.25183654, 57.37741978, 24.92962789, 47.85306044, 35.55674275, 43.62606531, 51.98445971, 57.10441923, 45.20539557, 43.22417529, 48.20941756, 37.12416781, 39.54238987, 45.31000358, 24.59001204, 32.61256929, 31.61553515, 55.76617515, 57.82479513, 34.12465645, 52.1634834, 50.140277, 34.5334757, 70.76112738, 47.22161503, 35.44101995, 54.50312705, 47.74706989, 21.04494842, 42.42698916, 57.8551517, 49.67127478, 67.6702045, 30.64335682, 31.87819093, 45.79096976, 42.72129981, 56.22043416, 22.12571532, 31.93377902, 31.9561172, 60.28281847, 37.49005649, 30.63141229, 22.82707918, 29.55804713, 55.79929136, 39.64043613, 31.79538118, 61.92391469, 19.30462724, 37.00041938, 61.26446455, 47.10048686, 34.70929308, 33.34157984, 49.28331646, 39.9565451, 48.80158593, 29.25279435, 49.96980394, 68.7766356, 49.61949286, 18.80600378, 52.93721773, 24.29791779, 67.69568275, 54.22725318, 35.67531845, 58.05037476, 70.54029077, 55.59508174, 42.07974012, 61.62117032, 44.47174079, 40.13197612, 61.19863058, 35.16748823, 54.79320966, 46.40640448, 41.99222891, 53.33216862, 19.04146695, 29.60278169, 38.43089591, 61.22497978, 32.04678119, 30.77915985, 38.02625789, 74.25140223, 30.44626923, 42.69951906, 28.99988779, 49.76041564, 30.86941271, 58.65788956, 62.64967161, 23.5689175, 42.21941421, 54.88455829, 38.10115824, 24.12341961, 32.84464782, 81.72102673, 42.42771851, 37.75191241, 32.05927543, 43.55812503, 64.79161154, 61.05179286, 53.24693267, 36.29056269, 61.49030629, 53.68500702, 65.93501988, 50.7243041, 51.72139759, 64.80610623, 58.2860023, 33.16444766, 42.7872046, 55.14190562, 39.14341079, 36.05577261, 30.03351742, 24.16526837, 47.94163599, 52.55045103, 56.60625705, 61.6878126, 23.13212844, 50.50369148, 47.79873905, 47.01238239, 35.9159739, 53.18067189, 48.42928497, 67.48879213, 37.37609292, 19.7749038, 47.87115046, 48.90378974)
p.values <- c(9.92E-11, 8.72E-17, 6.90E-11, 9.71E-13, 3.86E-10, 6.58E-19, 7.68E-08, 3.30E-09, 1.98E-20, 4.47E-16, 7.01E-08, 9.48E-12, 9.14E-12, 1.01E-14, 1.08E-11, 1.04E-18, 4.17E-12, 0.000331062, 4.97E-11, 8.14E-09, 2.90E-13, 4.53E-14, 4.27E-12, 1.14E-11, 2.46E-12, 5.02E-08, 2.99E-16, 3.31E-16, 7.74E-10, 1.39E-12, 2.58E-08, 1.86E-07, 9.29E-10, 3.75E-16, 3.64E-12, 3.94E-11, 1.08E-06, 6.97E-11, 7.38E-15, 1.02E-11, 2.07E-07, 1.38E-08, 3.84E-12, 4.86E-10, 1.81E-16, 1.20E-13, 1.35E-09, 1.34E-06, 6.06E-14, 6.68E-10, 2.21E-14, 1.97E-09, 1.26E-10, 2.44E-12, 1.00E-15, 9.74E-07, 1.52E-13, 1.07E-20, 3.39E-07, 1.21E-12, 5.26E-10, 3.34E-12, 2.35E-06, 1.68E-14, 2.74E-05, 7.79E-09, 1.83E-16, 5.84E-15, 1.07E-09, 1.75E-09, 2.53E-13, 5.74E-17, 1.27E-10, 4.17E-09, 7.26E-14, 3.16E-08, 1.52E-08, 7.35E-10, 2.91E-11, 2.08E-10, 3.25E-11, 1.60E-10, 2.36E-15, 9.47E-13, 6.65E-12, 3.35E-07, 1.44E-09, 5.53E-15, 1.41E-12, 2.04E-09, 9.65E-11, 2.99E-09, 4.49E-10, 4.34E-16, 2.86E-07, 1.54E-12, 5.56E-11, 3.49E-09, 3.42E-16, 1.98E-16, 3.27E-12, 1.23E-10, 8.01E-12, 0.000114784, 9.07E-07, 2.97E-15, 3.37E-11, 2.96E-11, 1.39E-10, 3.75E-11, 2.02E-12, 5.21E-10, 5.91E-11, 1.69E-12, 1.45E-09, 2.74E-10, 1.34E-14, 4.75E-15, 4.16E-13, 6.56E-08, 9.28E-12, 3.97E-10, 4.82E-10, 1.67E-09, 1.87E-12, 7.39E-09, 3.77E-09, 5.58E-09, 9.42E-10, 3.00E-14, 5.08E-11, 6.10E-11, 2.21E-07, 3.34E-12, 1.22E-11, 3.33E-17, 1.02E-11, 7.69E-11, 1.35E-14, 8.15E-12, 4.82E-12, 4.41E-13, 1.54E-12, 1.57E-14, 9.47E-07, 3.56E-05, 9.42E-10, 8.55E-12, 7.65E-15, 3.54E-10, 8.83E-12, 6.27E-14, 1.42E-14, 3.01E-14, 9.27E-11, 1.23E-07, 5.03E-08, 8.53E-12, 2.24E-10, 6.22E-07, 1.89E-09, 4.17E-15, 8.40E-17, 1.13E-20, 2.57E-11, 3.08E-11, 1.11E-11, 3.33E-11, 2.16E-17, 7.05E-12, 1.47E-12, 2.02E-07, 2.18E-08, 1.55E-10, 1.26E-05, 3.34E-13, 5.06E-11, 7.08E-11, 2.62E-13, 4.33E-11, 6.79E-11, 3.44E-10, 8.45E-11, 4.92E-13, 8.85E-09, 1.95E-11, 1.43E-14, 5.78E-13, 3.62E-12, 2.69E-18, 3.38E-09, 1.73E-10, 2.24E-08, 2.77E-10, 4.55E-12, 1.04E-11, 2.24E-15, 1.34E-10, 1.52E-16, 3.14E-13, 3.82E-10, 3.50E-15, 2.37E-09, 1.65E-17, 1.45E-13, 5.12E-09, 4.67E-09, 1.88E-12, 1.30E-12, 3.57E-11, 3.39E-14, 1.62E-19, 4.22E-15, 7.92E-10, 2.94E-08, 3.79E-09, 1.21E-08, 9.54E-10, 5.38E-09, 1.89E-13, 4.00E-09, 1.00E-07, 2.64E-08, 4.79E-20, 6.12E-08, 4.89E-14, 8.15E-09, 3.11E-12, 3.53E-10, 1.22E-08, 5.02E-15, 1.19E-08, 2.09E-10, 2.33E-07, 2.41E-14, 5.36E-15, 1.12E-06, 2.53E-06, 2.90E-14, 4.02E-14, 1.98E-08, 2.41E-11, 3.17E-08, 7.22E-12, 6.08E-11, 4.69E-14, 2.84E-12, 4.97E-08, 1.11E-10, 4.51E-11, 1.40E-10, 4.01E-15, 1.01E-09, 4.04E-10, 3.57E-14, 3.29E-14, 1.06E-12, 3.61E-09, 6.50E-11, 4.16E-09, 1.18E-13, 2.46E-11, 2.63E-13, 3.89E-14, 3.55E-09, 7.98E-08, 7.52E-13, 5.67E-13, 5.74E-15, 2.08E-15, 3.07E-11, 7.56E-09, 1.10E-14, 1.30E-11, 5.67E-08, 3.09E-08, 9.27E-08, 2.02E-13, 1.11E-15, 2.26E-14, 3.86E-10, 6.62E-10, 9.60E-11, 8.51E-08, 7.93E-17, 6.12E-09, 3.27E-07, 1.04E-13, 6.08E-23, 1.99E-15, 5.37E-14, 7.53E-13, 3.01E-17, 1.04E-11, 5.80E-09, 6.84E-10, 4.28E-13, 6.00E-13, 2.62E-10, 3.56E-15, 3.79E-08, 1.91E-12, 2.29E-11, 4.35E-14, 2.08E-10, 1.83E-15, 1.40E-14, 8.00E-14, 1.22E-16, 9.41E-07, 6.66E-13, 5.31E-15, 1.36E-14, 9.09E-12, 4.28E-11, 2.55E-10, 2.39E-11, 1.66E-14, 5.80E-15, 1.98E-13, 4.14E-11, 2.90E-11, 6.75E-12, 1.66E-14, 7.20E-15, 3.54E-09, 6.11E-09, 4.11E-09, 6.36E-12, 1.01E-13, 1.53E-09, 2.03E-11, 5.79E-08, 4.43E-11, 5.38E-11, 1.71E-15, 6.32E-10, 4.72E-11, 1.30E-06, 1.10E-13, 1.06E-05, 1.04E-07, 9.08E-08, 3.16E-10, 5.85E-16, 2.51E-08, 6.12E-07, 4.20E-15, 1.18E-14, 8.96E-10, 8.58E-08, 1.51E-09, 2.63E-11, 2.78E-13, 3.65E-12, 2.53E-08, 1.39E-11, 1.12E-11, 2.12E-13, 4.90E-08, 2.41E-09, 2.79E-13, 6.26E-14, 2.21E-11, 8.54E-15, 1.11E-10, 2.13E-10, 5.52E-10, 1.66E-07, 8.99E-13, 5.70E-10, 4.59E-18, 5.06E-09, 5.05E-13, 1.86E-12, 8.29E-08, 0.001134145, 6.71E-10, 5.00E-13, 6.98E-12, 5.62E-12, 2.36E-11, 1.30E-12, 2.79E-15, 8.06E-10, 1.56E-08, 8.46E-13, 1.24E-11, 3.35E-16, 1.08E-08, 4.58E-09, 4.97E-17, 1.19E-10, 2.36E-13, 2.34E-09, 8.75E-06, 2.31E-09, 2.12E-18, 3.24E-15, 1.14E-09, 2.73E-14, 5.63E-09, 1.10E-13, 4.29E-16, 7.84E-12, 8.04E-12, 9.36E-14, 1.32E-11, 2.24E-06, 2.80E-15, 1.18E-12, 7.60E-12, 1.62E-13, 2.62E-09, 1.60E-11, 3.58E-11, 4.04E-14, 8.64E-07, 2.17E-11, 1.02E-12, 8.47E-11, 1.05E-08, 9.32E-08, 1.43E-08, 9.71E-08, 3.28E-10, 2.01E-14, 7.19E-10, 1.20E-17, 3.78E-12, 4.02E-18, 5.38E-08, 3.97E-11, 3.38E-08, 4.82E-11, 4.00E-11, 2.60E-13, 2.16E-13, 9.53E-11, 6.67E-14, 1.06E-11, 9.22E-14, 1.63E-12, 1.01E-11, 1.85E-17, 1.98E-10, 2.81E-09, 1.89E-10, 5.08E-09, 4.85E-16, 1.42E-08, 1.49E-08, 9.42E-12, 2.83E-13, 8.22E-17, 3.18E-06, 2.81E-11, 2.37E-10, 6.33E-13, 5.37E-11, 2.59E-07, 1.49E-05, 3.45E-07, 3.18E-16, 5.55E-11, 1.88E-08, 4.26E-15, 1.16E-13, 5.56E-07, 3.25E-13, 3.39E-08, 4.62E-11, 2.56E-11, 1.88E-10, 2.82E-11, 6.87E-17, 4.96E-12, 8.68E-13, 2.35E-10, 2.01E-11, 6.16E-14, 2.91E-14, 2.31E-12, 6.82E-09, 6.46E-11, 4.34E-12, 2.64E-14, 8.76E-11, 2.92E-16, 1.69E-11, 5.79E-10, 4.21E-12, 2.00E-09, 5.04E-14, 1.96E-10, 3.67E-11, 8.01E-15, 2.21E-09, 1.53E-10, 1.78E-09, 1.74E-11, 4.68E-12, 6.14E-06, 4.41E-05, 6.03E-10, 5.19E-10, 4.04E-17, 1.77E-08, 1.27E-11, 6.70E-12, 2.10E-08, 5.62E-11, 3.55E-10, 4.38E-06, 2.04E-08, 2.19E-13, 9.57E-18, 1.92E-08, 4.19E-10, 7.40E-12, 6.76E-15, 1.04E-14, 6.06E-06, 1.24E-13, 3.13E-16, 2.00E-12, 5.43E-07, 8.30E-15, 2.87E-06, 1.55E-15, 4.93E-10, 2.37E-14, 4.01E-07, 4.47E-15, 9.27E-11, 1.82E-06, 3.27E-12, 1.31E-12, 7.58E-11, 4.56E-11, 1.29E-10, 3.02E-09, 3.38E-12, 3.25E-08, 1.05E-13, 3.13E-17, 4.00E-09, 3.46E-11, 1.14E-11, 2.95E-08, 4.28E-12, 5.43E-09, 7.24E-10, 1.83E-11, 1.74E-10, 1.67E-11, 3.90E-12, 1.57E-15, 5.34E-05, 1.79E-13, 1.17E-11, 1.57E-11, 2.50E-13, 2.04E-13, 8.64E-06, 8.86E-11, 1.54E-11, 9.88E-10, 1.84E-11, 1.88E-12, 4.34E-08, 2.86E-09, 2.71E-17, 4.30E-15, 8.18E-14, 8.15E-10, 2.65E-15, 3.91E-12, 6.54E-16, 3.33E-12, 7.13E-09, 1.46E-08, 8.58E-05, 9.33E-11, 4.17E-08, 7.69E-11, 3.00E-13, 3.71E-12, 9.57E-09, 6.79E-09, 3.21E-11, 1.35E-14, 2.78E-12, 1.76E-15, 1.96E-09, 2.64E-11, 1.50E-06, 2.42E-13, 7.32E-11, 1.10E-07, 3.16E-11, 7.49E-14, 2.77E-08, 5.22E-09, 1.30E-14, 2.90E-14, 8.03E-16, 5.06E-14, 4.82E-11, 2.54E-10, 3.15E-11, 2.87E-05, 1.43E-17, 1.15E-10, 1.64E-15, 1.01E-12, 1.80E-14, 2.86E-09, 7.06E-09, 1.11E-11, 4.49E-14, 2.77E-16, 3.83E-10, 2.79E-06, 6.56E-13, 9.11E-12, 3.47E-08, 9.28E-12, 1.09E-10, 7.56E-11, 1.41E-16, 4.02E-08, 4.46E-10, 1.63E-10, 7.78E-13, 6.37E-13, 1.01E-12, 1.84E-08, 4.94E-14, 4.80E-12, 5.02E-09, 4.26E-13, 3.48E-12, 1.84E-05, 1.16E-08, 8.79E-15, 1.70E-09, 4.19E-10, 3.87E-09, 4.21E-12, 2.14E-08, 7.27E-11, 2.39E-17, 4.83E-09, 5.56E-10, 1.96E-11, 1.70E-15, 1.22E-08, 9.21E-16, 2.21E-13, 1.31E-10, 7.76E-08, 3.56E-09, 1.15E-09, 1.68E-08, 1.41E-09, 1.44E-07, 1.53E-06, 1.11E-10, 7.85E-06, 1.01E-14, 2.75E-10, 6.02E-12, 6.23E-10, 1.50E-10, 2.94E-15, 3.35E-12, 1.41E-06, 4.51E-08, 8.45E-19, 9.79E-11, 6.99E-09, 3.06E-10, 5.04E-18, 6.22E-14, 2.18E-10, 1.48E-14, 6.29E-12, 7.83E-13, 1.57E-11, 9.13E-14, 1.70E-14, 6.05E-10, 1.12E-11, 7.19E-10, 1.51E-10, 1.97E-15, 2.33E-07, 2.13E-08, 3.69E-10, 6.24E-16, 9.17E-15, 1.54E-11, 4.87E-08, 7.55E-13, 1.11E-11, 6.58E-15, 1.95E-11, 1.15E-09, 2.18E-08, 2.86E-14, 4.55E-08, 4.74E-10, 8.38E-14, 2.73E-16, 2.91E-12, 9.83E-09, 6.88E-14, 8.34E-08, 1.52E-10, 1.08E-13, 5.30E-10, 2.99E-06, 5.92E-13, 1.10E-10, 1.82E-17, 1.66E-12, 1.03E-10, 8.50E-14, 1.08E-11, 4.14E-13, 1.59E-06, 7.31E-13, 1.26E-11, 4.45E-12, 3.18E-13, 6.87E-16, 4.25E-12, 7.07E-11, 1.15E-12, 2.65E-06, 4.25E-07, 3.52E-14, 1.14E-09, 6.45E-17, 2.84E-13, 1.83E-09, 2.74E-09, 4.07E-17, 4.28E-08, 9.33E-11, 1.01E-15, 3.99E-12, 6.10E-08, 5.63E-12, 2.22E-09, 2.76E-05, 6.48E-11, 1.31E-07, 7.70E-10, 5.97E-14, 9.53E-09, 1.03E-05, 3.31E-12, 1.41E-15, 3.54E-12, 5.61E-10, 1.39E-15, 4.44E-11, 4.93E-15, 7.08E-12, 1.69E-08, 1.64E-10, 1.02E-08, 3.65E-11, 4.41E-12, 2.71E-07, 1.88E-06, 1.04E-10, 4.83E-08, 2.91E-09, 1.10E-09, 1.51E-15, 2.86E-09, 2.68E-08, 2.94E-09, 1.96E-11, 5.83E-12, 6.11E-15, 3.02E-13, 8.63E-10, 8.35E-16, 3.28E-10, 5.16E-11, 3.89E-09, 1.08E-13, 3.88E-14, 4.09E-08, 4.23E-08, 2.43E-13, 1.83E-10, 1.37E-12, 3.12E-10, 9.16E-14, 2.93E-15, 3.06E-12, 1.22E-14, 7.30E-13, 1.38E-09, 1.36E-11, 2.78E-10, 7.10E-13, 2.60E-10, 2.43E-07, 2.08E-05, 1.13E-10, 1.04E-09, 1.06E-14, 8.29E-11, 3.00E-14, 4.71E-08, 8.34E-07, 2.48E-11, 3.47E-12, 5.13E-09, 9.76E-16, 2.19E-13, 1.33E-11, 9.32E-11, 6.36E-14, 7.25E-11, 1.36E-05, 2.18E-15, 7.90E-11, 9.41E-15, 5.95E-14, 2.50E-13, 3.47E-17, 1.42E-14, 1.85E-17, 2.44E-10, 5.97E-18, 9.87E-12, 3.05E-12, 1.38E-09, 1.30E-13, 3.17E-14, 1.99E-15, 4.34E-07, 1.04E-05, 1.88E-15, 1.34E-13, 8.23E-08, 5.02E-12, 1.90E-09, 3.24E-12, 8.89E-11, 0.000142133, 3.00E-07, 3.60E-14, 5.95E-07, 4.59E-12, 2.48E-09, 3.98E-11, 5.59E-13, 4.13E-14, 1.77E-11, 4.88E-11, 3.83E-12, 1.11E-09, 3.21E-10, 1.68E-11, 7.09E-07, 1.12E-08, 1.88E-08, 8.16E-14, 2.87E-14, 5.17E-09, 5.11E-13, 1.43E-12, 4.19E-09, 4.03E-17, 6.34E-12, 2.63E-09, 1.55E-13, 4.85E-12, 4.49E-06, 7.34E-11, 2.82E-14, 1.82E-12, 1.93E-16, 3.10E-08, 1.64E-08, 1.32E-11, 6.31E-11, 6.48E-14, 2.55E-06, 1.60E-08, 1.58E-08, 8.22E-15, 9.19E-10, 3.12E-08, 1.77E-06, 5.43E-08, 8.03E-14, 3.05E-10, 1.71E-08, 3.57E-15, 1.11E-05, 1.18E-09, 4.99E-15, 6.74E-12, 3.83E-09, 7.73E-09, 2.22E-12, 2.60E-10, 2.83E-12, 6.35E-08, 1.56E-12, 1.10E-16, 1.87E-12, 1.45E-05, 3.44E-13, 8.25E-07, 1.91E-16, 1.79E-13, 2.33E-09, 2.55E-14, 4.51E-17, 8.90E-14, 8.76E-11, 4.16E-15, 2.58E-11, 2.37E-10, 5.16E-15, 3.03E-09, 1.34E-13, 9.61E-12, 9.16E-11, 2.82E-13, 1.28E-05, 5.30E-08, 5.67E-10, 5.09E-15, 1.51E-08, 2.89E-08, 6.98E-10, 6.88E-18, 3.43E-08, 6.38E-11, 7.24E-08, 1.74E-12, 2.76E-08, 1.88E-14, 2.47E-15, 1.21E-06, 8.16E-11, 1.28E-13, 6.72E-10, 9.04E-07, 9.98E-09, 1.57E-19, 7.33E-11, 8.03E-10, 1.50E-08, 4.12E-11, 8.33E-16, 5.56E-15, 2.94E-13, 1.70E-09, 4.45E-15, 2.35E-13, 4.66E-16, 1.06E-12, 6.40E-13, 8.26E-16, 2.27E-14, 8.47E-09, 6.10E-11, 1.12E-13, 3.94E-10, 1.92E-09, 4.25E-08, 8.84E-07, 4.39E-12, 4.19E-13, 5.32E-14, 4.02E-15, 1.51E-06, 1.19E-12, 4.72E-12, 7.05E-12, 2.06E-09, 3.04E-13, 3.42E-12, 2.12E-16, 9.74E-10, 8.71E-06, 4.55E-12, 2.69E-12)
While p-values range from 6.08038E-23 to 0.001134145, the bootstrapped p-value I get is 0.4995005 and I don't understand why. I am using the following function to find the bootstrapped p-value:
(1+sum(logit.boot$t[,2] > logit.boot$t0[2]))/(1+logit.boot$R)
where logit.boot$t[,2] takes on values from the p.values vector, logit.boot$t0[2] equals 2.664684e-11 and logit.boot$R = 1000.
EDIT
Here is the code I used for bootstrapping:
logit.bootstrap <- function(data, indices){
d <- data[indices, ]
Mf1 <- glm(Y ~ A + B + C, data = d, family = "binomial")
data.setM1 <- na.omit(d[, all.vars(formula(Mf1))])
M1.io <- glm(Y ~ A + B, data = data.setM1, family = "binomial")
my.test <- lrtest(Mf1, M1.io)
return(c(my.test$"Chisq"[2], my.test$"Pr(>Chisq)"[2]))
}
logit.boot <- boot(data=my.data, statistic=logit.bootstrap, R=1000) # 10'000 samples
In the result of the boot function, t0 should the p value on the original data, and t is some p values which are generated from random resampling/permutation on the original data. And in your case, you shouldn't use (1+sum(logit.boot$t[,2] > logit.boot$t0[2]))/(1+logit.boot$R) to get information from your bootstrapped p values, you may use quantile(logit.boot$t[,2], c(0.025,0.975)) or something like this to obtain a bootstrapped 95% confidence interval on your p value. This is not very meaningful, since the meaning of p value is already a probability (confidence level), why do you bother to obtain a confidence interval for p value? And the validness of the bootstrap method relies on the correctness of your parametric model. So if you want to use non-parametric approach toward this problem, I think you need to find some other approaches instead of this one.
How to fit a loess curve over this decomposed time series data in R?
We have time series data with some seasonality from the past 4 years. We want to predict the general rise in trend next year. For this, we decomposed the time series and observed the trend line:
However, this trend line is placed in the middle of the values rather than the values themselves. We are not satisfied with simply extrapolating this trend line since it falls very short of expected traffic.
Since we are interested in only the general rise in trend and not the seasonality, we remove the seasonality from the components:
mydata <- read.csv("values.csv")
mydataseries <- ts(mydata, start=c(2012,1,1),frequency = 365.25
mydataseriescomponents <- decompose(mydataseries)
mydataseriesminusseasons <- mydataseries - mydataseriescomponents$seasonal
We are now trying to fit a loess() curve in R around this time series data minus the seasonal component, that is mydataseriesminusseasons:
855154881.9
1027395443
1132284155
944870172.3
898459083.9
845115286.7
204393180.2
-75788428.32
-184120868.7
-164634776
-190543808.7
-43973009.39
-452418843.2
-1065106918
-1194545584
-1250333168
-379435027.4
-151057609.1
134304962.6
37020062.65
-307740042.8
-309480234.5
-388529539.7
-379333445.3
-193124460
-663765015.3
-100597898.3
-327949890.1
-429500583.5
-1321506072
-1444202356
-369913100.7
-715237274.1
-83507361.25
-328296509.9
-409957935.1
-1351211680
-533631519.6
-882845870.8
-711202595.9
-147504975.3
-361087020.5
-392960561
-456709751.2
-1065209981
-2062128692
-1112973164
-914751976.5
-1377070656
-1903432323
-2110392180
-1984477186
-1434481986
-988902503.8
-944477878.3
-1388889111
-1559730054
-1668752977
-1590459585
-1528272998
-850923022
-542720361.8
-791860111.9
-1057854181
-852120155.6
-810625768.1
-56185632.59
576968488.7
691371629.9
160486150.2
-308533580.1
-261619133.2
-404620378.3
740114544.5
1006206606
622867052.1
-71219884.82
-39173777.42
-533631907.8
-709943766.3
-565909169.9
-968499911.2
-1056539190
-1266350530
-1445031918
-1579156208
-1550356091
-1316401212
-899907725.4
-681646243
-996858524.4
-1261573594
-1620842348
-1529532466
-1107218074
287538766
303061371.4
-382291371.7
-639757694.3
-737369705.5
-732118809.2
-393825621.1
-810602326.3
-1044759123
-1410535140
-1636325717
-1646415420
-1514426422
-1241825553
31448256.14
64762654.11
-513468522.6
-676115681.9
-191498426
-86743267.18
80585792.24
538395944.8
262807408.8
-156283609.7
-72157485.31
-56386961.34
-14257457.85
233362524.8
495465812.9
354441383.3
74975465.29
28985451.46
187381822.2
517857439.4
940857149.4
1397097814
1965769992
1735207534
1568036010
1496104390
1647402384
928936108
1372629124
1417356139
1168474219
1080545493
996524274.5
988026158.1
1060004940
1315522041
1444552768
1438503151
1174067300
1063743311
1086528293
1310565938
1542624917
1504331433
1280216655
1726223821
1670308928
1120574360
1084897870
1285908119
1232985139
1101293494
1081075441
881717495.1
975005126.7
1058436909
1290433280
1288259727
1038856967
1006240656
891910425.2
920601850.4
981585788.8
1536507833
1546018200
1149902811
1230460988
1191817042
1161287112
1273239958
1590385882
1538451838
1242359113
913747727.7
888013837.2
934884821.1
1198395868
1277044767
1235456131
963043028
925500611.4
921906159.7
957528730.9
934517634.5
1183155135
1155142736
893512199.3
829955361
839182104.7
904102148.6
972879836.6
1227504663
1192420215
904137203.4
944756516
1375037708
1328414157
1779424344
1921248556
1899629843
1773839270
1565431502
1592386775
1825650269
1404794363
1393784791
1416771770
1288386476
1289239680
815093905.4
681990799.1
783953618.4
1103067833
1182429773
933897907.8
874114515.6
787403408.5
860338795
1052657878
1297180989
1245476158
1014316130
1004606484
735958898.4
575505161.4
588750452.3
803692654.7
833093724.3
531996754.3
199110366.8
-18901281.1
-6413198.146
-33290860.64
252073940.6
203247618.9
-164091120.7
-501720846.4
-865543851.3
-856086326
-612355346.3
400846624.1
511862726
288735892.8
-127533264.3
-940372573.2
-155917893.1
-699989731.6
-320401187.5
-277824927.9
-705440125.7
-1042182377
-1206106831
-1339738778
-165282703.3
222304361.4
136773204.1
-224139914.3
-271723348.1
-288223577.8
-285482682.4
-1057479814
-1059799528
-960773269.6
-1378106942
-1682462197
-1880270405
-1877392694
-1621410466
-1218560480
387109224.3
85424610.03
-558048286.6
-432347510.8
-107370046
224698124.4
732906395.5
-523071744.4
-1239504492
-1798916179
-2065361083
-1987860160
-1725292219
-1109123667
-971275176.7
-1420498329
-1889158526
-1136864728
-1002777887
647298007.9
1277421457
248465769.8
-377527211.2
-724681515.2
-967478152.4
-947666627.1
-734506065.3
-408053802.2
-241421840.8
-363715586.3
-829237162.5
-929529173.2
-260432788.4
-67756618.13
242997207.2
567370574.9
128921994
-159670249.1
-226982681.8
-240623081
-26502753.8
311495394.1
477285225.3
198408156.8
-195753985.4
-271887709.5
-196538766.7
-72464893.44
180860168.9
360960891.9
147945699.2
-29352193.17
325127464.9
960490365.6
1453481987
1424677454
1207446280
537281902.1
-167977127.5
-505612366.5
-1460096451
-1554214240
-1472813276
-1158137635
-1079634747
-1400116741
-1623247577
-1720625509
-1291068926
-541823228.2
423680238
828606261.2
651175802
588260033.4
511022741
644493024.6
696908085.1
949150687.9
1037411745
754948966.3
790033725.6
1053226989
1170333305
1056602964
1081413405
1135864583
971795777.8
855154881.9
1027395443
1132284155
944870172.3
896319497.9
845115286.7
204393180.2
-75788428.32
-184120868.7
-164634776
-190543808.7
-43973009.39
-452418843.2
-1065106918
-1194545584
-1250333168
-379435027.4
-151057609.1
139705076.6
37020062.65
-307740042.8
-309480234.5
-388529539.7
-379333445.3
-193124460
1308430606
-100597898.3
-327949890.1
-429500583.5
-1321506072
-1444202356
-369913100.7
1419976593
180125360.7
-328296509.9
-409957935.1
-1351211680
-533631519.6
1246061777
1639641696
272442819.7
-361087020.5
-392960561
-456709751.2
1014735303
478499133.8
-748455756.5
-914751976.5
-1377070656
-1903432323
-2110392180
-1984477186
-1434481986
-988902503.8
-944477878.3
-1388889111
-1559730054
-1668752977
-1590459585
-1528272998
-850923022
-542720361.8
-791860111.9
-1057854181
-852120155.6
-810625768.1
-56185632.59
576968488.7
681069818.9
160486150.2
-308533580.1
-261619133.2
-404620378.3
740114544.5
1006206606
622867052.1
-71219884.82
-39173777.42
-533631907.8
-709943766.3
-565909169.9
1641890896
1588111061
880069519.6
1090752417
1278890698
1335767912
1519822543
1812984581
1334545973
426228457.6
456951496.9
1228693380
1312355639
1637418032
691968559
303061371.4
-382291371.7
-639757694.3
-737369705.5
-732118809.2
-408383535.1
1630144553
1548715666
747397223.2
884268460.1
1205537489
1226277745
1637844137
477586396.1
64762654.11
-513468522.6
-676115681.9
-191498426
-86743267.18
80585792.24
538395944.8
262807408.8
-156283609.7
-72157485.31
-56386961.34
-14257457.85
233362524.8
495465812.9
354441383.3
74975465.29
28985451.46
187381822.2
517857439.4
940857149.4
1397097814
1479492584
1206235886
1230335054
1209244303
1772074811
1693077942
1461745247
1417356139
1168474219
1080545493
996524274.5
988026158.1
1059643199
1314652245
1440434637
1438503151
1174067300
1063743311
1086528293
1310565938
1542624917
1504331433
1280216655
1726223821
1670308928
1863627651
1882123871
2055914335
1992301669
1579375771
1533925079
1762629244
1838997943
1897919645
2107505072
1997289608
1464337865
1415561153
1676756233
1700040307
1783398157
1624554189
1546018200
1149902811
1230460988
1191817042
1161373475
1273239958
1590385882
1538451838
1242359113
913747727.7
888013837.2
934884821.1
1198395868
1277044767
1235456131
963043028
925500611.4
921906159.7
957528730.9
934517634.5
1183155135
1151401692
893512199.3
829955361
839182104.7
904102148.6
972879836.6
1227504663
1192420215
904137203.4
944756516
1375037708
1328414157
1193178073
1253502729
1251928059
1185271006
1244858293
1231640212
1171269572
1343448619
1393784791
1416771770
1288386476
1289239680
815093905.4
681990799.1
783953618.4
1103067833
1182429773
933897907.8
874114515.6
787403408.5
860338795
1052657878
1297180989
1245476158
1014316130
1004606484
735958898.4
575505161.4
588750452.3
803692654.7
833093724.3
531996754.3
199110366.8
-18901281.1
-6413198.146
-33290860.64
252073940.6
203247618.9
-164091120.7
-501720846.4
-865543851.3
-856086326
-612355346.3
400846624.1
511862726
288735892.8
-127533264.3
-940372573.2
-155917893.1
-699989731.6
-320401187.5
-277824927.9
-705440125.7
-1042182377
-1206106831
-1339738778
-165282703.3
222304361.4
136773204.1
-224139914.3
-271723348.1
-288223577.8
-285482682.4
410687500.7
1631841798
1553281634
713309450.5
649367763.9
960215164.3
1049107976
1148989485
1678375251
1164709303
183895982
465208739.4
766073381.2
586658157
853178118.4
1317822533
1674419406
693138111.6
610057420.2
928861116.6
1021382975
1317392280
1950890283
1753467250
679132043.8
530667904.1
1923321662
2445945599
2224779498
2205640478
2617939193
1771677175
1773936492
2444894838
2401384689
2491268212
2581667225
2449579792
1593738389
1956098689
2378730864
235226517.6
-67756618.13
242997207.2
567370574.9
128921994
-159670249.1
-226982681.8
-240623081
-26502753.8
311495394.1
477285225.3
198408156.8
-195753985.4
-271887709.5
-196538766.7
-72464893.44
180860168.9
360960891.9
147945699.2
-29352193.17
325127464.9
960490365.6
1453481987
1424677454
1207446280
537281902.1
-167977127.5
-505612366.5
1404374892
1613823539
1849271413
1936762445
1513383047
1256267888
1553901607
1655449233
1979063830
2224098910
2147758870
1612601292
1277373178
1489383946
1406268621
1510782787
1555473157
1721503681
1473734831
1214127421
1427159815
1454163770
1559572239
1693448937
1686056516
1534649610
1402815411
1597472633
1479661974
1572474919
1758527502
1912156129
2007806540
2577055685
3079863351
3189154952
3084552356
3020184398
3019956688
2361199448
1906290720
1878596097
1894542086
54727454.63
-151057609.1
134304962.6
37020062.65
-307740042.8
-309480234.5
-388529539.7
-379333445.3
-193124460
-663765015.3
-100597898.3
-327949890.1
-429500583.5
1538626202
1943756793
142960155.3
-715237274.1
-83507361.25
-328296509.9
-409957935.1
1469349471
-6715352.565
-882845870.8
-711202595.9
-147504975.3
-361087020.5
-392960561
-456709751.2
-1065125015
1006714631
2153142496
1804378762
1673270778
2693842453
3029003743
3091803034
2506604564
2457560591
2027821010
1756470132
2064844611
2119623546
2162263280
2002164168
1694641809
616667238.2
178971439.1
187991217
342780020.4
452558045.9
1221341191
1928282804
1898438088
2350801273
3271266186
3102579205
3404316683
1296736764
1006206606
622867052.1
-71219884.82
-39173777.42
777976197.2
831608580.7
1173972977
574093131.8
145884449.2
125630390.6
140647407.2
-1948845.745
-20569346.72
170376510.8
416953573.6
394780276
210086267.6
240846310.9
-4692367.437
13267344.2
462459409.2
1638331860
1368251762
844946221.3
908678722.7
858991700.5
821130809.8
1087001122
501346560.7
49030467.37
-115293555.8
-42916860.95
-94506694.82
65744337.46
214422930.7
1246493179
948746467.1
450984956.4
906168925.1
55072855.99
-86742286.18
80586773.24
538396925.8
262808389.8
-156282628.7
-72156504.31
-56385980.34
-14256476.85
233363505.8
495466793.9
354442365.3
74976446.29
28986433.46
187382803.2
517858420.4
940858130.4
1397098796
1479493565
1206236868
1230336035
1209245284
1021481495
1358007248
1908140672
1690817796
1450430426
1573572004
1548289864
1518408336
1551340078
1800682764
1691240562
1675755235
1422581675
1551622923
1630779709
1361558060
1542625898
1504332415
1280217636
1726224802
1670309909
1120575341
1084898851
1285909100
1226741796
1101294475
1081076422
881718476.1
975006107.7
1058437890
1290434261
1288260708
1038857948
1006241637
891911406.2
920602831.4
981586769.8
1536508814
1546019181
1149903793
1230461969
1191818023
1161288092
1273240939
1590386863
1538452819
1242360094
913748708.7
888014818.2
934885802.1
1198396849
1277045748
1235457112
963044009
925501591.4
921907140.7
957529712.9
934518615.5
1183156116
1151402673
893513180.3
829956342
839183085.7
904103129.6
972880817.6
1227505644
1192421196
904138184.4
944757497
1375038689
1328415139
1193189782
1253504218
1251929547
1185272494
1244859781
1231641700
1171271060
1343450107
1393786279
1416773258
1288387964
1289241168
1276942382
1223342782
1265179173
1582950274
1549362846
1289741511
1522844687
1575762386
1623981295
1140018366
1297182477
1245477646
1014317618
1004607972
3339996446
2617836416
2598501271
2725463672
2379954942
2430902194
3387783330
4057325882
3941464221
4026010913
3593315188
3218647755
2899429341
3156298547
3453939489
3535652872
3957490786
2127115167
1652343230
1644069350
1408460965
3374914231
1354451507
3451778958
4125068975
3062203905
2860803149
3523217492
3446260128
4086491216
1353224655
1786887688
1329952369
1262946794
1434089723
1426765711
1394483892
3184736742
3585370070
2492338381
2870647454
3422458727
3496074936
3440427221
3488297885
3411694273
1862226333
2347786389
2984492965
2832984919
2706245540
2975365013
2750770215
2329108891
2668972592
3485974677
3498308730
3651172133
4020199587
3953317903
2733969884
2903235323
3953139670
4668898648
4652525612
4073702669
3715845753
3017343417
4181519935
4751974434
4619926728
4594101972
4732221475
4318140179
3780497707
4451918052
5012718317
4833179055
5653878208
5807608123
5396558840
4198373572
4884875793
5677377778
5912605893
5944449107
5747882280
5227572313
4478777171
4801671568
5862958412
6148995450
6002089979
5948770595
5519491673
4863435104
5079255850
5868770156
5542293262
4248733096
2766041977
2732273377
2763007960
3709017196
5826881003
5418281457
4464795375
4353979464
4041806357
3642750843
3991233296
4574288639
4505497973
4507525662
3759850718
2770466566
1884370202
2017997394
2536894291
2394368239
2561217637
2329522814
2238153751
1823017685
1985828850
2534460960
2292248472
2008277139
1792241233
1777656572
1763197595
1863541282
2165268053
2093475484
2045555081
1853852293
1858329332
1755509088
1713779365
1790957115
1574452597
1579659809
1670558649
1766876466
1623856630
2694719637
3768836252
3933201414
3980190714
4955559887
4938681336
4365493305
4568797166
5264823884
5274383182
5433866314
5417386015
5045165577
4013353224
4858457348
5313474135
5517720717
4442688396
4160415512
4888461743
3958104821
4567324340
5323056776
5488909281
4557331369
5223549360
4326783309
3764876526
4571173289
5423886480
5493146693
5626783987
4769215133
3238864341
3322242772
3840761593
4440231932
3952228746
3826102990
3642814617
3683433336
3290239564
3679644227
4401886050
4271007289
4331924299
4217934600
4322830334
3958867188
4735975510
5434096780
5704487461
5359613448
5223744170
3712806728
2381705672
2316848053
2396344551
1960261326
2098710139
1955760687
2933140424
2967171559
3746036475
5150516737
5103739176
4799974487
4940976107
4470787625
2874748403
3370617116
4116000776
3907376783
3879136871
3858305272
3498190930
2968939532
3482738547
4592541370
4548237451
4039300654
3954893868
3196099484
2977703338
3645589701
4866174649
5072864386
5232041565
5275606802
4693688082
3245094491
3790221336
4772983985
4581973930
4331144672
4167962637
3622317542
3760217425
4483667233
5577532419
5303025177
5677698961
5720893489
5385575901
4466914817
5021515141
5865735715
5703914224
5678717816
5600943246
5110447244
4590128626
4877035122
5442054864
5540200590
5229513831
4574668437
3734155348
2825747731
2665659550
3218611753
3177069031
3225833078
2857378098
2605242926
2290489544
2556941031
3052363819
3023655743
3139325913
3183875762
3084641745
2584707861
2573111682
2596449197
3119292036
3103807890
3003440326
3050225234
2639037382
2719271674
3174072606
2288642761
2407051251
2769659370
2792708736
2422258872
2550055714
3094896689
3167462954
3145311598
2982505987
2846929030
2410677820
2528516371
3317404595
3405548963
3264859256
3217887704
3076794348
2682319143
2752574869
3546644862
3387139453
3465822066
3527697761
3304648740
2671176465
2775904656
3369489444
3186975051
3265608930
3125970955
2336383605
2101672532
2227649209
3046717349
3161392531
3174174168
3069526988
3140929673
2396972936
2491933483
3274873818
3471396399
3449881421
3260944652
3059846681
2300326480
2410144424
3280639160
3165080129
1880734112
2026731745
1851573264
1593311358
1623039610
1888090770
1983714225
1990135314
1884610261
1966858510
1881518298
1817889199
2209334095
2213195958
3180360773
3506589476
3266458246
2314386847
2203488644
2979775043
2887785856
3029641480
2854946175
2971200632
2338656676
2508827357
3221768984
3271647981
1494100499
2557318665
2566743784
2024262236
2326858931
2898711691
2629767554
2418419575
2529673112
2517326919
2394071701
2882872031
3958018274
4398615549
4850186215
4769908627
3875047506
3686152334
3944845885
4407300286
5483421647
5150638137
5610203814
4608824221
3183456836
4173041229
5250468129
5281994707
5707036828
5352424541
5755606494
4561838923
5206514812
6017273753
5965257129
6031374315
6079040231
4379684817
2770833772
3855818791
4754914912
4592227424
4426284753
4410453960
4025381538
3174239722
3637497533
4441638488
4174099821
3906145103
3894032411
3032325960
2296152859
3634154800
5013642209
4862468183
4819578542
4516969809
3607317085
2438327317
3732908830
5107431888
4696140901
1856348267
1237864483
406427050.9
166776835.3
1498808575
1828048545
1624152578
1349901570
1420774156
998226407.7
1012022910
1532830520
1859891754
1422784584
1300680502
1969725485
1936125754
1730921235
2284540775
2480696530
2271292433
2177291835
2174315253
1943550394
1789789173
2208387982
2445973421
2180115879
2053590351
2057128187
1868753587
1794479025
2093710971
2310482091
1882157273
1506021669
1535469156
2038211764
2132884419
2539631965
2937476532
2237006540
1897259543
801631140.7
589473184.2
116334380.3
674275361.3
1282712559
1119202007
1004676722
781599593.7
1184057471
1405174402
2414122535
3215047803
3117104932
2843730513
2338714670
1830982498
1282357057
1398445510
1709185553
1343346425
1299455992
1382754660
1451688601
1355952002
1395566041
1646971917
1518132724
1378268909
1383505613
1477733701
1396182586
1340692285
1513095775
952400088.2
654084246.7
626929432.3
700786606
523798987.3
1212948425
1567710490
1072360130
1057312922
876650104.6
1532682661
1439342205
1801524855
2099113224
1992935512
2063753470
1849349346
1589462235
1341435771
1147947080
2071354378
2006092710
2105408195
1234206054
656144711.4
1208902786
1232216105
2213555415
2017146411
1991371378
994748835.5
1523018258
551904050.2
4489317425
7011882149
7509029269
7405531928
6627086822
5118361296
2929515389
4386298474
6193577031
6201888044
5600929516
5249637157
4071570693
3812427933
4814958529
6581889904
6706593436
6538328968
5770674289
5146640592
3869631764
5168489883
7218970135
7319257013
6818980598
6581026441
4463128268
2549197744
2751600290
3586655201
3073231312
2610229695
2664878334
2445857027
3333286739
5342397336
8218514365
7400486992
7351969323
6170498161
6166815740
4938259613
5123232548
6952344511
7484834076
7975757356
6042108897
3954139391
2561409558
3636287452
6659637045
6610145475
6303751643
5790815423
5286756045
532871596.2
2403662106
2933984545
2177296758
1954486433
1713519574
1459802901
4020142932
5383799348
6858179971
7112822894
6327294007
5759923625
5042533123
3796835736
5750880779
7453820307
7335397513
6962686301
7790660114
6724956363
5003169135
6701050938
7992041249
7756326177
7913301179
7741411043
6785177818
6284270117
6963921883
7785974652
7574222100
7588499293
5788919572
4105788142
2859825648
2924210471
4069907537
3895077405
3979654780
2075329893
1979403277
1632081713
1877890303
2412917344
3763581680
3726439732
3673898152
3345262416
2191053212
2548906553
3841133834
3846853011
3492242728
3479185981
3290714847
2514060721
2645599322
2791855040
3791586130
However, we are not familiar with the form of the loess() function and it seems to expect a formula which we do not know. We tried the following:
y <- c(mydata)
x <- 1:1614
lo <- loess(y~x)
We get the error: Error in model.frame.default(formula = y ~ x) :
invalid type (list) for variable 'y'
How do we fit a loess curve around the values to get the general trend line? Also, after removing the seasonal component we have some negative values for some reason. Would this affect the loess() curve?