Getting unknown error in SudokuSolver Backtracking problem - recursion

I was trying a Backtracking problem (SudokuSolver) I am getting an error which no-one could resolve or even understand. Hence I am seeking help here.
The problem is as follows:
And the main function/part of my code is:
Here my removerow, removecol and removebox functions are removing the numbers which have already occurred in that row, column and sub-box of 3x3 respectively.
The error I am getting is very long that is:
Can someone please help me with this? I'm stuck at it since 2 days. Ready to provide further clarification if needed.
Please find the entire code here: link

I have created a test class, to try understand what your functions are doing - or rather, to make sure they do what I think they do...
import unittest
from Otter.Help_Parth.entire_code import tellbox
from Otter.Help_Parth.entire_code import removerow
from Otter.Help_Parth.entire_code import removecol
from Otter.Help_Parth.entire_code import removebox
class MyTestCase(unittest.TestCase):
def test_tellbox(self):
""" Put each of the 81 x,y combination to tellbox
and retrieve the 81 corresponding box-numbers
assert that the numbers are as expected. """
lst_results = list()
for i in range(9):
for j in range(9):
lst_results.append(tellbox(i, j))
self.assertEqual(lst_results, [0, 0, 0, 3, 3, 3, 6, 6, 6, 0, 0, 0, 3, 3, 3, 6, 6, 6, 0, 0, 0, 3, 3, 3, 6, 6, 6, 1, 1, 1, 4, 4, 4, 7, 7, 7, 1, 1, 1, 4, 4, 4, 7, 7, 7, 1, 1, 1, 4, 4, 4, 7, 7, 7, 2, 2, 2, 5, 5, 5, 8, 8, 8, 2, 2, 2, 5, 5, 5, 8, 8, 8, 2, 2, 2, 5, 5, 5, 8, 8, 8])
def test_removerow(self):
""" Given a partly filled board, a list and an x,y coordinate
try call removerow() and assert that the returned list are as expected. """
board = [[1, 2, 3, 4, 5, 6, 7, 8, 9],
[4, 5, 6, 7, 8, 9, 1, 2, 3],
[7, 8, 9, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0]]
l = [1, 2, 3, 4, 5, 6, 7, 8, 9]
x, y = 2, 4
res = removerow(l, board, x, y)
self.assertEqual([2, 3, 4, 5, 6], res)
def test_removecol(self):
""" Given a partly filled board, a list and an x,y coordinate
try call removecol() and assert that the returned list are as expected. """
board = [[1, 2, 3, 4, 5, 6, 7, 8, 9],
[4, 5, 6, 7, 8, 9, 1, 2, 3],
[7, 8, 9, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0]]
l = [1, 2, 3, 4, 5, 6, 7, 8, 9]
x, y = 2, 4
res = removecol(l, board, x, y)
self.assertEqual([1, 2, 3, 4, 6, 7, 9], res)
def test_removebox(self):
""" Given a partly filled board, a list and an x,y coordinate
try call removebox() and assert that the returned list are as expected. """
board = [[1, 2, 3, 4, 5, 6, 7, 8, 9],
[4, 5, 6, 7, 8, 9, 1, 2, 3],
[7, 8, 9, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0]]
l = [1, 2, 3, 4, 5, 6, 7, 8, 9]
x, y = 2, 4
bno = tellbox(x, y)
res = removebox(l, board, x, y, bno)
self.assertEqual([2, 3], res)
if __name__ == '__main__':
unittest.main()
It all goes quite well, until removebox().
If you can read and understand my test_removebox(self) Are you kind to confirm that, under the given circumstances, it should in fact return [2, 3]?
Otherwise, please tell what it should return, and why?
When I run the test class, it returns the following:
NB: To be able to run the test class against your code, you need to change it like this:
if __name__ == '__main__':
board = [[ int(ele) for ele in input().split() ] for i in range(9)]
ans=solveSudoku(board,0,0)
if ans:
print('true')
else:
print('false')

Both your code-sample and the output seems incomplete.
We don't know your variables, so I assume 'board' is a list of 81 integers, but what is the mysterious (and newer used) boxl?
And why would you set board[x][y] = 0 (in the line with the comment), when you have just gone through the trouble of setting it = ele?
We would be able to help you better, if you provided the entire code, in a run-able form.

Sorry! I just realized that I answered to the wrong post!

Related

Using R to create forest plots of coefficients for regressions on subsamples

I have a dataset of chess positions made up of 100 groups, with each group taking one of 50 positions ("Position_number") and one of two colours ("stm_white"). I want to run a linear regression for each Position_number subsample, where stm_white is the explanatory variable and stm_perform is the outcome variable. Then, I want to display the coefficient of stm_white and the associated confidence interval for each regression in a forest plot. The idea is to be able to easily see which Position_number subsample gives significant coefficients for stm_white, and to compare coefficients across positions. For example, the plot would have 50 y-axis categories labelled with each position number, the x-axis would represent the coefficient range, and the plot would display a horizontal confidence bar for each position number.
Where I'm stuck:
Getting the confidence interval bounds for each regression
Plotting each of the 50 coefficients (with confidence intervals) on one plot. (I think this is called a forest plot?)
This is how I current get a list of the coefficients for each regression:
fits <- by(df, df[,"Position_number"],
function(x) lm(stm_perform ~ stm_white, data = x))
# Combine coefficients from each model
do.call("rbind", lapply(fits, coef))
And here is a sample of 10 positions (apologies if there's a better way to show reproducible data):
>dput(droplevels(dfMWE[,c("Position_number","stm_white","stm_perform")]))
structure(list(Position_number = c(0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6,
6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9,
9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
10, 10, 10, 10, 10, 10), stm_white = c(0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1), stm_perform = c(0.224847134350316, -0.252000458803946,
0.263005239459311, -0.337712202569111, 0.525880930891169, -0.5,
0.514387184165999, 0.520136722035817, -0.471249436107731, -0.557311633762293,
-0.382774969095054, -0.256365477992672, -0.592466230584332, 0.420100239642119,
0.35728693116738, -0.239203909010858, 0.492804918290949, -0.377349804212738,
0.498560888290847, 0.650604627933873, 0.244481117928803, 0.225852022298169,
0.448376452689039, 0.305090287270497, 0.275461757157464, 0.0232950364735793,
-0.117225030904946, 0.103523492101814, 0.098301745397805, 0.435599509759579,
-0.323024628921732, -0.790798102797238, 0.326223812111678, -0.331305043692668,
0.300230596737942, -0.340292005855252, 0.196181480575316, -0.0606495585093978,
0.789844179758131, -0.0862623926308338, -0.560150145231903, 0.697345078589853,
-0.425719796345476, 0.65321716721887, -0.878090073942596, 0.393712176214572,
0.636076899687882, 0.530184680003902, -0.567228844342952, 0.767024918145021,
-0.207303615824231, -0.332581578126777, -0.511510891217792, 0.227871326531416,
-0.0140876421179904, -0.891010911045765, -0.617225030904946,
-0.335142021445235, -0.517262524432376, 0.676301669492737, 0.375998241382333,
-0.0882899718631629, -0.154706189382, -0.108431333126633, 0.204584592662721,
0.475554538879339, 0.0840205872617279, -0.403370826694226, -0.74253555894307,
0.182570385474772, -0.484175014735265, -0.332581578126777, -0.427127748605496,
0.474119069108831, -0.0668284645696687, -0.0262098994728823,
-0.255269593134965, -0.313699742316688, -0.485612815834001, 0.302654921410147,
-0.425719796345476, 0.65321716721887, 0.393712176214572, 0.60766106412682,
0.530184680003902, 0.384135895746244, 0.564400490240421, 0.767024918145021,
0.702182602090521, 0.518699777929559, -0.281243170101218, -0.283576305897061,
0.349395372066127, -0.596629173305774, 0.0849108889395813, -0.264122555898524,
0.593855385236178, -0.418698521631085, 0.269754586702576, -0.719919005947152,
0.510072446927438, -0.0728722513945044, -0.0849108889395813,
0.0650557537775339, 0.063669188530584, -0.527315973006493, -0.716423694102939,
-0.518699777929559, 0.349395372066127, -0.518699777929559, 0.420100239642119,
-0.361262250888275, 0.431358608116332, 0.104596852632671, 0.198558626418023,
0.753386077785615, 0.418698521631085, -0.492804918290949, -0.636076899687882,
-0.294218640287997, 0.617225030904946, -0.333860575416878, -0.544494573083008,
-0.738109032540419, -0.192575818328721, -0.442688366237707, 0.455505426916992,
0.13344335621046, 0.116471711943561, 0.836830966002895, -0.125024693001636,
0.400603203290743, -0.363923100312118, -0.157741327529574, -0.281243170101218,
-0.326223812111678, -0.548774335859742, 0.104058949158278, -0.618584122089031,
-0.148779202375097, -0.543066492022212, -0.790798102797238, -0.541637702714763,
0.166337530816562, -0.431358608116332, -0.471249436107731, -0.531618297828107,
-0.135452994588696, 0.444109038883147, -0.309993792719686, 0.472684026993507,
-0.672509643334985, -0.455505426916992, -0.0304828450187082,
-0.668694956307332, 0.213036720610531, -0.370611452782498, -0.100361684849949,
-0.167940159469667, -0.256580594295053, 0.41031649686005, 0.544494573083008,
-0.675040201040299, 0.683816314193659, 0.397841906825283, 0.384135895746244,
0.634743335052317, 0.518699777929559, -0.598013765769344, -0.524445461120661,
-0.613136820153143, 0.12949974225673, -0.337712202569111, -0.189904841395243,
0.588289971863163, 0.434184796930767, -0.703385003471829, 0.505756208411145,
0.445530625978324, -0.167137309739621, 0.437015271896404, -0.550199353253537,
-0.489927553072562, -0.791748837508184, 0.434184796930767, 0.264122555898524,
-0.282408276808469, -0.574280203654524, 0.167940159469667, -0.439849854768097,
-0.604912902007957, 0.420100239642119, 0.35728693116738, 0.239220254140668,
-0.276612130560829, -0.25746444105693, 0.593855385236178, -0.632070012100074,
0.314483587504712, 0.650604627933873, -0.226860086923233, -0.702182602090521,
0.25746444105693, -0.174474012638818, 0.0166045907672774, 0.535915926945102,
0.141635395826102, 0.420100239642119, 0.557311633762293, 0.593855385236178,
0.6961287704296, 0.0444945730830079, -0.234005329233511, 0.448376452689039,
-0.86655664378954, 0.22107824319756, 0.148051654147426, 0.543066492022212,
-0.448376452689039, 0.373300918333268)), row.names = c(NA, -220L
), groups = structure(list(Position_number = c(0, 1, 2, 3, 4,
5, 6, 7, 8, 9, 10), .rows = structure(list(1:20, 21:40, 41:60,
61:80, 81:100, 101:120, 121:140, 141:160, 161:180, 181:200,
201:220), ptype = integer(0), class = c("vctrs_list_of",
"vctrs_vctr", "list"))), row.names = c(NA, 11L), class = c("tbl_df",
"tbl", "data.frame"), .drop = TRUE), class = c("grouped_df",
"tbl_df", "tbl", "data.frame"))
confint() can get you the confidence interval of a model.
forestplot() from the forestplot R package can make you a forest plot.
library(dplyr)
library(forestplot)
results <- lapply(unique(df$Position_number), function(pos) {
fit = filter(df, Position_number == pos) %>%
lm(data = ., stm_perform ~ stm_white)
stm_white_lm_index = 2 # the second term in lm() output is "stm_white"
coefficient = coef(fit)[stm_white_lm_index]
lb = confint(fit)[stm_white_lm_index,1] # lower bound confidence
ub = confint(fit)[stm_white_lm_index,2] # upper bound confidence
output = data.frame(Position_number = pos, coefficient, lb, ub)
return(output)
}) %>% bind_rows() # bind_rows() combines output from each model in the list
with(results, forestplot(Position_number, coefficient, lb, ub))
The forest plot shows the "Position_number" labels on the left and the regression coefficients of "stm_white" with the 95% confidence intervals plotted. You can further customize the plot. See forestplot::forestplot() or this introduction by Max Gordon for details.

Error message when reorganizing duplicated columns in a dataframe with specifics names

Ive been struggling to solve this.
This is my dataframe:
df<-structure(list(CESP6.MRVE3 = c(3.39731078, 3.351139056, 3.541515582,
3.329084161, 3.117085569, 2.630336741, 3.263587913, 3.2854739,
2.115273321, 1.255217018, 0.558756742, -0.478780166, -0.667471284,
-0.814286812, -1.489226751, -1.957340764, -2.397829895, -2.106376737,
-1.799451962, -0.699075436, -0.686257093, -0.397822857, -0.079821449,
-0.532685698, -0.502453448, 0.124038498, -0.183209859, -0.207727845,
-0.421813003, -1.550648396, -0.940915675, -0.282337163, -0.282104914,
0.051723363, -0.095957822, -0.356446953, -0.586070428, -0.003895889,
-0.706760138, -0.499624387, -0.265363282, 0.33515752, -0.056408244,
-1.048118284, -1.251337786, -1.19362493, -1.519219549, -1.663016155,
-1.036524208, -1.006369535, -1.066137285, -0.991731905, -1.463586221,
-1.419036564, -1.121612261, -1.571668563, -1.494677088, -1.21322393,
-0.758818549, -0.418730576, -0.080007787, 0.240368738, 0.050667688,
-0.157657302, -0.327569328, -0.305106237, -0.070845132, 0.339675669,
0.435446235, 0.323159091, 0.305477906, 0.500815644, 0.697451866,
0.504088089, 0.185252695, 0.149946628, 0.098092312, -0.119733149,
-0.03216457, -0.430422859, 0.056069088, 0.079253559, 0.651283822,
0.302448429, 0.282680678, 0.137663163, 0.037606861, 0.175752544,
0.619292269, 0.252188188, 0.629112963, 0.504017872, 0.236836216,
-0.235883757, -0.3413341, -0.004198349, 0.018409018, 0.710295005,
0.514988938, 0.4701157, 0.528261383, 0.569070737, -0.062206474,
-0.597656818, -0.783107161, -0.670433093, -0.638114278, -0.705795463,
-0.793188095, -0.502023489, -0.202656896, -0.335732122, -0.624201387,
-0.282459676, -0.598342848, -0.705957332, -0.547667373, -0.703550544,
-0.958645635, -0.74899049, -0.367393055, 0.188666063, 0.852927168,
1.13423605, 1.400150892, 1.028440852, 0.96449997, 1.732501378,
2.084243089, 1.743609682, 1.587293682, 1.309112969, 1.048557137,
1.193750599, 1.337290323, 0.911695704, 0.793437415, 0.976832863,
0.682248176, 0.479172951, 0.59335648, 0.825741995, 0.201656837,
0.087715955, -0.200253782, -0.281464293, -0.423751438, -0.849923161,
-0.548758555, -0.775574083, -0.812678164, -0.918994164, -0.712146965,
-0.987741584, -0.999307348, -0.264546715, -0.110574162, 0.445984485,
0.453985892, 0.342420128, 0.344738943, 0.105759274, 0.000519907,
-0.4401135, -0.669592699, -0.851879843, -0.5891282, -0.32810787,
-0.253702489, -0.260806569, -0.02863203, 0.183398233, 0.067082233,
0.157891587, -0.261587611, -0.320423005, -0.704508163, -0.89442019,
-0.835053597, -0.499715859, -0.48286866, -0.675799609, -0.894779279,
-1.288575885, -1.754459056, -1.278111386, -1.262629372, -1.163618032,
-0.725183796, -0.505528651, -0.866084482, -1.142400482, -0.924687626,
-1.223090192, -0.949406191, -0.314712258, -0.191172534, 0.25963682,
0.058215332, -0.397812115, -0.393406735, -0.201598324, -0.067770047,
0.584115939, 0.347944216, 0.773070977, 0.653014674, 1.011737463,
0.958085133, 0.961624857, 1.128838184, 1.255330131, 1.110456892,
1.18112197, 1.675882603, 2.111364617, 1.567423735, 1.148732617,
1.60076288, 1.300850854, 1.642159736, 0.943546194, 0.645076929,
0.905164902, 1.041656849, 0.744552768, 1.02046761, 0.953718782,
0.627979887, 0.420442978, 0.16096378, -0.26340993, -0.277350812,
-0.239205128, -0.364511195, -0.034423222, -0.376277538, -0.197843302,
0.077061607, -0.304504157, -0.2299545, -1.078212789, -0.954884041,
-0.682276674, -0.435351899, -0.449226081, -0.174387872, -0.15653074,
-1.569106437, -1.558230383), ABEV3.JBSS3 = c(-0.125505311, -0.381380131,
-0.294037578, -0.298621426, -0.277723674, 0.387057047, 0.237627028,
-0.176649161, -0.102841696, -0.204853657, -0.010728477, -0.088211983,
0.291087137, -0.008015111, -0.500672558, -0.327838349, -0.381776712,
-0.444424104, -1.109018008, -1.502966426, -1.311740904, -1.240197666,
-1.028982199, -0.656137935, 1.150241417, 1.038229456, 1.044291093,
0.66550656, 0.792223738, 1.085375662, 0.698835245, 0.35779654,
0.052567206, 0.612173986, 0.083389453, 0.319778861, -0.009651158,
-0.512944035, -0.334300455, -0.687285673, -0.249287579, -0.843563768,
-0.350401788, -0.258858551, -0.224415656, -0.168036304, 0.283834704,
0.714414741, 0.161747239, -0.066064038, -0.374848571, -0.193950819,
-0.130798896, -0.071837601, -0.208030136, 0.039322473, -0.03913429,
0.248536034, -0.052184956, -0.222250405, -0.172315854, 0.492137096,
0.095606735, -0.194468769, -0.44615296, -0.284927438, -0.294029686,
-0.388295819, -0.078033498, -0.263580547, -0.260100853, -0.300811787,
0.256223106, 0.450348286, 0.327373124, 0.170197277, 0.040449544,
0.541357351, 0.129345389, 0.848962225, 0.541458608, 0.374610532,
0.276471485, 0.500596665, 0.462785388, 0.219492511, 0.589427062,
0.190642529, 0.064439938, 0.586956432, 0.136880927, 0.06906965,
-0.110678083, 0.292483896, 0.230154219, -0.0457206, -0.18159542,
-0.597798011, -0.384963802, -0.235357022, -0.228649899, -0.143879233,
-0.104917938, 0.0021069, -0.023450205, -0.023515654, -0.189718245,
0.384734706, 0.199177601, 0.213302781, 0.097427962, -0.441038856,
-0.135622705, -0.243433982, -0.495118173, -0.271310708, 0.030242585,
0.139531651, 0.436238773, 0.493263611, 0.217706506, 0.29989523,
0.237565553, 0.472981704, 0.566133628, 0.630258808, 0.63567496,
0.614964025, 0.536179492, 0.643532101, 0.744112137, 0.422100176,
-0.059911786, 0.03937728, 0.356412173, -0.134954303, 0.076588935,
-0.64833274, -0.674525275, -0.693299752, -0.453365201, -0.547303564,
-0.419950956, 0.089010339, 0.096362947, -0.129839643, -0.425386692,
-0.175452141, -0.193908904, -0.04075698, -0.194705398, -0.169934733,
-0.194200866, -0.047831569, 0.282748467, 0.23397399, 0.277135969,
0.233187551, 0.242158901, 0.164019854, 0.303626634, 0.454852156,
0.668987392, 0.807303201, 0.883682553, 1.13426259, 0.662896114,
0.843813977, 0.119865559, -0.090855431, -0.138666708, -0.033896042,
0.243456566, -0.103719281, -0.4641125, -0.323532464, -0.151026025,
0.359881782, 0.246578849, 0.140386314, 0.550648636, 0.674446045,
0.215661512, 0.294305092, -0.039315556, 0.12093671, -0.180429765,
0.400150271, 0.187492824, -0.08289034, -0.053918989, -0.33560318,
-0.30663183, -0.385088593, -0.155154042, -0.466520517, -0.092395337,
-0.321507641, -0.363191832, -0.394220481, -0.877205699, -0.64888989,
-0.393145968, -0.713231528, -1.132026116, -1.415646764, -0.897320899,
-0.568359605, -0.672625738, -0.239146044, 0.204333651, -0.085741854,
0.14225624, 0.26089982, -0.044657285, 0.363986295, 0.579402447,
0.411908885, 0.466679551, 0.512413417, 0.83848511, 0.994556802,
0.753835812, 0.724415849, 0.053694859, 0.08878324, 0.150008762,
0.395107198, 0.110195579, 0.093347503, -1.15091863, -0.282910481,
-0.267494329, -0.215633377, 0.022036947, -0.42029273, -0.194231093,
-0.195269798, -0.321790104, -0.20927361, -0.2899946, -0.183287477,
-0.255627209, -0.132475285, -0.411259818, -0.449716581, -0.291400772,
-0.083074907, -0.093468127, 0.092911226), ABEV3.GOAU4 = c(-0.096399151,
-0.430291395, -0.181638865, -0.199604415, -0.159431687, 0.126334543,
0.089393571, 0.136005281, -0.03229346, 0.172789718, 0.262458491,
-0.002791521, -0.021944077, -0.341429824, -0.525159639, -0.464133097,
-0.517862913, -0.67650318, -1.187509124, -1.416482581, -1.415968328,
-1.382063821, -1.091037279, -1.096116528, 0.720332754, 0.756782036,
0.796100948, 0.446452774, 0.171031998, 0.690521674, 0.704946805,
0.315973347, -0.031650677, 0.570725299, 0.068873875, 0.097688834,
-0.240430809, -0.740941134, -0.567215724, -0.573140453, -0.243983969,
-0.621932849, -0.263288653, -0.200571151, 0.036715276, 0.173668511,
0.347069065, 0.654867781, 0.597926809, 0.429790496, 0.196410541,
0.31726632, 0.459138342, 0.569311069, 0.279142271, 0.348981807,
0.16238236, 0.575433053, 0.428321318, 0.104258311, -0.019633933,
0.422912806, 0.266988077, 0.066998376, -0.116219151, -0.141298401,
-0.19722313, -0.1914569, -0.236365387, -0.491794497, -0.421288579,
-0.556880117, -0.229081401, -0.068908674, -0.212288629, -0.153465335,
-0.244479614, -0.023461407, -0.245150402, 0.308404245, 0.475357481,
0.380619757, 0.41113401, 0.592493743, 0.530633985, 0.305546401,
0.523686642, 0.160648213, 0.09591049, 0.421847483, 0.072874025,
0.106599435, 0.027276116, 0.232026103, 0.16728838, -0.080847933,
-0.246268709, -0.57456745, -0.356768734, -0.237099961, -0.399463673,
-0.2755675, 0.048149575, 0.304419759, 0.318665792, 0.226293745,
0.119352773, 0.42783454, 0.279877325, 0.32614751, 0.271059925,
-0.023507036, -0.013488402, -0.09907994, -0.29331371, -0.232295503,
-0.111114867, -0.16721036, 0.081604599, 0.206354586, 0.100592285,
0.286520943, 0.223986468, 0.446020918, 0.63331568, 0.568748719,
0.534344186, 0.447394879, 0.567396844, 0.563333837, 0.57638453,
0.429272794, 0.111144816, 0.089114295, 0.064547334, -0.286800135,
-0.064424159, -0.383909906, -0.66322489, -0.7757677, -0.517977319,
-0.71085332, -0.822704743, -0.207450802, -0.395237254, -0.352869613,
-0.887948863, -0.750150147, -0.751164425, -0.497259918, -0.400306682,
-0.512670394, -0.313522244, 0.239024494, 0.349880274, 0.379190851,
0.461891683, 0.624779948, 0.335123438, 0.374621449, 0.570216916,
0.579202638, 0.462424094, 0.78597874, 0.892257259, 0.960568264,
0.65125328, 0.363279395, -0.041466663, -0.155862861, -0.201283637,
-0.209411615, 0.040248824, 0.006015054, -0.183820553, -0.135176358,
0.13330541, 0.37533986, 0.636691258, 0.385335454, 0.52855691,
0.674160712, 0.478056884, 0.437213369, 0.120272397, 0.191469702,
0.122658672, 0.639945099, 0.628256104, 0.16096527, -0.108691239,
-0.141242384, -0.424118385, -0.467506675, -0.550203578, -0.750193279,
-0.372565326, -0.353742033, -0.329162808, -0.623567341, -0.724240093,
-0.257807481, -0.2342445, -0.1342342, -0.123378421, -0.385912896,
-0.151162909, 0.094611656, 0.096987632, 0.557835076, 0.889877861,
0.727343386, 0.861922611, 0.872095338, 0.610747869, 0.767709439,
0.921272421, 0.966526362, 1.036186801, 1.18415628, 1.129751747,
0.801794532, 0.745528276, 0.483164564, -0.102247876, -0.020375854,
-0.249007786, -0.317981244, -0.465776031, -0.340009801, -1.570170265,
-1.063900081, -1.037629896, -0.921009852, -0.63286961, -1.026932617,
-0.820491669, -0.776928688, -0.946585198, -0.885233799, -0.704386355,
-0.566254448, -0.368276634, -0.18623385, -0.294874117, -0.373172858,
-0.119951402, -0.30977034, -0.171638433, 0.042436838), CESP6.PETR4 = c(2.698640815,
2.652045837, 2.937365971, 2.693377102, 2.525824567, 2.41656399,
2.862150684, 2.900076928, 1.863639826, 1.008959961, 0.514595849,
-0.256362475, -0.403757133, -0.396946322, -0.584390173, -1.167579363,
-1.610877236, -1.383908548, -1.022309956, 0.109713073, -0.055233353,
0.190886461, 0.309187948, -0.156291597, -0.30059622, 0.286481151,
0.296480384, 0.163715631, -0.082612787, -1.340165322, -0.513087952,
0.20085019, 0.239042993, 0.382556698, 0.304313167, 0.1043124,
0.028408418, 0.758674211, 0.05223711, -0.033558189, 0.366707604,
0.694160217, 0.395916686, -0.577539063, -0.72881314, -0.754184002,
-0.987906311, -1.25343505, -0.688963788, -0.811146228, -0.74705021,
-0.734494061, -1.208907367, -1.25060588, -0.915127869, -1.405019953,
-1.62368792, -1.329858462, -1.009641863, -0.634321728, -0.350541464,
0.141639442, -0.545804409, -0.553248261, -0.633673464, -0.507395774,
-0.27893641, 0.310638386, 0.208673313, 0.225799877, 0.295641233,
0.113251723, 0.585166069, 0.603783309, 0.151709553, 0.028362486,
0.032725064, -0.136476023, -0.315568427, -0.659823856, -0.117800827,
0.221773969, 0.007795396, -0.302471931, -0.410873338, -0.39885031,
-1.085810235, -1.054211642, -0.806443275, -1.079632465, -0.657076316,
-0.557718886, -0.835380104, -1.277295984, -1.509685494, -1.144474042,
-1.155481559, -0.669095185, -0.644999167, -0.439679033, -0.185158578,
-0.231111753, -0.492336636, -0.797549622, -0.951696368, -0.476968843,
-0.262181828, -0.301116356, -0.358826767, -0.386270618, 0.215001924,
0.150164182, 0.118623546, 0.378731462, 0.158306258, -0.203076502,
-0.003926142, -0.095999899, -0.682752753, -0.681154161, -0.402003801,
0.228528551, 0.841775696, 1.033265605, 1.425179951, 1.053639314,
0.964704787, 1.641515597, 1.956835732, 1.330349437, 1.193487899,
1.195777487, 1.355085724, 1.504127401, 1.592053644, 1.398489212,
1.356839892, 1.237747488, 1.19881296, 1.2674302, 1.194556764,
1.195780113, 0.869500888, 0.80290591, 0.45232283, 0.187376404,
-0.056079345, -0.631874644, -0.631233608, -0.944047554, -0.955055071,
-0.992183169, -0.604474292, -0.697772165, -0.932501224, -0.428138645,
-0.298988286, 0.360635704, 0.634623039, 0.544039959, 0.367129227,
0.123298235, -0.054096423, -0.63590362, -0.921432358, -1.232015439,
-0.869034853, -0.513398966, -0.428503268, -0.346213679, -0.196530199,
0.07495971, 0.100171161, 0.352885186, 0.094375095, 0.056022881,
-0.44168753, -0.591846174, -0.519931829, -0.223229701, -0.391739792,
-0.527268531, -0.678759974, -0.923015403, -1.438119705, -1.252582204,
-1.313698404, -0.771359622, -0.587737234, -0.445023209, -0.593691176,
-0.827946605, -0.598371809, -0.935924344, -0.972835075, -0.444267029,
-0.493892553, -0.156973059, -0.352768357, -0.797448223, -1.040262169,
-0.991062616, -1.074676242, -0.601162537, -0.851321181, -0.469880465,
-0.580039109, -0.362162059, -0.43987247, -0.419606677, 0.092574228,
0.224863817, 0.37539617, 0.537419199, 0.865078882, 1.16970802,
0.496785391, -0.043373253, 0.531413762, 0.343594667, 0.626733129,
0.227532042, 0.32162806, 0.350196106, 0.460837142, 0.329987502,
0.442543651, 0.346373109, 0.236372342, 0.015098265, -0.060805717,
-0.328466934, 0.134454161, 0.246901626, 0.160148771, 0.629457008,
0.497541128, 0.895891808, 1.232485252, 1.229078697, 1.190252852,
0.495464304, 0.580735244, 0.968928048, 1.225314422, 0.944622658,
1.717385877, 1.537335917, 0.04526216, -0.29505436), CESP6.MRVE3.new = c(2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2), ABEV3.JBSS3.new = c(0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), ABEV3.GOAU4.new = c(0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), CESP6.PETR4.new = c(2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2)), class = "data.frame", row.names = c(NA,
-250L))
As you can seeI have columns named with begin names plus ".new"
I want do put them together, It is possible to do this with dplyr package. So the output will be:
CESP6.MRVE3|CESP6.MRVE3.new|ABEV3.JBSS3|ABEV3.JBSS3.new|ABEV3.GOAU4|ABEV3.GOAU4.new|CESP6.PETR4|CESP6.PETR4.new
This is what I am doing:
df %>%
select(flatten_chr(map(colnames(df), ~c(.x , paste0(.x , ".new")))))
This is the wrong messge I am receveing:
Error: Unknown columns `CESP6.MRVE3.new.new`, `ABEV3.JBSS3.new.new`, `ABEV3.GOAU4.new.new` and `CESP6.PETR4.new.new`
In addition: Warning message:
'glue::collapse' is deprecated.
Use 'glue_collapse' instead.
See help("Deprecated") and help("glue-deprecated").
What am I doing wrong??? Any help?

Histogram of a list of matrices for each entry in R

My function is so complex returning a list of matrices (lower triangular matrices). I would like to get the histogram of the entries of the matrices. That is, for all the matrices, I would like to histogram the values at row 1 and col 1. Then, get the histogram of the entry at the row 1 and col 2 and so on till I get the histogram for each entry.
In another words, suppose that the entries at the row 2 and col 1 in the 3 matrices are as follows:
2.4, 3.5, 2.56
For example, suppose I have these matrices:
mat1 <- c(0, 3, 3, 4, 4,
0, 0, 3, 4, 5,
0, 0, 0, 4, 6,
0, 0, 0, 0, 3,
0, 0, 0, 0, 0)
mat1 <- matrix(mat1, 5, 5)
mat2 <- c(0, 5, 6, 4, 4,
0, 0, 3, 5, 5,
0, 0, 0, 6, 6,
0, 0, 0, 0, 3,
0, 0, 0, 0, 0)
mat2 <- matrix(mat2, 5, 5)
mat3 <- c(0, 4, 3.5, 3, 2.9,
0, 0, 3, 3.9, 3 ,
0, 0, 0, 3.5, 2.5,
0, 0, 0, 0, 1.8,
0, 0, 0, 0, 0)
mat3 <- matrix(mat3, 5, 5)
Then I need to get a histogram for each entry (in total, I should get 10 histograms, since I have 10 element in each matrix (non-zero entries). Any help please?

Error using gof() function from LogisticDx package in r

Hi I keep getting the following error message when trying to use the gof() function from the LogisticDx package.
Error in factor(G, labels = dx1[, format(max(P), digits = 3), by = G]$V1) :
invalid 'labels'; length 6 should be 1 or 5
I can not figure out what causes this error. You can find the code causing the error as well as code that works below.
If you need more information just let me knows
H
R version and loaded libraries
R version 3.2.3 (2015-12-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 15.04
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] ROCR_1.0-7 gplots_2.17.0 LogisticDx_0.2 xtable_1.8-2 pander_0.5.2
[6] plyr_1.8.3 Amelia_1.7.4 mice_2.25 Rcpp_0.11.6 knitr_1.11
Code that produces error
PD <- c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0)
E <- c(4, 4, 3, 1, 0, 5, 1, 5, 5, 5, 1, 5, 4, 2, 1, 1, 1, 1, 5,
0, 5, 5, 5, 5, 0, 0, 4, 5, 2, 4, 5, 4, 5, 3, 0, 5, 3, 3,
5, 2, 4, 0, 0, 5, 1, 0, 3, 2, 5, 1, 2, 4, 0, 2, 5, 5, 4,
3, 5, 2, 0, 2, 0, 1, 0, 5, 2, 0, 3, 0, 0, 0, 1, 0, 0, 2,
3, 0, 0, 1, 0, 0, 0, 0, 0, 2, 0, 1, 3, 0, 2, 0, 3, 0, 0,
0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 0, 0, 2, 0, 0,
0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 2, 1, 0, 0)
test.data <- data.frame(PD,E)
factor(test.data$PD)
g.error <- gof(glm(PD ~ E,family=binomial,data=test.data),plot=FALSE)
Code that works
data(uis)
g.works <- gof(glm(RACE ~ NDRGTX,family=binomial,data=uis),plot=FALSE)
The issue here relates to the number of groups in the Hosmer–Lemeshow test being performed. It appears that the default argument for gof is g=10, which doesn't seem to work for 129 observations (it does work if you add one more row).
It should work if you set the number of groups to 9.
gof(glm(PD ~ E,family=binomial,data=test.data), plot=FALSE, g=9)
Maybe someone else can answer why this is the case?
The problem occurs in line 397 in the source code
https://github.com/dardisco/LogisticDx/blob/master/R/gof.R
### Hosmer-Lemeshow
## sort by probability
dx1 <- dx1[order(P), ]
## base group size
gs1 <- dx1[, sum(n) %/% g] # Line 393
g1 <- rep(gs1, g)
## remainer
mod1 <- dx1[, sum(n) %% g]
g1[seq(1, g-1, by=g/mod1)] <- gs1 + 1 # Line 397
Here the length of seq(1, g-1, by=g/mod1) does not necessarily equal the remainer of the division of line 393 making sum(g1) not equal to n (ex n=129, g=10). This causes the error on line 399
dx1[, "G" := factor(G,
labels=dx1[, format(max(P), digits=3), by=G]$V1)]
Since the last dx1[129,G] is NA when n=129 and g=10. A solution could be
g1[seq(1, mod1, by = 1)] <- gs1+1

What is causing these upticks in this ggplot?

Example data:
df <- structure(
list(
group = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 3, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 7, 7, 7, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11),
val = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.000141111115226522, 0, 0, 0, 0.00127000000793487, 0.00070555554702878, 0.000141111115226522, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.00127000000793487, 0.000282222230453044, 0, 0, 0.000141111115226522, 0.000282222230453044, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.00070555554702878, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.000141111115226522, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
),
.Names = c("group", "val"),
row.names = c(NA, -1000L),
class = c("data.table", "data.frame"),
.internal.selfref = <pointer: 0xe4d468>
)
I can plot this as a timeseries per group, like so:
ggplot(df, aes(x=unlist(by(val, group, seq_along)), y=val, group=group)) +
geom_line(alpha=0.5)
but I want to plot a rolling mean of the data, like so:
library(zoo)
ggplot(df, aes(x=unlist(by(val, group, seq_along)),
y=rollmean(val, 48, fill=NA), group=group)) +
geom_line(alpha=0.5)
But this adds upticks to the end of each line, that do not exist in the data:
The upticks at 130 and 670 do not exist in the data, nor do they exist in the rolling mean, as you can see with rollmean(df[group==5, val], 48, fill=NA). So what is causing them?
The first uptick does occur at exactly 132. In your rolling mean, you chose the default align, which sets it to center, meaning that each point is the mean of the previous k/2 and the future k/2 points. Since you set k=48, it means that point 132 will be the mean of (132-24):(132+24). You can verify that the first non-zero point is indeed 156.
# First non-zero value
min(which(df$val!=0))
# 156
You can also verify that the first non-zero value in the rolling mean is 132.
df$rollmean <- rollmean(df$val, 48, fill=NA)
min(which(df$rollmean!=0))
# 132
Additionally, it looks like you are applying your rolling mean across all groups, which you almost certainly don't want. Try splitting by group, like you did with by to create the time variable. Here is an example:
# Set a time variable before hand
df$time <- with(df, unlist(by(val, group, seq_along)))
df$group <- as.factor(df$group)
k=48
# Remove those groups wtihout enough values for rolling mean of k window
df.subset <- df[df$group %in% names(which(table(df$group) >= k)),]
# Calculate a rolling mean on each group
df.subset$rollmean <- unlist(by(df.subset$val, df.subset$group, FUN=rollmean, k=k, fill=NA))
# Plot
ggplot(df.subset, aes(x=time,
y=rollmean,
colour=group)) + geom_line()

Resources