Related
I have a timeseries data which contain some peaks and valleys which significantly differ from the threshold level (an example vector is provided below).
The peak/valley height/width may vary as well as the noise level.
I am interested in finding & reporting both peaks and valleys.
Currently I am using a function based on this thread:
Peak signal detection in realtime timeseries data
However, I would like to improve it according to my needs: I would like to introduce an additional parameter: I would like to ignore the peaks/valleys if they are not significant enough (below certain height - peaks, not enough downward). On the attached picture, I put red circles on peaks I would like to be reported; unmarked ones should be ignored).
Here is an example vector:
signal <- c(659, 613, 586, 642, 685, 695, 691, 733, 638, 708, 706, 691, 703, 712, 715, 700, 693, 682, 717, 711, 722, 700, 704, 704, 715, 691, 670, 684, 689, 711, 680, 692, 686, 710, 702, 699, 702, 715, 691, 670, 684, 689, 711, 673, 688, 699, 677, 701, 680, 692, 686, 710, 702, 699, 717, 691, 703, 712, 715, 700, 693, 682, 717, 711, 722, 700, 704, 704, 715, 691, 670, 684, 689, 711, 680, 692, 686, 710, 702, 699, 702, 706, 651, 712, 722, 734, 705, 714, 691, 704, 704, 669, 712, 715, 689, 715, 691, 670, 684, 689, 711, 651, 712, 722, 734, 705, 714, 691, 686, 676, 690, 693, 702, 694, 682, 693, 724, 693, 707, 684, 687, 705, 680, 680, 705, 680, 693, 700, 704, 704, 669, 712, 715, 689, 715, 691, 670, 684, 689, 711, 673, 678, 699, 677, 680, 682, 676, 690, 658, 675, 663, 667, 682, 673, 675, 656, 652, 563, 544, 542, 540, 532, 538, 505, 526, 565, 629, 720, 713, 720, 720, 773, 732, 740, 695, 689, 723, 685, 726, 710, 684, 693, 715, 692, 683, 712, 707, 693, 699, 717, 703, 687, 682, 690, 716, 708, 713, 700, 676, 708, 691, 717, 711, 722, 688, 695, 641, 666, 638, 639, 600, 635, 609, 653, 671, 649, 716, 708, 713, 700, 676, 708, 691, 717, 711, 722, 700, 704, 704, 669, 712, 715, 689, 715, 691, 670, 684, 689, 711, 673, 688, 699, 677, 701, 680, 692, 686, 710, 702, 699, 717, 691, 703, 712, 715, 700, 693, 682, 717, 711, 722, 700, 704, 704, 715, 691, 670, 684, 689, 711, 680, 692, 686, 710, 702, 699, 702, 715, 691, 670, 684, 689, 711, 673, 688, 699, 677, 701, 680, 692, 686, 710, 702, 699, 717, 691, 703, 712, 715, 700, 693, 682, 717, 711, 722, 700, 704, 704, 715, 691, 670, 684, 769, 767, 740, 752, 686, 710, 702, 699, 702, 706, 651, 712, 722, 734, 705, 714, 691, 704, 704, 669, 712, 715, 689, 715, 691, 670, 684, 689, 711, 704, 669, 712, 715, 689, 715, 691, 670, 684, 689, 711, 673, 688, 699, 677, 701, 680, 692, 686, 710, 702, 699, 717, 691, 703, 712, 715, 700, 693, 682, 717, 711, 722, 700, 704, 704, 715, 691, 670, 684, 689, 711, 680, 692, 686, 710, 702, 699, 702, 715, 691, 670, 684, 689, 711, 673, 688, 699, 677, 701, 680, 692, 686, 710, 702, 699, 665, 630, 808, 686, 787, 781, 796, 815, 786, 793, 664, 717, 691, 703, 712, 715, 700, 693, 682, 717, 711, 722, 700, 704, 704, 715, 691, 670, 684, 689, 711, 680, 692, 686, 710, 702, 699, 702, 706, 651, 712, 722, 734, 705, 714, 691, 704, 704, 669, 712, 715, 689, 715, 691, 670, 684, 689, 711)
And here is a depiction of what I want (red circles - peaks to be reported):
I am the author of the original algorithm you were referring to.
To answer your question, let's first discuss the characteristics of your data:
The timeseries is stationary: the average value seems to be constant around 700
There are infrequent peaks, both up and down
The peaks do not change the distribution or the trend of the data
The peaks are clustered: if there is a peak (signal), the peak persists for approximately 20-30 data points
Because your timeseries is stationary, you should put the influence parameter close to zero. Because there are infrequent peaks, you should put the threshold high (above 3 std. devs). Because the peaks are clustered, you should put the lag parameter as high as you can.
For example, using:
lag <- 130
threshold <- 4
influence <- 0
Yields the following:
Here is the accompanying code:
ThresholdingAlgo <- function(y,lag,threshold,influence) {
signals <- rep(0,length(y))
filteredY <- y[0:lag]
avgFilter <- NULL
stdFilter <- NULL
avgFilter[lag] <- mean(y[0:lag], na.rm=TRUE)
stdFilter[lag] <- sd(y[0:lag], na.rm=TRUE)
for (i in (lag+1):length(y)){
if (abs(y[i]-avgFilter[i-1]) > threshold*stdFilter[i-1]) {
if (y[i] > avgFilter[i-1]) {
signals[i] <- 1;
} else {
signals[i] <- -1;
}
filteredY[i] <- influence*y[i]+(1-influence)*filteredY[i-1]
} else {
signals[i] <- 0
filteredY[i] <- y[i]
}
avgFilter[i] <- mean(filteredY[(i-lag):i], na.rm=TRUE)
stdFilter[i] <- sd(filteredY[(i-lag):i], na.rm=TRUE)
}
return(list("signals"=signals,"avgFilter"=avgFilter,"stdFilter"=stdFilter))
}
y <- c(659, 613, 586, 642, 685, 695, 691, 733, 638, 708, 706, 691, 703, 712, 715, 700, 693, 682, 717, 711, 722, 700, 704, 704, 715, 691, 670, 684, 689, 711, 680, 692, 686, 710, 702, 699, 702, 715, 691, 670, 684, 689, 711, 673, 688, 699, 677, 701, 680, 692, 686, 710, 702, 699, 717, 691, 703, 712, 715, 700, 693, 682, 717, 711, 722, 700, 704, 704, 715, 691, 670, 684, 689, 711, 680, 692, 686, 710, 702, 699, 702, 706, 651, 712, 722, 734, 705, 714, 691, 704, 704, 669, 712, 715, 689, 715, 691, 670, 684, 689, 711, 651, 712, 722, 734, 705, 714, 691, 686, 676, 690, 693, 702, 694, 682, 693, 724, 693, 707, 684, 687, 705, 680, 680, 705, 680, 693, 700, 704, 704, 669, 712, 715, 689, 715, 691, 670, 684, 689, 711, 673, 678, 699, 677, 680, 682, 676, 690, 658, 675, 663, 667, 682, 673, 675, 656, 652, 563, 544, 542, 540, 532, 538, 505, 526, 565, 629, 720, 713, 720, 720, 773, 732, 740, 695, 689, 723, 685, 726, 710, 684, 693, 715, 692, 683, 712, 707, 693, 699, 717, 703, 687, 682, 690, 716, 708, 713, 700, 676, 708, 691, 717, 711, 722, 688, 695, 641, 666, 638, 639, 600, 635, 609, 653, 671, 649, 716, 708, 713, 700, 676, 708, 691, 717, 711, 722, 700, 704, 704, 669, 712, 715, 689, 715, 691, 670, 684, 689, 711, 673, 688, 699, 677, 701, 680, 692, 686, 710, 702, 699, 717, 691, 703, 712, 715, 700, 693, 682, 717, 711, 722, 700, 704, 704, 715, 691, 670, 684, 689, 711, 680, 692, 686, 710, 702, 699, 702, 715, 691, 670, 684, 689, 711, 673, 688, 699, 677, 701, 680, 692, 686, 710, 702, 699, 717, 691, 703, 712, 715, 700, 693, 682, 717, 711, 722, 700, 704, 704, 715, 691, 670, 684, 769, 767, 740, 752, 686, 710, 702, 699, 702, 706, 651, 712, 722, 734, 705, 714, 691, 704, 704, 669, 712, 715, 689, 715, 691, 670, 684, 689, 711, 704, 669, 712, 715, 689, 715, 691, 670, 684, 689, 711, 673, 688, 699, 677, 701, 680, 692, 686, 710, 702, 699, 717, 691, 703, 712, 715, 700, 693, 682, 717, 711, 722, 700, 704, 704, 715, 691, 670, 684, 689, 711, 680, 692, 686, 710, 702, 699, 702, 715, 691, 670, 684, 689, 711, 673, 688, 699, 677, 701, 680, 692, 686, 710, 702, 699, 665, 630, 808, 686, 787, 781, 796, 815, 786, 793, 664, 717, 691, 703, 712, 715, 700, 693, 682, 717, 711, 722, 700, 704, 704, 715, 691, 670, 684, 689, 711, 680, 692, 686, 710, 702, 699, 702, 706, 651, 712, 722, 734, 705, 714, 691, 704, 704, 669, 712, 715, 689, 715, 691, 670, 684, 689, 711)
lag <- 130
threshold <- 4
influence <- 0
# Run algo with lag = 30, threshold = 5, influence = 0
result <- ThresholdingAlgo(y,lag,threshold,influence)
# Plot result
par(mfrow = c(2,1),oma = c(2,2,0,0) + 0.1,mar = c(0,0,2,1) + 0.2)
plot(1:length(y),y,type="l",ylab="",xlab="")
lines(1:length(y),result$avgFilter,type="l",col="cyan",lwd=2)
lines(1:length(y),result$avgFilter+threshold*result$stdFilter,type="l",col="green",lwd=2)
lines(1:length(y),result$avgFilter-threshold*result$stdFilter,type="l",col="green",lwd=2)
plot(result$signals,type="S",col="red",ylab="",xlab="",ylim=c(-1.5,1.5),lwd=2)
}
Now if you also wish to find the minimum or maximum value within each signal, you can simply overlay the result array on your y (data) array and identify where it achieves the highest or lowest value.
For example (using explicit loops instead of a quick one-liner):
cluster$flag <- FALSE
cluster$sign <- FALSE
cluster$data <- NULL
cluster$index <- NULL
for(i in 1:length(result$signals)) {
if (abs(result$signals[i]) == 1.0) {
cluster$flag <- TRUE
cluster$sign <- result$signals[i]
cluster$data <- append(cluster$data, y[i])
cluster$index <- append(cluster$index, i)
} else {
if (cluster$flag) {
if (cluster$sign == 1) {
print(cat("The positive cluster starting at", cluster$index[1], "has maximum", max(cluster$data), "\n"), sep='')
} else {
print(cat("The negative cluster starting at", cluster$index[1], "has minimum", min(cluster$data), "\n"), sep='')
}
}
cluster$flag <- FALSE
cluster$sign <- NULL
cluster$data <- NULL
cluster$index <- NULL
}
}
This gives the following output:
The negative cluster starting at 158 has minimum 505
The positive cluster starting at 410 has maximum 808
The positive cluster starting at 412 has maximum 815
Hope that helps!
Let's start by plotting your data, along with its mean:
plot(signal, type = 'l')
abline(h = mean(signal), col = 'blue3', lty = 2)
Since you want the upwards peak near the end of the series to be included but not the downwards spike at the very start, we need to find the number of standard deviations that would identify this peak but not the one at the start. This turns out to be about 3.5 standard deviations, as we can see if we plot the 3.5 standard deviation lines:
abline(h = mean(signal) + c(3.5, -3.5) * sd(signal), lty = 2, col = 'red')
Now comes the tricky part. We use run length encoding to identify which contiguous parts of the sequence are outside of the 3.5 standard deviations, and find the point of largest absolute deviation for each:
big <- abs(signal - mean(signal)) > 3.5 * sd(signal)
exceed <- split(seq_along(big), data.table::rleid(big))[rle(big)$value]
peaks <- sapply(exceed, function(x) x[which.max(abs(signal[x] - mean(signal)))])
Now the vector peaks will contain only the maximum or minimum point within each spike. Note although you only have two spikes in your sample data, this will work for however many spikes you have.
To demonstrate, let us plot the points:
points(peaks, signal[peaks], col = 'red', pch = 16)
Maybe you want something like this:
max.index <- which.max(signal)
max <- max(signal)
min.index <- which.min(signal)
min <- min(signal)
plot(signal, type = "l")
points(max.index, max, col = "red", pch = 16)
points(min.index, min, col = "red", pch = 16)
Output:
Since I just started getting familiar with forecasting, so I stumbled upon the example here based on which I have a few questions:
How can I forecast for the next 5 years?
What are the red and blue shaded areas around the forecast lines and what's the interpretation?
Why is there a break between the forecast lines and the historical lines?
What forecasting model does geom_forecast use?
lungDeaths data:
structure(c(2134, 1863, 1877, 1877, 1492, 1249, 1280, 1131, 1209,
1492, 1621, 1846, 2103, 2137, 2153, 1833, 1403, 1288, 1186, 1133,
1053, 1347, 1545, 2066, 2020, 2750, 2283, 1479, 1189, 1160, 1113,
970, 999, 1208, 1467, 2059, 2240, 1634, 1722, 1801, 1246, 1162,
1087, 1013, 959, 1179, 1229, 1655, 2019, 2284, 1942, 1423, 1340,
1187, 1098, 1004, 970, 1140, 1110, 1812, 2263, 1820, 1846, 1531,
1215, 1075, 1056, 975, 940, 1081, 1294, 1341, 901, 689, 827,
677, 522, 406, 441, 393, 387, 582, 578, 666, 830, 752, 785, 664,
467, 438, 421, 412, 343, 440, 531, 771, 767, 1141, 896, 532,
447, 420, 376, 330, 357, 445, 546, 764, 862, 660, 663, 643, 502,
392, 411, 348, 387, 385, 411, 638, 796, 853, 737, 546, 530, 446,
431, 362, 387, 430, 425, 679, 821, 785, 727, 612, 478, 429, 405,
379, 393, 411, 487, 574), .Dim = c(72L, 2L), .Dimnames = list(
NULL, c("mdeaths", "fdeaths")), .Tsp = c(1974, 1979.91666666667,
12), class = c("mts", "ts", "matrix"))
Code:
library(forecast)
# Data
lungDeaths = cbind(mdeaths, fdeaths)
# Plot
autoplot(lungDeaths) + geom_forecast()
Output:
To remove the gap you can use showgap:
If showgap=FALSE, the gap between the historical observations and the
forecasts is removed.
Code:
library(forecast)
autoplot(lungDeaths) +
geom_forecast(showgap = FALSE)
Output:
To forecast 5 years you can use h to set the number of forecasts:
autoplot(lungDeaths) +
geom_forecast(h = 60, showgap = FALSE)
Output:
To remove the confidence intervals use PI:
If FALSE, confidence intervals will not be plotted, giving only the
forecast line.
library(forecast)
autoplot(lungDeaths) +
geom_forecast(h = 60, showgap = FALSE, PI = FALSE)
Output:
I have some problems with resending IR signals from a remote to control my shutters.
I recorded the raw IR codes, but even another Arduino does not recieve anything. It does not print any data.
I am a bit confused about the library ESP8266irRemote. It needs a frequency for sending raw ir data. As the timings are given in ms, I do not understand what this frequncy is supposed to be. Where could I read this frequency from? What are some default values? -- EDIT cleared up, it is the carrier frequency. Seems like the default of 38kHz should be right.
And why could it be that my Arduino does not recieve anything? If I simply use an example for a Samsung TV, it receives everything fine.
Thanks for any help!
EDIT:
uint16_t up3[95] = {444, 1190, 442, 1190, 1256, 376, 1258, 374, 440, 1190, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 1282, 350, 440, 1192, 440, 1192, 440, 1190, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 438, 1194, 1256, 374, 1258, 374, 1256, 19240, 440, 1192, 440, 1192, 1282, 350, 1256, 376, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 1256, 374, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 438, 1192, 440, 1192, 438, 1192, 440, 1192, 440, 1192, 464, 1168, 1256, 376, 1256, 376, 1256}; // UNKNOWN 87FDCA19
uint16_t stop3[95] = {1288, 346, 448, 1182, 1214, 418, 1222, 410, 444, 1188, 438, 1194, 466, 1164, 448, 1184, 440, 1192, 438, 1192, 1258, 374, 380, 1252, 448, 1182, 466, 1166, 448, 1184, 466, 1166, 448, 1182, 404, 1228, 468, 1164, 378, 1252, 1280, 350, 1256, 376, 448, 1184, 1264, 19234, 1220, 414, 402, 1230, 1284, 348, 1252, 380, 406, 1226, 378, 1252, 404, 1228, 404, 1228, 404, 1228, 438, 1192, 1266, 366, 468, 1164, 406, 1226, 446, 1186, 448, 1184, 448, 1184, 378, 1252, 448, 1184, 400, 1232, 448, 1184, 1264, 368, 1254, 376, 468, 1164, 1264}; // UNKNOWN 6CE4F608
uint16_t dwn3[95] = {398, 1252, 1280, 352, 1284, 348, 1250, 380, 446, 1188, 462, 1170, 432, 1198, 378, 1254, 446, 1186, 442, 1188, 1282, 348, 402, 1230, 464, 1166, 434, 1196, 446, 1186, 446, 1186, 434, 1198, 462, 1168, 446, 1186, 446, 1186, 378, 1252, 400, 1230, 1218, 414, 378, 20118, 466, 1168, 1216, 414, 1262, 370, 1194, 436, 398, 1232, 398, 1232, 380, 1252, 464, 1168, 464, 1166, 466, 1164, 1196, 436, 400, 1232, 444, 1188, 400, 1230, 446, 1188, 466, 1164, 378, 1254, 446, 1186, 444, 1186, 466, 1166, 402, 1230, 458, 1172, 1282, 348, 464}; // UNKNOWN 2744EDAC
uint16_t up2[95] = {466, 1186, 444, 1186, 1262, 370, 444, 1186, 1260, 370, 446, 1186, 444, 1186, 446, 1186, 468, 1162, 446, 1186, 1262, 370, 444, 1188, 444, 1186, 444, 1188, 444, 1188, 444, 1186, 446, 1186, 444, 1188, 444, 1186, 444, 1188, 1262, 368, 1262, 370, 444, 1186, 1262, 19236, 446, 1186, 446, 1186, 1260, 370, 444, 1188, 1262, 370, 444, 1186, 446, 1186, 446, 1186, 446, 1186, 444, 1186, 1262, 370, 446, 1186, 444, 1188, 444, 1188, 446, 1186, 446, 1184, 446, 1186, 446, 1186, 446, 1186, 446, 1184, 1262, 370, 1260, 372, 446, 1186, 1260}; // UNKNOWN 2D1A9455
uint16_t stop2[95] = {1260, 374, 442, 1190, 1256, 376, 440, 1190, 1258, 374, 440, 1190, 440, 1192, 442, 1190, 440, 1192, 440, 1192, 1256, 374, 440, 1190, 440, 1192, 440, 1190, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 1256, 374, 1258, 374, 1256, 19240, 1258, 374, 440, 1192, 1256, 374, 440, 1192, 1256, 374, 440, 1192, 440, 1192, 440, 1190, 440, 1190, 440, 1192, 1256, 374, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 440, 1192, 440, 1190, 440, 1192, 440, 1192, 440, 1190, 440, 1192, 1256, 374, 1256, 376, 1256}; // UNKNOWN B54FF968
uint16_t dwn2[95] = {478, 1156, 1288, 342, 1288, 344, 450, 1182, 1288, 342, 450, 1182, 476, 1154, 452, 1180, 450, 1180, 450, 1182, 1290, 342, 450, 1182, 476, 1156, 478, 1154, 478, 1154, 474, 1158, 450, 1182, 450, 1182, 474, 1156, 450, 1180, 1292, 340, 476, 1156, 474, 1158, 450, 20048, 476, 1156, 1290, 340, 1266, 366, 450, 1182, 1266, 364, 450, 1182, 476, 1156, 476, 1156, 450, 1182, 474, 1156, 1266, 366, 450, 1182, 474, 1156, 476, 1156, 476, 1156, 474, 1156, 450, 1182, 450, 1182, 474, 1158, 474, 1158, 1266, 366, 450, 1180, 450, 1182, 450}; // UNKNOWN 983238A8
IRsend irsend(4);
void setup() {
// put your setup code here, to run once:
irsend.begin();
}
void loop() {
// put your main code here, to run repeatedly:
irsend.sendRaw(dwn3, 95, 999);
delay(10000);
}
That's the code I used. I recoded the raw arrays using the raw dump example provided with the esp8266ir library.
I cut the import part, but be assured, the correct headers were imported. The code compiles without any issue.
Thanks for the suggested edit. I am sorry about the first, not well organized question.
As you did not provide any code and not much information in general I can only guess.
Possible issues:
wrong emitter wavelength
wrong carrier frequency, typically between 30 and 60kHz. 38kHz is most common.
or some error in sending what you have recorded.
I suggest you first find out how a IR remote control works befor you attempt to build one yourself.
I am working to perform a bootstrap using the statistic median for dataset "file", containing only one column "Total". This is it:
Total <-
c(2089, 1567, 1336, 1616, 1590, 1649, 1341, 1614, 1590, 1621,
1621, 1631, 1295, 107, 18, 195, 2059, 870, 2371, 787, 98, 2422,
655, 1277, 1336, 2109, 1811, 1337, 1290, 1308, 1359, 1600, 1296,
693, 107, 1359, 89, 89, 89, 89, 2411, 1639, 89, 89, 1283, 89,
89, 89, 2341, 1012, 1295, 1853, 1277, 1571, 1288, 1300, 1619,
107, 555, 1612, 1300, 1300, 2093, 133, 1674, 988, 132, 647, 606,
544, 873, 274, 120, 1620, 1601, 1601, 906, 1603, 1613, 1592,
1603, 1610, 1321, 2380, 1575, 1575, 1277, 2354, 1561, 1579, 2367,
2341, 876, 1612, 1588, 2087, 1612, 890, 1586, 1580, 611, 1797,
2079, 1937, 189, 171, 706, 1647, 1642, 1278, 1650, 1623, 1647,
1661, 1692, 1632, 1684, 2474, 403, 842, 593, 98, 2354, 1265,
866, 1483, 2379, 1650, 1875, 1655, 1632, 1691, 1329, 867, 1632,
1693, 1623, 829, 1659, 1685, 666, 1585, 1659, 2169, 1623, 1645,
1654, 1698, 2172, 789, 1698, 579, 2443, 335, 132, 1952, 1265,
978, 1624, 979, 1729, 607, 181, 752, 424, 386, 309, 998, 1435,
2476, 392, 1657, 348, 1652, 1646, 1345, 2445, 1655, 840, 1624,
1652, 1321, 1321, 2201, 957, 917, 2458, 4096, 2458, 1346, 2459,
1634, 2459, 2459, 2459, 2508, 714, 2457, 2457, 1703, 669, 976,
1634, 2459, 2491, 2393, 625, 1763, 879, 886, 1085, 731, 924,
1649, 1216, 1647, 2470, 668, 2326, 757, 215, 276, 186, 901, 1402,
429, 554, 2457, 1643, 986, 730, 1028, 971, 1952, 1584, 1023,
1352, 839, 2434, 430, 2462, 1327, 1004, 385, 1099, 1067, 758,
679, 1423, 2495, 1664, 2495, 2495, 1345, 2530, 1754, 1804, 2525,
1652, 2536, 1646, 2529, 1380, 1845, 963, 1339, 2482, 1417, 1729,
1384, 1648, 344, 1648, 955, 609, 485, 1822, 513, 223, 222, 193,
1410, 1159, 586, 585, 2671, 2702, 2529, 2212, 1658, 741, 2529,
861, 1758, 905, 2529, 597, 1049, 2529, 619, 2620, 2596, 1688,
2590, 2545, 2590, 883, 287, 723, 2565, 1835, 1738, 2243, 1693,
2565, 250, 2529, 1880, 1777, 701, 444, 927, 1127, 825, 2726,
1977, 235, 241, 269, 660, 1523, 420, 678, 213, 544, 940, 983,
605, 2716, 1848, 1848, 182, 1225, 365, 993, 224, 267, 309, 271,
324, 178, 2657, 1772, 546, 456, 2637, 1771, 677, 1409, 653, 2359,
690, 828, 2742, 1812, 2777, 552, 1572, 2742, 2792, 2819, 1753,
265, 1901, 1753, 2716, 2800, 2742, 453, 2742, 586, 1920, 929,
1897, 2742, 1859, 1899, 1106, 1135, 759, 730, 1838, 863, 1929,
2751, 2751, 2751, 2751, 713, 430, 2788, 1784, 966, 2483, 1784,
1786, 2727, 857, 1798, 1815, 730, 390, 593, 1489, 1448, 1784,
1510, 2788, 812, 856, 808, 941, 2797, 2757, 1852, 2757, 2412,
486, 1034, 615, 845, 974, 727, 969, 2916, 1841, 1926, 1926, 533,
446, 733, 696, 1214, 1857, 1907, 2824, 2631, 3556, 2496, 1617,
1000, 707, 936, 761, 960, 1936, 857, 423, 1130, 1165, 2453, 338,
988, 1869, 1951, 1932, 2820, 2742, 628, 447, 866, 637, 932, 2742,
1795, 2881, 695, 762, 2778, 427, 714, 2781, 1865, 1861, 678,
1465, 1770, 845, 356, 817, 385, 1820, 2692, 1787, 1510, 1814,
857, 2616, 204, 465, 1773, 2754, 1793, 1773, 1900, 185, 2706,
1162, 766, 2742, 1816, 2742, 1790, 1803, 1795, 1026, 334, 832,
478, 1849, 2679, 1773, 797, 2649, 1814, 1808, 99, 2037, 2616,
2719, 1813, 2637, 2648, 1813, 865, 1717, 2588, 2711, 2818, 1828,
2553, 2720, 1791, 1780, 2706, 2565, 1717, 1881, 1037, 329, 893,
723, 1821, 2692, 2586, 2729, 1755, 1793, 2670, 2602, 2638, 2684,
1813, 1755, 1755, 2626, 832, 739, 724, 1968, 2598, 2627, 851,
749, 684, 625, 2673, 2778, 1764, 2644, 1800, 1792, 511, 2776,
1890, 1764, 2776, 1040, 1049, 2699, 2061, 897, 1764, 274, 2755,
1912, 2581, 1780, 820, 1803, 2692, 2783, 572, 2751, 2699, 1830,
1875, 633, 1083)
Then I tried to use the bootstrap function:
> boot (Total, median, 1000)
ORDINARY NONPARAMETRIC BOOTSTRAP
Call:
boot(data = Total, statistic = median, R = 1000)
Bootstrap Statistics :
original bias std. error
t1* 1603 0 0
There were 50 or more warnings (use warnings() to see the first 50)
The warning message was:
the condition has length > 1 and only the first element will be used
Can you please advise me how do I perform bootstrap to generate 95% confidence intervals for the median? I am a beginner in this and your help would be much appreciated.
Thank you so much in advance.
Admittedly the boot function from the boot package has a slightly non-intuitive aspect to it. But if you read the documentation (or look at the examples in the documentation) you'll see specific instructions about the statistic argument:
In all other cases statistic must take at least two arguments. The
first argument passed will always be the original data. The second
will be a vector of indices, frequencies or weights which define the
bootstrap sample.
So instead of:
x <- rnorm(10)
boot(data = x,statistic = median,R = 1000)
You want this:
boot(data = x,statistic = function(x,i) median(x[i]),R = 1000)
Once you're that far, the function boot.ci() can be used to compute the confidence intervals (only some of them are available in this particular example I believe).
b <- boot(data = x,statistic = function(x,i) median(x[i]),R = 1000)
boot.ci(b)
Though the answer by #joran is right, since I already had code tested, with the CI computation, here it goes.
library(boot)
bootMedian <- function(data, indices) median(data[indices])
b <- boot(Total, bootMedian, R = 1000)
boot.ci(b)
This is how you would "roll your own" bootrap:
# number of bootstrap replicates
B <- 10000
# create empty storage container
result_vec <- vector(length=B)
for(b in 1:B) {
# draw a bootstrap sample
this_sample <- sample(Total, size=length(Total), replace=TRUE)
# calculate your statistic
m <- median(this_sample)
# save your calucated statistic
result_vec[b] <- m
}
# then probably draw a histogram of your bootstrapped replicates
hist(result_vec)
# get 95% confidence interval
result_vec <- result_vec[order(result_vec)]
lower_bound <- result_vec[round(0.025*B)]
upper_bound <- result_vec[round(0.0975*B)]
I use the standard normal random generator in this code:
B <- i
bs.result <- matrix(NA, nrow=i, ncol=...)
for (b in 1:i) {
sample.n <- rnorm(n, mean-..., sd=...)
optim.b <- optim(c(mu=0, sd=1), loglik, control=list(fnscale=-1), z=sample.n)
bs.result <- c(optim.b$par, optim.b$converge)
}
With the last column of the table you can check whether your optimize function had converged.
I'm trying to use TSP package with GA. I want to do something similar to this
My code:
library(GA)
library(globalOptTests)
library(TSP)
data("USCA50")
fitFun <-
function(x)
-tour_length(solve_TSP(USCA50))
dist <- as.matrix(USCA50)
GA <- ga(
type = "permutation",
fitness = fitFun,
distMatrix = dist,
min =1,
max = 50
)
The error I get:
Error in fitness(Pop[i, ], ...) :
unused argument (distMatrix = c(0, 1167, 1579, 437, 3575, 1453, 226, 2976, 1107, 1006, 1046, 891, 1488, 1030, 1803, 190, 1122, 1373, 1860, 523, 1047, 1152, 370, 1453, 1629, 1323, 1032, 654, 1462, 752, 993, 813, 1178, 1705, 816, 1206, 1285, 1641, 1578, 1703, 1343, 1317, 1647, 1157, 1479, 1703, 1166, 1211, 795, 1572, 1167, 0, 413, 1422, 2895, 316, 1172, 3094, 140, 382, 189, 530, 392, 526, 635, 1174, 2056, 286, 692, 910, 207, 211, 1035, 303, 2046, 2164, 1385, 845, 297, 597, 1033, 393, 1766, 546, 386, 1076,
153, 476, 432, 546, 184, 184, 481, 1579, 1686, 543, 20, 2008, 527, 434, 1579, 413, 0, 1832, 2766, 167, 1585, 3265, 508, 677, 547, 842, 229, 775, 229, 1575, 2451, 275, 289, 1277, 582, 514, 1420, 207, 2347, 2544, 1720, 1189, 116, 947, 1350, 800, 2117, 138, 777, 1338, 334, 62, 106, 145, 260, 312, 128, 1911, 1961, 136, 413, 2384, 913, 131, 437, 1422, 1832, 0, 3437, 1732, 272, 2607, 1327, 1355, 1345, 1269, 1787, 1409, 2041, 615, 697, 1670, 2093, 954, 1256, 1345, 807, 1672, 1242, 8
Is there something wrong with my GA package? RStudio doesn't show me this parameter but somehow others are able to run it.