I'm trying to visualize a signal and its frequency spectrum in Julia.
I found the FFTW package that provides the FFT and DSP for the frequencies.
Here is what I'm trying, with a sinusoidal signal:
using Plots
using FFTW
using DSP
# Number of points
N = 2^14 - 1
# Sample rate
fs = 1 / (1.1 * N)
# Start time
t0 = 0
tmax = t0 + N * fs
# time coordinate
t = [t0:fs:tmax;]
# signal
signal = sin.(2π * 60 * t) # sin (2π f t)
# Fourier Transform of it
F = fft(signal)
freqs = fftfreq(length(t), fs)
freqs = fftshift(freqs)
# plots
time_domain = plot(t, signal, title = "Signal")
freq_domain = plot(freqs, abs.(F), title = "Spectrum")
plot(time_domain, freq_domain, layout = 2)
savefig("Wave.pdf")
I expected to see a nice plot with a peak in the 60 Hz, but all I got was a weird result:
I'm ignoring the negative frequencies for now.
How should I do that in Julia?
What you call fs in your code is not your sampling rate but the inverse of it: the sampling period.
The function fftfreq takes the sampling rate as its second argument. Since what you give as the second argument is the sampling period, the frequencies returned by the function are incorrectly scaled by (1/(Ts^2)).
I renamed fs to Ts and changed the second argument to fftfreq to the sampling rate 1.0/Ts. I think you also need to shift the result of fft.
# Number of points
N = 2^14 - 1
# Sample period
Ts = 1 / (1.1 * N)
# Start time
t0 = 0
tmax = t0 + N * Ts
# time coordinate
t = t0:Ts:tmax
# signal
signal = sin.(2π * 60 .* t) # sin (2π f t)
# Fourier Transform of it
F = fft(signal) |> fftshift
freqs = fftfreq(length(t), 1.0/Ts) |> fftshift
# plots
time_domain = plot(t, signal, title = "Signal")
freq_domain = plot(freqs, abs.(F), title = "Spectrum", xlim=(-1000, +1000))
plot(time_domain, freq_domain, layout = 2)
savefig("Wave.pdf")
Related
suppose I have a data like y and I fit a smooth function to this data with Fourier basis
y<- c(1,2,5,8,9,2,5)
x <- seq_along(y)
Fo <- create.fourier.basis(c(0, 7), 4)
precfd = smooth.basis(x,y,Fo)
plotfit.fd(y, x, precfd$fd)
precfd <- smooth.basis(x, y, Fo);coef(precfd)
the out put of last line gives me this:
const 411.1060285
sin1 -30.5584033
cos1 6.5740933
sin2 26.2855849
cos2 -26.0153965
I know what is the coefficient but what in const? in original formula there is no constant part as this link say:
http://lampx.tugraz.at/~hadley/num/ch3/3.3a.php
The first basis function in create.fourier.basis is a constant function to allow for a non-zero mean (intercept) in the data. From the documentation of the create.fourier.basis function:
The first basis function is the unit function with the value one everywhere. The next two are the sine/cosine pair with period defined in the argument period. The fourth and fifth are the sin/cosine series with period one half of period. And so forth. The number of basis functions is usually odd.
You can drop the first (unit) basis function in create.fourier.basis with the argument dropind = 1. Below some example code that illustrates which basis functions are used in create.fourier.basis. Note: the scaling of the basis functions depends on the period argument in create.fourier.basis.
Example 1: non-zero mean
library(fda)
## time sequence
tt <- seq(from = 0, to = 1, length = 100)
## basis functions
phi_0 <- 1
phi_1 <- function(t) sin(2 * pi * t) / sqrt(1 / 2)
phi_2 <- function(t) cos(2 * pi * t) / sqrt(1 / 2)
## signal
f1 <- 10 * phi_0 + 5 * phi_1(tt) - 5 * phi_2(tt)
## noise
eps <- rnorm(100)
## data
X1 <- f1 + eps
## create Fourier basis with intercept
four.basis1 <- create.fourier.basis(rangeval = range(tt), nbasis = 3)
## evaluate values basis functions
## eval.basis(tt, four.basis1)
## fit Fourier basis to data
four.fit1 <- smooth.basis(tt, X1, four.basis1)
coef(four.fit1)
Example 2: zero mean
## signal
f2 <- 5 * phi_1(tt) - 5 * phi_2(tt)
## data
X2 <- f2 + eps
## create Fourier basis without intercept
four.basis2 <- create.fourier.basis(rangeval = range(tt), nbasis = 3, dropind = 1)
## evaluate values basis functions
## eval.basis(tt, four.basis2)
## fit Fourier basis to data
four.fit2 <- smooth.basis(tt, X2, four.basis2)
coef(four.fit2)
time <- 1:12
y <- c(0,0,0,0,0,0,0,12,34,69,100,100)
mdl <- gcFitModel(time,y,control = grofit.control(fit.opt = "m", model.type = "gompertz"))
I get my parameters from the above mdl to fit the gompertz equation
y <- A* exp(-exp(mu * e * (lambda - time)/mu + 1))
mu <- 36.162016
lambda <- 7.9800164
A <- 100
time <- 1:12
time here is in a step of 15 days. For e.g time = 1 implies mid-day of a 15 day period, time 2 implies mid day of the next 15 days period, time 3 implies implies the mid-day of the next 15 days period and so on.
I fitted the following function:
e <- exp(1)
y <- 100 * exp(-exp(mu * e * (lambda - time)/mu + 1))
plot(time,y)
The lambda controls the movement along the x-axis.
I am looking to modify this curve so that I get more data points by converting the ids into weeks i.e instead of mid-point of every 15 days, I want to get y for every 7 days. How can I do this?
Your code doesn't include any fitting routines, so I am assuming this is a question about plotting.
Here is an example of a plot with 100 points between time = 1 and time = 12.
time <- seq(1, 12, length.out = 100);
mu <- 34.55844
y <- 100 * exp(-exp(mu * e * (3 - time)/mu + 1))
plot(time,y)
My task is to simulate a compound Poisson process defined as:
where
is a Poisson process and Y_i are Gamma(shape,scale) distributed. This is my R code:
# parameter for Poisson distribution.
lambda = 1
# parameters for Gamma distribution.
shape = 7.5
scale = 1
comp.pois = function(t.max, lambda) {
stopifnot(t.max >= 0 && t.max %% 1 == 0)
# offset ns by 1 because first y is 0.
# generate N(t), that is number of arrivals until time t.
ns = cumsum(rpois(n = t.max, lambda = lambda)) + 1
# generate gamma distributed random variables Y_i.
ys = c(0, rgamma(n = max(ns), shape = shape, scale = scale))
# generate all X(t) for t <= t.max.
return(c(0, cumsum(x = ys[ns])))
}
Compute a random sample of X(10) and compare means and variances.
# sample size.
size = 1000
t = 10
# ts is a vector of sample values for X(10).
ts = sapply(1:size, function(i) comp.pois(t, lambda)[t])
# sample mean and variance:
(mean.s = mean(ts))
(var.s = var(ts))
# theoretical mean and variance:
(mean.t = lambda * t * shape * scale)
(var.t = (shape + 1) * shape * scale^2)
output:
> # sample:
> (mean.s = mean(ts))
[1] 63.38403
> (var.s = var(ts))
[1] 184.3264
> # theoretical:
> (mean.t = lambda * t * shape * scale)
[1] 75
> (var.t = (shape + 1) * shape * scale^2)
[1] 63.75
This variance is gigantic, but I cannot spot my mistake. Please help. Thank you.
EDIT:
I used the following algorithm to generate the N(t). I don't know why it is supposed to be better. I took it from Rizzo, Maria L. Statistical computing with R. CRC Press, 2007. The mean is good, but the variance is even worse. I tried sampling from the Gamma distribution only once for the entire simulation (although I'm pretty sure this does not reflect the problem very well) and the mean was off by around 10-40 for t = 10. When resampling for every X(t) (which is what the following code does), the mean is very exact. As pointed out, the variance is horrifying. This is probably not a good solution, but I suppose it is as good as it gets.
lambda = 3
shape = 6
scale = 2
size = 10000
eps = 1e-8
t = 10
# with probability 1-eps, n or less gamma distributed random variables are needed.
n = qpois(1-eps, lambda = lambda * t)
# sample from the gamma distribution. Not sure if it's ok to use the same sample every time.
# with this, the mean is of by about 10%.
# ys = c(rgamma(n = n, shape = shape, scale = scale))
# the interarrival times are exponentially distributed with rate lambda.
pp.exp = function (t0) {
# not sure how many Tn are needed :/
Tn = rexp(1000, lambda)
Sn = cumsum(Tn)
return(min(which(Sn > t0)) - 1)
}
# generate N(t) which follow the poisson process.
ns = sapply(1:size, function (i) pp.exp(t))
# generate X(t) as in the problem description.
xs = sapply(ns, function (n) {
ys = c(rgamma(n = n, shape = shape, scale = scale))
sum(ys[1:n])
})
output (t=10) in this case:
> # compare mean and variance of 'size' samples of X(t) for verification.
> # sample:
> (mean.s = mean(xs))
[1] 359.864
> (var.s = var(xs))
[1] 4933.277
> # theoretical:
> (mean.t = lambda * t * shape * scale)
[1] 360
> (var.t = (shape + 1) * shape * scale^2)
[1] 168
The spectrum of my data set shows 3 periodic components in the time serie. I would like to substract the periodic components to keep the data without these periodicities.
It points out periodic events with the periodicity of (1/144 = daily), (1/72 = 1/2daily), and (1/6 = hourly).
My idea was to find out the Fourier components (mag and phase) of my dataset and to extract the Fourier components for these 3 specific frequencies and to create a new signal with is :
Data - PeriodicSignal_1h - PeriodicSignal_1/2day - PeriodicSignal_1day
I try with fft but I do not know how to extract the signal at these specific frequencies.
My dataset is complicated but I'm working on an example to understand the process. Here is the example :
samplingFrequency = 1000;
timeInterval = 1/samplingFrequency;
signalIndex = seq(0, 1, by=timeInterval);
N = 1000
a1 = 2;
a2 = 3;
f1 = 10;
f2 = 20;
signal1 = a1 * sin(2 * pi * f1 * signalIndex);
signal2 = a2 * sin(2 * pi * f2 * signalIndex);
inputSignal = signal1 + signal2;
Y <- fft(inputSignal)
mag <-sqrt(Re(Y)^2+Im(Y)^2)*2/length(inputSignal)
phase <-atan(Im(Y)/Re(Y)) Yr <- Re(Y) Yi <- Im(Y)
I'm trying to extract the mag and the phase of the signal with frequency f1. And I would like to generate a new signal with is :
ImputSignal - Signal_f1
I believe the following does what you are looking for ... I changed some of the variable names. At the bottom is the frequency selection that you asked about.
Set up the time and frequency parameters
samplingFrequency = 1000;
f_Hz = samplingFrequency
N = 1000
df_Hz = f_Hz / N
T = 1 / df_Hz
dt=T/N
t = dt*(seq(1,N)-1)
Generate a fake signal, no noise
a1 = 2;
a2 = 3;
f1 = 10;
f2 = 20;
signal1 = a1 * sin(2 * pi * f1 * t);
signal2 = a2 * sin(2 * pi * f2 * t);
inputSignal = signal1 + signal2;
Plot the fake signal
plot(t, signal1,type='l',col='green',ylim=c(-6,6))
lines(t, signal2,col='red')
lines(t, inputSignal,col='black')
Get the fft, and plot positive frequency portion
Y <- fft(inputSignal)
m <- floor(N/2)-1
posFreqIndices <- 2:(m+1)
negFreqIndices <- N:(m+3)
mag <-sqrt(Re(Y)^2+Im(Y)^2)*2/length(inputSignal)
phase <-atan(Im(Y)/Re(Y))
Yr <- Re(Y)
Yi <- Im(Y)
freq <- seq(df_Hz,f_Hz/2-df_Hz,df_Hz)
plot(freq,mag[posFreqIndices],type='l',xlab='Freq (Hz)', ylab='Magnitude',xlim=c(0,30))
# plot(freq,10*log10(mag[posFreqIndices]),type='l',xlab='Freq (Hz)', ylab='Magnitude (db)',xlim=c(0,30))
# plot(freq,phase[posFreqIndices]*180/pi,type='l',xlab='Freq (Hz)', ylab='Phase (deg)',xlim=c(0,30))
Identify the frequencies for the filtered signal based on amplitude
ampSelectIndices <- which(mag>1.9 & mag < 2.1)
Generate the filtered fft for the selected frequencies
YAmpSelect <- Y*0
YAmpSelect[ampSelectIndices] = Y[ampSelectIndices]
Calculate the inverse fft
yAmpSelect = Re(fft(YAmpSelect, inverse = TRUE))/length(YAmpSelect)
Plot the filtered signal
plot(t,yAmpSelect,t='l',xlab='t (sec)',ylab='Filtered for mag ~ 2')
Plot the original signal minus the filtered signal
plot(t,inputSignal-yAmpSelect,type='l')
The fft is calculated with frequencies folded. The following checks the unfolding process, this check works for real valued signals (not complex value time signals). The process is correct for complex valued time signals.
checkFreqWrapping = all.equal(mag[posFreqIndices], mag[negFreqIndices])
stopifnot(checkFreqWrapping)
Select fft values by frequency
freqSelectIndices_a <- which(9.95 < freq & freq < 10.05)
freqSelectIndices = union(posFreqIndices[freqSelectIndices_a],negFreqIndices[freqSelectIndices_a])
Create the fft for selected frequencies
YFreqSelect <- Y*0
YFreqSelect[freqSelectIndices ] = Y[freqSelectIndices ]
Calculate the time signal, plot it.
yFreqSelect = Re(fft(YFreqSelect, inverse = TRUE))/length(YFreqSelect)
plot(t,yFreqSelect,t='l',xlab='t (sec)',ylab='Filtered for mag ~ 2')
plot(t,inputSignal-yFreqSelect,type='l')
OK, I think that explains how to select fft values based on frequencies... Good luck...
I came across an interesting presentation on page 32, and I started out to replicate and understand a code presented
The code from the presentation is as follows:
#Unicredit banks code
library(evir)
library(fExtremes)
# Quantile function of lognormal-GPD severity distribution
qlnorm.gpd = function(p, theta, theta.gpd, u)
{
Fu = plnorm(u, meanlog=theta[1], sdlog=theta[2])
x = ifelse(p<Fu,
qlnorm( p=p, meanlog=theta[1], sdlog=theta[2] ),
qgpd( p=(p - Fu) / (1 - Fu) , xi=theta.gpd[1], mu=theta.gpd[2], beta=theta.gpd[3]) )
return(x)
}
# Random sampling function of lognormal-GPD severity distribution
rlnorm.gpd = function(n, theta, theta.gpd, u)
{
r = qlnorm.gpd(runif(n), theta, theta.gpd, u)
}
set.seed(1000)
nSim = 1000000 # Number of simulated annual losses
H = 1500 # Threshold body-tail
lambda = 791.7354 # Parameter of Poisson body
theta1 = 2.5 # Parameter mu of lognormal (body)
theta2 = 2 # Parameter sigma of lognormal (body)
theta1.tail = 0.5 # Shape parameter of GPD (tail)
theta2.tail = H # Location parameter of GPD (tail)
theta3.tail = 1000 # Scale parameter of GPD (tail)
sj = rep(0,nSim) # Annual loss distribution inizialization
freq = rpois(nSim, lambda) # Random sampling from Poisson
for(i in 1:nSim) # Convolution with Monte Carlo method
sj[i] = sum(rlnorm.gpd(n=freq[i], theta=c(theta1,theta2), theta.gpd=c(theta1.tail, theta2.tail, theta3.tail), u=H))
However I get this error which I cannot resolve:
Error: min(p, na.rm = TRUE) >= 0 is not TRUE
APPENDED Question
Many thanks to Shadow.
I dont know how to change function reference. Is it as easy as qgpd.fExtremes to qgpd.evir?
Thanks to Shadow again to pointing this out.
For anyone who wishes to change reference to function from different package (In the above example from fExtremes to evir its as simple as adding evir:::(function).
Example:
evir:::qgpd( p=(p - Fu) / (1 - Fu) , xi=theta.gpd[1], mu=theta.gpd[2], beta=theta.gpd[3]) )
The reason you get an error here is that the packages fExtremes and evir both implement different versions of the function qgpd. In the evir version, p can be less than 0, while the fExtremes package only implements qgpd for p>=0.
The easiest solution to this is to change the qgpd function call to evir:::qgpd.