Related
I am trying to make documentation for NeuraNetDiffEq.jl. I find an ODE solution with Flux.jl from this amazing tutorial here https://mitmath.github.io/18S096SciML/lecture2/ml .
using Flux
using DifferentialEquations
using LinearAlgebra
using Plots
using Statistics
NNODE = Chain(x -> [x],
Dense(1, 32, tanh),
Dense(32, 1),
first)
NNODE(1.0)
g(t) = t * NNODE(t) + 1f0
ϵ = sqrt(eps(Float32))
loss() = mean(abs2(((g(t + ϵ) - g(t)) / ϵ) - cos(2π * t)) for t in 0:1f-2:1f0)
opt = Flux.Descent(0.01)
data = Iterators.repeated((), 5000)
iter = 0
cb = function () # callback function to observe training
global iter += 1
if iter % 500 == 0
display(loss())
end
end
display(loss())
Flux.train!(loss, Flux.params(NNODE), data, opt; cb=cb)
t = 0:0.001:1.0
plot(t,g.(t),label="NN")
plot!(t,1.0 .+ sin.(2π .* t) / 2π, label="True")
I am having trouble understanding the parameters involved to invoke training process for NueralNetDiffEq.jl as in:
function DiffEqBase.solve(
prob::DiffEqBase.AbstractODEProblem,
alg::NeuralNetDiffEqAlgorithm,
args...;
dt,
timeseries_errors = true,
save_everystep=true,
adaptive=false,
abstol = 1f-6,
verbose = false,
maxiters = 100)
What would be a valid input for alg parameter? What would be the equivalent code in NeuralNetDiffEq.jl for the above ODE example?
I would like to calculate RSI 14 in line with the tradingview chart.
According to there wiki this should be the solution:
https://www.tradingview.com/wiki/Talk:Relative_Strength_Index_(RSI)
I implemented this is in a object called RSI:
Calling within object RSI:
self.df['rsi1'] = self.calculate_RSI_method_1(self.df, period=self.period)
Implementation of the code the calculation:
def calculate_RSI_method_1(self, ohlc: pd.DataFrame, period: int = 14) -> pd.Series:
delta = ohlc["close"].diff()
ohlc['up'] = delta.copy()
ohlc['down'] = delta.copy()
ohlc['up'] = pd.to_numeric(ohlc['up'])
ohlc['down'] = pd.to_numeric(ohlc['down'])
ohlc['up'][ohlc['up'] < 0] = 0
ohlc['down'][ohlc['down'] > 0] = 0
# This one below is not correct, but why?
ohlc['_gain'] = ohlc['up'].ewm(com=(period - 1), min_periods=period).mean()
ohlc['_loss'] = ohlc['down'].abs().ewm(com=(period - 1), min_periods=period).mean()
ohlc['RS`'] = ohlc['_gain']/ohlc['_loss']
ohlc['rsi'] = pd.Series(100 - (100 / (1 + ohlc['RS`'])))
self.currentvalue = round(self.df['rsi'].iloc[-1], 8)
print (self.currentvalue)
self.exportspreadsheetfordebugging(ohlc, 'calculate_RSI_method_1', self.symbol)
I tested several other solution like e.g but non return a good value:
https://github.com/peerchemist/finta
https://gist.github.com/jmoz/1f93b264650376131ed65875782df386
Therefore I created a unittest based on :
https://school.stockcharts.com/doku.php?id=technical_indicators:relative_strength_index_rsi
I created an input file: (See excel image below)
and a output file: (See excel image below)
Running the unittest (unittest code not included here) should result in but is only checking the last value.
if result == 37.77295211:
log.info("Unit test 001 - PASSED")
return True
else:
log.error("Unit test 001 - NOT PASSED")
return False
But again I cannot pass the test.
I checked all values by help with excel.
So now i'm a little bit lost.
If I'm following this question:
Calculate RSI indicator from pandas DataFrame?
But this will not give any value in the gain.
a) How should the calculation be in order to align the unittest?
b) How should the calculation be in order to align with tradingview?
Here is a Python implementation of the current RSI indicator version in TradingView:
https://github.com/lukaszbinden/rsi_tradingview/blob/main/rsi.py
I had same issue in calculating RSI and the result was different from TradingView,
I have found RSI Step 2 formula described in InvestoPedia and I changed the code as below:
N = 14
close_price0 = float(klines[0][4])
gain_avg0 = loss_avg0 = close_price0
for kline in klines[1:]:
close_price = float(kline[4])
if close_price > close_price0:
gain = close_price - close_price0
loss = 0
else:
gain = 0
loss = close_price0 - close_price
close_price0 = close_price
gain_avg = (gain_avg0 * (N - 1) + gain) / N
loss_avg = (loss_avg0 * (N - 1) + loss) / N
rsi = 100 - 100 / (1 + gain_avg / loss_avg)
gain_avg0 = gain_avg
loss_avg0 = loss_avg
N is the number of period for calculating RSI (by default = 14)
the code is put in a loop to calculate all RSI values for a series.
For those who are experience the same.
My raw data contained ticks where the volume is zero. Filtering this OLHCV rows will directly give the good results.
I am developing an adhoc multinomial logistic model using Julia.
It works fine (although I am sure it could be improved!)
I have written the likelihood function and use Optim to estimate the parameters and the standard errors.
I would like now to develop some robust estimates. Coming from R, I would be using the sandwich package. I did not find any equivalent in Julia. So I could develop something specific I guess.
For this I would need the value of the gradient for each observation (row). I do not find a way to do that using Optim. (When I use gradient!(func, x), I get the sum of the gradients across rows, which is not what I am looking for)
Is there a way to do that using OnceDifferentiable or TwiceDifferentiable ?
Alternatively, is there a package equivalent to R Sandwich that would have escaped from my google researches?
The code I have developed so far:
LLIK_MNL = function(init::Array{Float64})
b = init
u1 = X1*b
u2 = X2*b
u3 = X3*b
umax = max.(u1, u2, u3)
umin = min.(u1, u2, u3)
ucensor = (umax + umin)/2
exu1 = exp.(u1 - ucensor)
exu2 = exp.(u2 - ucensor)
exu3 = exp.(u3 - ucensor)
sexu = exu1 .+ exu2 .+ exu3
Pr=(choice1 .* exu1 + choice2 .* exu2 + choice3 .* exu3) ./ sexu
LL = sum(log.(Pr))
return -LL
end
func = TwiceDifferentiable(var -> LLIK_MNL(var), beta_ini)
opt = optimize(func, beta_ini)
est_MNL = Optim.minimizer(opt)
numerical_hessian = hessian!(func, est_MNL)
var_cov_matrix = inv(numerical_hessian)
temp = diag(var_cov_matrix)
t_stats = est_MNL ./ sqrt.(temp)
pp = 2 * cdf.(Normal(), -abs.(t_stats))
hcat(est_MNL, sqrt.(temp), t_stats, round.(pp, 4))
I have a data-frame consisting of Time Series.
Date Index | Time Series 1 | Time Series 2 | ... and so on
I have used pyRserve to run a forecasting function using R.
I want to implement parallel processing using celery.
I have written the worker code in the following context.
def pipeR(k #input variable):
conn = pyRserve.connect(host = 'localhost', port = 6311)
# OPENING THE CONNECTION TO R
conn.r.i = k
# ASSIGNING THE PYTHON VARIABLE TO THAT OF IN THE R ENVIRONMENT
conn.voideval\('''
WKR_Func <- forecst(a)
{
...# FORECASTS THE TIMESERIES IN COLUMN a OF THE DATAFRAME
}
''')
conn.eval('forecst(i)')
# CALLING THE FUNCTION IN R
group(pipeR.s(k) for k in [...list of column headers...])()
To implement parallel processing, can I have a single port for all the worker process (like I did in the above code, port:6311) or should I have different ports for different worker processes ??
I'm currently getting an error
Error in socketConnection("localhost", port=port, server=TRUE, blocking=TRUE, : cannot open the connection
in R.
The problem got resolved when I opened different ports for each worker process...
def pipeR( k, Frequency, Horizon, Split, wd_path):
# GENERATING A RANDOM PORT
port = randint(1000,9999)
# OPENING THE PORT IN THE R ENVIRONMENT
conn0 = pyRserve.connect(host = 'localhost', port = 6311)
conn0.r.port = port
conn0.voidEval\
('''
library(Rserve)
Rserve(port = port, args = '--no-save')
''')
# OPENING THE PORT IN THE PYTHON ENVIRONMENT
conn = pyRserve.connect(host = 'localhost', port = port)
# ASSIGNING THE PYTHON VARIABLE TO THAT OF IN THE R ENVIRONMENT
conn.r.i = k
conn.voideval\
('''
WKR_Func <- forecst(a)
{
...# FORECASTS THE TIMESERIES IN COLUMN a OF THE DATAFRAME
}
''')
conn.eval/('forecst(i)')
conn0.close()
I have been using (or attempting to, most of the modules I have I have borrowed from other Wikis as my understanding of Lua is limited at best) Lua-based modules and their invoking templates (i.e., the ones that invoke them with {{#invoke:}}) on my Wiki for some time and I have an infobox for chemical compounds (my Chembox) which includes inputs for the numbers of atoms of each chemical element present in the compound (i.e., those numbers used in the compound's molecular formula) and the molar mass of the compound. I was thinking, since it is possible to calculate the molar mass of the compound based on its molecular formula it might be possible to create a Lua module that can do this automatically for me, eliminating the need for me to input the molar mass myself into these infoboxes.
So my question is basically, how do I do this?
Research
My background is in mathematics, so I felt the most straight-forward way of doing this is to set up two vectors, A and B, and perform a dot-product between them. A would contain the user-defined variables, namely, those provided to the template that invokes the module. For example, say we were talking about ethanol (C2H6O) then the template may look like:
{{Molar mass calculator
|C = 2
|H = 6
|O = 1
}}
. B would contain the average atomic mass of each element in grams per mol (g/mol). This vector will be self-provided (i.e., by me, I have a list of 84 elements found in sufficient quantities in nature for these standard atomic weights to be available, with their standard atomic weights also listed. So I will provide them in the Lua code if you just show me where to add them to the code.).
I have looked up dot products in Lua to see if this is possible and how to do them and found this code (http://rosettacode.org/wiki/Dot_product#Lua):
function dotprod(a, b)
local ret = 0
for i = 1, #a do
ret = ret + a[i] * b[i]
end
return ret
end
print(dotprod({1, 3, -5}, {4, -2, 1}))
I believe this is what you are looking for :)
Populate the "AtomicWeightLookup" table with your own values.
There is an example call to the "Calculate" function at the end of the program.
local AtomicWeightLookup = {
C = 12.01,
H = 1.001,
O = 16
}
local function Calculate(Input)
-- Input Example: {C = 2, H = 6, O = 1}
local Result = 0
-- Iterate through Input table
for Element,Quantity in next,Input do
-- If element is not found in table, assume 0 weight.
local AtomicWeight = AtomicWeightLookup[Element] or 0
-- Multiply
Result = Result + Quantity * AtomicWeight
end
return Result
end
-- EXAMPLE
print(Calculate({C = 2, H = 6, O = 1}))
Via emailing the Wikitech-I mailing list (wikitech-I#lists.sourceforge.net) I received an answer. Module:Molar mass calculator is to have the code (the round function is designed to round the end result to the nearest three decimal places, as I later decided that I wanted this capability too):
local p = {};
function p.calc(frame)
local atomicWeights = mw.loadData( 'Module:Standard atomic weight' )
local total = 0
for element, quantity in pairs( frame:getParent().args ) do
if quantity ~= "" then
local atomicWeight = atomicWeights[element] or error( 'Element "' .. element .. '" is not known. Please add it to [[Module:Standard atomic weight]] if it\'s not a typo.' )
quantity = tonumber( quantity ) or error( 'Quantity for ' .. element .. ' is not a number.' )
total = total + atomicWeight * quantity
end
end
function round(num, idp)
local mult = 10^(idp or 0)
return math.floor(num * mult + 0.5) / mult
end
return round(total,3)
end
return p
Module:Standard atomic weight has the code:
local M = {}
M['Ag'] = 107.8682 -- Silver (Ag)
M['As'] = 74.921595 -- Arsenic (As)
M['Au'] = 196.966569 -- Gold (Au)
M['B'] = 10.8135 -- Boron (B)
M['Ba'] = 137.327 -- Barium (Ba)
M['Bi'] = 208.9804 -- Bismuth (Bi)
M['Br'] = 79.904 -- Bromine (Br)
M['C'] = 12.0106 -- Carbon (C)
M['Ca'] = 40.078 -- Calcium (Ca)
M['Cl'] = 35.4515 -- Chlorine (Cl)
M['Co'] = 58.933194 -- Cobalt (Co)
M['Cu'] = 63.546 -- Copper (Cu)
M['F'] = 18.998403163 -- Fluorine (F)
M['Fe'] = 55.845 -- Iron (Fe)
M['Ga'] = 69.723 -- Gallium (Ga)
M['H'] = 1.007975 -- Hydrogen (H)
M['Hg'] = 200.592 -- Mercury (Hg)
M['I'] = 126.90447 -- Iodine (I)
M['K'] = 39.0983 -- Potassium (K)
M['Li'] = 6.9675 -- Lithium (Li)
M['Mg'] = 24.3055 -- Magnesium (Mg)
M['Mn'] = 54.938044 -- Manganese (Mn)
M['N'] = 14.006855 -- Nitrogen (N)
M['Na'] = 22.98976928 -- Sodium (Na)
M['Ni'] = 58.6934 -- Nickel (Ni)
M['O'] = 15.9994 -- Oxygen (O)
M['P'] = 30.973761998 -- Phosphorus (P)
M['Pb'] = 207.2 -- Lead (Pb)
M['Pt'] = 195.084 -- Platinum (Pt)
M['S'] = 32.0675 -- Sulfur (S)
M['Se'] = 78.971 -- Selenium (Se)
M['Sr'] = 87.62 -- Strontium (Sr)
M['Tl'] = 204.3835 -- Thallium (Tl)
M['Zn'] = 65.38 -- Zinc (Zn)
return M
and Template:Molar mass calculator:
{{#invoke:Molar mass calculator|calc}} g/mol