Here is my code in Julia and I would like to improve its speed since it is slow for large dataset. I provided the code with a small example so it can be executed and produce the results. I think that bottleneck is using find function in the loop which causes the code to be very slow but I don't know how I can replace it with sth faster.
A = [[1,2,3,4,5], [2,3,4,5,6,7,8], [4,7,8,9], [9,10], [2,3,4,5]]
mx = maximum(maximum(ar))
idx_new = zeros(Int, mx)
flag = ones(Int, mx);
Hscore = rand(1, length(A))
thresh = 0.2 * sum(Hscore)
acc_q = 0
pos = sortperm(vec(Hscore))
iter = 1
while acc_q < thresh
acc_q = acc_q + Hscore[pos[iter]]
nd = A[pos[iter]]
fd_flag = flag[nd]
cc = in.(fd_flag, 2)
node = nd[findall(x->x==0, cc)]
dd = nd[findall(x->x!=0, cc)]
TF = isempty(dd)
if TF == true
q_val = Hscore[pos[iter]]
acc_q = acc_q + q_val
idx_new[vec(node)] .= (val + 1)
flag[node] .= 2
val = val + 1;
iter = iter + 1
end # end of if TF
end ## end of while loop
While "please improve my code" is not a right question style for StackOverflow, generally when searching many times for element among many many options these are the first two that you might consider:
Sort the list of elements (with sort!) and use searchsorted to find the desired element
Use Set(mylist) to create a hash set and than search within the set.
Related
I am trying to implement ST-HOSVD algorithm in Julia because I could not found library which contains ST-HOSVD.
See this paper in Algorithm 1 in page7.
https://people.cs.kuleuven.be/~nick.vannieuwenhoven/papers/01-STHOSVD.pdf
I cannot reproduce input (4,4,4,4) tensor by approximated tensor whose tucker rank is (2,2,2,2).
I think I have some mistake in indexes of matrix or tensor elements, but I could not locate it.
How to fix it?
If you know library of ST-HOSVD, let me know.
ST-HOSVD is really common way to reduce information. I hope the question helps many Julia user.
using TensorToolbox
function STHOSVD(A, reqrank)
N = ndims(A)
S = copy(A)
Sk = undef
Uk = []
for k = 1:N
if k == 1
Sk = tenmat(S, k)
end
Sk_svd = svd(Sk)
U1 = Sk_svd.U[ :, 1:reqrank[k] ]
V1t = Sk_svd.V[1:reqrank[k], : ]
Sigma1 = diagm( Sk_svd.S[1:reqrank[k]] )
Sk = Sigma1 * V1t
push!(Uk, U1)
end
X = ttm(Sk, Uk[1], 1)
for k=2:N
X = ttm(X, Uk[k], k)
end
return X
end
A = rand(4,4,4,4)
X = X_STHOSVD(A, [2,2,2,2])
EDIT
Here, Sk = tenmat(S, k) is mode n matricization of tensor S.
S∈R^{I_1×I_2×…×I_N}, S_k∈R^{I_k×(Π_{m≠k}^{N} I_m)}
The function is contained in TensorToolbox.jl. See "Basis" in Readme.
The definition of mode-k Matricization can be seen the paper in page 460.
It works.
I have seen 26 page in this slide
using TensorToolbox
using LinearAlgebra
using Arpack
function STHOSVD(T, reqrank)
N = ndims(T)
tensor_shape = size(T)
for i = 1 : N
T_i = tenmat(T, i)
if reqrank[i] == tensor_shape[i]
USV = svd(T_i)
else
USV = svds(T_i; nsv=reqrank[i] )[1]
end
T = ttm( T, USV.U * USV.U', i)
end
return T
end
I would like to calculate RSI 14 in line with the tradingview chart.
According to there wiki this should be the solution:
https://www.tradingview.com/wiki/Talk:Relative_Strength_Index_(RSI)
I implemented this is in a object called RSI:
Calling within object RSI:
self.df['rsi1'] = self.calculate_RSI_method_1(self.df, period=self.period)
Implementation of the code the calculation:
def calculate_RSI_method_1(self, ohlc: pd.DataFrame, period: int = 14) -> pd.Series:
delta = ohlc["close"].diff()
ohlc['up'] = delta.copy()
ohlc['down'] = delta.copy()
ohlc['up'] = pd.to_numeric(ohlc['up'])
ohlc['down'] = pd.to_numeric(ohlc['down'])
ohlc['up'][ohlc['up'] < 0] = 0
ohlc['down'][ohlc['down'] > 0] = 0
# This one below is not correct, but why?
ohlc['_gain'] = ohlc['up'].ewm(com=(period - 1), min_periods=period).mean()
ohlc['_loss'] = ohlc['down'].abs().ewm(com=(period - 1), min_periods=period).mean()
ohlc['RS`'] = ohlc['_gain']/ohlc['_loss']
ohlc['rsi'] = pd.Series(100 - (100 / (1 + ohlc['RS`'])))
self.currentvalue = round(self.df['rsi'].iloc[-1], 8)
print (self.currentvalue)
self.exportspreadsheetfordebugging(ohlc, 'calculate_RSI_method_1', self.symbol)
I tested several other solution like e.g but non return a good value:
https://github.com/peerchemist/finta
https://gist.github.com/jmoz/1f93b264650376131ed65875782df386
Therefore I created a unittest based on :
https://school.stockcharts.com/doku.php?id=technical_indicators:relative_strength_index_rsi
I created an input file: (See excel image below)
and a output file: (See excel image below)
Running the unittest (unittest code not included here) should result in but is only checking the last value.
if result == 37.77295211:
log.info("Unit test 001 - PASSED")
return True
else:
log.error("Unit test 001 - NOT PASSED")
return False
But again I cannot pass the test.
I checked all values by help with excel.
So now i'm a little bit lost.
If I'm following this question:
Calculate RSI indicator from pandas DataFrame?
But this will not give any value in the gain.
a) How should the calculation be in order to align the unittest?
b) How should the calculation be in order to align with tradingview?
Here is a Python implementation of the current RSI indicator version in TradingView:
https://github.com/lukaszbinden/rsi_tradingview/blob/main/rsi.py
I had same issue in calculating RSI and the result was different from TradingView,
I have found RSI Step 2 formula described in InvestoPedia and I changed the code as below:
N = 14
close_price0 = float(klines[0][4])
gain_avg0 = loss_avg0 = close_price0
for kline in klines[1:]:
close_price = float(kline[4])
if close_price > close_price0:
gain = close_price - close_price0
loss = 0
else:
gain = 0
loss = close_price0 - close_price
close_price0 = close_price
gain_avg = (gain_avg0 * (N - 1) + gain) / N
loss_avg = (loss_avg0 * (N - 1) + loss) / N
rsi = 100 - 100 / (1 + gain_avg / loss_avg)
gain_avg0 = gain_avg
loss_avg0 = loss_avg
N is the number of period for calculating RSI (by default = 14)
the code is put in a loop to calculate all RSI values for a series.
For those who are experience the same.
My raw data contained ticks where the volume is zero. Filtering this OLHCV rows will directly give the good results.
Heres my code:
k = str(input())
s = str(input())
x = 0
if len(k)<= len(s):
while x < len(k):
if k[x] == s[x]:
k = k.replace(k[x], "")
s = s.replace(s[x], "")
x = x+1
else:
x = x+1
it says that the else is not working and i don't know how to debugg it. Thank you.
I think this method is much more elegant than the original one, if the strings are very long and performance is important, I suggest to keep in mind that strings in Python are immutable - you can optimize the code by using one of the suggested methods in the following article
k = "String1Avc"
s = "String2Abc"
new_k = ""
new_s = ""
for i in range(min(len(k), len(s))):
if k[i] != s[i]:
new_k += k[i]
new_s += s[i]
print(new_k)
print(new_s)
output:
1v
2b
I am building a metaheuristic in Julia for study purpose.
The purpose is to find the best order of boxes.
1) I start with an initial order (random order) defined as. Order = InitOrder before my while loop.
2) For each iteration in the while loop I set CurrentOrder = Order
3) When the CurrentOrder is changed, Order changes too. Why does Order change value without being assigned? And how do I avoid it?
Version:
JuliaPro 1.0.2.1
Editor: Atom
while ( (time_ns()-timestart)/1.0e9 < RunLength && done == false ) #Stopping Criteria
done = true #Starting point
IterationCount = IterationCount + 1
BestCurrentValue = sum(H) #Worst case solutio
CurrentOrder = Order #(From,To)
for n1=1:N
for n2=1:N
if n1 != n2
(CurrentOrder,CopyTo) = SwapBox(CurrentOrder,n1,n2) #Swap boxes
(CurrentLayout,L) = DeltaCopy(CurrentLayout,CopyTo,CurrentOrder) #Delta Copy to minimise calculations
(TempLayout,L) = BLV(BinW,CurrentLayout,CopyTo,CurrentOrder,W,H,L) #Evalueate by BLV
if L < BestCurrentValue #check if TempLayout is better than Best Current
BestCurrentValue = L
BestCurrentOrder = CurrentOrder
BestCurrentLayout = CurrentLayout
end #if L<...
end #if n1 != n2
##############################################################################
CurrentOrder = Order
##############################################################################
end #n2 in N
end #n1 in N
if BestCurrentValue < BestValue
done = false #Look further
BestValue = BestCurrentValue
BestOrder = BestCurrentOrder
BestLayout = BestCurrentLayout
Order = BestOrder
end #if BestCurrentValue...
end #while
Your assignment NewOrder=Order does not copy any information in memory, it just says that the variable NewOrder should point to the same memory location as Order. Changing one of these variables will thus also change the other. If you want to copy a variable you could use NewOrder=deepcopy(Order)
got a while loop going, and that's working fine.
However I also need to add another condition.
I need the loop to keep going until it satisfies the while loop, but then I also need to add that this can only get repeated x times.
I think you would have to make a for loop to do x times, is it possible to put a while loop in this?
Basically how can I make a loop either reach the goal or stop after x loops??
The expression in while needs to be TRUE for the loop to continue. You can use | or & to add extra conditions. This loop runs 99 times or until sum of random variables is less than 100.
counter <- 0
result <- 0
while(counter < 100 | sum(result) < 100) {
result <- sum(result, rnorm(1, mean = 5, sd = 1))
counter <- sum(counter, 1)
}
> result
[1] 101.5264
> counter
[1] 21
Just pass the current iterator value as an argument to your function. That way you can break the recursion if that reaches a particular value.
But why do you have a while loop if you use recursion, for example:
add_one_recursive = function(number) {
number = number + 1
cat("New number = ", number, "\n")
if(number > 10) {
return(number)
} else {
return(add_one_recursive(number))
}
}
final_number = add_one_recursive(0)
New number = 1
New number = 2
New number = 3
New number = 4
New number = 5
New number = 6
New number = 7
New number = 8
New number = 9
New number = 10
New number = 11
Does not require an explicit loop at all.