Related
How I can calculate accurate energy if I have Power, Current , Voltage values
This is the code of energy calculation, the result's it's wrong so how I can fix that
I want to measure apparent energy, I don't have a problem in V , I, P values
if(millis() >= energyLastSample + 1)
{
energySampleCount = energySampleCount + 1;
energyLastSample = millis();
}
if(energySampleCount >= 1000)
{
apparent_energy_l1 = apparent_power_l1/3600.0;
finalEnergyValue_l1 = finalEnergyValue_l1 + apparent_energy_l1;
apparent_energy_l2 = apparent_power_l2/3600.0;
finalEnergyValue_l2 = finalEnergyValue_l2 + apparent_energy_l2;
apparent_energy_l3 = apparent_power_l3/3600.0;
finalEnergyValue_l3 = finalEnergyValue_l3 + apparent_energy_l3;
// Serial.print(finalEnergyValue,2);
// Serial.println("test");
energySampleCount = 0 ;
}
energy_total= finalEnergyValue_l1+finalEnergyValue_l2+finalEnergyValue_l3;
}
Some tips about power calculation using Arduino or any microcontroller,
open-source code or project,
guidelines to solve my problem
Note that energy (W x t) is a measurement of power over time, while power is a measurement of work, meaning that you cannot simply divide power by 3600 (which would be the factor to convert from seconds to hours) to get an energy value. Power (W) is a measurement of how much work for example a device is currently doing. If you want to calculate the Energy consumed by a device, you will have to continuously measure the Power, for example in 1s intervals, and add it to a counter. Then you have a value which represents Ws - Watt seconds. You can then calculate the Wh consumed from that value.
Example:
You have a device which consumes 300W of power. You keep that device running for exactly 3 hours. If you measure the power consumption every second as described, you will have measured 3240000 Ws. 3240000 Ws / 3600 = 900Wh / 1000 = 0,9 kWh. You can of course change your measurement interval to fit your needs in regard to accuracy.
Pseudocode:
if ( millis() >= lastmillis + 1000 )
{
lastmillis = millis();
wattseconds = wattseconds + power; #increment energy counter by current power
kilowatthours = wattseconds / 3600000;
print(kilowatthours)
}
You could of course use a one second interrupt with an external RTC to get a more accurate timing.
I am learning how to use Discrete Fourier Transform(DFT) to find the period about a^x mod(N), in which x is a positive integer, a is any prime number, and N is the product of two prime factors p and q.
For example, the period of 2^x mod(15) is 4,
>>> for x in range(8):
... print(2**x % 15)
...
Output: 1 2 4 8 1 2 4 8
^-- the next period
and the result of DFT is as following,
(cited from O'Reilly Programming Quantum Computers chapter 12)
There are 4 spikes with 4-unit spacing, and I think the latter 4 means that period is 4.
But, when N is 35 and period is 12
>>> for x in range(16):
... print(2**x % 35)
...
Output: 1 2 4 8 16 32 29 23 11 22 9 18 1 2 4 8
^-- the next period
In this case, there are 8 spikes greater than 100, whose locations are 0, 5, 6, 11, 32, 53, 58, 59, respectively.
Does the location sequence imply the magic number 12? And how to understand "12 evenly spaced spikes" from the righthand graph?
(cited from O'Reilly Programming Quantum Computers chapter 12)
see How to compute Discrete Fourier Transform? and all the sublinks especially How do I obtain the frequencies of each value in an FFT?.
As you can see i-th element of DFT result (counting from 0 to n-1 including) represent Niquist frequency
f(i) = i * fsampling / n
And DFT result uses only those sinusoidal frequencies. So if your signal does have different one (even slightly different frequency or shape) aliasing occurs.
Aliased sinusoid creates 2 frequencies in DFT output one higher and one lower frequency.
Any sharp edge is translated to many frequencies (usually continuous spectrum like your last example)
The f(0) is no frequency and represents DC offset.
On top of all this if the input of your DFT is real domain then the DFT result is symmetric meaning you can use only first half of the result as the second is just mirror image (not including the f(0)) which makes sense as you can not represent bigger than fsampling/2 frequency in real domain data.
Conclusion:
You can not obtain frequency of signal used by DFT as there is infinite number of ways how such signal can be computed. DFT is reconstructing the signal using sinwaves and your signal is definately no sinwave so the results will not match what you think.
Matching niquist frequencies to yours is done by correctly chosing the n for DFT however without knowing the frequency ahead you can not do this ...
It may be possible to compute the singular sinwave frequency from its 2 aliases however your signal is no sinwave so that is not applicable for your case anyway.
I would use different approaches to determine frequency of integer numeric signal:
compute histogram of signal
so count how many of each number there is
test possible frequencies
You can brute force all possible periods of signal and test if consequent periods are the same however for big data is this not optimal...
We can use histogram to speed this up. So if you look at the counts cnt(ix) from histogram for periodic signal of frequency f and period T in data of size n then the period of signal should be a common divider of all the counts
T = n/f
k*f = GCD(all non zero cnt[i])
where k divides the GCD result. However in case n is not exact multiple of T or the signal has noise or slight deviations in it this will not work. However we can at least estimate the GCD and test all frequencies around which will be still faster than brute force.
So for each count (not accounting for noise) it should comply this:
cnt(ix) = ~ n/(f*k)
k = { 1,2,3,4,...,n/f}
so:
f = ~ n/(cnt(ix)*k)
so if you got signal like this:
1,1,1,2,2,2,2,3,3,1,1,1,2,2,2,2,3,3,1
then histogram would be cnt[]={0,7,8,4,0,0,0,0,...} and n=19 so computing f in periods per n for each used element leads to:
f(ix) = n/(cnt(ix)*k)
f(1) = 19/(7*k) = ~ 2.714/k
f(2) = 19/(8*k) = ~ 2.375/k
f(3) = 19/(4*k) = ~ 4.750/k
Now the real frequency should be a common divider (CD) of results so taking biggest and smallest counts rounded up and down (ignoring noise) leads to these options:
f = CD(2,4) = 2
f = CD(3,4) = none
f = CD(2,5) = none
f = CD(3,5) = none
so now test frequency (luckily its just one valid in this case) 2 periods per 19 samples meaning T = ~ 9.5 so test rounded up and down ...
signal(t+ 0)=1,1,1,2,2,2,2,3,3,1,1,1,2,2,2,2,3,3,1
signal(t+ 9)=1,1,1,2,2,2,2,3,3,1 // check 9 elements
signal(t+10)=1,1,2,2,2,2,3,3,1,? // check 10 elements
As you can see signal(t...t+9)==signal(t+9...t+9+9) meaning the period is T=9.
I'm very new to machine learning so I apologize if the answer to this is very obvious.
I'm using a decision tree, using the rpart package, to attempt to predict when a structure fire may result in a fatality using a variety of variables related to that structure fire such as what was the cause, the extent of damage etc.
The chance of a fatality resulting from structure fire is about 1 in 100.
In short I have about 154,000 observations in my training set. I have noticed that when I use the full training set, that the complexity parameter cp has to be reduced all the way down to .0003.
> rpart(Fatality~.,data=train_val,method="class", control=rpart.control(minsplit=50,minbucket = 1, cp=0.00035))
n= 154181
node), split, n, loss, yval, (yprob)
* denotes terminal node
1) root 154181 1881 0 (0.987800053 0.012199947)
2) losscat=Minor_Loss,Med_Loss 105538 567 0 (0.994627528 0.005372472) *
3) losscat=Major_Loss,Total_Loss 48643 1314 0 (0.972986863 0.027013137)
6) HUM_FAC_1=3,6,N, 46102 1070 0 (0.976790595 0.023209405) *
7) HUM_FAC_1=1,2,4,5,7 2541 244 0 (0.903974813 0.096025187)
14) AREA_ORIG=21,24,26,47,72,74,75,76,Other 1846 126 0 (0.931744312 0.068255688)
28) CAUSE_CODE=1,2,5,6,7,8,9,10,12,14,15 1105 45 0 (0.959276018 0.040723982) *
29) CAUSE_CODE=3,4,11,13,16 741 81 0 (0.890688259 0.109311741)
58) FIRST_IGN=10,12,15,17,18,Other,UU 690 68 0 (0.901449275 0.098550725) *
59) FIRST_IGN=00,21,76,81 51 13 0 (0.745098039 0.254901961)
118) INC_TYPE=111,121 48 10 0 (0.791666667 0.208333333) *
119) INC_TYPE=112,120 3 0 1 (0.000000000 1.000000000) *
15) AREA_ORIG=14,UU 695 118 0 (0.830215827 0.169784173)
30) CAUSE_CODE=1,2,4,7,8,10,11,12,13,14,15,16 607 86 0 (0.858319605 0.141680395) *
31) CAUSE_CODE=3,5,6,9 88 32 0 (0.636363636 0.363636364)
62) HUM_FAC_1=1,2 77 24 0 (0.688311688 0.311688312) *
63) HUM_FAC_1=4,5,7 11 3 1 (0.272727273 0.727272727) *
However, when I just grab the first 10,000 observations (no meaningful order) I can now run with a cp of .01
> rpart(Fatality~., data = test, method = "class",
+ control=rpart.control(minsplit=10,minbucket = 1, cp=0.01))
n= 10000
node), split, n, loss, yval, (yprob)
* denotes terminal node
1) root 10000 112 0 (0.988800000 0.011200000)
2) losscat=Minor_Loss,Med_Loss 6889 26 0 (0.996225867 0.003774133) *
3) losscat=Major_Loss,Total_Loss 3111 86 0 (0.972356156 0.027643844)
6) HUM_FAC_1=3,7,N 2860 66 0 (0.976923077 0.023076923) *
7) HUM_FAC_1=1,2,4,5,6 251 20 0 (0.920318725 0.079681275)
14) CAUSE_CODE=1,3,4,6,7,8,9,10,11,14,15 146 3 0 (0.979452055 0.020547945) *
15) CAUSE_CODE=5,13,16 105 17 0 (0.838095238 0.161904762)
30) weekday=Friday,Monday,Saturday,Tuesday,Wednesday 73 6 0 (0.917808219 0.082191781) *
31) weekday=Sunday,Thursday 32 11 0 (0.656250000 0.343750000)
62) AREA_ORIG=21,26,47,Other 17 2 0 (0.882352941 0.117647059) *
63) AREA_ORIG=14,24,UU 15 6 1 (0.400000000 0.600000000)
126) month=2,6,7,9 7 1 0 (0.857142857 0.142857143) *
127) month=1,4,10,12 8 0 1 (0.000000000 1.000000000) *
Why is it that a greater number of observations is resulting in me
having to reduce complexity? Intuitively I would think it should be
opposite.
Is having to reduce cp to .003 "bad"?
Generally, is there any other advice for improving the effectiveness of a decision tree, especially when predicting something that has such low probability in the first place?
cp, from what I read, is a parameter that is used to decide when to stop adding more leaves to the tree (for a node to be considered for another split, the improvement of the relative error by allowing a new split must by more than that cp threshold). Thus, the lower the number, the more leaves it can add. More observations implies that there is an opportunity to lower the threshold, I'm not sure I understand that you "have to" reduce cp... but I could be wrong. If this is a very rare event and your data doesn't lend itself to showing significant improvement in the early stages of the model, it may require that you "increase the sensitivity" by lowering the cp... but you probably know your data better than me.
If you're modeling a rare event, no. If it's not a rare event, the lower your cp the more likely you are to overfit to the bias of your sample. I don't think that minbucket=1 ever leads to a model that is interpretable, either... for similar reasons.
Decision Trees, to me, don't make very much sense beyond 3-4 levels unless you really believe that these hard cuts truly create criteria that justify a final "bucket"/node or a prediction (e.g. if I wanted to bucket you into something financial like a loan or insurance product that fits your risk profile, and my actuaries made hard cuts to split the prospects). After you've split your data 3-4 times, producing a minimum of 8-16 nodes at the bottom of your tree, you've essentially built a model that could be thought of as 3rd or 4th order interactions of independent categorical variables. If you put 20 statisticians (not econo-missed's) in a room and ask them about the number of times they've seen significant 3rd or 4th order interactions in a model, they'd probably scratch their heads. Have you tried any other methods? Or started with dimension reduction? More importantly, what inferences are you trying to make about the data?
I want to calculate the max network throughput on 1G Ethernet link. I understand how to estimate max rate in packets/sec units for 64-bytes frame:
IFG 12 bytes
MAC Preamble 8 bytes
MAC DA 6 bytes
MAC SA 6 bytes
MAC type 2 bytes
Payload 46 bytes
FCS 4 bytes
Total Frame size -> 84 bytes
Now for 1G link we get:
1,000,000,000 bits/sec * 8 bits/byte => 1,488,096 fps
As I understand, this is a data link performance, correct?
But how to calculate throughput in megabits per second for different packets size, i.e. 64,128...1518? Also, how to calculate UDP/TCP throughput, since I have to consider headers overhead.
Thanks.
Max throughput over Ethernet = (Payload_size / (Payload_size + 38)) * Link bitrate
I.e. if you send 50 bytes of payload data, max throughput would be (50 / 88) * 1,000,000,000 for a 1G link, or about 568 Mbit/s. If you send 1000 bytes of payload, max throughput is (1000/1038) * 1,000,000,000 = 963 Mbit/s.
IP+UDP adds 28 bytes of headers, so if you're looking for data throughput over UDP, you should use this formula:
Max throughput over UDP = (Payload_size / (Payload_size + 66)) * Link bitrate
And IP+TCP adds 40 bytes of headers, so that would be:
Max throughput over TCP = (Payload_size / (Payload_size + 78)) * Link bitrate
Note that these are optimistic calculations. I.e. in reality, you might have extra options in the header data that increases the size of the headers, lowering payload throughput. You could also have packet loss that causes performance to drop.
Check out the Wikipedia article on the ethernet frame, and particularly the "Maximum throughput" section:
http://en.wikipedia.org/wiki/Ethernet_frame
On recent interview I was asked the following question. There is a function random2(), wich returns 0 or 1 with equal probability (0.5). Write implementation of random4() and random3() using random2().
It was easy to implement random4() like this
if(random2())
return random2();
return random2() + 2;
But I had difficulties with random3(). The only realization I could represent:
uint32_t sum = 0;
for (uint32_t i = 0; i != N; ++i)
sum += random2();
return sum % 3;
This implementation of random4() is based only my intuition only. I'm not sure if it is correct actually, because I can't mathematically prove its correctness. Can somebody help me with this question, please.
random3:
Not sure if this is the most efficient way, but here's my take:
x = random2 + 2*random2
What can happen:
0 + 0 = 0
0 + 2 = 2
1 + 0 = 1
1 + 2 = 3
The above are all the possibilities of what can happen, thus each has equal probability, so...
(p(x=c) is the probability that x = c)
p(x=0) = 0.25
p(x=1) = 0.25
p(x=2) = 0.25
p(x=3) = 0.25
Now while x = 3, we just keep generating another number, thus giving equal probability to 0,1,2. More technically, you would distribute the probability from x=3 across all of them repeatedly such that p(x=3) tends to 0, thus the probability of the others will tend to 0.33 each.
Code:
do
val = random2() + 2*random2();
while (val != 3);
return val;
random4:
Let's run through your code:
if(random2())
return random2();
return random2() + 2;
First call has 50% chance of 1 (true) => returns either 0 or 1 with 50% * 50% probability, thus 25% each
First call has 50% chance of 0 (false) => returns either 2 or 3 with 50% * 50% probability, thus 25% each
Thus your code generates 0,1,2,3 with equal probability.
Update inspired by e4e5f4's answer:
For a more deterministic answer than the one I provided above...
Generate some large number by calling random2 a bunch of times and mod the result by the desired number.
This won't be exactly the right probability for each, but it will be close.
So, for a 32-bit integer by calling random2 32 times, target = 3:
Total numbers: 4294967296
Number of x's such that x%3 = 1 or 2: 1431655765
Number of x's such that x%3 = 0: 1431655766
Probability of 1 or 2 (each): 0.33333333325572311878204345703125
Probability of 0: 0.3333333334885537624359130859375
So within 0.00000002% of the correct probability, seems pretty close.
Code:
sum = 0;
for (int i = 0; i < 32; i++)
sum = 2*sum + random2();
return sum % N;
Note:
As pjr pointed out, this is, in general, far less efficient than the rejection method above. The probability of getting to the same number of calls of random2 (i.e. 32) (assuming this is the slowest operation) with the rejection method is 0.25^(32/2) = 0.0000000002 = 0.00000002%. This together with the fact that this method isn't exact, gives way more preference to the rejection method. Lower this number decreases the running time, but increases the error, and it would probably need to be lowered quite a bit (thus reaching a high error) to approach the average running time of the rejection method.
It is useful to note the above algorithm has a maximum running time. The rejection method does not. If your random number generator is totally broken for some reason, it could keep generating the rejected number and run for quite a while or forever with the rejection method, but the for-loop above will run 32 times, regardless of what happens.
Using modulo(%) is not recommended because it introduces bias. Mapping will be nice only if n is power of 2. Otherwise some kind of rejection is involved as suggested by other answer.
Another generic approach would be to emulate built-in PRNGs by -
Generate 32 random2() and map it to a 32-bit integer
Get random number in range (0,1) by dividing it by max integer value
Simply multiply this number by n (=3,4...73 so on) and floor to get desired output