I have a progression of format (b + 1) * 2.
1, 4, 10, 22, 46, 94, 190...
In code I can do it like:
k = 1
for i in range(n):
k += 1
k *= 2
But doing it with loop is a bit inefficient. Is there math formula to find it's n-th member?
If the starting number is b1, then
2-nd term is 2*b1 + 2
3-rd term is 4*b1 + 6
4-th term is 8*b1 + 14
5-nd term is 16*b1 + 30 ..
n-th term is
b1 * 2^(n-1) + 2^n - 2
for your example b0 = 1
b(n) = 3*2^(n-1) - 2
Related
I'm trying to calculate 19v^2 + 49v + 8 to the 67th power over the finite field Z/67Z using Sage where v = sqrt(-2).
Here's what I have so far (using t instead of v):
R.<t> = PolynomialRing(GF(67))
poly = (19 * (t^2) + 49*t + 8)
poly^67
This works fine.
But now, I want to evaluate the resulting polynomial with the value sqrt(-2).
I'm not sure how to proceed. Any ideas?
Computing with a square root of -2 amounts to working modulo
the polynomial t^2 + 2.
The function power_mod can be used for that.
Instead of first powering and then reducing modulo t^2 + 2,
which would be wasteful, it performs the whole powering
process modulo t^2 + 2, which is a lot more efficient.
Here are two ways to write the (same) computation.
sage: t = polygen(GF(67))
sage: p = 19 * t^2 + 49 * t + 8
sage: power_mod(p, 67, t^2 + 2)
sage: R.<t> = GF(67)[]
sage: p = R([8, 49, 19])
sage: power_mod(p, 67, R([2, 0, 1]))
I need perform the matrix multiplicatión between differents layers in a neural network. That is: W0, W1, W2, ... Wn are the weights of the neural netwotk and the input is data. Resulting matrices are:
Out1 = data * W0
Out2 = Out1 * W1
Out3 = Out2 * W2
.
.
.
OutN = Out(N-1) * Wn
I Know the absolute max value in the weights matrices and also I know that the input data range values are from 0 to 1 (input are normalizated). The matrix multiplication is in fixed point with 16 bits. The weights are scalated to the optimal format point. For example: if the absolute maximun value in W0 is 2.5 I know that the minimun number of bits in the integer part is 2 and the bits in fractional part will be 14. Because the data input is in the range [0,1] also I know the integer and fractional bits are 1.15.
My question is: How can I know the mininum number of bits in the integer part in the resultant matrix to avoid overflow? Is there anyway to study and infer the maximun value in a matrix multiplication? I know about determinant and norm of a matrix, but, I think the problem is in the consecutive negatives or positives values in the matrix rows an columns. For example, if I have this row vector and this column vector, and the result is in 8 bits fixed point:
A = [1, 2, 3, 4, 5, 6, -7, -8]
B = [1, 2, 3, 4, 5, 6, 7, 8]
A * B = (1*1) + (2*2) + (3*3) + (4*4) + (5*5) + (6*6) + (7*-7) + (8*8) = 90 - 49 + -68
When the sum accumulator is below than 64, occurs overflow altough the final result be contained between [-64,63].
Another example: If I have have this row vector and this column vector, and the result is in 8 bits fixed point:
A = [1, -2, 3, -4, 5, -6, 7, -8]
B = [1, 2, 3, 4, 5, 6, 7, 8]
A * B = (1*1) - (2*2) + (3*3) - (4*4) + (5*5) - (6*6) + (7*7) - (8*8) = -36
The sum accumulator in any moment exceeds the maximun range for 8 bits.
To sum up: I'm looking for a way to analize the weights matrices to avoid the overflow in the sum accumulator. The way that I do the matrix multiplication is (only a example if matrices A and B has been scalated to 1.15 format):
A1 --> 1.15 bits
B1 --> 1.15 bits
A2 --> 1.15 bits
B2 --> 1.15 bits
mult_1 = (A1 * B1) >> 2^15; // Right shift to alineate the operands
mult_2 = (A2 * B2) >> 2^15; // Right shift to alineate the operands
sum_acc = mult_1 + mult_2; // Sum accumulator
let consider n=100 dimensional dot product (which is part of any matrix multiplication or convolution) of %3.13 fixed point format as an example.
Integer bits
max value in %4.13 is slightly below 2^4 so let consider it would be: 15.999999
Now n dimensional dot product has n multiplications and n-1 additions.
15.999999*15.999999 + 15.999999*15.999999 + .... + 15.999999*15.999999
Each multiplication will sum up the integer bits
15.999999*15.999999 = 255.999999 -> ceil(log2(255)) = 8 = 2*(4)-> %8.13
Now this value is 99 times added so its the same as:
255.999999*99 = 25343.999999 -> ceil(log2(25343)) = 15 = ceil(8+log2(99)) -> %15.13
So if n is number of dimensions and i is number of integer bits the result needs:
i' = ceil((i*2)+log2(n-1))
integer bits... so:
%1.? -> 99*( 1.999999^2) = 395.99 -> % 9.?
%2.? -> 99*( 3.999999^2) = 1583.99 -> %11.?
%3.? -> 99*( 7.999999^2) = 6335.99 -> %13.?
%4.? -> 99*(15.999999^2) = 25343.99 -> %15.?
i(1) = ceil((1*2)+log2(99)) = ceil(2+6.626) = 9
i(2) = ceil((2*2)+log2(99)) = ceil(4+6.626) = 11
i(3) = ceil((3*2)+log2(99)) = ceil(6+6.626) = 13
i(4) = ceil((4*2)+log2(99)) = ceil(8+6.626) = 15
Fractional bits
ok let see what hapens with multiplication:
0.1b^2 = 0.01b -> %?.1 -> %?.2
0.01b^2 = 0.0001b -> %?.2 -> %?.4
0.001b^2 = 0.000001b -> %?.3 -> %?.6
so f' = 2*f where f is number of fractional bits. The addition is not changing the bitwidth:
0.1b*2 = 1.0b -> %?.1 -> %?.1
0.01b*2 = 0.1b -> %?.2 -> %?.2
0.001b*2 = 0.01b -> %?.3 -> %?.3
as the result will not be smaller then operands. So when applying fractional part to the dot product we will have:
i' = ceil((i*2)+log2(n-1))
f' = 2*f
I'm having trouble understanding the meaning of a function f(x) representing the number of operations performed in some code.
int sum = 0; // + 1
for (int i = 0; i < n; i++)
for (int j = 1; j <= i; j++)
sum = sum + 1; // n * (n + 1) / 2
(Please note that there is no 2 in the numerator on that last comment, but there is in the function below.)
Then my notes say that f(x) = 2n(n + 1) / 2 + 1 = O(n^2)
I understand that because there are two for loops, that whatever f(x) is, it will be = O(n^2), but why is the time estimate what it is? How does the j<= i give you n*(n+1)? What about the 2 in the denominator?
Think about, across the entire execution of this code, how many times the inner loop will run. Notice that
when i = 0, it runs zero times;
when i = 1, it runs one time;
when i = 2, it runs two times;
when i = 3, it runs three times;
...; and
when i = n - 1, it runs n - 1 times.
This means that the total number of times the innermost loop runs is given by
0 + 1 + 2 + 3 + 4 + ... + (n - 1)
This is a famous summation and it solves to
0 + 1 + 2 + 3 + 4 + ... + (n - 1) = n(n - 1) / 2
This is the n - 1st triangular number and it's worth committing this to memory.
The number given - n(n + 1) / 2 - seems to be incorrect, but it's pretty close to the true number. I think they assumed the loop would run 1 + 2 + 3 + ... + n times rather than 0 + 1 + 2 + ... + n - 1 times.
From this it's a bit easier to see where the O(n2) term comes from. Notice that n(n - 1) / 2 = n2 / 2 - n / 2, so in big-O land where we drop constants and low-order terms we're left with n2.
Given a pyramid like:
0
1 2
3 4 5
6 7 8 9
...
and given the index of the pyramid i where i represents the ith number of the pyramid, is there a way to find the index of the row to which the ith element belongs? (e.g. if i = 6,7,8,9, it is in the 3rd row, starting from row 0)
There's a connection between the row numbers and the triangular numbers. The nth triangular number, denoted Tn, is given by Tn = n(n-1)/2. The first couple triangular numbers are 0, 1, 3, 6, 10, 15, etc., and if you'll notice, the starts of each row are given by the nth triangular number (the fact that they come from this triangle is where this name comes from.)
So really, the goal here is to determine the largest n such that Tn ≤ i. Without doing any clever math, you could solve this in time O(√n) by just computing T0, T1, T2, etc. until you find something bigger than i. Even better, you could binary search for it in time O(log n) by computing T1, T2, T4, T8, etc. until you overshoot, then binary searching on the range you found.
Alternatively, we could try to solve for this directly. Suppose we want to find the choice of n such that
n(n + 1) / 2 = i
Expanding, we get
n2 / 2 + n / 2 = i.
Equivalently,
n2 / 2 + n / 2 - i = 0,
or, more easily:
n2 + n - 2i = 0.
Now we use the quadratic formula:
n = (-1 ± √(1 + 8i)) / 2
The negative root we can ignore, so the value of n we want is
n = (-1 + √(1 + 8i)) / 2.
This number won't necessarily be an integer, so to find the row you want, we just round down:
row = ⌊(-1 + √(1 + 8i)) / 2⌋.
In code:
int row = int((-1 + sqrt(1 + 8 * i)) / 2);
Let's confirm that this works by testing it out a bit. Where does 9 go? Well, we have
(-1 + √(1 + 72)) / 2 = (-1 + √73) / 2 = 3.77
Rounding down, we see it goes in row 3 - which is correct!
Trying another one, where does 55 go? Well,
(-1 + √(1 + 440)) / 2 = (√441 - 1) / 2 = 10
So it should go in row 10. The tenth triangular number is T10 = 55, so in fact, 55 starts off that row. Looks like it works!
I get row = math.floor (√(2i + 0.25) - 0.5) where i is your number
Essentially the same as the guy above but I reduced n2 + n to (n + 0.5)2 - 0.25
I think ith element belongs nth row where n is number of n(n+1)/2 <= i < (n+1)(n+2)/2
For example, if i = 6, then n = 3 because n(n+1)/2 <= 6
and if i = 8, then n = 3 because n(n+1)/2 <= 8
I have this recurrence formula:
P(n) = ( P(n-1) + 2^(n/2) ) % (X)
s.t. P(1) = 2;
where n/2 is computer integer division i.e. floor of x/2
Since i am taking mod X, this relation should repeat at least with in X outputs.
but it can start repeating before that.
How to find this value?
It needn't repeat within x terms, consider x = 3:
P(1) = 2
P(2) = (P(1) + 2^(2/2)) % 3 = 4 % 3 = 1
P(3) = (P(2) + 2^(3/2)) % 3 = (1 + 2) % 3 = 0
P(4) = (P(3) + 2^(4/2)) % 3 = 4 % 3 = 1
P(5) = (P(4) + 2^(5/2)) % 3 = (1 + 4) % 3 = 2
P(6) = (P(5) + 2^(6/2)) % 3 = (2 + 8) % 3 = 1
P(7) = (P(6) + 2^(7/2)) % 3 = (1 + 8) % 3 = 0
P(8) = (P(7) + 2^(8/2)) % 3 = 16 % 3 = 1
P(9) = (P(8) + 2^(9/2)) % 3 = (1 + 16) % 3 = 2
P(10) = (P(9) + 2^(10/2)) % 3 = (2 + 32) % 3 = 1
P(11) = (P(10) + 2^(11/2)) % 3 = (1 + 32) % 3 = 0
P(12) = (P(11) + 2^(12/2)) % 3 = (0 + 64) % 3 = 1
and you see that the period is 4.
Generally (suppose X is odd, it's a bit more involved for even X), let k be the period of 2 modulo X, i.e. k > 0, 2^k % X = 1, and k is minimal with these properties (see below).
Consider all arithmetic modulo X. Then
n
P(n) = 2 + ∑ 2^(j/2)
j=2
It is easier to see when we separately consider odd and even n:
m m
P(2*m+1) = 2 + 2 * ∑ 2^i = 2 * ∑ 2^i = 2*(2^(m+1) - 1) = 2^((n+2)/2) + 2^((n+1)/2) - 2
i=1 i=0
since each 2^j appears twice, for j = 2*i and j = 2*i+1. For even n = 2*m, there's one summand 2^m missing, so
P(2*m) = 2^(m+1) + 2^m - 2 = 2^((n+2)/2) + 2^((n+1)/2) - 2
and we see that the length of the period is 2*k, since the changing parts 2^((n+1)/2) and 2^((n+2)/2) have that period. The period immediately begins, there is no pre-period part (there can be a pre-period for even X).
Now k <= φ(X) by Euler's generalisation of Fermat's theorem, so the period is at most 2 * φ(X).
(φ is Euler's totient function, i.e. φ(n) is the number of integers 1 <= k <= n with gcd(n,k) = 1.)
What makes it possible that the period is longer than X is that P(n+1) is not completely determined by P(n), the value of n also plays a role in determining P(n+1), in this case the dependence is simple, each power of 2 being used twice in succession doubles the period of the pure powers of 2.
Consider the sequence a[k] = (2^k) % X for odd X > 1. It has the simple recurrence
a[0] = 1
a[k+1] = (2 * a[k]) % X
so each value completely determines the next, thus the entire following part of the sequence. (Since X is assumed odd, it also determines the previous value [if k > 0] and thus the entire previous part of the sequence. With H = (X+1)/2, we have a[k-1] = (H * a[k]) % X.)
Hence if the sequence assumes one value twice (and since there are only X possible values, that must happen within the first X+1 values), at indices i and j = i+p > i, say, the sequence repeats and we have a[k+p] = a[k] for all k >= i. For odd X, we can go back in the sequence, therefore a[k+p] = a[k] also holds for 0 <= k < i. Thus the first value that occurs twice in the sequence is a[0] = 1.
Let p be the smallest positive integer with a[p] = 1. Then p is the length of the smallest period of the sequence a, and a[k] = 1 if and only if k is a multiple of p, thus the set of periods of a is the set of multiples of p. Euler's theorem says that a[φ(X)] = 1, from that we can conclude that p is a divisor of φ(X), in particular p <= φ(X) < X.
Now back to the original sequence.
P(n) = 2 + a[1] + a[1] + a[2] + a[2] + ... + a[n/2]
= a[0] + a[0] + a[1] + a[1] + a[2] + a[2] + ... + a[n/2]
Since each a[k] is used twice in succession, it is natural to examine the subsequences for even and odd indices separately,
E[m] = P(2*m)
O[m] = P(2*m+1)
then the transition from one value to the next is more regular. For the even indices we find
E[m+1] = E[m] + a[m] + a[m+1] = E[m] + 3*a[m]
and for the odd indices
O[m+1] = O[m] + a[m+1] + a[m+1] = O[m] + 2*a[m+1]
Now if we ignore the modulus for the moment, both E and O are geometric sums, so there's an easy closed formula for the terms. They have been given above (in slightly different form),
E[m] = 3 * 2^m - 2 = 3 * a[m] - 2
O[m] = 2 * 2^(m+1) - 2 = 2 * a[m+1] - 2 = a[m+2] - 2
So we see that O has the same (minimal) period as a, namely p, and E also has that period. Unless maybe if X is divisible by 3, that is also the minimal (positive) period of E (if X is divisible by 3, the minimal positive period of E could be a proper divisor of p, for X = 3 e.g., E is constant).
Thus we see that 2*p is a period of the sequence P obtained by interlacing E and O.
It remains to be seen that 2*p is the minimal positive period of P. Let m be the minimal positive period. Then m is a divisor of 2*p.
Suppose m were odd, m = 2*j+1. Then
P(1) = P(m+1) = P(2*m+1)
P(2) = P(m+2) = P(2*m+2)
and consequently
P(2) - P(1) = P(m+2) - P(m+1) = P(2*m+2) - P(2*m+1)
But P(2) - P(1) = a[1] and
P(m+2) - P(m+1) = a[(m+2)/2] = a[j+1]
P(2*m+2) - P(2*m+1) = a[(2*m+2)/2] = a[m+1] = a[2*j+2]
So we must have a[1] = a[j+1], hence j is a period of a, and a[j+1] = a[2*j+2], hence j+1 is a period of a too. But that means that 1 is a period of a, which implies X = 1, a contradiction.
Therefore m is even, m = 2*j. But then j is a period of O (and of E), thus a multiple of p. On the other hand, m <= 2*p implies j <= p, and the only (positive) multiple of p satisfying that inequality is p itself, hence j = p, m = 2*p.