I've a mechanism of Encrypting and Decrypting in ASP.NET 4.6 / VB.NET
In Decrypting i use
.......
y="zoSObQFhc+GJSUrnAmagZ2/uuunFrCUv" ' String received
Dim cipherBytes As Byte() = Convert.FromBase64String(y)
Dim msDecrypt As MemoryStream = New MemoryStream(cipherBytes)
Dim csDecrypt As CryptoStream = New CryptoStream(msDecrypt, New TripleDESCryptoServiceProvider().CreateDecryptor(Key, IV), CryptoStreamMode.Read)
Dim fromEncrypt As Byte() = New Byte(cipherBytes.Length - 1) {}
csDecrypt.Read(fromEncrypt, 0, fromEncrypt.Length)
Dim x as string = Encoding.Unicode.GetString(fromEncrypt)
I passed a crypted (with variable y) string and the result is correct and get back the orinal word "procedure" (without quotes) in variable x.
Print x produces "procedure" (without quotes)
Only the Lenght of x is 12.
Now if try to check
if x = "procedure"....
Test is false.
Where i get wrong ?
After applying
System.Text.Encoding.UTF8.GetBytes(x).Select(Function(b) Hex(b)))
Results:
String Decrypted: 70 72 6F 63 65 64 75 72 61 0 0 0
Fixed String 70 72 6F 63 65 64 75 72 61
Following is all process to go and back (Enc/Dec)
String to Encrypt: procedura
{Bytes Length=18}
(0): 112 (1): 0 (2): 114 (3): 0 (4): 111 (5): 0
(6): 99 (7): 0 (8): 101 (9): 0 (10): 100
(11): 0 (12): 117 (13): 0 (14): 114
(15): 0 (16): 97 (17): 0
After Encryption
{Bytes Length=24}
(0): 69 (1): 124 (2): 24 (3): 126
(4): 25 (5): 33 (6): 230 (7): 191
(8): 101 (9): 220 (10): 115 (11): 122
(12): 174 (13): 156
(14): 156 (15): 213 (16): 62 (17): 119
(18): 31 (19): 177 (20): 139 (21): 17
(22): 132 (23): 88
After Convert.ToBase64String:
RXwYfhkh5r9l3HN6rpyc1T53H7GLEYRY
String Received To Decrypt
RXwYfhkh5r9l3HN6rpyc1T53H7GLEYRY
Dim cipherBytes As Byte() = Convert.FromBase64String(Data)
>? cipherBytes
{Bytes Length=24}
(0): 69 (1): 124 (2): 24 (3): 126
(4): 25 (5): 33 (6): 230 (7): 191
(8): 101 (9): 220 (10): 115 (11): 122
(12): 174 (13): 156 (14): 156 (15): 213
(16): 62 (17): 119 (18): 31 (19): 177
(20): 139 (21): 17 (22): 132 (23): 88
After Decryption
{Bytes Length=24}
(0): 112 (1): 0 (2): 114 (3): 0 (4): 111 (5): 0
(6): 99 (7): 0 (8): 101 (9): 0 (10): 100 (11): 0
(12): 117 (13): 0 (14): 114 (15): 0 (16): 97 (17): 0
(18): 0 (19): 0 (20): 0 (21): 0 (22): 0 (23): 0
String Decrypted procedura
Related
> my_query <- paste("select * from", query_table, "where Arrived_Date_Time >=", arrived_earliest_date, "and Arrived_Date_Time < ", arrived_latest_date)
> dfDataIn <- sqlQuery(NSSP, my_query, stringsAsFactors=FALSE)
> odbcCloseAll()
> table(dfDataIn$Discharge_Disposition)
1 2 3 4 5 6 7 8 9 20 21
64059 336 1522 32 306 1166 2343 1 35423 312 36
30 41 43 50 51 61 62 63 64 65 66
26 18 295 133 200 5 270 76 3 1121 811
70 100
249 24
Actually dfDataIn$Discharge_Disposition is a character variable, and most importantly, most 1 here are supposed to be "01" in the database, whereas only minority are truly "1" in the database. (similarly for 2-9)
Is there any way to read the data in the right format?
You could try as.is = TRUE.
dfDataIn <- sqlQuery(NSSP, my_query, as.is = TRUE)
This will bring the data as is from the data source.
I have variable lr_temp and it is RAW(8) type. Making on it RawToHex(lr_temp) I receive string: BDAB59967DF11ECC (8 bytes presented in hex). My problem is to receive integer value for each byte:
dbms_output.put_line(RawToHex(lr_temp));
FOR i IN 1 .. 8
LOOP
dbms_output.put_line(ascii(utl_raw.substr(lr_temp, i, 1)));
END LOOP;
I receive:
BDAB59967DF11ECC
66
65
53
57
55
70
49
67
But BD is 189 not 66, AB is 171 not 65 and so on..
dbms_output.put_line(RawToHex(lr_temp));
FOR i IN 1 .. lcn_data_length
LOOP
dbms_output.put_line(utl_raw.cast_to_binary_integer(utl_raw.substr(lr_temp, i, 1)));
END LOOP;
and the output:
BDAB59967DF11ECC
189
171
89
150
125
241
30
204
What I need:
I have a huge data frame with the following columns (and some more, but these are not important). Here's an example:
user_id video_id group_id x y
1 1 0 0 39 108
2 1 0 0 39 108
3 1 10 0 135 180
4 2 0 0 20 123
User, video and group IDs are factors, of course. For example, there are 20 videos, but each of them has several "observations" for each user and group.
I'd like to transform this data frame into the following format, where there are as many x.N, y.N as there are users (N).
video_id x.1 y.1 x.2 y.2 …
0 39 108 20 123
So, for video 0, the x and y values from user 1 are in columns x.1 and y.1, respectively. For user 2, their values are in columns x.2, y.2, and so on.
What I've tried:
I made myself a list of data frames that are solely composed of all the x, y observations for each video_id:
summaryList = dlply(allData, .(user_id), function(x) unique(x[c("video_id","x","y")]) )
That's how it looks like:
List of 15
$ 1 :'data.frame': 20 obs. of 3 variables:
..$ video_id: Factor w/ 20 levels "0","1","2","3",..: 1 11 8 5 12 9 20 13 7 10 ...
..$ x : int [1:20] 39 135 86 122 28 167 203 433 549 490 ...
..$ y : int [1:20] 108 180 164 103 187 128 185 355 360 368 ...
$ 2 :'data.frame': 20 obs. of 3 variables:
..$ video_id: Factor w/ 20 levels "0","1","2","3",..: 2 14 15 4 20 6 19 3 13 18 ...
..$ x : int [1:20] 128 688 435 218 528 362 299 134 83 417 ...
..$ y : int [1:20] 165 117 135 179 96 328 332 563 623 476 ...
Where I'm stuck:
What's left to do is:
Merge each data frame from the summaryList with each other, based on the video_id. I can't find a nice way to access the actual data frames in the list, which are summaryList[1]$`1`, summaryList[2]$`2`, et cetera.
#James found out a partial solution:
Reduce(function(x,y) merge(x,y,by="video_id"),summaryList)
Ensure the column names are renamed after the user ID and not kept as-is. Right now my summaryList doesn't contain any info about the user ID, and the output of Reduce has duplicate column names like x.x y.x x.y y.y x.x y.x and so on.
How do I go about doing this? Or is there any easier way to get to the result than what I'm currently doing?
I am still somewhat confused. However, I guess you simply want to melt and dcast.
library(reshape2)
d <- melt(allData,id.vars=c("user_id","video_id"), measure.vars=c("x","y"))
dcast(d,video_id~user_id+variable,value.var="value",fun.aggregate=mean)
Resulting in:
video_id 1_x 1_y 2_x 2_y 3_x 3_y 4_x 4_y 5_x 5_y 6_x 6_y 7_x 7_y 8_x 8_y 9_x 9_y 10_x 10_y 11_x 11_y 12_x 12_y 14_x 14_y 15_x 15_y 16_x 16_y
1 0 39 108 899 132 61 357 149 298 1105 415 148 208 442 200 210 134 58 244 910 403 152 52 1092 617 1012 114 1105 424 548 394
2 1 1125 70 128 165 1151 390 171 587 623 623 80 643 866 310 994 114 854 129 781 306 672 -1 1096 354 525 524 150
Reduce does the trick:
reducedData <- Reduce(function(x,y) merge(x,y,by="video_id"),summaryList)
… but you need to fix the names afterwards:
names(reducedData)[-1] <- do.call(function(...) paste(...,sep="."),expand.grid(letters[24:25],names(summaryList)))
The result is:
video_id x.1 y.1 x.2 y.2 x.3 y.3 x.4 y.4 x.5 y.5 x.6 y.6 x.7 y.7 x.8
1 0 39 108 899 132 61 357 149 298 1105 415 148 208 442 200 210
2 1 1125 70 128 165 1151 390 171 587 623 623 80 643 866 310 994
My objective is to list the drift coefficient from a random walk with drift forecast function, applied to a set of historical data (below). Specifically I am trying to gather the drift coefficient starting from the random walk with drift model of the first year, then cumulatively to the last, recording the coefficient each time, meaning iteratively or each additional year (recording this into a list? if that is appropriate). To be clear each new random walk forecast is including all the previous years.
The data is a list of 241 consumption levels, and I am attempting to discern how the drift coefficent would change over the course of iteratively progressing from n=1 to n=241
Where for example the random walk with drift model is Y[t] = c + Y[t-1] + Z[t] where Z[t] is a normal error and c is the coefficient i am looking for. My current attempts at this involve a for loop function and extracting the c coefficient from the rwf() function from the "Forecast" package in R.
To extract this, I am doing as such
rwf(x, h = 1, drift = TRUE)$model[[1]]
which extracts the drift coefficient.
The problem is, my attempts at subsetting the data within the rwf call have failed, and I also don't believe, through trial and error and research, that rwf() supports the subset argument, as an lm model does for example. In this sense my attempts at looping the function have also failed.
An example of such code is
for (i in 1:5){print((rwf(x[1:i], h = 1, drift = TRUE))$model[[1]])}
which gives me the following error
Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) :
0 (non-NA) cases
In addition: Warning message:
In is.na(rows) : is.na() applied to non-(list or vector) of type 'NULL'
Any help would be much appreciated.
I read SO a lot for help but this is my first time asking a question.
The data is as follows
PCE
1 1306.7
2 1309.6
3 1335.3
4 1341.8
5 1389.2
6 1405.7
7 1414.2
8 1411.0
9 1401.6
10 1406.7
11 1425.0
12 1444.4
13 1474.7
14 1507.8
15 1536.6
16 1555.6
17 1575.2
18 1577.8
19 1583.0
20 1586.6
21 1608.4
22 1619.5
23 1622.4
24 1635.3
25 1636.1
26 1613.9
27 1627.1
28 1653.8
29 1675.6
30 1706.7
31 1732.9
32 1751.0
33 1752.9
34 1769.7
35 1792.1
36 1785.0
37 1787.4
38 1786.9
39 1813.4
40 1822.2
41 1858.7
42 1878.5
43 1901.6
44 1917.0
45 1944.2
46 1957.3
47 1976.0
48 2002.9
49 2019.6
50 2059.5
51 2095.8
52 2134.3
53 2140.2
54 2187.8
55 2212.0
56 2250.0
57 2313.2
58 2347.4
59 2353.5
60 2380.4
61 2390.3
62 2404.2
63 2437.0
64 2449.5
65 2464.6
66 2523.4
67 2562.1
68 2610.3
69 2622.3
70 2651.7
71 2668.6
72 2681.5
73 2702.9
74 2719.5
75 2731.9
76 2755.9
77 2748.4
78 2800.9
79 2826.6
80 2849.1
81 2896.5
82 2935.2
83 2991.2
84 3037.4
85 3108.6
86 3165.5
87 3163.9
88 3175.3
89 3166.0
90 3138.3
91 3149.2
92 3162.2
93 3115.8
94 3142.0
95 3194.4
96 3239.9
97 3274.2
98 3339.6
99 3370.3
100 3405.9
101 3450.3
102 3489.7
103 3509.0
104 3542.5
105 3595.9
106 3616.9
107 3694.2
108 3709.7
109 3739.6
110 3758.5
111 3756.3
112 3793.2
113 3803.3
114 3796.7
115 3710.5
116 3750.3
117 3800.3
118 3821.1
119 3821.1
120 3836.6
121 3807.6
122 3832.2
123 3845.9
124 3875.4
125 3946.1
126 3984.8
127 4063.9
128 4135.7
129 4201.3
130 4237.3
131 4297.9
132 4331.1
133 4388.1
134 4462.5
135 4503.2
136 4588.7
137 4598.8
138 4637.2
139 4686.6
140 4768.5
141 4797.2
142 4789.9
143 4854.0
144 4908.2
145 4920.0
146 5002.2
147 5038.5
148 5078.3
149 5138.1
150 5156.9
151 5180.0
152 5233.7
153 5259.3
154 5300.9
155 5318.4
156 5338.6
157 5297.0
158 5282.0
159 5322.2
160 5342.6
161 5340.2
162 5432.0
163 5464.2
164 5524.6
165 5592.0
166 5614.7
167 5668.6
168 5730.1
169 5781.1
170 5845.5
171 5888.8
172 5936.0
173 5994.6
174 6001.6
175 6050.8
176 6104.9
177 6147.8
178 6204.0
179 6274.2
180 6311.8
181 6363.2
182 6427.3
183 6453.3
184 6563.0
185 6638.1
186 6704.1
187 6819.5
188 6909.9
189 7015.9
190 7085.1
191 7196.6
192 7283.1
193 7385.8
194 7497.8
195 7568.3
196 7642.4
197 7710.0
198 7740.8
199 7770.0
200 7804.2
201 7926.4
202 7953.7
203 7994.1
204 8048.3
205 8076.9
206 8117.7
207 8198.1
208 8308.5
209 8353.7
210 8427.6
211 8465.1
212 8539.1
213 8631.3
214 8700.1
215 8786.2
216 8852.9
217 8874.9
218 8965.8
219 9019.8
220 9073.9
221 9158.3
222 9209.2
223 9244.5
224 9285.2
225 9312.6
226 9289.1
227 9285.8
228 9196.0
229 9076.0
230 9040.9
231 8998.5
232 9050.3
233 9060.2
234 9121.2
235 9186.9
236 9247.1
237 9328.4
238 9376.7
239 9392.7
240 9433.5
241 9482.1
You need at least two points to fit your model. Here's how I'd approach the problem after reading your data into a data.frame named x:
library(forecast)
drifts <- sapply(2:nrow(x), function(zz) rwf(x[1:zz,], drift = TRUE)$model$drift)
I'm not sure if this is what you were expecting or not, but here's a plot of your drift values:
I had thought that R had a standard overhead for storing objects (24 bytes, it seems, at least for integer vectors), but a simple test revealed that it's more complex than I realized. For instance, taking integer vectors up to length 100 (using random sampling, hoping to avoid any sneaky sequence compression tricks that might be out there), I found that different length vectors could have the same size, as follows:
> N = 100
> V = vector(length = 100)
> for(L in 1:N){
+ z = sample(N, L, replace = TRUE)
+ V[L] = object.size(z)
+ }
>
> options('width'=88)
> V
[1] 48 48 56 56 72 72 72 72 88 88 88 88 104 104 104 104 168 168 168 168
[21] 168 168 168 168 168 168 168 168 168 168 168 168 176 176 184 184 192 192 200 200
[41] 208 208 216 216 224 224 232 232 240 240 248 248 256 256 264 264 272 272 280 280
[61] 288 288 296 296 304 304 312 312 320 320 328 328 336 336 344 344 352 352 360 360
[81] 368 368 376 376 384 384 392 392 400 400 408 408 416 416 424 424 432 432 440 440
I'm very impressed by the 152 values that shows up (observation: 152 = 128 + 24, though 280 = 256 + 24 isn't as prominent). Can someone explain how these allocations arise? I have been unable to find a clear definition in the documentation, though V cells come up.
Even if you try N <- 10000, all values occur exactly twice, except for vectors of length :
5 to 8 (56 bytes)
9 to 12 (72 bytes)
13 to 16 (88 bytes)
17 to 32 (152 bytes)
The fact that the number of bytes occurs twice, comes from the simple fact that the memory is allocated in pieces of 8 bytes (referred to as Vcells in ?gc ) and integers take only 4 bytes.
Next to that, the internal structure of objects in R makes a distinguishment between small and large vectors for allocating memory. Small vectors are allocated in bigger blocks of about 2Kb, whereas larger vectors are allocated individually. The ‘small’ vectors consist of 6 defined classes, based on length, and are able to store vector data of up to 8, 16, 32, 48, 64 and 128 bytes. As an integer takes only 4 bytes, you have 2, 4, 8, 12, 16 and 32 integers you can store in these 6 classes. This explains the pattern you see.
The extra number of bytes is for the header (which forms the Ncells in ?gc). If you're really interested in all this, read through the R Internals manual.
And, as you guessed, the 24 extra bytes are from the headers (or Ncells ). It's in fact a bit more complicated than that, but the exact details can be found in the R internals manual