Is there a way to figure out the encoding scheme Ethereum uses? It looks like bitcoin used Base 64 encoding as shown here . Just trying to figure out regarding the Ethereum
From here
Recursive Length Prefix (RLP) serialization is used extensively in
Ethereum's execution clients. RLP standardizes the transfer of data
between nodes in a space-efficient format. The purpose of RLP is to
encode arbitrarily nested arrays of binary data, and RLP is the
primary encoding method used to serialize objects in Ethereum's
execution layer. The only purpose of RLP is to encode structure;
encoding specific data types (e.g. strings, floats) is left up to
higher-order protocols; but positive RLP integers must be represented
in big-endian binary form with no leading zeroes (thus making the
integer value zero equivalent to the empty byte array). Deserialized
positive integers with leading zeroes get treated as invalid. The
integer representation of string length must also be encoded this way,
as well as integers in the payload.
Related
I deal with lots of mathematical expressions in a certain Julia script and would like to know if storing such a formula as a String is ok, or whether using the Symbol data type is better. Thinking about scalability and keeping memory requirements to a minimum. Thanks!
Update: the application involves a machine learning model. Ideally, it should be applicable to big data too, hence the need for scalability.
In a string, each character is stored based on its number of codeunits, eg. 1 for ascii. The same is true for the characters of a Symbol. So that is a wash; do what fits your use best, probably Symbols since you are manipulating expressions.
An expression like :(x + y) is stored as a list of Any, with space allocated according to the sizeof each item in the expression.
In an expression like :(7 + 4 * 9) versus a string like "7 + 4 * 9" there are two conflicting issues. First, 7 is stored as 1 byte in the string, but 8 bytes in the expression since there are 64-bit Ints in play. On the other hand, whitespace takes up 1 byte each space in the string, but does not use memory in the expression. And a number like 123.123456789 takes up 14 bytes in the string and 8 in the expression (64 bit floats).
I think that, again, this is close to being even, and depends on the specific strings you are parsing. You could, as you work with the program, store both, compare memory usage of the resulting arrays, and drop one type of storage if you feel you should.
I have a question about aes keys.
I have a binary file which contains an aes256 key (32 bytes) at an unknown offset.
Would it be somehow possible to find this key in the file? Is it somehow possible to tell whether the next 32 bytes would be a valid aes key?
Thanks in advance
EDIT:
Thanks for all of your answers,
The key is stored in the file as normal bytes.
I finally managed to create a way to get it.
I basically filter out all strings, which actually made it work.
Thanks again
Well, yes and no. AES-256 keys should consist of just 32 bytes that are indistinguishable from random. Most files do not consist of just random bytes, so it could be possible t find a sequence that is most likely random, and this could be that key you are looking for. However, it might very well be that there are other random sequences in the file, or sequences that look like random but aren't random at all (such as the binary representation of the number Pi).
It may also be that you are unlucky and that the AES key doesn't look all that random. Or that the key is stored in hexadecimals (text) rather than binary byte values. Then there is the issue of finding the exact offset that might be the problem (is that initial byte with value 0x20 indicating the size of the AES key, a space character or part of the key value)?
Most files have a specific format, so you should first have a look at that. Just looking for random sequences may give you both false positives (rather likely) or false negatives (less likely). If you expect 64 bytes of randomness (two keys) then I suggest you search for that first, as it brings down the chance of false positives by a rather large amount.
No - unless you have a way to verify the key against a known plaintext/ciphertext pair - an AES key is not distinguishable from random noise. Any set of 16, 24 or 32 bytes is a valid AES key.
I am trying to find a straightforward way to store negative values in EEPROM, integer values ranging from -20 to 20. I have been using EEPROM.write and EEPROM.read functions to store strings one character at a time, but I am having trouble with negative numbers. I figure I only need one byte for this value.
It's just matter of number representation. You just have to use correct data types to print or use:
Version 1: int8_t data = EEPROM.read(addr);
Version 2:
byte data = EEPROM.read(addr);
Serial.print((int8_t)data);
EEPROM.write can be used directly with int8_t: EEPROM.write(int8_value);
Or, if you wan't int, put/get methods can be used for it (even for structs containing POD types only or so)
I want to encrypt and decrypt ASCII messages using an RSA algorithm written in assembly.
I read that for security and efficiency reasons the encryption is normally not called character-wise but a number of characters is grouped and encrypted together (e.g. wikipedia says that 3 chars are grouped).
Let us assume that we want to encrypt the message "aaa" grouping 2 characters.
"aaa" is stored as 61616100.
If we group two characters and encrypt the resulting halfwords the result for the 6161 block can in fact be something like 0053. This will result in an artificial second '\0' character which corrupts the resulting message.
Is there any way to work around this problem?
Using padding or anything similar is unfortunately not an option since I am required to use the same function for encrypting and decrypting.
The output of RSA is a number. Usually this number is encoded as an octet string (or byte array). You should not treat the result as a character string. You need to treat it as a byte array with the same length as the modulus (or at least the length of the modulus in bytes).
Besides the result containing a zero (null-terminator) the characters may have any value, including non-printable characters such as control characters and 7F. If you want to treat the result as a printable string, convert to hex or base64.
I searched a lot to find an encryption algorithm which its encrypted results do not include slash character. Anything I've tested so far (like this, this and this) generate strings which include slash character and therefore they make asp.net (web forms) routing misunderstand the way it should interpret the route.
Can you please help by introducing a symmetric encryption algorithm which generate encrypted strings that can safely be used for encrypting query strings without misguiding asp.net routing?
Encryption algorithms generally produce random (looking) bytes. These bytes can have any value. You can encode this value, for instance using hexadecimals or base 64. With hexadecimals you have already code that only contains 0..9 and a..f (in upper or lower case). However, hexadecimal encoding is not very efficient, doubling the ciphertext.
Base 64 uses 64 characters: A..Z, a..z, 0..9, + and /, and sometimes a padding character =. It is however very easy to replace the URL unsafe + and / characters with other ones, e.g. - and _ according to RFC 4648. You can also remove any = characters at the end, although you may have to put them back (until you get a multiple of 4 base 64 characters) depending on the base 64 decoding routine. Base 64 uses 4 characters for 3 bytes, so it expands the ciphertext by 33%.