In golang, can I print the value of a memory address from a given string?
For example, if run the following code:
a := "A String"
fmt.Println(&a)
It prints 0x1040c108.
How could I take a string such as 0x1040c108 and print the value of that string stored in the memory? Something like fmt.Println(*0x1040c108)
Is this possible?
This can be done, but it is a really really REALLY bad idea. Anytime you are importing the unsafe package, you are either doing something wrong, or something really hardcore. I'm hesitant to even answer this, but here goes.
https://play.golang.org/p/unkb-s8IzAo
package main
import (
"fmt"
"strconv"
"unsafe"
)
func main() {
// original example manually examined the printed address and used the value
// updated to preserve forward compatibility due to runtime changes shifting the address over time
hi := "HI"
// getting address as string dynamically to preserve compatibility
address := fmt.Sprint(&hi)
fmt.Printf("Address of var hi: %s\n", address)
// convert to uintptr
var adr uint64
adr, err := strconv.ParseUint(address, 0, 64)
if err != nil {
panic(err)
}
var ptr uintptr = uintptr(adr)
fmt.Printf("String at address: %s\n", address)
fmt.Printf("Value: %s\n", ptrToString(ptr))
}
func ptrToString(ptr uintptr) string {
p := unsafe.Pointer(ptr)
return *(*string)(p)
}
And yes, this was pretty much taken almost line for line from the unsafe godoc. https://godoc.org/unsafe
Also note that if/when your memory reference is NOT a go string, everything will come crashing down catastrophically. And that go vet is configured to send you an angry message for doing this, reinforcing that this is indeed a bad idea.
UPDATE: Updated example to run on playground as of go 1.15.1, which either the playground or go itself has changed the way the memory is addressed. Or the more likely case that changes in core libs/runtime will shift the address across versions. It now dynamically obtains the address vs a manually hardcoded value.
package main
import "C"
import (
"log"
"strconv"
"unsafe"
)
func main() {
// parse the string into an integer value
addr, _ := strconv.ParseInt("0x1040c108", 0, 64)
// cast the integer to a c string pointer
ptr := (*C.char)(unsafe.Pointer(uintptr(addr)))
// convert to a go string (this will segfault)
str := C.GoString(ptr)
// print it
log.Println(str)
}
Yes!! you can store the address in a pointer variable and print its value by derefrencing it
i := "something"
ptr := &i
fmt.Println(*ptr)
For accessing the memory using a hard coded address such as 0x1040c108, it is necessary for your program to have access to that memory address otherwise, you will get an error saying invalid indirection of a pointer or segmentation fault.
I'm trying to encrypt some data in go but it's hardly ever the correct cipher.BlockSize.
Is there a "built-in" way to add padding or should I be using a function to add it manually?
This is my solution now:
// encrypt() encrypts the message, but sometimes the
// message isn't the proper length, so we add padding.
func encrypt(msg []byte, key []byte) []byte {
cipher, err := aes.NewCipher(key)
if err != nil {
log.Fatal(err)
}
if len(msg) < cipher.BlockSize() {
var endLength = cipher.BlockSize() - len(msg)
ending := make([]byte, endLength, endLength)
msg = append(msg[:], ending[:]...)
cipher.Encrypt(msg, msg)
} else {
var endLength = len(msg) % cipher.BlockSize()
ending := make([]byte, endLength, endLength)
msg = append(msg[:], ending[:]...)
cipher.Encrypt(msg, msg)
}
return msg
}
Looking at Package cipher it appears like you may have to add the padding yourself, see PKCS#7 padding.
Essentially add the required padding bytes with the value of each byte the number of padding byte added.
Note that you need to add padding consistently and that means that if the data to be encrypted is an exact multiple of the block size an entire block of padding must be added since there is no way to know from the data if padding has been added or not, it is a common mistake to try to out-smart this. Consider if the last byte is 0x00, is that padding or data?
here's my solution
// padOrTrim returns (size) bytes from input (bb)
// Short bb gets zeros prefixed, Long bb gets left/MSB bits trimmed
func padOrTrim(bb []byte, size int) []byte {
l := len(bb)
if l == size {
return bb
}
if l > size {
return bb[l-size:]
}
tmp := make([]byte, size)
copy(tmp[size-l:], bb)
return tmp
}
In the code below (also at http://play.golang.org/p/77fRvrDa4A but takes "too long to process" in the browser there) the 124 byte version of the sourceText won't encrypt because: "message too long for RSA public key size" of 1024. It, and the longer 124 byte sourceText version, work with 2048 bit key size.
My question is how does one exactly calculate the key size in rsa.GenerateKey given the byte length of the source text? (A small paragraph size of text takes nearly 10 seconds at 4096 key size, and I don't know the length of the sourceText until runtime.)
There's a very brief discussion of this at https://stackoverflow.com/a/11750658/3691075, but it's not clear to me as I'm not a crypto guy.
My goal is to encrypt, store in a DB and decrypt about 300-byte long JSON strings. I control both the sending and the receiving end. Text is encrypted once, and decrypted many times. Any hints of strategy would be appreciated.
package main
import (
"crypto/md5"
"crypto/rand"
"crypto/rsa"
"fmt"
"hash"
"log"
"time"
)
func main() {
startingTime := time.Now()
var err error
var privateKey *rsa.PrivateKey
var publicKey *rsa.PublicKey
var sourceText, encryptedText, decryptedText, label []byte
// SHORT TEXT 92 bytes
sourceText = []byte(`{347,7,3,8,7,0,7,5,6,4,1,6,5,6,7,3,7,7,7,6,5,3,5,3,3,5,4,3,2,10,3,7,5,6,65,350914,760415,33}`)
fmt.Printf("\nsourceText byte length:\n%d\n", len(sourceText))
// LONGER TEXT 124 bytes
// sourceText = []byte(`{347,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,65,350914,760415,33}`)
// fmt.Printf("\nsourceText byte length:\n%d\n", len(sourceText))
if privateKey, err = rsa.GenerateKey(rand.Reader, 1024); err != nil {
log.Fatal(err)
}
// fmt.Printf("\nprivateKey:\n%s\n", privateKey)
privateKey.Precompute()
if err = privateKey.Validate(); err != nil {
log.Fatal(err)
}
publicKey = &privateKey.PublicKey
encryptedText = encrypt(publicKey, sourceText, label)
decryptedText = decrypt(privateKey, encryptedText, label)
fmt.Printf("\nsourceText: \n%s\n", string(sourceText))
fmt.Printf("\nencryptedText: \n%x\n", encryptedText)
fmt.Printf("\ndecryptedText: \n%s\n", decryptedText)
fmt.Printf("\nDone in %v.\n\n", time.Now().Sub(startingTime))
}
func encrypt(publicKey *rsa.PublicKey, sourceText, label []byte) (encryptedText []byte) {
var err error
var md5_hash hash.Hash
md5_hash = md5.New()
if encryptedText, err = rsa.EncryptOAEP(md5_hash, rand.Reader, publicKey, sourceText, label); err != nil {
log.Fatal(err)
}
return
}
func decrypt(privateKey *rsa.PrivateKey, encryptedText, label []byte) (decryptedText []byte) {
var err error
var md5_hash hash.Hash
md5_hash = md5.New()
if decryptedText, err = rsa.DecryptOAEP(md5_hash, rand.Reader, privateKey, encryptedText, label); err != nil {
log.Fatal(err)
}
return
}
One does not usually calculate the RSA key size based on payload. One simply needs to select one RSA key size based on a compromise between security (bigger is better) and performance (smaller is better). If that is done, use hybrid encryption in conjunction with AES or another symmetric cipher to actually encrypt the data.
If the payload doesn't exceed 300 bytes and you're using OAEP (at least 42 bytes of padding), then you can easily calculate the minimum key size:
(300 + 42) * 8 = 2736 bit
That's already a reasonable size key. It provides good security according to today's norms and is fairly fast. There is no need to apply a hybrid encryption scheme for this.
Now, you may notice that the key size isn't a power of 2. This is not a problem. You should however use a key size that is a multiple of 64 bit, because processors use 32-bit and 64-bit primitives to do the actual calculation, so you can increase the security without a performance penalty. The next such key size would be:
ceil((300 + 42) * 8 / 64.0) * 64 = 2752 bit
Here are some experimental results what some languages/frameworks accept (not performance-wise) as the key size:
Golang: multiple of 1 bit and >= 1001 (sic!) [used ideone.com]
PyCrypto: multiple of 256 bit and >= 1024 [local install]
C#: multiple of 16 bit and >= 512 [used ideone.com]
Groovy: multiple of 1 bit and >= 512 [local install]
Java: multiple of 1 bit and >= 512 [used ideone.com: Java & Java7]
PHP/OpenSSL Ext: multiple of 128 bit and >= 640 [used ideone.com]
Crypto++: multiple of 1 bit and >= 16 [local install with maximal validation toughness of 3]
Before you decide to use some kind of specific key size, you should check that all frameworks support that size. As you see, there are vastly varying results.
I tried to write some performance tests of key generation, encryption and decryption with different key sizes: 512, 513, 514, 516, 520, 528, 544, 576. Since I don't know any go, it would be hard to get the timing right. So I settled for Java and Crypto++. The Crypto++ code is probably very buggy, because key generation for 520-bit and 528-bit keys is up to seven orders of magnitude faster than for the other key sizes which is more or less constant for the small key size window.
In Java the key generation was pretty clear in that the generation of a 513-bit key was 2-3 times slower than for a 512-bit key. Other than that the results are nearly linear. The graph is normalized and the numbers of iterations is 1000 for the full keygen-enc-dec cycle.
The decryption makes a little dip at 544-bit which is a multiple of 32-bit. Since it was executed on a 32-bit debian, this might mean that indeed there are some performance improvements, but on the other hand the encryption was slower for that key size.
Since this benchmark wasn't done in Go, I won't give any advice on how small the overhead can be.
I have a problem with decryption when I try to decrypt the same byte slice again.
Example of code for clarification:
package main
import (
"fmt"
"crypto/cipher"
"crypto/des"
)
const (
// tripleKey is TripleDES key string (3x8 bytes)
tripleKey = "12345678asdfghjkzxcvbnmq"
)
var (
encrypter cipher.BlockMode
decrypter cipher.BlockMode
)
func init() {
// tripleDESChiper is chiper block based on tripleKey used for encryption/decryption
tripleDESChiper, err := des.NewTripleDESCipher([]byte(tripleKey))
if err != nil {
panic(err)
}
// iv is Initialization Vector used for encrypter/decrypter creation
ciphertext := []byte("0123456789qwerty")
iv := ciphertext[:des.BlockSize]
// create encrypter and decrypter
encrypter = cipher.NewCBCEncrypter(tripleDESChiper, iv)
decrypter = cipher.NewCBCDecrypter(tripleDESChiper, iv)
}
func main() {
message := "12345678qwertyuia12345678zxcvbnm,12345678poiuytr"
data := []byte(message)
hash := encrypt(data)
decoded1 := decrypt(hash)
decoded2 := decrypt(hash)
decoded3 := decrypt(hash)
decoded4 := decrypt(hash)
fmt.Printf("encrypted data : %x\n", data)
fmt.Printf("1 try of decryption result : %x\n", decoded1)
fmt.Printf("2 try of decryption result : %x\n", decoded2)
fmt.Printf("3 try of decryption result : %x\n", decoded3)
fmt.Printf("4 try of decryption result : %x\n", decoded4)
}
func encrypt(msg []byte) []byte {
encrypted := make([]byte, len(msg))
encrypter.CryptBlocks(encrypted, msg)
return encrypted
}
func decrypt(hash []byte) []byte {
decrypted := make([]byte, len(hash))
decrypter.CryptBlocks(decrypted, hash)
return decrypted
}
This code is also available and runnable
on the playground.
It gives the following result:
encrypted data : 313233343536373871776572747975696131323334353637387a786376626e6d2c3132333435363738706f6975797472
1 try of decryption result : 313233343536373871776572747975696131323334353637387a786376626e6d2c3132333435363738706f6975797472
2 try of decryption result : 5e66fa74456402c271776572747975696131323334353637387a786376626e6d2c3132333435363738706f6975797472
3 try of decryption result : 5e66fa74456402c271776572747975696131323334353637387a786376626e6d2c3132333435363738706f6975797472
4 try of decryption result : 5e66fa74456402c271776572747975696131323334353637387a786376626e6d2c3132333435363738706f6975797472
As you can see the first decryption works well and returns valid result,
but all other tries returns the wrong result.
The first 16 bytes of result is not as in source byte slice.
Can somebody describe what I am doing wrong?
Short version: don't reuse the decrypter object.
Longer version: You're using a cipher in CBC mode: when encrypting the data, the plaintext for block N is XOR-ed with the ciphertext for block N-1 (or the IV, on the first block). On decryption this is done in reverse.
This means that when you try and reuse your decrypter object you don't get the correct results because the state isn't correct - it is decrypting the blocks as if they were subsequent blocks in your message. A peculiarity of CBC is that an incorrect IV will only affect the first decrypted block.
I've writing a client-side app in Go that needs to interact with a C program on the server-side. The client does an AES CFB encrypt and the server decrypts. Unfortunately the server-side has a bug with reusing an initialization vector. It tries to do 3 decrypt operations based on:-
key1, iv
key2, iv
key3, iv
Due to this issue the iv is actually modified between decrypt operations. My problem now is how to reproduce this behaviour on the client side using Go.
By inserting a Println into the encrypt function below, I can see the cfb struct which, I think, contains the modified IV for the next block but because it's a stream interface, I'm not sure how to extract it into a byte slice. Any suggestions?
Thanks
package main
import (
"fmt"
"encoding/hex"
"crypto/cipher"
"crypto/aes"
)
func encrypt_aes_cfb(plain, key, iv []byte) (encrypted []byte) {
block, err := aes.NewCipher(key)
if err != nil {
panic(err)
}
encrypted = make([]byte, len(plain))
stream := cipher.NewCFBEncrypter(block, iv)
stream.XORKeyStream(encrypted, plain)
fmt.Println(stream)
return
}
func main() {
plain := []byte("Hello world...16Hello world...32")
key := make([]byte, 32)
iv := make([]byte, 16)
enc := encrypt_aes_cfb(plain, key, iv)
fmt.Println("Key: ", hex.EncodeToString(key))
fmt.Println("IV: ", hex.EncodeToString(iv))
fmt.Println("Enc: ", hex.EncodeToString(enc))
}
Going down the path you're hinting at is a bit ugly, and prone to break when the implementation changes.
You can get the IV from the stream by:
s := reflect.Indirect(reflect.ValueOf(stream))
lastIV := s.FieldByName("next").Bytes()
But, there's an easier way! Concatenate the plain text inputs, so that the stream for the second starts with the IV from the end of the first (and so on).
Playground Example
combined := append(plain, plain2...)
encCombined := encrypt_aes_cfb(combined, key, iv)
enc := encCombined[:len(plain)]
enc2 := encCombined[len(plain):]