Storing arbitrary bytes in RocksDB - rocksdb

Question

RocksDB states that it can store arbitrary data but the API only supports std::string types. I want to store std::vector<T> values and it appears if I want to do this then I must convert it into a std::string.
Is there a less brittle way to storing arbitrary types?

Answer 1

I tend to use the following to pack/unpack structures/classes to std::string using a template to automatically adjust to their size.
template <typename T>
std::string Pack(const T* data)
{
std::string d(sizeof(T), L'\0');
memcpy(&d[0], data, d.size());
return d;
}
template <typename T>
std::unique_ptr<T> Unpack(const std::string& data)
{
if (data.size() != sizeof(T))
return nullptr;
auto d = std::make_unique<T>();
memcpy(d.get(), data.data(), data.size());
return d;
}
So the following client code can pack and unpack a structure into the database:
// Test structure
BOB b = {};
b.a = 12;
b.b = 144;
b.c[0] = 's';
b.c[1] = '\0';
// Write to the db
status = pDb->Put(rocksdb::WriteOptions(), key, Pack(&b));
// Read from db with same key
std::string result;
status = pDb->Get(rocksdb::ReadOptions(), key, &result);
std::unique_ptr<BOB> pBob = Unpack<BOB>(result);
if (b.a == pBob->a && b.b == pBob->b && b.c[0] == pBob->c[0])
{
printf("Structure matches!\n");
}
else
{
printf("Structure doesn't match!\n");
}

Answer 2

As a key-value store, RocksDB can store "arbitrary byte array". For your use case, you would need to have some way to serialize your std::vector<T> to a byte array so that it can be put into a std::string. Note that the input std::string does not need to be zero-terminated.