I want to store serialized objects (or whatever) in a key/value cache.
Now I do something like this :
public string getValue(int param1, string param2, etc )
{
string key = param1+"_"+param2+"_"+etc;
string tmp = getFromCache();
if (tmp == null)
{
tmp = getFromAnotherPlace();
addToCache( key, tmp);
}
return tmp;
}
I think it can be awkward. How can I design the key?
if i understood the question, i think the simplest and smartest way to make a key is to use an unidirectional hash function as MD5, SHA1 ecc...
At least two reason for doing this:
The resulting key is unique for sure!(actually both MD5 and SHA1 have been cracked (= )
The resulting key has a fixed lenght!
You have to give your object as argument of the function and you have your unique key.
I don t know very much c# but i am quite sure you can find an unidirectional hash function builted-in.
First of all your key seems to be composed out of a lot of characters. Keep in mind that the key name also occupies memory (1byte / char) so try to keep it as short as possible. I've seen situations where the key name was larger than the value, which can happen if you have cases where you store an empty array or an empty value.
The key structure. I guess from your example that the object you want to store is identified by the params (one being the item id maybe, or maybe filters for a search [...]). Start with a prefix. The prefix should be the name of the object class (or a simplified name depicting the object in general).
Most of the time, keys will have a prefix + identifier. In your example you have multiple identifiers. If one of them is a unique id, go with only prefix + id and it should be enough.
If the object is large and you don't always use all of it then change your strategy to a multiple key storage. Use one main key for storing the most common values, or for storing the components of the object, values of which are stored in separate keys. Make use of pipes and get the whole object in one connection using one "multiple" query :
mainKey = prefix + objectId;
object = getFromCache(mainKey);
startCachePipeline();
foreach (object[properties] as property) {
object->property = getFromCache(prefix + objectId + property);
}
endCachePipeline();
The structure for an example "Person" object would then be something like :
person_33 = array(
properties => array(age, height, weight)
);
person_33_age = 28;
person_33_height = 6;
person_33_weight = 150;
Memcached uses memory most efficient when objects stored inside are of similar sizes. The bigger the size difference between objects (not talking about 1 lost big object or singular cases, although memory gets wasted then as well) the more wasted memory.
Hope it helps!
Related
I was going to use Hashtable but some existing answer said only LinkedHashMap preserve the insertion order. So, it seems that I can get the insertion order with the entries or keys properties.
My question is, when the map has n elements, if I want to remove the first n/2 elements, is there a better way than looping through the keys and repeatedly calling remove(key)? That is, something like this
val a = LinkedHashMap<Int, Int>();
val n = 10;
for(i in 1 .. n)
{
a[i] = i*10;
}
a.removeRange(0,n/2);
instead of
val a = LinkedHashMap<Int, Int>();
val n = 10;
for(i in 1 .. n)
{
a[i] = i*10;
}
var i = 0;
var keysToRemove= ArrayList<Int>();
for(k in a.keys)
{
if(i >= n/2)
break;
else
i++
keysToRemove.add(k);
}
for(k in keysToRemove)
{
a.remove(k);
}
The purpose of this is that I use the map as a cache, and when the cache is full, I want to purge the oldest half of the entries. I do not have to use LinkedHashMap as long as I can:
Find the value using a key, efficiently.
Remove a range of entries at once.
There's no method in the class that makes this possible. The source code doesn't have any operations for ranges of keys or entries. Since the linking is built on top of the HashMap logic, individual entries still have to be individuatlly found by a hashed key lookup to remove them, so being able to remove a range couldn't be done faster in a LinkedHashMap, which is unlike the analogy of a LinkedList to an ArrayList.
For simpler code that's equivalent to what you're doing:
a.keys.take(a.size / 2).forEach(a::remove)
If you don't want to use a library for a cache set, LinkedHashSet is designed so you can easily build your own by subclassing. For instance, a basic one that simply removes the oldest entry when you add elements above a certain collection size:
class CacheHashMap<K, V>(private var maxSize: Int): LinkedHashMap<K, V>() {
override fun removeEldestEntry(eldest: MutableMap.MutableEntry<K, V>?): Boolean =
size == maxSize
}
Also, if you set accessOrder to true in your constructor call, it orders by last used to most recently used entry, which might be more apt for your situation than insertion order.
EDIT: sorry I missed the part about using this as an LRU cache, for that use case, TreeMap will not be suitable.
If insertion order is just incidental for you, and what you want is in fact the actual order of comparable keys, you should use a TreeMap instead.
However, the specific use case of removing half the keys might not be supported directly. You will rather find methods to remove keys below/above a certain value, and get the highest/lowest keys.
I am reading the source code of map in go1.10.3.It seemed there exist corresponding method about operation such as:
makemap(t *maptype, hint int, h *hmap) *hmap ==> m = make(map[xx]yy)
mapaccess1(t *maptype, h *hmap, key unsafe.Pointer)==> m['key']
but I cant find the correspond method for the operation which add key/value as below:
m['xx']='yy'
there exist a method called mapassign which has some similarity with this
operation.
mapassign(t *maptype, h *hmap, key unsafe.Pointer) unsafe.Pointer
this will add a new key to the map, but as we can see, the input arguments has no value. And another question is when it has already this key, it maybe update this key.
if !alg.equal(key, k) {
continue
}
// already have a mapping for key. Update it.
if t.needkeyupdate {//why??
typedmemmove(t.key, k, key)
}
since the two key is equal, why should update it?
summary:
1. the relation between add key/value operation and method mapassign?
2. why it maybe need to update the key since the insert key and the key which has already exist is equal in mapassign method?
In the operation m[k] = v, the caller copies the value v to the address returned by mapassign.
The comments in the function needkeyupdate explain why some types need key updates: floating point & complex -0 and 0 are equal, but different values; string might have smaller backing store.
I have an app that I'm converting from Swift2 to Swift3. One thing I need to do is the following:
dict.enumerateKeysAndObjects({ (key, rotationString, stop) -> Void in
//code goes here to use key as an integer
)
How do I do that? Also, rotationString will also be an integer.
Background:
In the Swift2 version of the app I was saving data to defaults thusly:
func saveGame() {
let defaults = UserDefaults.standard
defaults.set(blackRotations, forKey: "blackRotations")
defaults.set(whiteRotations, forKey: "whiteRotations")
etc.
Here, whiteRotations and blackRotations were NSDictionary objects where the actual data being saved had keys that were NSNumbers and values that were also NSNumbers.
So, what I need to do is to handle loading the saved game from UserDefaults. The values for blackRotations and whiteRotations were dictionaries when the game was saved. The structure of this dictionary was a bunch of integer pairs. You can think of them as integer lookup tables saved as dictionaries.
So all I really am asking for is how to load this data from UserDefaults and be able to treat both the keys and values as integers.
Possible solution? I'm still working on trying to get this to work, but I'll post here anyway, in the hope that it will help to illustrate what I am trying to do.
In the method that loads the saved state, I think I need to do something along these lines:
if let blackDict = (defaults.dictionary(forKey: "blackRotations") as? [String:Int]) {
for (keyString, rotation) in blackDict {
blackRotations[Int(keyString)!] = rotation
}
}
I have std::map which contains list of values associated with a Key. Actual implementation contains many such Keys. Is there a similar way in Lua Table implementation which could hold multiple values for a specific key. If so how to write and read from that table.
I referred the How do I create a Lua Table in C++, and pass it to a Lua function?
I have only access to set and get values which is on my C++ code which was written more generic and cannot create table in C++. (Third party C++ code).
All I have is I can get KeyType, Key, and Value using the
luaState = luaL_newstate();
lua_register(luaState, "getValue", get_value);
lua_register(luaState, "setValue", set_value);.
The C++ code has something like
typedef std::set<const char *> TValueNames;
std::map<const char *, TValueNames> keyValueList;
By referring to Lua document I understood I can create a table with Key as index and assign value as its data. But I need to know how to assign multiple value(data) for one Key.
https://www.lua.org/pil/2.5.html
The example lua script implementation is like,
local keyType = getValue("KeyType");
local Key = getValue("Key");
local Value = getValue("Value");
KeyValueTable = {}
KeyValueTable[Key] = Value;
I need to create something which could hold information like,
["Key1"] = {"Value1,Value2,Value3"};
["Key2"] = {"Value2,Value3,Value4,Value5"};
As you know, a key in a Lua table can only refer to one value, but you can easily make that value a table to hold multiple values. To more faithfully represent the set in the C++ structure, we can make the values into keys in the inner table.
local function setValue(self, key, value)
self[key] = self[key] or {}
self[key][value] = true
end
local function removeValue(self, key, value)
if type(self[key]) == 'table' then
self[key][value] = nil
end
end
local function checkValue(self, key, value)
if type(self[key]) == 'table' then
return self[key][value]
end
end
I have a QHash defined as follows
QHash<QString, QString> hashLookup;
I have inserted a few values to this hash as follows:
hashLookup.insert("OMG", "Oh my God!");
hashLookup.insert("LOL", "Laugh out loud");
hashLookup.insert("RIP", "Rest in peace");
// and so on
I have a few QStrings as follows:
QString a = "OMG_1";
QString b = "LOL_A";
QStirng c = "OMG_YOU";
QString d = "RIP_two";
I am supposed to find if these values exist in hashLookup, i.e, since OMG_1 contains OMG, I should be able to retrieve Oh my God!.
I have tried to do this using
if(hashLookup.contains(a)
//do something
which ofcourse tries to look for a key OMG which is not present in the lookup table and does not return anything. Is partial matching of key values possible in Qt? If yes, how should I go about implementing this.
There is no opportunity in QHash class to extract values by partial matching of key, because QHash use hash function (Qt documentation: qHash) which:
The qHash() function computes a numeric value based on a key. It can
use any algorithm imaginable, as long as it always returns the same
value if given the same argument. In other words, if e1 == e2, then
qHash(e1) == qHash(e2) must hold as well. However, to obtain good
performance, the qHash() function should attempt to return different
hash values for different keys to the largest extent possible.
Different keys give almost always different hash.
In your task you can run on QHash keys and make comparison with QString functionality. Something like this:
QString getHashValue(const QString& strKey, const QHash<QString, QString>& hashLookup)
{
QList<QString> uniqueKeys = hashLookup.uniqueKeys();
foreach(const QString& key, uniqueKeys)
{
if(strKey.contains(key))
return hashLookup.value(key);
}
}
...
getHashValue("OMG_1", hashLookup);
First, in your example the QHash.contains(QString key) method tries to find OMG_1, which in fact it will not find.
You may implement a method which will take a expanded key and tries to locate any subkey of the given value in the hash. Here you have to define some rules I think or it may not return the intendend value.
Think of following example: the hash contains the keys OMG and OM. To match the provided expanded key you implement something like this
bool hashContainsExpanded(const QString &key) const {
if (!hash.contains(key) && key.length() > 1)
return hasContainsExpanded(key.substring(0, key.length() - 1));
return hash.contains(key);
}
This method will let you find the key OMG but not OM which is contained in this key. You may also implement a method which will take the first character of the provided expanded key and test it for containment. If not found, it will take the second and test again and so on. This will match OM in favour of OMG.
Also keep in mind, that you may work later with the matched key and thus you should return it instead of only returning true.