I have checked the Docs but I got confused a bit. When storing a long integer such as 265628143862153200. Would it be more efficient to store it as a string of integer.
This is what I need help with is the below calculation corrent?
Integer:
length of 265628143862153200 *8 ?
String:
length of 265628143862153200 +1 ?
The Firebase console is not meant to be part of administrative workflows. It's just for discovery and development. If you have production-grade procedures to follow, you should only write code for that using the provided SDKs. Typically, developers make their own admin sites to deal with data in Firesotre.
Also, you should know that JavaScript integers are not "big" enough to store data to the full size provided by Firestore. If you want to use the full size of a number field, you will have to use a system that supports Firestore's full 64 bit signed integer type.
If you must store numbers bigger than either limit, and be able to modify them by hand, consider instead storing multiple numbers, similar to the way Firestore's timestamp stores seconds and nanoseconds as separate numbers, so that the full value can overflow greater than signed 64 bits.
Related
I'm building a system that will eventually store extremely large number values (double or floating point types, not int) and I'm using firestore to store my data. Is there a cap for how big numbers can grow in a firestore database? I can't seem to find any documentation anywhere.
As per the documentation, Firestore allows only 64 bit signed integers. A 64-bit signed integer. It has a minimum value of -9,223,372,036,854,775,808 and a maximum value of 9,223,372,036,854,775,807 (inclusive).
Same for float: Firestore allows 64-bit double precision, IEEE 754
I just tried adding a random number over that limit and you get.
Then I tried requesting the document and that's the data you get:
It'll be hard to get the precise value if you store it as a number. You may be interested storing the number in string format if that numbers are that large. But do keep the 1 MiB (1,048,576 bytes) limit of a document in mind.
Working with the "new" Firebase Cloud Messaging, I would like to reliably save client device registration_id tokens to the local server database so that the server software can send them push notifications.
What is the smallest size of database field that I should use to save 100% of client registration tokens generated?
I have found two different libraries that use TextField and VarChar(255) but nothing categorically defining the max length. In addition, I would like the server code to do a quick length check when receiving tokens to ensure they "look" right - what would be a good min length and set of characters to check for?
I think this part of FCM is still the same as GCM. Therefore, you should refer to this answer by #TrevorJohns:
The documentation doesn't specify any pattern, therefore any valid string is allowed. The format may change in the future; please do not validate this input against any pattern, as this may cause your app to break if this happens.
As with the "registration_id" field, the upper bound on size is the max size for a cookie, which is 4K (4096 bytes).
Emphasizing on the The format may change in the future part, I would suggest to stay safe and have a beyond the usual max (mentioned above) length. Since the format and length of a registration token may also vary.
For the usual length and characters, you can refer to these two answers the latter being much more definitive:
I hasn't seen any official information about format of GCM registrationId, but I've analyzed our database of such IDs and can make following conclusions:
in most cases length of a registrationID equals 162 symbols, but can be variations to 119 symbols, maybe other lengths too;
it consists only from this chars: [0-9a-zA-Z\-\_]*
every regID contains one or both of "delimiters": - (minus) or _ (underline)
I am now using Firebase Cloud Messaging instead of GCM.
The length of the registration_id I've got is 152.
I've also got ":" at the very beginning each time like what jamesc mentioned (e.g. bk3RNwTe3H0:CI2k_HHwgIpoDKCIZvvDMExUdFQ3P1).
I make the token as varchar(255) which is working for me.
However, the length of registration_id has no relationship with size
of 4k. You are allowed to send whatever size of the data through
network. Usually, cookies are limited to 4096 bytes, which consist of
name, value, expiry date etc.
This is a real fcm token:
c2aK9KHmw8E:APA91bF7MY9bNnvGAXgbHN58lyDxc9KnuXNXwsqUs4uV4GyeF06HM1hMm-etu63S_4C-GnEtHAxJPJJC4H__VcIk90A69qQz65toFejxyncceg0_j5xwoFWvPQ5pzKo69rUnuCl1GSSv
as you can see the length of token is: 152
I don't think the upper limit for a registration ID is 4K. It should be safe to assume that it is much lower than that.
The upper limit for a notification payload is 4KB (link), and the notification payload includes the token (link). Since the payload also needs to include the title, body, and other data too, the registration ID should be small.
That's what I understand from the docs ¯\_(ツ)_/¯
The last tokens I got were 163-chars long. I think it's safe to assume that they will never exceed 255 chars. Some comments in the other answer reported much higher lengths!
Update
So far, in 4 months that I'm running my app, there are over 100k registration IDs, and every single one of them is 163-chars long. It's very possible that Google maintains the ID length stable in order not to crash apps. Hence, I'd suggest
getting a few registration IDs in your local machine
measuring their length and verifying it's constant (or at least it doesn't change significantly)
picking a safe initial value, slightly higher than the ID length
I think it's unlikely for the length to change now, but I'll keep an eye. Please let me know if you noticed IDs of different lengths in your apps!
My application requires a key value store. Following are some of the details regarding key values:
1) Number of keys (data type: string) can either be 256, 1024 or 4096.
2) Data type of values against each key is a list of integers.
3) The list of integers (value) against each key can vary in size
4) The largest size of the value can be around 10,000,000 integers
5) Some keys might contain very small list of integers
The application needs fast access to the list of integers against a specified key . However, this step is not frequent in the working of the application.
I need suggestions for best Key value stores for my case. I need fast retrieval of values against key and value size can be around 512 MB or more.
I checked Redis but it requires the store to be stored in memory. However, in the given scenario I think I should look for disk based key value stores.
LevelDB can fit your use case very well, as you have limited number of keys (given you have enough disk space for your requirements), and might not need a distributed solution.
One thing you need to specify is if (and how) you wish to modify the lists once in the db, as levelDB and many other general key-val stores do not have such atomic transactions.
If you are looking for a distributed db, cassandra is good, as it will also let you insert/remove individual list elements.
TL;DR: I'm building a data set to share between iOS and Android. Should I tweak the integer sizes to match their actual ranges, or just make everything Integer and use Int in Swift and long in both Java and Swift?
In a typical SQL database, storing a large number of 4-byte integers would take ~4x more space than a 1-byte integer[1]. However, I read in this answer that integers are stored bit-packed and in the Realm Java help that The integer types byte, short, int, and long are all mapped to the same type (long actually) within Realm. So, reading between the lines, it seems that the on disk storage will be the same regardless of what integer sub-type I use.
So, from a pure Realm / database perspective should I just use Int & long in Swift & Java respectively? (I.e. leaving aside language differences, like casting, in-memory size etc.)
If an integer field is indexed, does that make any difference to the type chosen?
PS: Many thanks to the Realm team for their great docs and good support here on SO!
[1] Yes, I know it's more complicated than that.
Your interpretation is correct: the same underlying storage type is used for all integer types in Realm, and that storage type adjusts the number of bits it uses per value based on the range of values stored. For example, if you only store values in the range 0-15 then each value will use fewer bits than if you store values in the range 0-65,535. Similarly, all indexes on integer properties use a common storage type.
I'm storing phone country codes. They range from 1 to about 300. What's going to be more performant for datatype: int or string? I'm using SQL server 2008 and linq-to-sql.
Thanks.
Note: Whoa, really wierd - you asked about phone codes and I wrote about ZIP codes. Sorry about that! I think the advice still stands though...
Original answer: Performance will most likely be negligible - assign the proper type based on what the data is. ZIP codes, while numeric (in the US at least), aren't numbers - they should be stored as strings.
It is very important to understand the semantic nature of the data you are storing. Once you understand what something is then you can begin to reason about how it should be stored. I am assuming that currently you are storing only the first 5 numbers of a US postal code (like this: 12345).
If you were to store this data as a number this would work. Then imagine that your manager tells you that there is a new requirement that the app you are building will start to collect ZIP codes in the ZIP+4 format (which looks like this: 12345-6789). Now you are stuck with a nasty refactoring that involves either changing the type in the database to varchar(10) or doing some crazy voodoo in your app to strip out the dash when you save the ZIP code and then add it back in for display later.
If you're really worried about space and performance then you could use a smallint (which equates to a int16). This will mean that the data will only take 2 bytes of storage (and 2 bytes in memory).
Given an option where I know the datatype will always be integer, I'll go for integer albeit smaller size - smallint / tinyint (depending on the required range).
I don't expect much difference in performance though.
How are you going to be using them and do any have leading zeros?
If you are going to be combining with phone numbers that are usually stored as string, you want to store them as a string as well or you will waste processing power converting them in every query.
If you aren't planning on doing math or joins with it, it is problably a bad idea to store as a number. Your data set is likely so small and the strings so tiny (300 is the max value) that using an int would probably gain you nothing in a join either.
Country codes are strings (notwithstanding that they use only the characters 0..9) and should be stored as such.
They are so few that you don't need to be concerned about this, though it would be simpler to apply a check constraint with an integer type.
my rule of thumb has always been.. do I need an average? For example, you can store a zip code as integer, but are you ever going to need the average zip code? Probably not. As such, store as char.. unless you may need more than 5 characters, in which case store as varchar.