Working with the "new" Firebase Cloud Messaging, I would like to reliably save client device registration_id tokens to the local server database so that the server software can send them push notifications.
What is the smallest size of database field that I should use to save 100% of client registration tokens generated?
I have found two different libraries that use TextField and VarChar(255) but nothing categorically defining the max length. In addition, I would like the server code to do a quick length check when receiving tokens to ensure they "look" right - what would be a good min length and set of characters to check for?
I think this part of FCM is still the same as GCM. Therefore, you should refer to this answer by #TrevorJohns:
The documentation doesn't specify any pattern, therefore any valid string is allowed. The format may change in the future; please do not validate this input against any pattern, as this may cause your app to break if this happens.
As with the "registration_id" field, the upper bound on size is the max size for a cookie, which is 4K (4096 bytes).
Emphasizing on the The format may change in the future part, I would suggest to stay safe and have a beyond the usual max (mentioned above) length. Since the format and length of a registration token may also vary.
For the usual length and characters, you can refer to these two answers the latter being much more definitive:
I hasn't seen any official information about format of GCM registrationId, but I've analyzed our database of such IDs and can make following conclusions:
in most cases length of a registrationID equals 162 symbols, but can be variations to 119 symbols, maybe other lengths too;
it consists only from this chars: [0-9a-zA-Z\-\_]*
every regID contains one or both of "delimiters": - (minus) or _ (underline)
I am now using Firebase Cloud Messaging instead of GCM.
The length of the registration_id I've got is 152.
I've also got ":" at the very beginning each time like what jamesc mentioned (e.g. bk3RNwTe3H0:CI2k_HHwgIpoDKCIZvvDMExUdFQ3P1).
I make the token as varchar(255) which is working for me.
However, the length of registration_id has no relationship with size
of 4k. You are allowed to send whatever size of the data through
network. Usually, cookies are limited to 4096 bytes, which consist of
name, value, expiry date etc.
This is a real fcm token:
c2aK9KHmw8E:APA91bF7MY9bNnvGAXgbHN58lyDxc9KnuXNXwsqUs4uV4GyeF06HM1hMm-etu63S_4C-GnEtHAxJPJJC4H__VcIk90A69qQz65toFejxyncceg0_j5xwoFWvPQ5pzKo69rUnuCl1GSSv
as you can see the length of token is: 152
I don't think the upper limit for a registration ID is 4K. It should be safe to assume that it is much lower than that.
The upper limit for a notification payload is 4KB (link), and the notification payload includes the token (link). Since the payload also needs to include the title, body, and other data too, the registration ID should be small.
That's what I understand from the docs ¯\_(ツ)_/¯
The last tokens I got were 163-chars long. I think it's safe to assume that they will never exceed 255 chars. Some comments in the other answer reported much higher lengths!
Update
So far, in 4 months that I'm running my app, there are over 100k registration IDs, and every single one of them is 163-chars long. It's very possible that Google maintains the ID length stable in order not to crash apps. Hence, I'd suggest
getting a few registration IDs in your local machine
measuring their length and verifying it's constant (or at least it doesn't change significantly)
picking a safe initial value, slightly higher than the ID length
I think it's unlikely for the length to change now, but I'll keep an eye. Please let me know if you noticed IDs of different lengths in your apps!
Related
I have checked the Docs but I got confused a bit. When storing a long integer such as 265628143862153200. Would it be more efficient to store it as a string of integer.
This is what I need help with is the below calculation corrent?
Integer:
length of 265628143862153200 *8 ?
String:
length of 265628143862153200 +1 ?
The Firebase console is not meant to be part of administrative workflows. It's just for discovery and development. If you have production-grade procedures to follow, you should only write code for that using the provided SDKs. Typically, developers make their own admin sites to deal with data in Firesotre.
Also, you should know that JavaScript integers are not "big" enough to store data to the full size provided by Firestore. If you want to use the full size of a number field, you will have to use a system that supports Firestore's full 64 bit signed integer type.
If you must store numbers bigger than either limit, and be able to modify them by hand, consider instead storing multiple numbers, similar to the way Firestore's timestamp stores seconds and nanoseconds as separate numbers, so that the full value can overflow greater than signed 64 bits.
I would like to generate a multipart byte range response. Is there a way for me to do it without scanning each segment I am about to send out, since I need to generate multipart boundary strings?
For example, I can have a user request a byterange that would have me fetch and scan 2GB of data, which in my case involves me loading that data into my (slow) VM as strings and so forth. Ideally I would like to simply state in the response that a part has a length of a certain number of bytes, and be done with it. Is there any tooling that could provide me with this option? I see that many developers just grab a UUID as the boundary and are probably willing to risk a tiny probability that it will appear somewhere within the part, but that risk seems to be small enough multiple people are taking it?
To explain in more detail: scanning the parts ahead of time (before generating the response) is not really feasible in my case since I need to fetch them via HTTP from an upstream service. This means that I effectively have to prefetch the entire part first to compute a non-matching multipart boundary, and only then can I splice that part into the response.
Assuming the data can be arbitrary, I don’t see how you could guarantee absence of collisions without scanning the data.
If the format of the data is very limited (like... base 64 encoded?), you may be able to pick a boundary that is known to be an illegal sequence of bytes in that format.
Even if your boundary does collide with the data, it must be followed by headers such as Content-Range, which is even more improbable, so the client is likely to treat it as an error rather than consume the wrong data.
Major Web servers use very simple strategies. Apache grabs 8 random bytes at startup and renders them in hexadecimal. nginx uses a sequential counter left-padded with zeroes.
UUIDs are designed to avoid collisions with other UUIDs, not with arbitrary data. A UUID is no more likely to be a good boundary than a completely random string of the same length. Moreover, some UUID variants include information that you may not want to disclose, such as your machine’s MAC address.
Ideally I would like to simply state in the response that a part has a length of a certain number of bytes, and be done with it. Is there any tooling that could provide me with this option?
Maybe you can avoid supporting multiple ranges and simply tell the clients to request each range separately. In that case, you don’t use the multipart format, so there is no problem.
If you do want to send multiple ranges in one response, then RFC 7233 requires the multipart format, which requires the boundary string.
You can, of course, invent your own mechanism instead of that of RFC 7233. In that case:
You cannot use 206 (Partial Content). You must use 200 (OK) or some other applicable status code.
You cannot use the multipart/byteranges media type. You must come up with your own media type.
You cannot use the Range request header.
Because a 200 (OK) response to a GET request is supposed to carry a (full) representation of the resource, you must do one of the following:
encode the requested ranges in the URL; or
use something like POST instead of GET; or
use a custom, non-standard status code instead of 200 (OK); or
(not sure if this is a correct approach) use media type parameters, send them in Accept, and add Accept to Vary.
The chunked transfer coding may be useful, but you cannot rely on it alone, because it is a property of the connection, not of the payload.
Update: GCM is deprecated, use FCM
What is the maximum length for a Registration ID issued by GCM servers? GCM documentation do not provide this info. Googling for this reveals that Registration ID is not fixed length in nature and can be up to 4K (4096 bytes) in length. But these are not official answers from Google. I am currently receiving Registration IDs which are 162 characters long. Can anybody help?
On android-gcm forum a google's developer confirms it's 4k
I am interested in know about this also. My reg id size is 183 chars. I suspect it won't be longer than 512 chars though, let alone 4K. Imagine sending bulk notification, a 4K reg id x 1000 = 4MB message size!
In the end, I just use the 'text' type in my MySQL table to store the registration id. So even if google send me a 1K, 2K, or 4K (very unlikely) reg id, I will be able to handle it.
Update: I have come across a new reg id size: 205.
This is what has said in GCM doc,
A JSON object whose fields represents the key-value pairs of the message's payload data. If present, the payload data it will be included in the Intent as application data, with the key being the extra's name. For instance, "data":{"score":"3x1"} would result in an intent extra named score whose value is the string 3x1.
There is no limit on the number of key/value pairs, though there is a limit on the total size of the message (4kb). The values could be any JSON object, but we recommend using strings, since the values will be converted to strings in the GCM server anyway.
If you want to include objects or other non-string data types (such as integers or booleans), you have to do the conversion to string yourself. Also note that the key cannot be a reserved word (from or any word starting with google.).
To complicate things slightly, there are some reserved words (such as collapse_key) that are technically allowed in payload data. However, if the request also contains the word, the value in the request will overwrite the value in the payload data. Hence using words that are defined as field names in this table is not recommended, even in cases where they are technically allowed. Optional.
I'm building a secure payment portal.
We currently have two applications that will be using this. One is a web application, the other a desktop app. Both of these require users to login/authenticate, the same credentials can be used for either application.
I want to build an automatic login mechanism that will fill in all the various login/order details and be able to call this from either app mentioned above. I've been thinking that the best way to do this is to pass this information encrypted through the URL. ie https://mysite.com/TakePayment.aspx?id=GT2jkjh3....
Since we don't want to integrate the payment processing too tightly into the desktop app to reduce our PCI scope, we decided to have it open the browser to a central, secured payment page through a simple shell execute with the full URL causing the default browser to open that page.
Originally we were using AES for the encryption, but this is currently being re-examined as we would prefer not having to give out the key to the end user (AES is symmetric, symmetric encryption = both parties need the private key, why bother even encrypting then since we're going to be distributing the app?) So I'm looking at switching it over to use Public Key Encryption with the built in RSA routines within .NET
After coding up the RSA portion I noticed most examples on the net used 1024bits for the key-length, I went with this and now have our portal working with public key encryption, however the URLs generated are much much longer than when I was using AES so it made me start researching what the max limits for URLs are. http://www.boutell.com/newfaq/misc/urllength.html Says that IE is the limiting browser at about 2048 characters in the path portion. My initial tests with the RSA encryption show my urls will be around 1400 chars long.
My questions boil down to this:
1) Is there a better way for passing information from a desktop app to a website that I'm not thinking of? I'd prefer it be just as easy to use from another web page as it is from the desktop, hence my current solution.
2) Is 1024 bit RSA keys necessary? Or overkill for something like this? A shorter key would mean shorter encrypted text right?
3) Are there any other unforeseen problems with URLs in the 1200-1400 character range? Proxies? Firewalls? Web-Accelerators?
Thanks
Update 12/11/2011:
Come to find out, the method that we ended up going with here ended up biting us in the ass recently (or rather we found out about it today, even though the problem was a very sporadic and difficult one to track down..)
The plain text token that we encrypted was originally rather small, only a hundred bytes or so. This is what resulted in my test URLs being approximately 1400 bytes long. Through feature creep we've been required to add more data to the token, and the average URL length jumped to 1700-1800 in length.
Once the length of our plain text hits 173 characters long and above however, the URL length jumps again, this time up to 2080+ or so, which now causes problems for IE. After some investigation in how RSA encryption works, this should have been totally expected, but was an oversight on my part originally.
We're using 1024 bit RSA encryption, which means that the maximum data block size that can be encrypted is 1024/8 - 24 = 86 bytes, every 86 bytes needs to be "chopped up" and encrypted separately, so at 86 * 2 = 172, we're only encrypting two blocks, above that we're encrypting three, four, five, etc. By passing 172, our cipher text length grew so long the URL's are now too long.. I'm probably messing up the explanation a little here, but that's the general gist of it..
It seems we'll be looking at designing a better way for this to work, as it can be expected they'll want "more features" to be added in the future and thus our token will grow ever larger...
Assuming this is all logged in a database can you not pass the data back and forth using SSL web services. Then in the case of being able to quickly go from the desktop app to the web app make a rpc call to the website to generate a random key, pass that to the user and call a web page using that. Make the key valid for say 10 seconds meaning should a key be captured and broken it will have become invalid?
I have little experience with this kind of thing so I'm expecting many holes to be poked in the idea.
Has anyone used Google translation API ? What is the max length limit for using it?
The limit was 500... now it is 5000 chars.
source
500 characters
source
At the moment, the throttle limit is 100,000 characters per day. Looks like you can apply to have that limit increased/removed.
I've used it to translate Japanese to English.
I don't believe the 500 char limit is true if you use http://code.google.com/p/jquery-translate/, but one thing that is true is you're restricted as to the number of requests you can make within a certain period of time. They also try to detect whether or not you're sending a lot of requests with a similar period, almost like a mini "denial of service" attack.
So when I did this I wrote a client with a random length sleep between requests. I also ran it on a grid so all the requests didn't come from a single IP address.
I had to translate ~2000 Java messages from a resource bundle from Japanese to English. It worked out pretty nicely, as long as the text was single words. Longer phrases with context came out awkwardly.
Please have look at this link it will give the correct answer at the bottom of the page.
https://developers.google.com/translate/v2/faq
What is the maximum number of characters per request?
The maximum size of each text to be translated is 5000 characters, not including any HTML tags.
You can send source strings of up to 5,000 characters, but there are a
few provisos that are sometimes lost.
You can only send the 5,000 characters via the POST method.
If you use GET method, you are limited to 2,000-character length limit on urls. If a url is longer than that, Google's servers will just reject it.
Note: 2,000-character limit including the path and the rest
of the query string as well + you must count uri encoding (for instance every space becomes a %20, every quotation
mark a %22)
The Cloud Translation API is optimized for translating of smaller requests. The recommended maximum length for each request is 5K characters (code points). However, the more characters that you include, the higher the response latency. For Cloud Translation - Advanced, the maximum number of code points for a single request is 30K. Cloud Translation - Basic has a maximum request size of 100K bytes.
https://cloud.google.com/translate/quotas