Given that I already have a guaranteed unique string, is it possible to generate a shorter version of this string while keeping the newly generated strings also unique?
The existing unique code is the id generated by firebase firestore, the ID is too long and I prefer not to use it as referral code. I was hoping to limit the length of the string around 8-10 chars.
EX of firebase id: NIzTFZUC64Voc4ttiiNsBF0GNWF3
I need a much shorter version of this string
I also don't want the codes to be case sensitive
Related
I have a collection where the documents are uniquely identified by a date, and I want to get the n most recent documents. My first thought was to use the date as a document ID, and then my query would sort by ID in descending order. Something like .orderBy(FieldPath.documentId, descending: true).limit(n). This does not work, because it requires an index, which can't be created because __name__ only indexes are not supported.
My next attempt was to use .limitToLast(n) with the default sort, which is documented here.
By default, Cloud Firestore retrieves all documents that satisfy the query in ascending order by document ID
According to that snippet from the docs, .limitToLast(n) should work. However, because I didn't specify a sort, it says I can't limit the results. To fix this, I tried .orderBy(FieldPath.documentId).limitToLast(n), which should be equivalent. This, for some reason, gives me an error saying I need an index. I can't create it for the same reason I couldn't create the previous one, but I don't think I should need to because they must already have an index like that in order to implement the default ordering.
Should I just give up and copy the document ID into the document as a field, so I can sort that way? I know it should be easy from an algorithms perspective to do what I'm trying to do, but I haven't been able to figure out how to do it using the API. Am I missing something?
Edit: I didn't realize this was important, but I'm using the flutterfire firestore library.
A few points. It is ALWAYS a good practice to use random, well distributed documentId's in firestore for scale and efficiency. Related to that, there is effectively NO WAY to query by documentId - and in the few circumstances you can use it (especially for a range, which is possible but VERY tricky, as it requires inequalities, and you can only do inequalities on one field). IF there's a reason to search on an ID, yes it is PERFECTLY appropriate to store in the document as well - in fact, my wrapper library always does this.
the correct notation, btw, would be FieldPath.documentId() (method, not constant) - alternatively, __name__ - but I believe this only works in Queries. The reason it requested a new index is without the () it assumed you had a field named FieldPath with a subfield named documentid.
Further: FieldPath.documentId() does NOT generate the documentId at the server - it generates the FULL PATH to the document - see Firestore collection group query on documentId for a more complete explanation.
So net:
=> documentId's should be as random as possible within a collection; it's generally best to let Firestore generate them for you.
=> a valid exception is when you have ONE AND ONLY ONE sub-document under another - for example, every "user" document might have one and only one "forms of Id" document as a subcollection. It is valid to use the SAME ID as the parent document in this exceptional case.
=> anything you want to query should be a FIELD in a document,and generally simple fields.
=> WORD TO THE WISE: Firestore "arrays" are ABSOLUTELY NOT ARRAYS. They are ORDERED LISTS, generally in the order they were added to the array. The SDK presents them to the CLIENT as arrays, but Firestore it self does not STORE them as ACTUAL ARRAYS - THE NUMBER YOU SEE IN THE CONSOLE is the order, not an index. matching elements in an array (arrayContains, e.g.) requires matching the WHOLE element - if you store an ordered list of objects, you CANNOT query the "array" on sub-elements.
From what I've found:
FieldPath.documentId does not match on the documentId, but on the refPath (which it gets automatically if passed a document reference).
As such, since the documents are to be sorted by timestamp, it would be more ideal to create a timestamp fieldvalue for createdAt rather than a human-readable string which is prone to string length sorting over the value of the string.
From there, you can simply sort by date and limit to last. You can keep the document ID's as you intend.
I read somewhere that db.collection("mycollection").document().getId(); gives document Id in mycollection without hitting cloudstore database. But how it is possible to create unique Id without knowing document id of already existing doucments or hitting couldstore?
The auto-ID that is generated when you call document() is a fairly basic UUID (universally unique identifier). Such identifier are statistically guaranteed to be unique. In my words: there is so much random information in there, that the chances of two calls generating the same value are infinitesimally small.
So Firestore doesn't actually call the server to check whether the ID it generates is unique. It instead relies on the mathematical properties of picking a single value out of a sufficiently large and random set to be very certain it is unique.
Assuming I have a list of data I would like to store with Firebase realtime database, and search it later.
What would be the best way to store the data and query it to get the best performance?
My data is a list of names (containing a lot of names).
["Bob", "Rob", ...]
Note that I have multiple clients searching in a given time.
If the names are supposed to be unique and order doesn't matter, you'll want to store them as a mathematical set. In the Firebase Realtime Database you'll model this as:
"usernames": {
"Bob": true,
"Rob": true
}
A few things of note about this model:
We use the names as keys, which means that each name is by definition unique (since each key can exist only once in its containing node).
The true values have no specific meaning. They are just needed, since Firebase can't store a key without a value.
Certain characters (such as . and /) cannot be used in keys. If a name contains such characters, you will have to filter them out (or encode them) in the key. For example someone named Jes.sie will have to be stored as Jes.sie (lossy) or e.g. Jes%2Esie (with URL encoding).
In such cases you could store the original unfiltered/unencoded name as the value. So: "Jes%2Esie": "Jes.sie".
A few more general notes about (text) searching in the Firebase Realtime Database:
Firebase can only do prefix matches, it has no support for searching strings that contain or end with a certain substrings. This means that in the original data it can search for everything starting with an B (with orderByKey().startAt("R").endAt("R\uF7FF")), but it can't search for everything ending with ob.
Searches are case-sensitive. If you want to be able to search case-insensitive, consider storing the keys as all-lowercase:
"usernames": {
"bob": "Bob",
"rob": "Rob",
"jes%2esie": "Jes.sie"
}
If you need better support for text-search, consider integrating a third-party search engine. Common recommendations are Elastic-search (self-hosted) or Algolia (cloud-based).
For more information on many of these topics, see:
this article on NoSQL data modeling
the video series Firebase for SQL developers
Cloud Firestore Case Insensitive Sorting Using Query (while written for Firestore, the same applies here)
For an app I build, I want to use the ID generated by Firebase push as an email address local part. Since the dash (-) is not allowed as first character, I would like to replace it with another character.
This has to be reversible though. Therefore I want to know, which characters does the Firebase push ID consist of?
So far I have seen:
alpha (a-z and A-Z and 0-9)
underscore (_)
dash (-)
Sample: -KD3rcGMuucRDjKOTK3O
Are there any other characters which might be contained in the ID?
Do firebase IDs always start with a dash?
There are probably a lot of better ways to generate a unique email address than by using Firebase's push ids and then mangling them. That said, if you want to learn more about how Firebase generates its push ids, read this blog post: The 2^120 Ways to Ensure Unique Identifiers. It also explains why you should not rely on push ids to be unguessable/secure.
An important thing to realize from that post is that the first 8 characters of a push id contain an encoded timestamp, which is also the reason they always start with the same characters if you generate them close to each other.
The post also contains a link to a gist of the JavaScript code to generate a push id.
The set of characters that Firebase selects from is:
-0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz;
As you can see the - is just the first character in this dictionary, which is the only reason the push ids (currently) all start with a -. At some point in the future they will start with a 0, then a 1, etc. If you take the code in the gist, you could calculate when each of those roll-overs happen.
Finally: I once wrote an answer on how to get the timestamp back from a push id. Doing so is not recommended, but it can be a fun experiment: Can you get the timestamp from a Firebase realtime database key?
Not strictly a response to the question asked but related: based on #Frank's answer above it seems like a regex that will always match a Firebase push ID will look something like this:
const regex = /[a-zA-Z0-9-_;]*/gm;
This regex assumes that the ID in the string will be delimited by /. The - and ; added to cover the remaining character set. Remove the gm pattern flags if you are only after the first match.
I had a problem where I needed to extract the push ID from an URL. The push ID appeared after another known ID. The regex for such a situation can look like this:
let regex = new RegExp(`(?<=${known_ID}\/)[a-zA-Z0-9-_;]*`);
I have a webpage Default.aspx which generate the id for each new user after that the id will be subbmitted to database on button click on Default.aspx...
if onother user is also entering the same time the id will be the same ... till they press button on default.aspx
How to get rid of this issue...so that ... each user will be alloted the unique id ...
i m using the read write code to generate unique id ..
You could use a Guid as ids. And to generate an unique id:
Guid id = Guid.NewGuid();
Another possibility is to use an automatically incremented primary column in the database so that it is the database that generates the unique identifiers.
Three options
Use a GUID: Guid.NewGuid() will generate unique GUIDs. GUIDs are, of course, much longer than an integer.
Use intelocked operations to increment a shared counter. Interlocked.Increment is thread safe. This will only work if all the requests happen in the same AppDomain: either process cycling on a refresh of the code will create a new AppDomain and restart the count.
Use an IDENTITY column in the database. The database is designed to handle this, within the request that inserts the new row, use SCOPE_IDENTITY to select the value of the identity to update in memory data (ORMs should handle this for you). (This is SQL Server, other databases have equivalent functionality.)
Of there #3 is almost certainly best.
You could generate a Guid:
Guid.NewGuid()
Or you could let the database generate it for you upon insert. One way to do this is via a Sequence. See the wikipedia article for Surrogate Keys
From the article:
A surrogate key in a database is a unique identifier for either an entity in the modeled world or an object in the database. The surrogate key is not derived from application data.
The Sequence/auto-incremented column option is going to be simpler, and easier to remember when manually querying your DB (during debugging), but the DBA at my work says he's gotten 20% increases in performance by switching to Guids. He was using Oracle, and his database was huge, though :)
I use a utility static method to generate id's, basically use the full datetime(including seconds) and generate a random number of say 3 or 4 characters and return the whole thing, then you can save it to the database.