I am newer to the nosql world and am having a hard time with a certain aspect of denormalization and how my database should be architected. I will be using Firebase, but I believe this problem extends to denormalization in general.
The issue that I have is how to mark user "likes" against a piece of data when that data (lets call it "strings") is localized in many languages. Ideally I would do something like:
{
en: {
strings: {
0: "String 0 in English",
1: "String 1 in English",
...
}
},
es: {
strings: {
0: "String 0 in Spanish",
1: "String 1 in Spanish",
...
}
}
}
This way I could access the desired text in the desired language with just the query path. Easy - Except for liking. Being a global and localized set of data, if a Spanish user likes "string[0]", it should be a like against "string[0]" for all languages. I don't even know where to store "likeCount" in a setup like this.
Another approach I thought of was having the localized strings together like:
{
strings: {
0: {
likeCount: 110,
string: {
en: "String 0 in English",
es: "String 0 in Spanish"
}
},
1: {
likeCount: 13,
string: {
en: "String 0 in English",
es: "String 1 in Spanish"
}
},
...
}
}
As you can see, it is more obvious here that the "likeCount" can be attached directly to the object in this case. But now, when I query for any or multiple "strings", I will be getting ALL the languages back and the client will then have to display which one applies. With large strings and many languages, this feels like a lot of overhead.
My question then is, what is the best way to structure data in a case like this so that the strings and the likeCounts are fast and efficient?
If it helps and you want to go above and beyond with an answer/example, I will want a "user" object to reference all liked strings, but I will NOT require that a "string" also reference the users that have liked it.
EDIT:
One other case I could consider is doing manual join requests in the client. If I used the architecture in option 1 above to easily get the localized text, I could add an additional top-level object like:
{
en: {
strings: {
0: "String 0 in English",
1: "String 1 in English",
...
}
},
es: {
strings: {
0: "String 0 in Spanish",
1: "String 1 in Spanish",
...
}
},
likes: {
0: 110,
1: 13,
...
}
}
Then when in the client I could make two simultaneous requests and merge/join the results. Does this approach sound reasonable for Firebase? I do not think the overhead or latency would really suffer here.
Related
I'm using Nokalva ASN.1/C++ 6.7 compiler, and I need to set 'partial decoding' to decode some X2AP PER messages that are quite long and to extract just some field values. I'm using ASN.1 x2ap_15.3.asn specification.
To set 'partial decoding' I need to use the OSS.DataCallback directive:
----
but I don't know how to define the 'absoluteReference'
Can somebody with experience in Nokalva ASN.1/C++ 6.7 compiler please help me defining 'absoluteReference'?
This is, for example, a json representation of a Handover Request message, and I would be interested in decoding just the fields 'UE-X2AP-ID', 'Cause' and 'GUMMEI'.
value1 X2AP-PDU ::= initiatingMessage : {
procedureCode 0,
criticality reject,
value HandoverRequest : {
protocolIEs {
{
id 10,
criticality reject,
value UE-X2AP-ID : 1
},
{
id 5,
criticality ignore,
value Cause : misc : control-processing-overload
},
{
id 11,
criticality reject,
value ECGI : {
pLMN-Identity '123456'H,
eUTRANcellIdentifier '0000000000000000000011111111'B
}
},
{
id 23,
criticality reject,
value GUMMEI : {
gU-Group-ID {
pLMN-Identity '123456'H,
mME-Group-ID '1234'H
},
mME-Code '11'H
}
}
...
}
}
This is a question about a particular commercial vendor's ASN.1 Tool which is better sent directly to the vendor's support email address. Please direct your question to support#oss.com instead of using this forum.
I'm trying to get started with Firebase and I just want to make sure that this data structure is optimized for Firebase.
The conversation object/tree/whatever looks like this:
conversations: {
"-JRHTHaKuITFIhnj02kE": {
user_one_id: "054bd9ea-5e05-442b-a03d-4ff8e763030b",
user_two_id: "0b1b89b7-2580-4d39-ae6e-22ba6773e004",
user_one_name: "Christina",
user_two_name: "Conor",
user_one_typing: false,
user_two_typing: false,
last_message_text: "Hey girl, what are you doing?",
last_message_type: "TEXT",
last_message_date: 0
}
}
and the messages object looks like so:
messages: {
"-JRHTHaKuITFIhnj02kE": {
conversation: "-JRHTHaKuITFIhnj02kE",
sender: "054bd9ea-5e05-442b-a03d-4ff8e763030b",
message: "Hey girl, what are you doing?",
message_type: "TEXT",
message_date: 0
}
}
Is storing the name relative to the user in the conversation object needed, or can I easily look up the name of the user by the users UID on the fly? Other than the name question, is this good? I don't want to get started with a really bad data structure.
Note: Yes, i know the UID for the conversation & message are the same, I got tired of making up variables.
I usually model the data that I need to show in a single screen in a single location in the database. That makes it possible to retrieve that data with a single read/listener.
Following that train of thought it makes sense to keep the user name in the conversation node. In fact, I usually keep the username in each message node too. The latter prevents the need for a lookup, although in this case I might be expanding the data model a bit far for the sake of keep the code as simple as possible.
For the naming of the chat: if this is a fairly standard chat app, then user may expect to have a persistent 1:1 chat with each other, so that every time you and I chat, we end up in the same room. A good approach for accomplishing that in the data model, can be found in this answer: Best way to manage Chat channels in Firebase
I don't think you structured it right. You should bare in mind "What if" complete analysis.
Though, I would recommend structuring it this way (I made it up for fun, not really tested in-terms of performance when getting a huge traffic. but you can always do denormalization to increase performance when needed):
{
"conversation-messages" : {
"--JpntMPN_iPC3pKDUX9Z" : {
"-Jpnjg_7eom7pMG6LDe1" : {
"message" : "hey! Who are you?",
"timestamp" : 1432165992987,
"type" : "text",
"userId" : "user:-Jpnjcdp6YXM0auS1BAT"
},
"-JpnjibdwWpf1k-zS3SD" : {
"message" : "Arya Stark. You?",
"timestamp" : 1432166001453,
"type" : "text",
"userId" : "user:-OuJffgdYY0jshTFD"
},
"-JpnkqRjkz5oT9sTrKYU" : {
"message" : "no one. a man has no name.",
"timestamp" : 1432166295571,
"type" : "text",
"userId" : "user:-Jpnjcdp6YXM0auS1BAT"
}
}
},
"conversations-metadata" : { // to show the conversation list from all users for each user
"-JpntMPN_iPC3pKDUX9Z" : {
"id": "-JpntMPN_iPC3pKDUX9Z",
"date":995043959933,
"lastMsg": "no one. a man has no name.",
"messages_id": "-JpntMPN_iPC3pKDUX9Z"
}
},
"users" : {
"user:-Jpnjcdp6YXM0auS1BAT" : {
"id" : "user:-Jpnjcdp6YXM0auS1BAT",
"name" : "many-faced foo",
"ProfileImg" : "...."
"conversations":{
"user:-Yabba_Dabba_Doo" : {
"conversation_id": "-JpntMPN_iPC3pKDUX9Z",
"read" : false
}
}
},
"user:-Yabba_Dabba_Doo" : {
"id" : "user:-Yabba_Dabba_Doo",
"name" : "Arya Stark",
"ProfileImg" : "...."
"conversations":{
"user:-Jpnjcdp6YXM0auS1BAT" : {
"conversation_id": "-JpntMPN_iPC3pKDUX9Z",
"read" : true
}
}
}
}
}
I have been researching Firebase as an alternative to the recently deprecated Dropbox Datastore API. I read the articles about structuring data, but I’m still a little unclear.
I have a bunch of users:
users
- name
- email
...and each user has three database “tables”, aircraft, entries, and customFields.
aircraft
- name
- category
- make
entries
- flightDate
- departure
- destination
customFields
- name
- type
So would my Firebase data structure look something like this?
{
“users”: {
“bob”: {
“name”: …
“email”: …
},
“sally”: {
“name”: …
“email”: …
}
},
“aircraft”:{
???
},
“entries”:{
???
},
“customFields”:{
???
}
}
Thanks in advance.
Are you familiar with OOP? Each "table" is an object. Personally I would do something as follows. Since I don't understand what you're trying to achieve with the database and their objects, this may not be correct:
{
"user": {
"name": "bob",
"aircraft": {
"name": "name"
},
"entries": {
"flightdate": "27/05/2015"
}
}
}
Think in objects, not tables. Think parent and child.
But in your example, if each object (user, aircraft, entries etc.) was plurals, you can treat them as a "table", it would just be an array of objects:
{
"aircrafts":[
{
"id":1,
"name": "name"
},
{
"id":2,
"name": "name"
}
]
}
Edit: My first example was if each user had an aircraft, in retrospect it was silly, but my point still stands.
We are thinking about migrating from Pusher to Firebase. We are having troubling thinking about how Pusher channels would be represented in Firebase.
In Pusher we have a channel per user. So a user might be in a user-1 channel, another might be in a user-2 channel.
Then our backend/server would send a message to both these users via Pusher.trigger(message, ['user-1', 'user-2']).
I think this would usually be done like this:
{
web_page_1: {
user_1: {
messages: [{}, {}, ..],
},
user_2: {
messages: [{}, {}, ..],
},
...
},
web_page_2: {
user_2: {
messages: [{}, {}, ..],
},
user_3: {
messages: [{}, {}, ..],
}
},
....
}
Here the problem is: User 1 and User 2 for the same page might have a lot of messages in common. Is there a way to reduce this duplication, since these messages can get rather large, sending and storing them per user can get expensive. Also User 1 should not be able to read the messages of User 2.
It would be nice to do something like this:
{
web_page_1: {
message_1: {
user_ids: [1,2,3]
content: {},
},
message_2: {
recipient_ids: [3,4,5]
content: {},
}
...
},
web_page_2: {
message_1: {
user_ids: [1,2,3]
content: {},
},
message_2: {
user_ids: [3,4,5]
content: {},
}
},
....
}
But then, how would the security policy be applied such that a message can only be read by the user_ids specified in it.
Any pointers would be really appreciated.
If multi-cast is your use-case and the messages get large, I would indeed split the messages from the users and add message-references to the users like you show.
Root
Users
provider:344923
Name: Akshay Rawat
Messages
1: true
2: true
3: true
provider:209103
Name: Frank van Puffelen
Messages
1: true
Messages
1: It's a beautiful day
2: The sun is shining
3: I feel good, I feel good
4: And nothing's gonna stop me now
In the above data you can see that you and I are users. The provider:... is our uid, but can be anything that allows you to identify the current user. You've received messages 1, 2 and 3, while I have only received message 3. Neither of us has received message 4.
I took the Web_page level out to simplify things a bit. If you really need that level, you can add it back. The basic approach will remain the same.
You security rules can then use these message-references to see if the use can read a specific message:
{
"rules": {
"Messages": {
"$message_id": {
".read": "root.child('Users/'+auth.uid+'/Messages').hasChild($message_id)"
}
}
}
This rule defines the security for any child under messages (identified by $message_id). We grant read access if the $message_id is references as a message for the current user (auth.uid).
Let's say I have an object in the test bucket in my Riak installation with the following structure:
{
"animals": {
"dog": "woof",
"cat: "miaow",
"cow": "moo"
}
}
When performing a search request for this object, the structure of the search results is as follows:
{
"responseHeader": {
"status": 0,
"QTime": 3,
"params": {
"q": "animals_cow:moo",
"q.op": "or",
"filter":"",
"wt": "json"
}
},
"response": {
"numFound": 1,
"start": 0,
"maxScore": "0.353553",
"docs": [
{
"id": "test",
"index": "test",
"fields": {
"animals_cat": "miaow",
"animals_cow": "moo",
"animals_dog": "woof"
},
"props": {}
}
]
}
}
As you can see, the way the object is stored, the cat, cow and dog keys are nested within animals. However, when the search results come back, none of the keys are nested, and are simply separated by _.
My question is this: Is there any way provided by Riak to "reverse format" the search, and return the fields of the object in the correct (nested) format? This becomes a problem when storing and returning user data that might possibly contain _.
I do see that the latest version of Riak (beta release) provides a search schema, but I can't seem to see whether my question would be answered by this.
What you receive back in the search result is what the object looked like after passing through the json analyzer. If you need the data formatted differently, you can use a custom analyzer. However, this will only affect newly put data.
For existing data, you can use the id field and issue a get request for the original object, or use the solr query as input to a MapReduce job.