My application requires a monitoring service which I am hosting on Heroku which monitors, corrects, and modifies data based on changes to the users account. However there are some fields on the users account which are updated at rapid speeds, and having all of these accounts spam my monitoring service with data is very inefficient.
Is it possible to ignore fields when listening to child_changed so that the callback is not fired and the data is never downloaded? Depending on the users speed, I could be downloading hundreds of accounts a few times a second on my monitoring application and that's just something that will go down in flames.
Here's an example of what I'm looking for:
{
name: 'Jimmy',
birthday: 'Mar 23, 1976',
biography: 'Random text',
interests: {
// ...
}
}
Perhaps I want to ignore the biography field, therefor the child_added feature should download new account information ONLY if one of the following values are changed:
name
brithdate
interests
and the data should be ignored and not downloaded if the biography field is changed, because it's not a field that is relevant to my monitor, and just wastes bandwidth.
Firebase always synchronizes complete nodes. If you want to synchronize only a subset of each node from a list of nodes, you should separate that subset into its own top-level node.
profiles: {
uidJimmy: {
name: 'Jimmy',
birthday: 'Mar 23, 1976',
biography: 'Random text',
}
},
interests: {
uidJimmy: {
// ...
}
}
You'll find this to be a common these when using Firebase (or most other NoSQL databases): you have to model the data for the way that your application wants to consume it. I highly recommend reading this article on NoSQL data modeling.
Related
Can anyone give me any advice on how to create an LP pool on the Solana devnet?
I planned this job for testing swaps between specific two tokens on the devnet using the Raydium protocol.
So, I need to create a swap pool on the devnet first.
To achieve this, I did it like below.
First of all, to list on the serum market, I cloned the Raydium-Dex repository on my mac and changed the Serum dex's program id from the mainnet to the devnet, and I success registered on the devnet serum. (Custom token with SOL pairs)
As a result, I got a serum market program id.
After that, I cloned the Raydium-frontend repository to create a liquidity pool. And modified wellknownToken.config.ts so that my custom tokens could be possible to create a new pool.
Finally, I could access the create pool UI from the localhost web UI.
I clicked Initialize Liquidity Pool button on the UI and got a Toast message Create a new pool Transaction Sent apparently.
However, It is not worked well. Because I can not find the transaction hash on the Solscan website.
I tracked the button's click event codes and I figured out one thing.
One of the result value of Liquidity.makeCreatePoolTransaction function has a null value, especially, feePayer.
const { transaction: sdkTransaction1, signers: sdkSigners1 } = Liquidity.makeCreatePoolTransaction({
poolKeys: sdkAssociatedPoolKeys,
userKeys: { payer: owner }
})
const testTx = await loadTransaction({ transaction: sdkTransaction1, signers: sdkSigners1 })
console.log('feepayer', testTx.feePayer?.toBase58()) // null
I felt this is not the preferred (good) way to create a swap pool on the Solana devnet, but I can not find a better way to achieve this task.
What am I missing? or What am I should read or learn?
please give me some advice on how to do it to make it works.
Thanks.
It looks like the transaction created with Liquidity.makeCreatePoolTransaction hasn't been sent to the network, so it doesn't exist anywhere. Be sure to send and confirm the transaction before trying to load it. You can use something like:
const { transaction: sdkTransaction1, signers: sdkSigners1 } = Liquidity.makeCreatePoolTransaction({
poolKeys: sdkAssociatedPoolKeys,
userKeys: { payer: owner }
});
await sendAndConfirmTransaction(connection, transaction, [wallet]);
Note that this requires a connection to send and a wallet to sign. More info at https://github.com/solana-labs/solana/blob/9ac2245970de90af30bff9d1f7f35cd2d8f2bf6d/web3.js/src/util/send-and-confirm-transaction.ts#L18
You might run into other issues though, because the Raydium program isn't deployed to devnet: https://explorer.solana.com/address/675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8?cluster=devnet
I learned about firebase and cloud functions recently and have been able to develop simple applications with them.
I now want to expand my knowledge and really struggling with Trigger Email Extension.
On a specific event on my firebase, I want to fire an email to the user in a custom format, but I am unable to even activate the extension for now.
Can someone please explain with example please about these fields marked in the picture?
I had this question too, but got it resolved. Here's your answer:
"Email documents collection" is the collection that will be read to trigger the emails. I recommend leaving named "mail" unless you already have a collection named mail.
"Users collection (Optional)" refers to a collection (if any) that you want to use in tandem with a user auth system. I haven't had this specific use case yet, but I imagine once you understand how Trigger Email operates, it should be somewhat self-explanatory.
"Templates collection (Optional)" is helpful for templates in which you can use handlebar.js is automatically input specific information per user. (eg. <p>Hello, {{first_name}}</p> etc.) Similar to the previously mentioned collections, you can name it whatever you want.
How to create a template (I have yet to actually implement this, so take this with a grain of salt):
In your templates collection, you want to name each document with a memorable ID. Firebase gives the example:
{
subject: "#{{username}} is now following you!",
html: "Just writing to let you know that <code>#{{username}}</code> ({{name}}) is now following you.",
attachments: [
{
filename: "{{username}}.jpg",
path: "{{imagePath}}"
}
]
}
...specifying a good ID would be following. As you can see, the documents should be structured just like any other email you would send out.
Here is an example of using the above template in javascript:
firestore()
.collection("mail")
.add({
toUids: ["abc123"], // This relates to the Users Collection
template: {
name: "following", // Specify the template
// Specify the information for the Handlebars
// which can also be pulled from your users (toUids)
// if you have the data stored in a user collection.
// Of course that gets more into the world of a user auth system.
data: {
username: "ada",
name: "Ada Lovelace",
imagePath: "https://path-to-file/image-name.jpg"
},
},
})
I hope this helps. Let me know if you have an issues getting this set up.
I'm creating a database in Firebase to manage a local basketball league, for the first part of development I want to work on match management, mainly scoring and foul registration.
The thing with scoring is that a player can score 1, 2 or 3 points and there are different kinds of fouls, like regular fouls and technical fouls, I want to be able to differentiate between those.
Also, a small detail is that a person can play for different teams in different divisions
Here is my idea for the data structure in Firebase:
divisions:{
division1:{
name:"first division"
teams:{
team1:true
team2:true
}
}
}
teams:{
team1:{
name:"Team 1"
division: division1
players:{
player1:true
player2:true
}
matches:{
match1: true
}
}
}
players:{
player1:{
name:"Player 1"
phoneNumber:"555-XXXX"
address:"123 address"
teams:{ //A player can play for different teams in different divisions
team1:true
team2:true
}
}
}
matches:{
match1:{
date:10-20-2019
court:"West Park"
referee:"John Doe"
players:{
player1:{
/*Should I store points scored and fouls comitted in here and
the players collection?*/
}
player2:{...}
}
}
I'm unsure as to where to put the data regarding points and fouls, in the future I plan to use the database to create statistics and such, but in the meantime I just want to have a registry for matches including the players, their points score and fouls.
There is no singular correct way to model data in a NoSQL database. It all depends on the use-cases of the app you want to build. In fact, when using a NoSQL database, it is quite common to adapt your data model as you add new use-cases to your app.
On your current model, I would add an additional top-level data structure to store information about each individual match. Something like this:
matches: {
matchid1: {
teams: {
team1: true,
team2: true
},
events: {
"-Lasdkjhd31": {
time: "2m44",
type: "foul",
player: "player1id",
team: "team1"
}
}
}
}
But as said, that is dependent on the use-cases of the app. So my structure above allows the storing of event information for each match, which clearly would be useful if you want to show a timeline of what happened in each match.
Beyond general guidance, it's hard to be concrete. I do recommend that you read/watch these though:
NoSQL data modeling
Firebase for SQL developers
The Firebase documentation on data modeling
Getting to know Cloud Firestore (which is about Firestore, but the same logic often applies to Realtime Database too)
I am using Firestore to figure out, in real-time, each user's share of the cost of an item. Example:
/tickets/100/ticket-item/1:
{
name: 'Red Dead Redemption'
price: '5000'
payers (array of maps): [
{
name: 'John',
share: '1666'
},
{
name: 'Jane',
share: '1667'
},
{
name: 'Jack',
share: '1667'
}
]
}
Given that the max write rate to a document is 1/second, will the write always fail if two users add themselves to the same ticket item doc at the exact same time?
I know that this can be mitigated to an extent by using transactions, but a transaction will only re-execute a finite number of times. Let's say it re-executes up to 5 times. If 6 users write to same ticket item doc at the exact same time, can I expect one of these writes to fail?
I would appreciate any and all advice regarding how to handle this.
will the write always fail if two users add themselves to the same ticket item doc at the exact same time?
Yes it will. So if you are sure you'll have situations in which two or even more users will try to write/update data in a single document in the exact same time, I recommend you to be careful about this limitation because you might start to see some of this write operations to fail.
I know that this can be mitigated to an extent by using transactions
It's a good idea but please be aware that transactions will fail when the client is offline.
If 6 users write to same ticket item doc at the exact same time, can I expect one of these writes to fail?
As the docs states, a transaction will only re-execute a finite number of times. But please also note that in case of a transaction failure:
A failed transaction returns an error and does not write anything to the database.
So all you have to do is to take some action in case o transaction failure.
I'm researching same problem.
May be like a solution: moving "payers" into separate collection with a ticket_id field?
So you'll have no limitations.
From my last question let me decided to write huge amount of data to
firebase databases for testing purpose.
Here is the outcome
1000 record: Nothing significant happen it work fine.
10000 record: Response from all other read operation from firebase return only after the write operation complete.
100000 record : Same as the result of 10000 record but took more longer, I can't perform any firebase operation unless force close the app and reopen it. The screen started hang for some time might because I perform loop in main thread(ios).
1 million record : I'm afraid so never try.
The reason I need to write such amount of data is because I building some social app(android,ios,web) it use SQL before but I think it is time to switch to firebase. By studying this I having the idea on how to build a user feed without using the IN clause. The data structure look like this
users
user1
name: bob
user2
name: alice
follows:
user1: true
posts
post1
author: user1
text: 'Hi there'
feeds
user2
post1: true
As the example If one of the user having 61 million follower it will need to insert record to 61 million feeds/$uid/. Which the write operation barely survive with 100k. On this link it suggest not to do it in the client side but big point of firebase is it is backendless how I suppose to write beside from client side.
So my question is there any efficient way to achieve on how to not get other read operation interupt by this kind of massive write operation? Or there is way better data modeling for this?
I really need help. I really apperciated even just a comment.
Lets start with an example Firebase structure using observers and events to capture feeds from specific users.
We have a users node which stores data about all of the users, in this case just their name. We also have a feeds node which stores the feeds. In this case we could also call it messages as we are just storing messages users create.
users
uid_0
name: "Scott"
uid_1
name: "Frank"
uid_2
name: "Leroy"
feeds
feed_0
uid: uid_1
msg: "some message from uid_1"
feed_1
uid: uid_2
msg: "a message from uid_2"
Assume that Scott (uid_0) wants to 'subscribe' to any feeds from Frank and Leroy.
func addSomeFeeds() {
self.addFeed(uidFeed: "uid_1")
self.addFeed(uidFeed: "uid_2")
}
func addFeed(uidFeed: String) {
let feedsRef = self.ref.child("feeds")
let feedQuery = feedsRef.queryOrdered(byChild: "uid").queryEqual(toValue: uidFeed)
feedQuery.observe(.childAdded, with: { snapshot in
let feedDict = snapshot.value as! [String: Any]
let msg = feedDict["msg"] as! String
print(msg)
})
}
and the output
some message from uid_1
a message from uid_2
and then if uid_2 adds another feed (with a message)
Feed From uid_2
The above code is run on Scott's device and attaches an observer to the Feeds node that 'watches' for any feeds added by Frank or Leroy.
This could expanded and watch for changes via .childChanged so if the feed has multiple 'posts' in it any time a post is updated in a particular feed, Scott's app will be notified of the changes within that feed.
Another option is to add a more generic observer to the feeds node wheras the app would be notified for any feed added, changed or removed. In that case, when the app receives a snapshot of the feed, simply compare it to an array of feeds the user is interested in and if doesn't match one of those, ignore it, otherwise, notify the user.