What is the situation with fees when:
I have one channel
10 users belong to the channel
one of the people added a message to the channel so all users who are listening will receive the message
Costs:
1. Adding a message costs X1
2. What will the cost be for everyone to read?
Adding a message costs X1
Adding a message will cost you exactly one document write.
What will the cost be for everyone to read?
If all 10 users are reading that message, then the cost will of 10 document reads.
Related
If a transaction reads 3 docs and then updates 2 documents successfully but something after this causes the transaction to fail... will I be charged for the 3 reads and 2 writes that were made even though they are rolled back?
Edit---
Also will the get() below only cost 1 read? Where col2 is a subcollection of doc1.
db.collection('col1').doc('doc1').collection('col2').doc('doc2').get();
Edit 2
The firebase website states the following
For example, if a transaction reads documents and another client modifies any of those documents, Cloud Firestore retries the transaction. This feature ensures that the transaction runs on up-to-date and consistent data.
So say my transaction performs 10 reads on 10 different documents. If this gets called and during exectution some of the same documents are updated by other users, which will make the transaction retry, am I going to be hit with 10 * Number of retries for my reads?
Edit 3
I have read more about the transactions here https://firebase.google.com/docs/firestore/transaction-data-contention and it states that the server side transactions will lock the documents and wait for the transaction to finish.
q1) As the transaction is locking and not retrying over and over... will multiple concurrent calls to the firebase function that has a transaction not cost any extra reads/writes and will the functions just take longer to execute because of the lock?
q2) The webpage also has a banner at the bottom stating
Note: Only the server client libraries support transactions with read operations after write operations. For transactions in the mobile/web SDKs, document reads must come before document writes.
I just tried this on my firebase function and recieve the following error...
Error: Firestore transactions require all reads to be executed before all writes.
at Transaction.get (/srv/node_modules/#google-cloud/firestore/build/src/transaction.js:76:19)
I am using firebase admin version "^8.8.0", is performing reads after writes a feature that has been added in newer versions?
If transactions fail, will I still be charged?
Yes. A read was completed so you are charged for it. (I am unsure if there are any "rollback charges" - as the change now needs to be reversed.)
What is the cost of a sub-collection document read?
The doc1 was not read - so it would not be charged. You are charged for only one read.
I couldn't find a clear text in the documentation and these answers are from my personal usage for Firebase for over a couple years. A Firebasers confirmation would be helpful.
I am working on a flutter app that fetches 341 documents from the firestore, after 2 days of analysis I found out that my read requests are increasing too much. So I made a chart on the stackdriver metrics explorer from which I get to know that my app is just reading 341 docs a single time, it's the firebase console which is increasing my reads.
Now, comes to what are the questions that are bothering me,
1)How reads are considered when we see data on the console and how can I reduce my read requests? Basically there are 341 docs but it is showing more than 600 reads whenever I refresh my console.
2)As you can see in the picture there are two types of document reads 'LOOKUP' and 'QUERY', what's the exact difference between them?
3)I am getting data from the firestore with a single instance and when I open my app the chart shows 1 active client which is cool but in the next 5 minutes, the number of active clients starts to increase.
Can anybody please explain to me why this is happening?
For the last question, I tried to disable all the service accounts and then again opened my app but got the same thing again.
Firestore.instance.collection("Lectures").snapshots(includeMetadataChanges: true).listen((d){
print(d.metadata.isFromCache);//prints false everytime
print(d.documents.length);// 341
print(d.documentChanges.length);//341
});
This is the snippet I am using. When the app starts it runs only once.
I will try to answer your questions:
How reads are considered when we see data on the console and how can I
reduce my read requests? Basically there are 341 docs but it is
showing more than 600 reads whenever I refresh my console.
Reads are considered depending on your how you query your Firestore database in addition to your access to this database from the console so using of the Firebase console will incur reads and even if you leave the console open to do other stuff, when new changes to database occured these changes will incur reads also, automatically.and any document read from the server is going to be billed. It doesn't matter where the read came from. The console should be included in that.
Check this official documentation under the "Manage data" title you can see there is a note : "Note: Read, write, and delete operations performed in the console count towards your Cloud Firestore usage."
Saying that if you think there is an issue with this, you can contact Firebase support directly to have more detailed answers.
However, If you check the free plan of Firebase you can see that you have 50K free reads per day.
A workaround that I found for this (thanks to Dependar Sethi)
Bookmarking the Usage tab of the Firestore page. (So you basically
'Skip' the Data Tab)
Adding a dummy collection in a certain way that ensures it is the
first collection(alphabetically) which gets loaded by default on
the Firestore page.
you can find his full solution here.
Also, you can optimise your queries however you want to retreive only the data that you want using where() method and pagination with Firebase
As you can see in the picture there are two types of document reads
'LOOKUP' and 'QUERY', what's the exact difference between them?
I guess there are no important difference between them but "QUERY" is getting the actual data(when you call data() method) while "LOOKUP" is getting a reference of these data(without calling data() method).
I am getting data from the firestore with a single instance and when I
open my app the chart shows 1 active client which is cool but in the
next 5 minutes, the number of active clients starts to increase.
For this question, considering the metrics that you are choosing in Stackdriver I can see 3 connected clients. and as per the decription of "connected client" metric:
The number of active connections. Each mobile client will have one connection. Each listener in admin SDK will be one connection. Sampled every 60 seconds. After sampling, data is not visible for up to 240 seconds.
So please check: how many mobiles are connected to this instance and how many listeners do you have in your app. The sum of all of them is the actual number of connected clients that you are seeing in Stackdriver.
Simple gcloud dataflow pipeline:
PubsubIO.readStrings().fromSubscription -> Window -> ParDo -> DatastoreIO.v1().write()
When load is applied to the pubsub topic, the messages are read but not acked:
Jul 25, 2017 4:20:38 PM org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSource$PubsubReader stats
INFO: Pubsub projects/my-project/subscriptions/my-subscription has 1000 received messages, 950 current unread messages, 843346 current unread bytes, 970 current in-flight msgs, 28367ms oldest in-flight, 1 current in-flight checkpoints, 2 max in-flight checkpoints, 770B/s recent read, 1000 recent received, 0 recent extended, 0 recent late extended, 50 recent ACKed, 990 recent NACKed, 0 recent expired, 898ms recent message timestamp skew, 9224873061464212ms recent watermark skew, 0 recent late messages, 2017-07-25T23:16:49.437Z last reported watermark
What pipeline step should ack the messages?
stackdriver dashboard shows that there are some acks but the number of unacked messages stays stable.
no error messages in the trace indicating that the message processing failed.
entries show up in the datastore
Dataflow will only acknowledge PubSub messages after they are durably committed somewhere else. In a pipeline that consists of PubSub -> ParDo -> 1 or more sinks, this may be delayed by any of the sinks having problems (even if they are being retried, that will slow things down). This is part of ensuring that results seem to be processed effectively-once. See a previous question about when Dataflow acknowledges a message for more details.
One (easy) option to change this behavior is to add a GroupByKey (using a randomly generated key) after the PubSub source and before the sinks. This will cause the messages to be acknowledged earlier, but may perform worse, since PubSub is generally better at holding the unprocessed inputs than the GroupByKey.
As per this quote I found:
registration_ids – Type String array – (Optional) [Recipients of a message]
Multiple registration tokens, min 1 max 1000.
Is this the actual limit of device tokens I can send a single message to? And do messages to topics have the same limit?
ex:
{
"to": [reg_token_01, reg_token_02, ..., reg_token_1000],
"priority": "high",
"data": {
"title": "Hi Peeps!",
"message": "This is a special message for only for you... More details are available..."
}
}
As always, thanks for the info and direction!
Update: For v1, it seems that registration_ids is no longer supported. It is strongly suggested that topics be used instead.
Seeing as FCM is based from the GCM core, the maximum number of registration tokens you can send to when using the registration_ids parameter is 1000. I'm pretty sure you did see that in the official documentation.
So if ever you still intend to use the registration_ids parameter but you need to send it to more than 1000, you can follow what was #Eran said in his answer here:
If you need to send the same message to more than 1000 Registration IDs, you simply split the sending process into groups of 1000 Registration IDs. Each group would be sent in a separate request to GCM server.
However, when it comes to topics, there is no limit. There used to be, but it was scrapped years ago. I have mentioned it my previous answers before:
Answer 1:
Nope. As per their blog last December 2015:
We’re now happy to announce that we’re allowing unlimited free topics for your app. This means app developers can place an unlimited number of devices within each topic and create an unlimited number of topics.
Answer 2:
Nope. Seeing that FCM has GCM as its core, there is no limit in the number of Topics for any app. There used to be a 1 million limit, but it was removed. You can refer to this Google Developers Blog for that.
Also, when creating a Topic in FCM, it would seem that it takes a day for it to be available, as per this post.
Apparently, there are legacy API's to achieve it. See here Send FCM message to multiple registration tokens
The method sendToDevice accepts array of registration tokens
I want to programatically send SMS / text messages from my Meteor app; many people recommend twilio for this, and there are several Meteor Twilio packages (findable via atmosphere and/or googling/binging). I don't know if one of these packages is decidedly better than the other, but for now, at least, I'm using the abhiaayer:meteor-twilio package.
My concern is that, when you create a Twilio account, they assign you a "from" phone number (you can't, apparently, just use your own). Maybe I'm misunderstanding how this works, but before I go too far down this path, I want to know if the sender of the SMS -- IOW the user of my app -- will be able to use their phone number as the "from"/sender phone number.
After all, the whole point of my app is to allow the user to send out multiple identical texts (such as "where are you?" or "are you okay?") to friends/family, and then get a response back from them (to his phone, not to a Twilio-supplied number).
If it's possible to use Twilio with your own phone number as the "from"/sending number, has anyone got suggestions on Meteor package preferences (abhiaayer, andreioprisan, DispatchMe) and how that can be accomplished from Meteor?
I imagine the basic code would be pretty much the same, regardless of package used; e.g., here's an example from the andreioprisan package
twilio = Twilio(ACCOUNT_SID, AUTH_TOKEN);
twilio.sendSms({
to:'+16515556677', // Any number Twilio can deliver to
from: '+14506667788', // A number you bought from Twilio and can use for outbound communication
body: 'word to your mother.' // body of the SMS message
}, function(err, responseData) { //this function is executed when a response is received from Twilio
if (!err) { // "err" is an error received during the request, if any
// "responseData" is a JavaScript object containing data received from Twilio.
// A sample response from sending an SMS message is here (click "JSON" to see how the data appears in JavaScript):
// http://www.twilio.com/docs/api/rest/sending-sms#example-1
console.log(responseData.from); // outputs "+14506667788"
console.log(responseData.body); // outputs "word to your mother."
}
});
I did create a Twilio account, and have the "from" number (not my actual cell phone number) they assigned my account, and the SID, but I don't kow what the AUTH_TOKEN should be.
If what I intend is not possible from Twilio, what I may do is just write the app so that all the selected "to" numbers can be copied to the clipboard, so that the user can paste them into his SMS screen's "recipients" textbox; but I was hoping to afford the user the luxury of simply tapping a button.
I don't think you can do this (send from the user's own cell number using Twilio). If your app is an actual mobile app as opposed to a web app you can use the Android or iOS APIs for sending SMS from the user's phone. I'm not sure of the details of this and certainly with iOS it looks like you can't do so fully automatically for obvious reasons (you could just spam all the user's contacts for instance, and this would incur a monetary cost to the user).
There's a Cordova plugin here which can do what you want: https://github.com/cordova-sms/cordova-sms-plugin (I haven't used it so can't vouch for it but it seems to be actively maintained).
If it is a web app, your suggesting of copying the numbers to clipboard is by far the most straightforward solution and I would recommend that in the first instance. For completeness I've included some details and considerations below of how you can do 2-way communication with Twilio (or alternatives such as Nexmo) in a couple of ways, in case someone finds it helpful. It's not exactly trivial though. 1-way communication is a lot easier!
Send via API
From your app the user types a message and selects who to send it to. You app then connects to your server which uses the Twilio API to send N messages to the N recipients. These messages will appear to come from the Twilio-provided number.
Send via SMS
The user manually sends an SMS to your Twilio number and then you receive a webhook to your server from Twilio with the message details. You have to do the processing to work out who to forward the message to and then use the Twilio API to do so. Again the recipients will see a message from the Twilio-provided number.
Replies
When a recipient replies to the message, Twilio sends you a webhook with the details and you can determine who sent the original message and forward the reply back to the original sender.
Number Pooling
One of the obvious flaws here is that if multiple users send a message to the same person then there's no way of telling which message they are replying to. There's no message IDs passed with SMS so you have to use multiple sending numbers (one per unique sender to a particular recipient). The amount of numbers required is basically the most number of different senders one user would have to reply to (this is not generally possible to work out beforehand, so you would have to call the API to provision a new number on the fly).
To give a more concrete example say you have 2 users (S1 & S2) and 3 recipients (R1, R2 & R3). You have 1 Twilio-provided number (N1).
S1 sends a message to R1 via your app, you use N1 to send the message via the Twilio API. R1 receives the message from N1. If they reply, you receive a message to N1 from R1 so you know you need to forward it to S1.
S1 sends a message to R2 via your app, R2 has not yet received any messages so you can reuse N1 to send the message. R2 replies to N1 and again you can forward it to S1. If this is delivered in-app, no further problems, if the reply is forwarded via SMS then we'd need to provision a new number (N2) to enable S1 to reply to R2's reply.
S2 sends a message to R3 via your app, as before you can reuse N1 and still route the reply correctly.
Now if S2 sends a message to R1, we realise that R1 has already received messages from S1 using number N1. In this case we can't use N1 because we cannot identify who the reply was intended for. If we haven't already, we need to provision a new number (N2) and now we can send the message so R1. When R1 replies to N2 we know the reply needs to be forwarded to S2.
The more users sending to one recipient, the bigger (and more expensive) your number pool gets. It's probably worth implementing some kind of timeout (say 72 hours) in which the recipient can reply. So in this case if the timeout had expired after S1 sent the message to R1 via N1, we could reuse N1 for the communication between S2 and R1. Obviously this isn't entirely foolproof but it could reduce costs.
Identity
The other issue with sending from a pooled number. I send a message to Dan via an app, and he receives that message from a random number:
Hi Dan, how are you?
How does Dan know who sent the message? You'd have to add some identifier to every message (or at least the first in each conversation thread).