When using the Admin SDK of Firebase realtime database, it establishes a connection web socket connection over HTTPS to the server, which includes about 4Kb ~ 5Kb of overhead (downloads), no matter how small the set or updated data (storage). And every device establishes a new connection after a few minutes.
My app will set or update data frequently so I am assuming Admin SDK may not be a good choice. Is there any way to reduce the download size or make these distinct user connections stay more on the server rather than for a few minutes?
Note: I also checked the Rest API after the post above, similarly it does cost 4Kb ~ 5Kb of overhead at each connection
firebaser here
This overhead mostly comes (about 3½KB) from the SSL/TLS handshake that is needed to establish a secure connection. Since Firebase can only be accessed over a secure connection, there is no way to avoid this overhead. The best you can do is keep the connection open longer to minimize the number of times you need to reestablish the connection.
Related
Consider a chat system where one user is listening to multiple parent nodes within one chat conversation.
Group title
Group description
Messages
Does observing/listening to the 3 above mean that it adds up to 3 connections on the 200k concurrency limit? I can't seem to understand the proper definition of a concurrent connection.
Each app (strictly speaking: each FirebaseDatabase instance) keeps a single connection to the Firebase Realtime Database server, no many how many listeners it has open.
Also see the Firebase FAQ, which says this about it:
A simultaneous connection is equivalent to one mobile device, browser tab, or server app connected to the database.
To overcome latency, On "Startup.cs" of asp.net core 2.1, I am creating 2 static connections to Azure Redis & reuse those same connection instances during the application entire life cycle.
Is it is good practice to create multiple connections to one Azure redis instance? what is max. no. of connections? will multiple instance have billing implications? Is Azure redis usage charges based number of connections or as per the amount of data transfer? please confirm.
First, it is not a good practice to create two Azure Redis static connections in the application.
In general projects, Redis is not frequently used, but is instantiated and created when the business needs it, and released after use. If you need to use it frequently, you can instantiate it in Startup.cs when the project starts, and define an instance globally, so that there will be no frequent creation and deletion of instances.
For Azure Redis billing methods, you can refer to the official documentation. It is not based on the number of connections nor the amount of transmission. It is billed according to time.
It is actually recommend to use different connections to reflect the varying data packet sizes ie you could setup a connection with higher timeout for data that is bigger in size, as opposed to data that is small in size. This is recommended only when you have data packets being stored in redis of varying sizes ex: 1kb to 100 kb and you cannot reduce their size of the packet.
Having different connection ensures that pipelining that usually happens when fetching data does not result in cascading timeouts. Multi connection is also recommended in Microsoft docs, have a look here by scrolling to the bottom and see point 3
https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-troubleshoot-client#large-request-or-response-size
We are using Cosmos DB SDK whose version is 2.9.2. We perform Document CRUD operations. Usually, the end-to-end P95 latency is 20ms. But sometimes the latency is over 1000ms. The high latency period lasts for 10 hours to 1 day. The collection is not throttling.
We have get some background information from:
https://icm.ad.msft.net/imp/v3/incidents/details/171243015/home
https://icm.ad.msft.net/imp/v3/incidents/details/168242283/home
There are some diagnostics strings in the tickets.
We know that the client maintains a cache of the mapping of logical partition and physical replica address. This mapping may be outdated because of replicas movement or outage. So client tries to read from the second/third replica. However, this retry has significant impact on end to end latency. We also observe that the high latency/timeout can last for several hours, even days. I expect there’s some mechanism of refreshing mapping cache in the client. But it seems the client stops visiting more than one replica only after we redeploy our service.
Here are my questions:
How can the client tell whether it’s unable to connect to a certain replica? Will the client wait until timeout or server tells client that the replica is unavailable?
In which condition the mapping cache will be refreshed? We are using Session consistency and TCP mode.
Will restarting our service force the cache to be refreshed? Or refreshing only happens when the machine restarts?
When we find there’s replica outage, is there any way to quickly mitigate?
What operations are performed (Document CRUD or query)?
And what are the observed latencies & frequencies? Also please check if the collection is throttling (with custom throttling policy).
Client do manage the some metada and does handle its staleness efficiently with-in SLA bounds.
Can you please create a support ticket with account details and 'RequestDiagnostis' and we shall look into it.
Our firebase realtime database is reaching it's 100k connection limit.
We have a object with some 30 keys some of them are int & float and are updated more often while the other are string which are updated less frequent.
Right now we fetch the whole node but I'm thinking of dividing the object in 2 parts.
1.one for more often updated but less in size data.
2.Second for less frequent updated, larger in size data.
So we can reduce the database usage.
The question is will the simultaneous connections increase to double? as there will be now 2 listeners instead of 1?
The number of connections to Realtime Database is almost certainly not going to be related to the types of data you're storing, or the way that data is organized.
Every mobile client establishes exactly one websocket to Realtime Database, and that socket is only active when the app is running in the foreground (and maybe a bit outside of that in some cases). That socket is considered a single connection, and all traffic between the database and the client go over that socket. It doesn't matter what data is being transferred - there is only that one socket.
Don't confuse listeners with connections. All added listeners/observers in an app use the same socket. Reducing the number of listeners will not reduce the number of connections.
I have heard the term connection pooling and looked for some references by googling it... But can't get the idea when to use it....
When should i consider using
connection pooling?
What are the advantages and
disadvantagesof connection pooling?
Any suggestion....
The idea is that you do not open and close a single connection to your database, instead you create a "pool" of open connections and then reuse them. Once a single thread or procedure is done, it puts the connection back into the pool and, so that it is available to other threads. The idea behind it is that typically you don't have more than some 50 parallel connections and that opening a connection is time- and resource- consuming.
When should i consider using
connection pooling?
Always for production system.
What are the advantages and
disadvantages of connection pooling?
Advantages:
Performance. Use a fixed pool of connection and avoid the costly creation and release of connections.
Shared infrastructure. If your database is shared between several apps, you don't want one app to exhaust all connections. Pooling help to limit the number of connection per app.
Licensing. Depending on your database license, the number of concurrent client is limited. You can set a pool with the number of authorized connections. If no connection is available, client waits until one is available, or times out.
Connectivity issue. The connection pool that is between the client and the database, can provide handy features such as "ping" test, connection retry, etc. transparently for the client. In worse case, there is a time-out.
Monitoring. You can monitor the pool, see the number of active connections, etc.
Disadvantage:
You need to set it up and configure it, which is really peanuts usually.
You should use connection pooling whenever the time to establish a connection is greater than zero (pretty much always) and when there is a sufficient average usage such that the connection is likely to be used again before it times out.
Advantages are it's much faster to open/close new connections as they're not really opened and closed, they're just checked out/in to a pool.
Disadvantage would be in some connection pools you'll get an error if all pooled connections are in use. This usually is a good thing as it indicates a problem with the calling code not closing connections, but if you legitimately need more connections than are in the pool and haven't configured it properly, you could get errors where you wouldn't otherwise.
And of course there will be other pros and cons depending on the specific environment you're working in and database.
In .NET, if you are using the same connection string for data access then you already have connection pooling. The concept is to reuse an idle connection without having to tear it down & recreate it, thereby saving server resources.
This is of-course taking into consideration that you are closing open connections upon completion of your work.
connection pooling enables re-use of an existing, but not used database connection. by using it you eliminate the overhead of the connection/disconnection to the database server. it provides a significant performance boost in most cases. the only reason i can think of not to use it is if your software won't be connecting frequently enough to keep the connections alive or if there's some bug in the pooling layer.
If we required to communicate with the database multiple times then it is not recommended to create a separate Connection Object every time, because creating and destroying connection object impacts performance.
To overcome this problem we should use a connection pool.
If we want to communicate with the database then we request a connection pool to provide a connection. Once we got the connection, by using it we can communicate with the database.
After completing our work, we can return the connection object back to the pool instead of destroying it.
The main advantage of connection pooling is to reuse the same connection object multiple times.