I'm using firestore for a project in a developing country with intermittent internet access. The internet may go down for up to a day. The client has other systems that use an onsite instance (ie a server they have that runs SQLServer) - that way the systems are still operational if the internet access goes down.
Is it possible to set up Firestore similarly - so that apps that use Firestore can still access it without internet access? If not, what could be a recommended workaround?
No. Cloud Firestore is only available as a cloud-hosted database. There are no "on prem" solutions to run it outside of Google's cloud.
Firebase provides a local emulator, but that's only suitable for running local tests.
Please take a look at this video https://www.youtube.com/watch?v=oDvdAFP6OhQ
Firestore is not designed to work as a standalone database. However, it's designed to withstand longer periods of downtime, hours and even days. But if it's days - you should already consider a different solution.
In your case, I suspect what you'd like to have various clients synchronized even if the Internet is down. This will not work in Firestore as each client would have its own copy of the cache. Thus I wouldn't recommend using Firestore in your scenario.
Related
We are building an app for our teams out in the field that they collect their daily information using Firebase. However one of our concerns is poor connectivity. We are looking to build an Online/Offline button they can click to essentially work offline for when things slow down. We've built a workflow in which we query all the relevant information from Firestore.
I wanted to know if there was a way to tell Firestore to work directly on the cache only and not try to hit the servers directly. I don't want Firestore attempting to make server calls until they enable online again.
You shouldn't need to do this. If you use realtime listeners, they will already first return the data from the local cache, and only then reach out to the server to check for updates.
If you are performing one-time reads, the SDK will by default try to reach the server first (since it has only one chance to give you a value). If you want it to only check the local cache, you can pass an argument to the get call to do so.
You can also disable the network completely, in which case the client will never call on the network and only serve from the local cache. I recommend reading about that and more in the documentation on using Firestore offline.
My assumption was that Firestore would not consider the client cache when evaluating Read usage. However, when I have persistence enabled in my application, it seems I can run the same queries endlessly and not see any change in my usage metrics. I've tested this over and over - when persistence is disabled, I see the usage metrics go up.
Given the amount of testing I've done, I feel that I should be able to safely declare that Firestore must be recognizing my local cache and only charging me for Reads on updated documents. However, I cannot find documentation around this anywhere, and I have combed through the docs. I'd certainly think if Google wasn't charging me for these cached Reads that they would want to highlight that benefit, whereas they really only highlight enablePersistence as an offline benefit - so I am perplexed.
Does someone have some insight into what is going on here?
Example enable persistence: firebase.firestore().enablePersistence()
Example query: myCollection.where(condition).get()
According to this documentation
Offline data persistence feature caches a copy of the Cloud Firestore data that your app is actively using, so your app can access the data when the device is offline
So, it is clear that you will only be charged when calling get() on the server to-read documents, mentioned in this answer
Also, be aware that
Offline Persistance is enabled by default for Android and iOS.
The pricing model for Cloud Firestore
This is a really great question. To clarify what the question is asking: "If my app is online and makes a query, but the query is fully returned by the offline cache, will I still be charged?"
I suspect yes.
Can i use Firebase Emulators (Firestore, Functions) to serve them permanently on example VPS server?
Hosting + storage will be apache and from for example #angularfire I will call firestore and functions
Is this possible or i must pay for use functions :c
#iAmPoor
I won't say that it's impossible to use the emulator in this way, but I strongly recommend against it. The emulator was only meant to be used during development and testing on your local machine before pushing code to production. It's not meant to run persistently. You could run into problems if you try to use it this way, since it's not tested to work like this.
You should know that Firebase and Google Cloud products have a free tier where you don't pay anything at all unless you exceed the free limits. I suggest reviewing the billing structure to find out what you actually get for free per month. I think it's quite generous.
As Doug said it is not verified to be used as such but I have tested an on-premises solution this way (no internet, about 100 IoT clients on LAN). Hosting server handles load well. The storage emulator works fine for small data sizes. The max time I know it ran without a problem is about 3 months.
I don't know if my app will work in China and I don't know how to test it. I had read somewhere that there's a firebase Chinese domain .cn in operation. Is there a possibility to transfer the current project to the Chinese domain and make it work, if it doesn't already?
If I create a new project and select the Chinese servers for my firebase services then will it have any other adverse consequences for the rest of the world? This solution would literally force all the developers to have the firebase servers located in China, just so they don't miss out out on their Chinese user base, which I assume could be an issue to some.
I might not have the best ideas but any suggestion and a workaround would be appreciated.
There isn't a simple way to do that. Indeed, trying to set the servers there might a be way, but it's not a guarantee, since China Firewalls are constantly changing. You can try some onlines tools as this one here to verify if your domain is blocked, by adding the URL of your application.
Besides that, the best option is the one clarified in this other post here. You would create an API in Heroku and make the data and calls from Firebase go over this API, so it would make the connection. Another option you may try is using a proxy server - as very well explained in the article Firebase: Accessing Firestore and Firebase through a proxy server - so it gets complied and it's possible to be used in China.
The usage of VPN is not a very good option, as not of them work in China and the ones that work can easily vary, also, this could impact the usability, since it might affect performance and usage for the users there.
To summarize, there isn't a specific and fully correct answer to using Firestore in China, due to the fact that updates and changes on their rules are regular, but these are some alternatives that you can give it a try. It's more about trying the one that would be less damaging for your use case.
Let me know if the information helped you!
Might be helpful for someone...
There is a Firebase alternative for China - Huawei Mobile Services. It contains services like Auth Service, Cloud Functions, Cloud DB, Crash, Push Kit, Ad Kit, and many more...
Our company is thinking about moving to the cloud. Would we still be able to meet all our current requirements (below). We want to be able to easily scale in the future without high costs.
5 ASP.net 4.0 websites running (using sql databases, see below)
SQL Server 2008 Express (8 databases on this)
2 Scheduler services running (send nightly reports via email e.g. new orders in db)
MongoDB and Memcached are also installed on server
Currently the websites are on a separate server from the database server for security reasons.
We were thinking about Windows Azure and Amazon Web Services (AWS) as providers, which would best fit our requirements?
Are there any other factors we need to consider?
Re: SQL Databases: on Windows Azure this would map to SQL Azure. Costs start at $5/month for up to a 100 MB instance - and goes all the way up to 150 GB - and goes beyond that with Federations.
Re: 5 ASP.net 4.0 websites running: these map naturally into Windows Azure Web Roles. The "small" instance is $0.12/hour/instance, and you'll usually want two instances (to avoid single point of failure for a few scenarios). Depending on your load, you may be able to put all 5 sites on the same instances. If you have very low usage sites, consider the $0.05/hour/instance "extra small" instance.
Re: Currently the websites are on a seperate server from the database server for security reasons: of course this is also doable.
Re: 2 Scheduler services running: Running Windows Services is no problem.
Re: send nightly reports via email e.g. new orders in db: No problem doing, though is not baked into Windows Azure directly, but there are many simple ways to do this (even for free, such as via SendGrid).
Re: We want to be able to easily scale in the future without high costs: you will need to do the math regarding your actual costs, but Windows Azure can surely scale.
Re: MongoDB and Memcache are also installed on server: These can both be run on Azure. Check out https://github.com/mongodb/mongo for MongoDB. Also, the Azure Caching service is also avail (managed for you).
Re: We were thinking about Azure and Amazon as providers, which would best fit our requirements: These are functionally very similar (in capability and cost), with a few noteworthy differences.
Windows Azure is Platform as a Service meaning that you don't need to worry about Virtual Machines, but rather Applications. In other words, you upload your (basically) Zipped app package to the cloud for execution. With Amazon, you will be dealing with the Virtual Machine yourself. In Azure, you get a copy of Windows Server 2008 which is managed for you, but you can also do admin things to it if you need to. This is far less of an advantage if your app is an old messy install that isn't really clean (though may not be a good high-value cloud candidate anyway).
Windows Azure has an emulator that works great - F5 right from visual studio to work with storage system and VMs and more popular features.
Re: Are there any other factors we need to consider: Yes. With any cloud application, you need to be prepared to deal with scaling out (not up), dealing with transient retries (you may need to retry an operation to a cloud service - any cloud service). The benefits of this are much better (and more cost-effective) scalability and higher reliability (when you run across nodes, you don't have a single point of failure). Be sure to understand when/where storage on a VM is persistent vs. ephemeral. There are more considerations, but these are primary ones.
You may want to check out the Windows Azure Pricing calculator.
Good luck! And welcome to the cloud.
with the exception of the scaling question, and the 2 physical servers, you can move this functionality into a hosted environment and you will technically be in "the cloud". This could be a dedicated or VPS (Virtual Private Server), or even a shared server if you are small.
Those can allow for growth over time...you just need to upgrade what you have with the provider.
You also could use a colo-server with a hosting provider, which basically means you put your hardware in an hosting provider rack, and use their electricity and bandwidth. They charge based on bandwidth usage.
Since you are using SQL Express, remember that each database is limited to 8gb. So that will limit your growth at some point. That would entail an upgrade from Express to regular SQL if you don't want to re-engineer anything.
Have you considered AppHarbour? It has Memcached, MongoDB, SQL Server and so on, and is quicker to deploy to than Azure. I like Azure, but there is quite a learning curve and I have found the connection to SQL Azure to be pretty bad - which means re-engineering your DAL to use something like the SQL Transient Failure Library = a bit of a faff for existing projects.
AppHarbour does not have blob storage - so if you are uploading files you will need to use Azure Blob Storage or Amazon S3 or some equivalent as well.
Hope this helps.
Not an expert but being that Asp.net is a Microsoft product it should be easier to migrate to azure, although from what I have heard AWS shouldn't be difficult. Another thing you may want to consider is cost. Last time I checked AWS is significantly less costly unless you already pay for MSDN subscriptions.
All the requirements you sum up are not any issue to deploy in Windows Azure. You can find a lot of information on the internet on how to do this.
Keep in mind, if you want to deploy your services to windows azure, you'll need to do some code review of your applications to fix session state, output cache and so forth on your web applications.
Since you want to scale them out and they are sitting behind a non-sticky round-robin load balancer, you will run into issues with your session state if it is saved on the machine itself. You'll need to part session state to SQL Azure or to the Windows Azure table storage for example.
Installing MongoB and Memcache in Azure is not an issue, you'll find a lot of information on how to do it, but it'll require some to set up your role and the scripting
codingoutloud has given a very detailed answer. I would add two very key considerations to think about when moving any application to Azure (or, indeed, many other cloud providers).
Local state
With normal Azure, they reserve the right to shut down any one instance of a role at any time in order to move or upgrade it. This means you always need at least two instances of any one role and they will be transparently load balanced. If your current websites are currently running on individual servers then they may rely on session state or files in local directories etc. Now, there are ways around this (like putting session state in SQL, using the cookie provider for temp data, using a shared drive for files etc) or, indeed, bypassing a lot of the benefits of Azure and using their "virtual server" concepts which means you don't get the scale benefits etc.
But, sites that rely heavily on local state may be challenging to move to the cloud.
Time Zones
All Azure servers run on UTC time. If you are used to running on dedicated servers serving users from a single time zone then chances are that you use things like DateTime.Now() which won't really correspond to what the user wants.
I don't see any of the above as limitations of Azure, I find them very useful in forcing you to build global and scalable solutions from the start. However, when porting an existing application, the above may be quite a challenge to adapt to, even though there are workarounds.
As also mentioned elsewhere, there is a learning curve to Azure and somehow the documentation - plentiful as it is - just doesn't quite seem to help for some reason. Once you "get it", though, I find Azure really nice and there are a bunch of subtle features that will help you build scalable solutions, like the whole queuing infrastructure, the blob storage and the table storage. In some ways the learning is hampered by having too much choice.
Good luck!