I'm working with Azure's offline-sync API.
(It's REALLY GREAT so far, but since it's still new-ish it doesn't have comprehensive documentation, only tutorials. We need to craft dependable integration tests, and we're finding that tricky because we need to rely on published behavior in official docs for that... or dig into the source, but that is liable to change at any time.)
The samples do this:
var store = new MobileServiceSQLiteStore("localstore.db");
The comments mention "initializes local store".
I assume the local sync database is a "throw-away" asset, as it can be recreated at will.
Is the expected behavior that it will create the local SQLite file if it does not exist, or it will recreate the file each time the mobile app starts and that call is made?
The tutorials are augmented by the HOWTO documentation (available under Mobile > Develop - in the same area as the tutorials) and the GitHub Wiki and the github.io pages for the SDK.
The local store is created if it doesn't exist, and new fields are added to tables if they are needed. It's sometimes good to delete the database - for example, if you reduce the field count in your mobile app (the process only adds fields). If you do this, the database will be re-created when the app is next restarted.
Related
I am using Firebase for a long time (since 2018) and loving it. In that time there was not Location southamerica-east1 (São Paulo). Now I would like to store the project (web app, cloud function, and database) in southamerica to reduce cost and make it near to my end-users (also based in Brazil).
I have source control, all environment parameters values stored in Custom Environment Variables. The application works fine when no data is found. No concerns with backup data. No problem about downtime. This is not a critical app.
Anyway, I can't delete the application because I already have some users logged in there and IoT devices sending data through PubSub.
How can I rebuild my Firebase/Firestore/Web application/Function from the ground up, and make sure the new location is southamerica? If possible, I need to keep user and passwords, and web
Looking forward, (I don't think moving the bucked location would be the best solution here) but based on this page Select locations for your project I can't update the location, but since it is based on bucked location, if it doesn't break the project, I will use Google Cloud Transfer Page to Moving and renaming buckets
May is it a better solution than rebuild the app (Firebase/Firestore/Web application/Function)?
May I break my Firestore database or cloud function or web app?
May I lost my project domain or any other related URL parameter like authDomain, databaseURL, storageBucket?
May I need to update some web app parameter after the change?
They cannot be moved at present and migrating data is a manual process. Difficulty varies by product.
General guidance
Do not delete the old project before fully migrated.
Hosting
This migration is nearly trivial, with the understanding that there is likely a minor service interruption while moving custom domains.
Deploy to the new site
CNAME your custom domain to the new site (myproject.firebaseapp.com)
Delete custom domain from old site
Add custom domain to new site
Cloud Functions
This migration is trivial.
Create a local directory for your new project
Run firebase init and set up project normally (enable Functions)
Copy your Functions code into the new project's functions/ directory
Deploy to the new project
Database
This migration is tricky, difficult, and highly specific to your use case and tolerance for downtime. What follows is a general template to adapt.
Reference docs for import/export: Firestore import/export, Realtime Database backups
In the old project:
Lock the database using security rules to prevent changes
Export existing database
In the new project:
Import the database backup
You probably need to migrate existing users (see account export/import) as well so user ids stored in your DB will still reference the correct accounts
Point existing apps to the new project
If downtime is not an option, or if you'll be deploying a new mobile app version and need time for changes to propagate, then you'll need to set up a dual write model:
Dual sync: Create a Cloud Function on both the new and old database that duplicate all create/update/delete operations on the respective partner endpoint.
Sync pre-existing data: Perform the export/import process as above on all data created before the dual sync was implemented, excluding the step to lock the old database
Shut down your old mobile app version (once enough accounts have migrated)
Shut down the dual sync Functions and turn down the old site
Based in your information, the main issue is Firestore because other products are globally balanced like Cloud IoT Core and Hosting (these can't be configured on a specific region)
Other products like Functions can be redeployed with the same code and name into another region.
I think that you can create another project only to move the database to the new region and configure all Cloud resources to reach the new located database.
As a caveat, you need to add another domain/subdomain and create new credentials to work with the new project; this step can´t be skipped because it is required for authentication.
On the application side you can add the access to the new database
In case you need assistance during your migration you can start a case with GCP/Firestore support.
This is a hard pill to swallow, but maybe the costs and the time to migrate to another region will be higher than keeping your application as is working today.
What's the best way to change the Firebase data model while you have multiple versions of your iOS app in production?
Since there's no 'application server' layer in the middle any changes in the database model could break older versions of the app.
Performance Related Example of the problem:
In version 1.0 I was naively keeping everything related to a post under '/posts/'. Now in version 2.0 I want to take Firebase's recommendation and add a '/user-post' endpoint to quickly list all posts for a given user.
People using version 1.0 of the iOS app are not writing any data to '/user-posts' since that endpoint didn't used to exist. People using version 2.0 therefore don't see any posts created by people using the old version of the app.
In theory I could create a server somewhere that listens for changes on '/post/' and adds them to '/user-posts' as well. That seems hard to maintain over time though if you have a lot of different versions of your app.
New Feature Example of the problem:
Lets say in version 1.0 of your mobile app you write new blog posts to '/posts/'. Now in version 2.0 of your app you introduce a Teams feature and all posts need to be in '/team/team-id/posts'.
People who haven't upgraded to version 2.0 will still be writing to '/posts'. Those posts won't be visible to people using version 2.0 who are reading from '/team/team-id/posts'.
I realize you could keep both endpoints simultaneously (and index /posts based on team ID) but over time this seems hard to maintain.
Traditional solutions:
If I were using something like Django or Express I'd do a database migration and then update the server-side endpoints for creating blogposts.
That would make changes in the database from the clients. I could in theory add an application-server tier to my architecture with Firebase, but that doesn't seem like it's recommended: https://firebase.googleblog.com/2013/03/where-does-firebase-fit-in-your-app.html
I would suggest you use Firebase Remote Config to show an alert via UIAlertController or different screen if an update is available. You could force the user to update to the current version and you don't have problems later because no posts with the old code can be created.
To answer your question:
I would develop a different app, add it to the same Firebase project and then let this app convert all old data to the new data model. So you would do this one time after releasing the new version and the old user data is converted to the new data model and everything works smoothly. You could also have a property like databaseVersion for every object.
To prevent future problems you could have a general property named app-version in your Firebase Realtime Database. Before every post the app checks if there is a newer version. If not the user can add the post but if there is a newer version you could show an message/alert via UIAlertController
I'm tryign to use 51Degrees in a .NET project that I deploy to Azure. August 2011, they released v1.2.1.3 marked as "Azure Compatible":
Foundation can now be deployed on to the Windows Azure Cloud service.
See the release note for full details on requirements and how to
setup. Azure related changes include: Instead of a log file, log
entries are written to a log table Instead of a devices file, previous
device requests are written to a device table A new conditional
compilation symbol - 'AZURE'. AZURE enabled builds will not work in
traditional ASP.NET.
Since then there have been a dozen releases and they are up to v2.1.4.9. However, their documentaiton is super light on how to use it with Azure. In fact, there was a bug originally because v1.2.1.3 stated
To make use of the changes you must create a storage account called
‘fiftyonedegrees’. The foundation will then create two tables, one for
previous devices, and another for logs.
This isn't possible because Azure storage accounts need to be unique across all instances so everyone can't create ones named fifityonedegrees.
Their response was:
After rereading the blog it seems I've made an oversight in this
regard, and will update shortly.
The storage account that the Foundation looks for can be changed in
the Foundation source code. Go to Foundation/Properties/Constants.cs
and change the string 'AZURE_STORAGE_NAME' to the name of your storage
account.
However, I'm still at a loss at how to utilize it within my project. Here's my issues:
I'm not clear whether v1.2.1.3 is the only Azure compatible release, or every release after is Azure compatible. Their documentation doesn't say.
When I install 51Degrees via NuGet, my project doesn't get an App_Data folder created which contradicts their documentation. The web.config file even has entries in it that reference the App_Data folder such as <log logFile="~/App_Data/Log.txt" logLevel="Info"/>.
Based on the response to the Azure storage account bug I quoted earlier, they are sayign IN need to edit the file Foundation/Properties/Constants.cs. However, since I'm installing via NuGet and it's a DLL, NuGet is presumably the wrong approach? Do I need to download the source and compile it myself and wire it up to my project manually?
I'm generally new to .NET, NuGet, VS, etc so appreciate the help.
All versions are Azure compatible from 1.2.1.3 onwards. I'm assuming this is the blog post you were talking about. After you've created your azure storage account, you'll have to edit the Constants.cs file in the source code and add in your account name. It's my understanding that this means you'll have to get access to the source code and edit it directly. One you have done this you'll need to recompile for the software to work correctly. I'm not sure if there is a way to perform the same task using NuGet, but I'll look into it. Hope this helps.
I have recently deployed my web role to Windows Azure. In the properties of my WebRole I have set Enable Diagnostics.
I can also see that it correctly maps to a storage account once deployed by viewing the configuration file of the hosted service.
I have not setup anything else for diagnostics, I am unaware that I need to do anything else.
I am now setting up AzureWatch (by paraleap) to monitor my instances however it reports that WADPerformanceCountersTable does not exist.
I am very new to Azure, don't have a clue how the diganostics work and can't find anything on Google that shows me how. Could someone please show me the way.
Ok I figured it out and will leave this here for others to follow.
Step 1
If you follow http://dunnry.com/blog/2012/02/27/SettingUpDiagnosticsMonitoringInWindowsAzure.aspx Windows Azure Diagnostics will start saving data into your attached Blob storage, full of diagnostic information.
Special Note: These count towards your storage transaction, which is why you will see them go up.
Step 2
However I needed the WADPerformanceCounterTable, which should have been located in the tables section of the storage account but it never was created. I needed this to use services like AzureWatch to monitor and spin up or down instances.
Special Note: This is performance counters, a specific subset of diagnostic information and this isn't stored in the blob section by default.
Step 3
In your project you need to add which performance counters to monitor in the WebRole.cs.
Special Note: You won't have this if you just added an existing project to an Azure deployment project. Unless you specifically started the project from scratch and chose the Azure templates, you will need to create this manually. You would also need to add: Microsoft.WindowsAzure.Diagnostics, Microsoft.WindowsAzure.ServiceRuntime and Microsoft.WindowsAzure.StorageClient as References. Best way to see how it all works is to create a blank project from an Azure template and copy over the necessary items.
Step 4
Next you need to define which performance counters to monitor. As such here is a great sample: http://code.msdn.microsoft.com/windowsazure/Windows-Azure-PerformanceCo-7d80ebf9
Extra Reference
Microsoft also has a few steps you can follow here that might help out if things still aren't working: http://msdn.microsoft.com/en-us/library/windowsazure/hh411521.aspx
Take a look at:
http://dunnry.com/blog/2012/02/27/SettingUpDiagnosticsMonitoringInWindowsAzure.aspx
There is also a lot of information on:
http://msdn.microsoft.com/en-us/library/windowsazure/gg433048.aspx
I want to provide iCloud support for my wrapper around sqlite. Is not using coredata.
I wonder how enable iCloud for it. The database content is changed all the time (is for invoicing). Also, if is possible to have some kind of versioning will be great.
Exist any sample I can use to do this?
The short answer is no, you would need to use Core Data as you suspected. Apple has stated that sqlite is unsupported.
Edit: Check out the section on iCloud that's now in the iOS Application Programming Guide under Using iCloud in Conjunction with Databases
Using iCloud with a SQLite database is possible only if your app uses
Core Data to manage that database. Accessing live database files in
iCloud using the SQLite interfaces is not supported and will likely
corrupt your database. However, you can create a Core Data store based
on SQLite as long as you follow a few extra steps when setting up your
Core Data structures. You can also continue to use other types of Core
Data stores—that is, stores not based on SQLite—without any special
modifications.
You can't just put the SQLite database in the iCloud container, because it might get corrupted. (As you modify an SQLite DB, temporary files are created and renamed, so if the sync process starts copying those files, you'll get a corrupt database.)
If you don't want to move to Core Data, you can do what Core Data does: store your database in your document folder, and store a transaction log in the iCould container. Every time you change the database, you add those changes to a log file, so you can play them back and make equivalent changes on other devices.
This gets pretty complicated: aside from getting the log/reply logic right, you'll want to coalesce redundant changes and periodically collapse the log into a complete copy of the database.
You might have an easier time developing a solution if you can exploit knowledge of your application (Core Data has to solve the problem in the general case). For example, you could save invoices as separate files in the cloud container (text, Property List, XML, JSON, whatever), writing them out as the database changes and only importing ones if the system tells you they were created or changed.
In summary, your choice is either to migrate to Core Data or write a sync solution yourself. Which one is best depends on the particulars of your application.