Xamarin Offline Sync with AzureMobileServices: Initial offline load incredibly slow - sqlite

I'm successfully using Azure Mobile Services and Xamarin Forms to perform CRUD operations on an SQL DB hosted with Azure. The offline sync portion stores the data in an SQLite db on the phone. There's been a few bumps along the way to get it working as smoothly as we have it now, but this remains to be the last hurdle.
Problem
When the device has no connection (tested using Airplane mode on a variety of physical and emulated devices) - the first time it goes to access any of the offline data, it takes a very long time to return anything. This is the case if the data exists in the SQLite DB or not.
There is no exception thrown, or anything that I can see printed to the logs that indicates what the delay might be.
To give an idea, a PullAsync() on 20 rows might take 5 seconds while online, and that data is stored to the SQLite DB. After putting the device into offline mode, that same operation may take up to 60 seconds. These numbers are quite arbitrary, but the delay is noticeably much too long.
To add to this, this long load only occurs the very first time any Offline Sync method is called. After that, every method is near instant, as I would expect it to be - but why not the first time?
Expected Result
I would expect that because the data is stored on the device already, and no internet connection can be detected, it should return the data almost instantly.
Code
Sync Class
The GetPolicies() method is where the delay would occur.
This is a sample of one of the components. Every other component is the same format, but different data.
IMobileServiceSyncTable<policy_procedure> policyTable = SyncController.policyTable;
public async Task<List<policy_procedure>> GetPolicies(string companyId)
{
//SemaphoreSlim
await SyncController.dbOperation.WaitAsync();
try
{
await SyncController.Initialize();
await policyTable.PullAsync("policy_procedure", policyTable.Where(p => p.fk_company_id == companyId).Where(p=> p.signature!=null || p.signature!=""));
return await policyTable.ToListAsync();
}
catch (Exception ex)
{
//For some reason, when this method is called and the device is offline, it will fall into this catch block.
//I assume this is standard for offline sync, as it's trying to do a pull with no connection, causing it to fail.
//Through using breakpoints, the delay occurs even before it reaches this catch statement.
Console.WriteLine(ex);
return await policyTable.ToListAsync();
}
finally
{
SyncController.dbOperation.Release();
}
}
Sync Controller
public static SemaphoreSlim dbOperation = new SemaphoreSlim(1, 1);
public static MobileServiceClient client;
public static MobileServiceSQLiteStore store;
public static async Task Initialize()
{
try
{
//This line is not standard for Offline Sync.
//The plugin returns true or false for the devices current connectivity.
//It's my attempt to see if there is a connection, to eliminate the load time.
//This does immediately take it back to the try statement in GetPolicies
if (!CrossConnectivity.Current.IsConnected)
return;
if (client ? .SyncContext ? .IsInitialized ? ? false)
return;
client = new MobileServiceClient(AppSettings.azureUrl);
var path = "local.db"; //Normally uses company ID,
path = Path.Combine(MobileServiceClient.DefaultDatabasePath, path);
store = new MobileServiceSQLiteStore(path);
/************************/
#
region Table Definitions in local SQLite DB
//Define all the tables in the sqlite db
..
store.DefineTable < policy_procedure > ();
..#endregion
await client.SyncContext.InitializeAsync(store);
/************/
#
region Offline Sync Tables
..
policyTable = client.GetSyncTable < policy_procedure > ();
..#endregion
}
catch (Exception ex)
{
Console.WriteLine(ex)
}
}
What I've Tried
Well, I'm not too sure what's even causing this, so most of my attempts have been around forcing an exception before this wait time occurs, so that it can fall out of the GetPolicies try-catch, as the wait time appears to be on the PullAsync.
My most recent attempt at this is commented in the code above (SyncController), where I use James Montemagno's Connectivity Plugin to detect the phones network connectivity. (I've tested this separately, and this works correctly without delay.)

The short story is that you don't want to call PullAsync in your GetPolicies method if your device is offline. For example, you could do
try
{
await SyncController.Initialize();
if (CrossConnectivity.Current.IsConnected)
{
await policyTable.PullAsync("policy_procedure", policyTable.Where(p => p.fk_company_id == companyId).Where(p=> p.signature!=null || p.signature!=""));
}
return await policyTable.ToListAsync();
}
but you will also want to handle the case where this is the first time the app runs and so you don't have any records yet.

Related

Is it a good / working practice to use Firebase's documentReference.get(GetOptions(source: cache)) in Flutter?

My issue was, that with the default GetOptions (omitting the parameter), a request like the following could load seconds if not minutes if the client is offline:
await docRef.get()...
If I check if the client is offline and in this case purposefully change the Source to Source.cache, I have performance that is at least as good, if not better, than if the client was online.
Source _source = Source.serverAndCache;
try {
final result = await InternetAddress.lookup('example.com');
if (result.isNotEmpty && result[0].rawAddress.isNotEmpty) {
_source = Source.serverAndCache;
}
} on SocketException catch (_) {
_source = Source.cache;
}
and then use this variable in the following way:
docRef.get(GetOptions(source: _source))
.then(...
This code works perfectly for me now, but I am not sure, if there are any cases in which using the code like this could raise issues.
Also it seems like a lot of boilerplate code (I refactored it into a function so I can use it in any Database methods but still...)
If there are no issues with this, why wouldn't this be the Firebase default, since after trying the server for an unpredictably long time it switches to cache anyways.

Flutter connecting to multiple BLE devices Synchronously

I'm using flutter to work on an bluetooth low energy app, via the flutterBlue library, in which we are potentially connecting to multiple peripherals at the same time.
I am able to connect to multiple peripherals if I connect to them individually and send commands to all of them simultaneously.
For state management, my BluetoothHelper is the Model for my ScopedModel.
class BluetoothHelper extends Model {
bool isProcessing = false;
int val = 0;
FlutterBlue flutterBlue = FlutterBlue.instance; //bluetooth library instance
StreamSubscription scanSubscription;
Map<DeviceIdentifier, ScanResult> scanResults = new Map();
/// State
StreamSubscription stateSubscription;
BluetoothState state = BluetoothState.unknown;
/// Device
List<BluetoothDevice> devicesList = new List(); //todo
bool get isConnected => (deviceList.size != 0);
StreamSubscription deviceConnection;
StreamSubscription deviceStateSubscription;
List<BluetoothService> services = new List();
Map<Guid, StreamSubscription> valueChangedSubscriptions = {};
BluetoothDeviceState deviceState = BluetoothDeviceState.disconnected;
Future startScan(String uuid) async {
isProcessing = true;
if (val == 0) {
Future.delayed(Duration(milliseconds: 25), () => scanAndConnect(uuid));
val++;
} else {
Future.delayed(Duration(seconds: 4), () => scanAndConnect(uuid));
}
}
scanAndConnect(String uuid){
scanSubscription =
flutterBlue.scan(timeout: const Duration(seconds: 120), withServices: [
//new Guid('FB755D40-8DE5-481E-A369-21C0B3F39664')]
]).listen((scanResult) {
if (scanResult.device.id.toString() == uuid) {
scanResults[scanResult.device.id] = scanResult;
print("found! Attempting to connect" + scanResult.device.id.toString());
device = scanResult.device;
//connect(device);
connect(device);
}
}, onDone: stopScan);
}
Future connect(BluetoothDevice d) {
deviceConnection = flutterBlue.connect(d).listen(
null,
);
deviceStateSubscription = d.onStateChanged().listen((s) {
if (s == BluetoothDeviceState.connected) {
stopScan();
d.discoverServices().then((s) {
print("connected to ${device.id.toString()}");
services = s;
services.forEach((service) {
var characteristics = service.characteristics;
for (BluetoothCharacteristic c in characteristics) {
if (c.uuid.toString() == '') {//we look for the uuid we want to write to
String handshakeValue ; //value is initiliazed here in code
List<int> bytes = utf8.encode(handshakeValue);
d.writeCharacteristic(c, bytes,
type: CharacteristicWriteType.withResponse);
devicesList.add(d);
}
}
});
});
}
});
}
}
I am trying to loop throw all peripheral Unique Identifier (UID) and then have them connect one after the other programmatically.
This wasnt working out great. It would always end up connecting to the very last peripheral. Seems like the flutterblue instance can only scan for one uid at a time, and if it receives another request, it immediately drops the last request and moves to the new one.
I applied this same logic to the connection of an individual peripheral logic where I'd tap one peripheral and the second immediately and it'd connect to the second one. (I'm not currently blocking the UI or anything while the connection process takes place)
I need to wait till the first peripheral is connected before moving onto the next one.
The code above is the only way I've gotten my peripherals but there are huge problems with this code. It can currently only connect to 2 devices. It's using delays instead of callbacks to achieve connection by giving enough time for the scan and connect to happen before moving onto the second peripheral.
My first instinct was to make the convert the startScan and connect methods into async methods but this isnt working out well as I'd hope.
{await connect(device); } => gives "The built in Identifier "await" cant be used as a type. I could just be setting up the asyncs incorrectly.
I have looked around for alternatives and I've come upon Completers and Isolates. I'm not sure how relevant that might be.
UI SIDE :
I have the following method set for the ontap of a button wrapped within a scoped model descendant. This is going to reliably load peripheralUIDs list with a few uids and then connect to them one after the other.
connectAllPeripherals(BluetoothHelper model, List<String> peripheralUIDs) {
for(var uuid in peripheralUIDs) { //list of strings containing the uuids for the peripherals I want to connect to
model.startScan(uuid);
}
}
Don't know if this point is still an issue.
Assuming your issue hasn't since been fixed. I think the issue you have is trying to maintain the connections within Flutter (rather than just connecting multiple devices and letting Flutter_Blue/the hardware manage the connections).
I've got it happily connecting to multiple devices; after you've setup the instance maintaining a list of multiple device attributes.
i.e. I made a ble-device class which contained each of the following:
StreamSubscription deviceConnection;
StreamSubscription deviceStateSubscription;
List<BluetoothService> services = new List();
Map<Guid, StreamSubscription> valueChangedSubscriptions = {};
BluetoothDeviceState deviceState = BluetoothDeviceState.disconnected;
Maintaining a LinkedHashMap with a new object initialised from the class above for each device connected works nicely.
Other than that - Flutter_Blue will only allow 1 concurrent request call at a time (like reading a characteristic), but you can stack them pretty easily with
await
with the above, I'm able to poll multiple devices within a few milliseconds of each other.
Don't know if that helps - but with any luck, someone also coming across my problem will hit this and save some time.

Firestore stops updating after losing and regaining an internet connection

I have set up some listeners like so:
deviceListener = db.addSnapshotListener(this::handleDbChange)
When I have a stable internet connection, the handler fires on a data change and allows me to update my application. However, when I lose and regain an internet connection the handler ceases to fire. This doesn't always occur on the first loss of connection, but it always occurs after 2 or 3 drops in my connection.
I have tried removing the listeners and re-adding them when the network changes. Additionally, I tried getting the data directly after the network connection is reestablished:
db.get().add().addOnCompleteListener {
val snapshot = it.result
snapshot.toObject(Model::class.java)
}
But, this still serves the stale data. The only way I've found to correct this issue is restarting the app.
If anyone else has encountered this issue, I'd appreciate any insight you may have on how to solve it. FYI, I'm using the com.google.firebase:firebase-firestore:17.0.2 version of the library.
I know its a late reply, and i'm only a novice here (so I could be wrong), but for anyone else to come across this... it may be a combination of the problem I had:
Firebase Firestore batch command wont commit after regaining connection
And the problem someone else had:
Firestore doesn't immediately start listening to changes when Internet Connection Resumes
In summary:
Ensure you test without an emulator.
If you need live data, turn data persistence off.
And Firestore may use an uncontrollable timer to dictate when it reconnects its listeners after a connection is regained.
Use device instead of emulator as NicCoe has mentioned. I also suffered from a similar problem for a long time and finally found that Firestore works differently on device and emulator. (FYI, I'm using com.google.firebase:firebase-firestore:17.1.3) Most problems were solved after changing the test environment with the device. One small problem with the device I have found is that Firestore gives empty result several times after regaining an internet connection. And I solved it with this code:
override fun onCreateView(inflater: LayoutInflater, container: ViewGroup?, savedInstanceState: Bundle?): View? {
...
mRetryButton.setOnClickListener {
val pendingIntent = PendingIntent.getActivity(context, 0, Intent(context, ThisActivity::class.java), PendingIntent.FLAG_ONE_SHOT)
val alarmManager = context?.getSystemService(Context.ALARM_SERVICE) as AlarmManager
alarmManager.set(AlarmManager.RTC, System.currentTimeMillis() + 100, pendingIntent)
System.exit(0)
}
...
}
fun fetchData() {
FirebaseFirestore.getInstance().collection("col_name").get()
.addOnCompleteListener {
if (it.isSuccessful) {
val result = it.result!!
if (result.isEmpty && result.metadata.isFromCache) {
mRetryButton.visibility = View.VISIBLE
return#addOnCompleteListener
}
var docs = result.documents
...
} else {
Log.d(TAG, "Error getting documents: ", it.getException())
}
}
}

How do I make an Angular2 application update instantly on all connected devices as the events are fired?

I need to build a restaurant management mobile application with Angular2 and Ionic2 and a website for the same restaurant with Angular2 that constantly make http connections to store and retrieve data from the database in order to maintain the latest data.
For example, If an order request was fired from a waiter's mobile phone, the new order is posted to the database, which I can accomplish. the chef needs to get the instant notification that a new order has been created. Also, the data needs to be reflected in other employees' phones and also on the website.
All I can come up with is the use of setInterval, but since I've never done anything like this before, I'm not sure if this is the correct way.
component
orders: Order[];
constructor(private orderService: OrderService) {
setInterval(function() {
this.orderService
.getOrders()
.subscribe(
(orders: Order[]) => this.orders = orders,
(error: Response) => console.log(error)
)}, 3000);
}
placeanOrder(order) {
this.orderService.postOrder(order);
}
service
getOrders() Observable<any>{
this.http.get(...).map(...);
}
placeOrder(order) {
this.http.post(...).map(...);
}
When I try something like this, I get the same error logged to the console every second.
Cannot read property 'getOrders' of undefined
Why am I getting the error?
How would I convert the Observable json data retrieved from server into my interface data type, in this case, of type Order?
What is a better approach to this?
Cause wrong usage of callbacks! You have to use the arrow-syntax!
wrong: setInterval(function() {
right: setInterval(() => {
Just describe your function with correct types:
getOrders(): Observable<Order[]> {
Nothing else needed. Property-names needs to be the same (interface/json)!
It's ok to use a timer. "Better" could be a never-closed-connection to the server, so server could SEND you that data and client don't have to POLL.
https://www.websocket.org/
http://socket.io/

Implementing exception handling in a function which returns a Stream

I'm implementing a function which returns a Stream. I'm not sure how to implement the error handling, what is best practice?
For functions which return a Future, it's best practice never to throw a synchronous error. Is this also true for functions which return a Stream?
Here's an example of what I'm thinking:
Stream<int> count() {
var controller = new StreamController<int>();
int i = 0;
try {
doSomethingThatMightThrow();
new Timer.repeating(new Duration(seconds: 1), () => controller.add(i++));
} on Exception catch (e) {
controller.addError(e);
controller.close();
}
return controller.stream;
}
In general it is true for Streams as well. The main idea is, that users should only need to handle errors in one way. Your example moves all errors to the stream.
There are circumstances where immediate errors are better (for instance you could make the error is due to a programming error and should never be handled anyways, or if you want to guarantee that a Stream never produces errors), but sending the error through a stream is almost always a good thing.
Small nit: a Stream should usually (there are exceptions) not produce any data until somebody has started listening. In your example you are starting a Timer even though you don't even know if there will ever be a listener. I'm guessing the example is reduced and not representative of your real code, but it is something to look out for. The solution would be to use the StreamController's callbacks for pause and subscription changes.
I've updated the example to take on-board Florian's comments.
In my real use case, I don't ever want to buffer the results, so I'm throwing an UnsupportedError if the stream is paused.
I've made it a terminating stream, rather than an infinite one.
If the user of this function adds a listener asynchronously after a few seconds, then they will lose the first couple of results. They shouldn't do this. I guess that's something to document clearly. Though perhaps, I could also throw an error if the subscribe state changes after the first data has been received, but before a close has been received.
Stream<int> count(int max) {
var controller = new StreamController<int>(
onPauseStateChange: () => throw new UnsupportedError('count() Stream pausing not supported.'));
int i = 0;
try {
doSomethingThatMightThrow();
new Timer.repeating(new Duration(seconds: 1), () {
if (!controller.hasSubscribers)
return;
controller.add(i++);
if (i >= max)
controller.close();
});
} on Exception catch (e) {
controller.addError(e);
controller.close();
}
return controller.stream;
}

Resources