I'm using Azure Mobile App with Xamarin.Forms to create an offline capable mobile app.
My solution is based on https://adrianhall.github.io/develop-mobile-apps-with-csharp-and-azure/chapter3/client/
Here is the code that I use for offline sync :
public class AzureDataSource
{
private async Task InitializeAsync()
{
// Short circuit - local database is already initialized
if (client.SyncContext.IsInitialized)
{
return;
}
// Define the database schema
store.DefineTable<ArrayElement>();
store.DefineTable<InputAnswer>();
//Same thing with 16 others table
...
// Actually create the store and update the schema
await client.SyncContext.InitializeAsync(store, new MobileServiceSyncHandler());
}
public async Task SyncOfflineCacheAsync()
{
await InitializeAsync();
//Check if authenticated
if (client.CurrentUser != null)
{
// Push the Operations Queue to the mobile backend
await client.SyncContext.PushAsync();
// Pull each sync table
var arrayTable = await GetTableAsync<ArrayElement>();
await arrayTable.PullAsync();
var inputAnswerInstanceTable = await GetTableAsync<InputAnswer>();
await inputAnswerInstanceTable.PullAsync();
//Same thing with 16 others table
...
}
}
public async Task<IGenericTable<T>> GetTableAsync<T>() where T : TableData
{
await InitializeAsync();
return new AzureCloudTable<T>(client);
}
}
public class AzureCloudTable<T>
{
public AzureCloudTable(MobileServiceClient client)
{
this.client = client;
this.table = client.GetSyncTable<T>();
}
public async Task PullAsync()
{
//Query name used for incremental pull
string queryName = $"incsync_{typeof(T).Name}";
await table.PullAsync(queryName, table.CreateQuery());
}
}
The problem is that the syncing takes a lot of time even when there isn't anything to pull (8-9 seconds on Android devices and more than 25 seconds to pull the whole database).
I looked at Fiddler to find how much time takes the Mobile Apps BackEnd to respond and it is about 50 milliseconds per request so the problem doesn't seem to come from here.
Does anyone have the same trouble ? Is there something that I'm doing wrong or tips to improve my sync performance ?
Our particular issue was linked to our database migration. Every row in the database had the same updatedAt value. We ran an SQL script to modify these so that they were all unique.
This fix was actually for some other issue we had, where not all rows were being returned for some unknown reason, but we also saw a substantial speed improvement.
Also, another weird fix that improved loading times was the following.
After we had pulled all of the data the first time (which, understandably takes some time) - we did an UpdateAsync() on one of the rows that were returned, and we did not push it afterwards.
We've come to understand that the way offline sync works, is that it will pull anything that has a date newer than the most recent updated at. There was a small speed improvement associated with this.
Finally, the last thing we did to improve speed was to not fetch the data again, if it already had cached a copy in the view. This may not work for your use case though.
public List<Foo> fooList = new List<Foo>
public void DisplayAllFoo()
{
if(fooList.Count == 0)
fooList = await SyncClass.GetAllFoo();
foreach(var foo in fooList)
{
Console.WriteLine(foo.bar);
}
}
Edit 20th March 2019:
With these improvements in place, we are still seeing very slow sync operations, used in the same way as mentioned in the OP, also including the improvements listed in my answer here.
I encourage all to share their solutions or ideas on how this speed can be improved.
One of the reasons for the slow Pull() is when more than (10) rows get the same UpdatedAt value. This happens when you update the rows at once, for example running an SQL command.
One way to overcome this is to modify the default trigger on the tables. To ensure every row gets a unique UpdateAt, we did something like this:
ALTER TRIGGER [dbo].[TR_dbo_Items_InsertUpdateDelete] ON [dbo].[TableName]
AFTER INSERT, UPDATE, DELETE
AS
BEGIN
DECLARE #InsertedAndDeleted TABLE
(
Id NVARCHAR(128)
);
DECLARE #Count INT,
#Id NVARCHAR(128);
INSERT INTO #InsertedAndDeleted
SELECT Id
FROM inserted;
INSERT INTO #InsertedAndDeleted
SELECT Id
FROM deleted
WHERE Id NOT IN
(
SELECT Id
FROM #InsertedAndDeleted
);
--select * from #InsertedAndDeleted;
SELECT #Count = Count(*)
FROM #InsertedAndDeleted;
-- ************************ UpdatedAt ************************
-- while loop
WHILE #Count > 0
BEGIN
-- selecting
SELECT TOP (1) #Id = Id
FROM #InsertedAndDeleted;
-- updating
UPDATE [dbo].[TableName]
SET UpdatedAt = Convert(DATETIMEOFFSET, DateAdd(MILLISECOND, #Count, SysUtcDateTime()))
WHERE Id = #Id;
-- deleting
DELETE FROM #InsertedAndDeleted
WHERE id = #Id;
-- counter
SET #Count = #Count - 1;
END;
END;
Related
Token for email verification is created with User registration and needs to be deleted from database within 24 hours with crons job help. In a delete function using query builder, token gets deleted only if date value is manually provided in form of string: {delDate: "2021-02-08T17:59:48.485Z" }. Here, all tokens with date before or equal 2021-02-08 get deleted, and is working fine. But thats a static input, manually put in hard code!
Since this must be a dynamic input, I set up variable 'delTime',which stores current date minus 24 hrs in it, but it seems .where condition will not take a variable as value, and will not delete, as: {delDate: deltime}. In fact, 'delDate' consoles exactly the info I need, but it will only work in form of string.
There is a ton of content online teaching how to delete stuff with a static value in typeorm querybuilder, but so hard to find with dynamic values....
How else can I make this work in a dynamic way ?
async delete(req: Request, res: Response){
try {
const tokenRepository = getRepository(Token);
var delTime = new Date();
delTime.setDate( delTime.getDate() - 1 );
console.log(delTime) //consoles 24 hors ago
await tokenRepository
.createQueryBuilder()
.delete()
.from(Token)
.where("tokenDate <= :deleteTime", { deleteTime: delTime})//value dynamically stored in variable does not delete
//.where("tokenDate <= :deleteTime", { deleteTime: "2021-02-08T18:01:10.489Z"})//static hard code value deletes
//.where("tokenDate <= :delTime", { delTime})//Variable put this way will not work either...
.execute();
} catch (error) {
res.status(404).send("Tokens not found");
return;
}
res.status(200).send('Tokens deleted successfuly');
}
Your parameterized query looks correct.
Switch on TypeOrm Query Logging to see the generated SQL, maybe you will be able to see what's going wrong. Paste the generated SQL into the SQL query console to debug.
Alternatively you can write your delete query without parameters and let Sqlite calculate current date -1 day:
.where("tokenDate <= DateTime('Now', '-1 Day'"); // for Sqlite
Note 1: Sqlite DateTime('Now') uses UTC time, so your tokenDate should use UTC also.
Note 2: This syntax is specific to Sqlite, you can do the same in other databases but the syntax is different, e.g. Microsoft SQL Server:
.where("tokenDate <= DATEADD(day, -1, SYSUTCDATETIME()); // for MS SQL Server
I have a problem with my code, I wrote simple flutter app which is f note app, and I have included SQLite as a database , I run the app at first via the emulator and everything went cool , but when I tried to run it on my real device (which is an android device), the database did not respond (i.e I could not add new notes to the database ) and when I went back to run my app via the emulator .. the app did the same thing I found in my real device and in console I found this error
Error: Can't use 'SqfliteDatabaseException' because it is declared more than once.
I need help Please
I saw your code and the problem is that the exception you get is probably related to this one:
PlatformException(sqlite_error, UNIQUE constraint failed: Notetable.id
And it's because you need to manage the unicity of your primary key when you insert a new row. You can have a look at this SO question for a quick reference.
So just to quickly make your code working I've made this changes (please take this code only for reference, write a better one):
void createDataBase(Database db,int newVersion) async{
await db.execute('CREATE TABLE IF NOT EXISTS $noteTable ($col_id INTEGER PRIMARY KEY ,'+
'$col_title TEXT , $col_description TEXT , $col_date TEXT,$col_priority INTEGER)');
}
And
Future<int> insertData(myNote note)async{
var mdatabase = await database;
var _newNoteMap = note.convertToMap();
_newNoteMap['id'] = null;
var result = await mdatabase.insert(noteTable, _newNoteMap);
return result;
}
Pay attention that you always call a DB insert even when you update an existing note.
UPDATE: added additional modification (not listed before)
in databaseObject.dart
Map<String,dynamic> convertToMap(){
var mapObject = Map<String,dynamic>();
mapObject["id"] = _id;
mapObject["title"] = _title;
mapObject["description"] = _description;
mapObject["date"] = _date;
mapObject["priority"] = _priority;
return mapObject;
}
in Note.dart
if(res >= 1){
showAlertDialog('Status', "New note added successfully and the value of result is $res");
}
I am working on using the Firebase database in a Unity project. I read that you want to structure your database as flat as possible for preformance so for a leaderboard I am structuring it like this
users
uid
name
uid
name
leaderboard
uid
score
uid
score
I want a class that can get this data and trigger a callback when it is done. I run this in my constructor.
root.Child("leaderboard").OrderByChild("score").LimitToLast(_leaderboardCount).ValueChanged += onValueChanged;
To trigger the callback with the data I wrote this.
void onValueChanged(object sender, ValueChangedEventArgs args)
{
int index = 0;
List<string> names = new List<string>();
foreach (DataSnapshot snapshot in args.Snapshot.Children)
{
root.Child("players").Child(snapshot.Key).GetValueAsync().ContinueWith(task =>
{
if (task.Result.Exists)
{
names.Add(task.Result.Child("name").Value.ToString());
index++;
if (index == args.Snapshot.ChildrenCount)
{
_callback(names);
}
}
});
}
}
I am wondering if there is a better way to do this that I am missing? I'm worried if the tasks finish out of order my leaderboard will be jumbled. Can I get the names and scores in one snapshot?
Thanks!
You can only load:
a single complete node
a subset of all child nodes matching a certain condition under a single node
It seems that what you are trying to do is get a subset of the child nodes from multiple roots, which isn't possible. It actually looks like you're trying to do a join, which on Firebase is something you have to do with client-side code.
Note that loading the subsequent nodes is often a lot more efficient than developers expect, since Firebase can usually pipeline the requests (everything goes over a single web socket connection). For more on this, see http://stackoverflow.com/questions/35931526/speed-up-fetching-posts-for-my-social-network-app-by-using-query-instead-of-obse/35932786#35932786.
I have this function in my application. If the insert of Phrase fails then can someone tell me if the Audit entry still gets added? If that's the case then is there a way that I can package these into a single transaction that could be rolled back.
Also if it fails can I catch this and then still have the procedure exit with an exception?
[Route("Post")]
[ValidateModel]
public async Task<IHttpActionResult> Post([FromBody]Phrase phrase)
{
phrase.StatusId = (int)EStatus.Saved;
UpdateHepburn(phrase);
db.Phrases.Add(phrase);
var audit = new Audit()
{
Entity = (int)EEntity.Phrase,
Action = (int)EAudit.Insert,
Note = phrase.English,
UserId = userId,
Date = DateTime.UtcNow,
Id = phrase.PhraseId
};
db.Audits.Add(audit);
await db.SaveChangesAsync();
return Ok(phrase);
}
I have this function in my application. If the insert of Phrase fails
then can someone tell me if the Audit entry still gets added?
You have written your code in a correct way by calling await db.SaveChangesAsync(); only one time after doing all your modifications on the DbContext.
The answer to your question is: No, the Audit will not be added if Phrase fails.
Because you are calling await db.SaveChangesAsync(); after doing all your things with your entities, Entity Framework wil generate all the required SQL Queries and put them in a single SQL transaction which makes the whole queries as an atomic operation to your database. If one of the generated query e.g. Auditgenerated query failed then the transaction will be rolled back. So every modification that have been done to your database will be removed and so Entity Framework will let your database in a coherent state.
I have a requirement for to show the search result on the jsp with maxcount of 10 and it should have a pagination to traverse back and forward as pagination functionality.
Dynamodb has a lastevaluatedkey, but it doesn't help to go back to the previous page, though I can move to the next result set by the lastevaluatedKey.
Can anybody please help on this.
I am using Java SPRING and DynamoDB as the stack.
Thanks
Satya
To enable forward/backward, all you need is to keep
the first key, which is hash key + sort key of the first record of the previously returned page (null if you are about to query the first page).
and
the last key of the retrieved page, which is hash key + sort key of the last record of the previously returned page
Then to navigate forward or backward, you need to pass in below parameters in the query request:
Forward: last key as the ExclusiveStartKey, order = ascend
Backward: first key as the ExclusiveStartKey, order = descend
I have achieved this in a project in 2016. DynamoDB might provide some similar convenient APIs now, but I'm not sure as I haven't checked DynamoDB for a long time.
Building on Ray's answer, here's what I did. sortId is the sort key.
// query a page of items and create prev and next cursor
// cursor idea from this article: https://hackernoon.com/guys-were-doing-pagination-wrong-f6c18a91b232
async function queryCursor( cursor) {
const cursor1 = JSON.parse(JSON.stringify(cursor));
const pageResult = await queryPage( cursor1.params, cursor1.pageItems);
const result = {
Items: pageResult.Items,
Count: pageResult.Count
};
if ( cursor.params.ScanIndexForward) {
if (pageResult.LastEvaluatedKey) {
result.nextCursor = JSON.parse(JSON.stringify(cursor));
result.nextCursor.params.ExclusiveStartKey = pageResult.LastEvaluatedKey;
}
if ( cursor.params.ExclusiveStartKey) {
result.prevCursor = JSON.parse(JSON.stringify(cursor));
result.prevCursor.params.ScanIndexForward = !cursor.params.ScanIndexForward;
result.prevCursor.params.ExclusiveStartKey.sortId = pageResult.Items[0].sortId;
}
} else {
if (pageResult.LastEvaluatedKey) {
result.prevCursor = JSON.parse(JSON.stringify(cursor));
result.prevCursor.params.ExclusiveStartKey = pageResult.LastEvaluatedKey;
}
if ( cursor.params.ExclusiveStartKey) {
result.nextCursor = JSON.parse(JSON.stringify(cursor));
result.nextCursor.params.ScanIndexForward = !cursor.params.ScanIndexForward;
result.nextCursor.params.ExclusiveStartKey.sortId = pageResult.Items[0].sortId;
}
}
return result;
}
You will have to keep a record of the previous key in a session var, query string, or something similar you can access later, then execute the query using that key when you want to go backwards. Dynamo does not keep track of that for you.
For a simple stateless forward and reverse navigation with dynamodb check out my answer to a similar question: https://stackoverflow.com/a/64179187/93451.
In summary it returns the reverse navigation history in each response, allowing the user to explicitly move forward or back until either end.
GET /accounts -> first page
GET /accounts?next=A3r0ijKJ8 -> next page
GET /accounts?prev=R4tY69kUI -> previous page