I have a Datasheet model with ~ 4000 records. Occasionally, I'll need to delete all the data and reload from a Google sheet. I'm able to delete a page of data, one row at a time with this code:
var set = widget.datasource.items;
set.forEach(function f(i){i._delete();});
widget.datasource.load();
The problem is that this is slow and problematic with pages of more than around 20 records. Is there a way to bulk delete records? Thanks for any advice!
You can use the app.deleteRecrods() method in a server script:
function clearData() {
var query = app.models.Student.newQuery();
var records = query.run();
app.deleteRecords(records);
}
Call it from a client script:
function clearData() {
google.script.run.withSuccessHandler(function() {
console.log('Clear all Student data.');
}).clearData();
}
Related
I have a Google drive table data source which stores list of open positions. Now in the data source I've set "Query per size" field to 10 so that I can get 10 records per page. I've added a Pager as well to show pagination.
My query is I want to display like "Page 1 of X" to my end users and this X will vary based on certain search filters. What will the best way to achieve this in Appmaker?
I've tried counting total records in a data source as per below code but every time updating that with the search criteria and recounting it is not a proper solution.
//Server side
var newQuery = app.models.Company.newQuery();
var records = newQuery.run();
var totalCount =0;
for(var i=0;i<records.length;i++)
{
totalCount=totalCount+1;
}
return totalCount;
In case you don't have any filters in your table your server code can be as simple as
// Server script
function getPagesCount(pageSize) {
var recordsCount = app.models.MyModel.newQuery().run().length;
var pagesCount = Math.ceil(recordsCount / pageSize);
return pagesCount;
}
As an alternative you can consider creating Calculated Model with a single field PagesCount.
In case you have some filters associated with the table then you'll need to run the query for the pages number with exact same filters.
Most likely the entire setup will not work effectively with Drive Tables since there is no way to query records number without querying records themselves. With Cloud SQL data backend one can create Calculated SQL Model with lightweight native SQL query (here :PageSize is query parameter which should be equal to the query.limit of the actual datasource):
SELECT
Ceil(COUNT(1) / :PageSize) AS RecordsNumber
FROM
TableName
WHERE
...
I've achieved this using Calculated Model as suggested by Pavel.
Steps :
Create a calculated data source with one field count.
In that data source add one parameter searchQuery. This will contain users filter going forward. Currently I have only one search query in which user can search many things. So I've added one parameter only.
In this data source add following server script.
Code:
// Server script
function getTotalRecords(query) {
var receivedQuery = query.parameters.searchQuery;
// console.log('Received query:' + query.parameters.searchQuery);
var records = app.models.Company.newQuery();
records.parameters.SearchText = query.parameters.searchQuery;
if(receivedQuery !== null) {
records.where = '(Name contains? :SearchText or InternalId contains? ' +
':SearchText or LocationList contains? :SearchText )';
}
var recordsCount = records.run().length;
var calculatedModelRecords = [];
var draftRecord = app.models.RecordCount.newRecord();
draftRecord.count = ''+recordsCount;
calculatedModelRecords.push(draftRecord);
return calculatedModelRecords;
}
.
On the Appmaker page bind a label with this data source.
On search query/your filter applied event add following code which Reload this data source and assign value to Parameter.
// Client script
function updateRecordCount(newValue) {
var ds = app.datasources.RecordCount;
ds.query.parameters.searchQuery = newValue;
ds.unload();
ds.load();
}
Using GoogleAppMaker how to create a data source from google contacts. There is an employee HR example app but I want to similarly manage contacts (add, modify, delete) and use select criteria.
At this time this task is not trivial in App Maker and it is pretty much generic. We can change question wording to CRUD operations with 3rd party datasources. Let's break it into smaller parts and address them separately.
Read/list contacts
This task is relatively easy. You need to use Calculated Model to proxy Apps Scripts Contacts API response. Once you create model with subset of fields from the Contact response you can create datasource for the model and bind it to List or Table widget. You can also try to find some inspiration in Calculated Model Sample.
// Server side script
function getContacts_() {
var contacts = ContactsApp.getContacts();
var records = contacts.map(function(contact) {
var record = app.models.Contact.newRecord();
record.FirstName = contact.getGivenName();
record.LastName = contact.getFamilyName();
var companies = contact.getCompanies();
if (companies.length > 0) {
var company = companies[0];
record.Organization = company.getCompanyName();
record.Title = company.getJobTitle();
}
var emails = contact.getEmails();
if (emails.length > 0) {
record.Email = emails[0].getAddress();
}
var phones = contact.getPhones();
if (phones.length > 0) {
record.Phone = phones[0].getPhoneNumber();
}
return record;
});
return records;
}
Create/Update/Delete
Since Calculated Models have some limitations, we need to turn on our imagination to create, update and delete records from their datasources. The basic strategy will be calling server side scripts for CUD operations in response to user actions on client side. To get user's input from UI we will need to utilize page's Custom Properties, one property for each Contact field:
Here are some snippets that should explain the idea
Create
// Client script
function onSubmitContactClick(submitButton) {
var props = submitButton.root.properties;
var contact = {
FirstName: props.FirstName,
LastName: props.LastName,
Organization: props.Organization,
...
};
google.script.run
.withSuccessHandler(function() {
// Most likely we'll need to navigate user back to the
// page with contacts list and reload its datasource
// to reflect recent changes, because our `CUD` operations
// are fully detached from the list datasource
app.showPage(app.pages.Contacts);
app.datasources.Contacts.load();
})
.withFailureHandler(function() {
// TODO: Handle error
})
.createContact(contact);
}
// Server script
function createContact(contactDraft) {
var contact = ContactsApp.createContact(contactDraft.FirsName,
contactDraft.LastName,
contactDraft.Email);
contact.addCompany(contactDraft.Organization, contactDraft.Title);
contact.addPhone(ContactsApp.Field.WORK_PHONE, contactDraft.Phone);
}
Update
Idea to update contact records will be very similar to the new contact creation flow, so I skip it for now.
Delete
Assuming that delete button is located inside contacts table row.
// Client script
function onDeleteContactClick(deleteButton) {
var email = deleteButton.datasource.item.Email;
google.script.run
.withSuccessHandler(function() {
// To update contacts list we can either reload the entire
// datasource or explicitly remove deleted item on the client.
// Second option will work way faster.
var contactIndex = deleteButton.parent.childIndex;
app.datasources.Contacts.items.splice(contactIndex, 1);
})
.withFailureHandler(function() {
// TODO: Handle error
})
.deleteContact(contact);
}
// Server script
function deleteContact(email) {
var contact = ContactsApp.getContact(email);
ContactsApp.deleteContact(contact);
}
I'm using Azure Mobile App with Xamarin.Forms to create an offline capable mobile app.
My solution is based on https://adrianhall.github.io/develop-mobile-apps-with-csharp-and-azure/chapter3/client/
Here is the code that I use for offline sync :
public class AzureDataSource
{
private async Task InitializeAsync()
{
// Short circuit - local database is already initialized
if (client.SyncContext.IsInitialized)
{
return;
}
// Define the database schema
store.DefineTable<ArrayElement>();
store.DefineTable<InputAnswer>();
//Same thing with 16 others table
...
// Actually create the store and update the schema
await client.SyncContext.InitializeAsync(store, new MobileServiceSyncHandler());
}
public async Task SyncOfflineCacheAsync()
{
await InitializeAsync();
//Check if authenticated
if (client.CurrentUser != null)
{
// Push the Operations Queue to the mobile backend
await client.SyncContext.PushAsync();
// Pull each sync table
var arrayTable = await GetTableAsync<ArrayElement>();
await arrayTable.PullAsync();
var inputAnswerInstanceTable = await GetTableAsync<InputAnswer>();
await inputAnswerInstanceTable.PullAsync();
//Same thing with 16 others table
...
}
}
public async Task<IGenericTable<T>> GetTableAsync<T>() where T : TableData
{
await InitializeAsync();
return new AzureCloudTable<T>(client);
}
}
public class AzureCloudTable<T>
{
public AzureCloudTable(MobileServiceClient client)
{
this.client = client;
this.table = client.GetSyncTable<T>();
}
public async Task PullAsync()
{
//Query name used for incremental pull
string queryName = $"incsync_{typeof(T).Name}";
await table.PullAsync(queryName, table.CreateQuery());
}
}
The problem is that the syncing takes a lot of time even when there isn't anything to pull (8-9 seconds on Android devices and more than 25 seconds to pull the whole database).
I looked at Fiddler to find how much time takes the Mobile Apps BackEnd to respond and it is about 50 milliseconds per request so the problem doesn't seem to come from here.
Does anyone have the same trouble ? Is there something that I'm doing wrong or tips to improve my sync performance ?
Our particular issue was linked to our database migration. Every row in the database had the same updatedAt value. We ran an SQL script to modify these so that they were all unique.
This fix was actually for some other issue we had, where not all rows were being returned for some unknown reason, but we also saw a substantial speed improvement.
Also, another weird fix that improved loading times was the following.
After we had pulled all of the data the first time (which, understandably takes some time) - we did an UpdateAsync() on one of the rows that were returned, and we did not push it afterwards.
We've come to understand that the way offline sync works, is that it will pull anything that has a date newer than the most recent updated at. There was a small speed improvement associated with this.
Finally, the last thing we did to improve speed was to not fetch the data again, if it already had cached a copy in the view. This may not work for your use case though.
public List<Foo> fooList = new List<Foo>
public void DisplayAllFoo()
{
if(fooList.Count == 0)
fooList = await SyncClass.GetAllFoo();
foreach(var foo in fooList)
{
Console.WriteLine(foo.bar);
}
}
Edit 20th March 2019:
With these improvements in place, we are still seeing very slow sync operations, used in the same way as mentioned in the OP, also including the improvements listed in my answer here.
I encourage all to share their solutions or ideas on how this speed can be improved.
One of the reasons for the slow Pull() is when more than (10) rows get the same UpdatedAt value. This happens when you update the rows at once, for example running an SQL command.
One way to overcome this is to modify the default trigger on the tables. To ensure every row gets a unique UpdateAt, we did something like this:
ALTER TRIGGER [dbo].[TR_dbo_Items_InsertUpdateDelete] ON [dbo].[TableName]
AFTER INSERT, UPDATE, DELETE
AS
BEGIN
DECLARE #InsertedAndDeleted TABLE
(
Id NVARCHAR(128)
);
DECLARE #Count INT,
#Id NVARCHAR(128);
INSERT INTO #InsertedAndDeleted
SELECT Id
FROM inserted;
INSERT INTO #InsertedAndDeleted
SELECT Id
FROM deleted
WHERE Id NOT IN
(
SELECT Id
FROM #InsertedAndDeleted
);
--select * from #InsertedAndDeleted;
SELECT #Count = Count(*)
FROM #InsertedAndDeleted;
-- ************************ UpdatedAt ************************
-- while loop
WHILE #Count > 0
BEGIN
-- selecting
SELECT TOP (1) #Id = Id
FROM #InsertedAndDeleted;
-- updating
UPDATE [dbo].[TableName]
SET UpdatedAt = Convert(DATETIMEOFFSET, DateAdd(MILLISECOND, #Count, SysUtcDateTime()))
WHERE Id = #Id;
-- deleting
DELETE FROM #InsertedAndDeleted
WHERE id = #Id;
-- counter
SET #Count = #Count - 1;
END;
END;
I need help with trying to understand how to delete all data from a table and then try to automatically import a new sheet with data into the newly cleared down table.
I'm currently trying the unload() method client side but that doesn't seem to cleardown my tables
function ClearDown(){
app.datasources.P11d.unload(function(){});
console.log('Finish Delete');
}
I've also tried to create a server side function, which also doesn't appear to work
function ClearTable(){
var records = app.models.P11d.newQuery();
// records.run();
console.log('Server Function Ran');
app.deleteRecords(records.run());
}
This is ran from a client side function:
function Delete(){
google.script.run.withSuccessHandler(function(result){
}).ClearTable();
console.log('Function Ran');
}
Again this is all to no avail
With the import side I've tried to do the below:-
Client Side:
function ImportData(){
console.log('Begin');
var ss = SpreadsheetApp.openById('SHEET ID');
var values = ss.getSheetByName('P11d').getDataRange().getValues();
var ssData = [];
// app.datasources.P11d.unload(function(){});
for (var i = 0; i<values.length; i++){
var newRecord = app.models.P11d.newRecord();
// add all fields to the new record
newRecord.Reg_Number = values[i][0];
newRecord.Reg_Date = values[i][1];
newRecord.Model_Description = values[i][2];
newRecord.P11d_Value = values[i][3];
newRecord.EngineSize = values[i][4];
newRecord.Fuel = values[i][5];
newRecord.CO2 = values[i][6];
newRecord.SIPP = values[i][7];
newRecord.GTA_Code = values[i][8];
newRecord.Type = values[i][9];
ssData.push(newRecord);
// console.log(newRecord.MODEL_FIELD);
}
console.log('Finished');
// return the array of the model.newRecord objects that would be consumed by the Model query.
}
Please can someone help with this, at the moment the way the data is sent over to me adding new stuff into the Drive Table is causing many duplicates.
Thanks in advance,
You can delete all records, import, and read from a spreadsheet using the AMU Library
Copy and paste the server and client scripts into your app.
I'm sure that will make it much easier!
To delete all the data in a model using this:
Button onClick:
google.script.run.AMU.deleteAllData('ModelName');
The correct way to delete records on the server is:
app.models.MODEL_NAME.deleteRecords(key_array);
datasource.unload() simply unloads the widget on the client. It does not affect the database records.
A better way to write your records query on the server is:
var query = app.models.MODEL_NAME.newQuery();
query.filters.your_filter_here;
var records = query.run();
Note that you cannot return a single record or an array of records from anything but a calculated model function without using a function posted here. (You can return a single field of a record using stringify for any json data.)
I am currently working on a solution to create datasource independent tables needed in App Maker.
For the delete function on the server try to change your code just a little bit, this function at least used to work for me, however I have not needed to use it in some time.
function ClearTable(){
var records = app.models.P11d.newQuery().run();
console.log('Server Function Ran');
app.deleteRecords(records);
}
When we recieve orders from web it creates a sales id and stores it. But if i recieve order from web at same time in two instances, it creates two sales orders for the same web order. So how can i stop it?
I kept as Index for weborder number Allow Duplicates:No. But still it doesnt work. Any Suggestions?
(Added as a answer bit late, 'cause I'm slow that way :))
Send a unique identifier like a GUID from the web, save it in SalesTable and in insert check if it already exists - or make a unique index for the field, but you might log these attempted duplicates and it's easier to code it yourself in insert or validateWrite.
This is because the user presses submit button several times. You need to track the number of clicks on the button. For this you need to use js.
var submit = 0;
function checkIsRepeat(){
var isValid = Page_ClientValidate();
if(isValid) {
if(++ submit > 1){
alert('Yours message here');
return false;
}
}
return isValid;
}