Firebase count group by - firebase

Does Firebase supports grouped counting?
I would like to get counting for specific key grouped by the value.
Example of my data structure:
"playbackPosition" : {
"-JZ2c7v-imQXaEx2c4Cs" : {
"data" : "0"
},
"-JZ2cDta73UGAgwIOejL" : {
"data" : "25"
},
"-JZ2cJutDZu-7quLpLI2" : {
"data" : "50"
},
"-JZ2cO-Me0nGK5XLTwd-" : {
"data" : "75"
},
"-JZ2cSh-XnstYxbE_Zad" : {
"data" : "100"
},
"-JZ2cWxya0-kmTPCze4u" : {
"data" : "0"
},
"-JZ2c_wX-ODv43TKXkNG" : {
"data" : "25"
}
}
Required results based on the data key :
0 => 2
25 => 2
50 => 1
75 => 1
100 => 1
And of course I must consider that it will have thousands of children's, not only 7...
Thanks ahead!
EDIT
Deeper explanation of the app and the problem we want to solve.
We have video scripts which runs on different websites, each video session (a user session) sends events and data, you can see an example here, check the Network tab - https://secure.bwebi.co/videostir/regular.html
Our goal is to collect this data and create an analytics real time dashboard with few charts and graphs.
You can see our current data structure here
Example for graphs we need:
Completion rate
General - Bar graph showing overall number of views per clip duration pre defined periods.
Filters - date (start/end), unique/all, All urls / specific urls (embed on), All configurations / specific configurations/ ignore silent.
X axis - groups 0-25, 25-50,50-75,75-99, 100
Y axis - number of views
Views per day (with completion rate)
General - Multi lines graph showing number of views per day per duration periods.
Filters - date (start/end), unique/all, All urls / specific urls (embed on), All configurations / specific configurations / ignore silent.
X axis - Time in days
Y axis - Number of views
Lines for:
Total daily views
Daily views with 100% duration
Daily views with 75-99% duration
Daily views with 50-75% duration
Daily views with 25-50% duration
Daily views with 0-25% duration
Hope it's more clear now!

Group by is a SQL function. The reason SQL can't do real-time data is because this sort of method does not scale. Mongo provides similar functionality, but once again, it doesn't scale. You may notice a pattern here of why Firebase does not provide this sort of query function.
It would be extremely helpful if you provided some context of what you're actually attempting to accomplish here, what the rules of the app are, and what approaches you've ruled out, rather than just your presupposed solution of group by. There are probably other, possibly better, alternatives. See the XY problem.
Here are a couple generic alternatives derived by making sweeping assumptions about your use case.
Store the totals
This is the most scalable solution. Store your data as follows:
/playbacks/$id/<playback data>
/group_totals/$group/<count>
When writing to playbacks, also update the count for the appropriate group:
var fb = new Firebase(URL);
function addPlayback(rec) {
var ref = fb.child('playbacks').push(rec, function(err) {
if( err ) throw err;
incrementCount(rec.data);
});
}
function incrementCount(count) {
fb.child('group_totals/' + count).transaction(function(currentVal) {
return (currentVal||0)+1;
});
}
Now when you want to get the total for a group, you can simply look up the value at group_totals/$group. Similarly, you can store ids for records that belong to each group and utilize that index to grab only the records for a given group.
Use priorities to fetch and group
A simpler approach would be to give each record a priority based on the group/data value.
var fb = new Firebase(URL);
function addPlayback(rec) {
rec['.priority'] = rec.data;
var ref = fb.child('playbacks').push(rec, function(err) {
if( err ) throw err;
});
}
Now to grab a set of records for a given group:
var fb = new Firebase(URL);
function getGroup(groupValue, callback) {
fb.child('playbackPosition').startAt(groupValue).endAt(groupValue).once('value', callback);
}
function logGroupCount(groupValue, callback) {
getGroup(groupValue, function(snap) {
console.log('there are ' + snap.numChildren() + ' items in group ' +groupValue);
});
}

A am not professional programmer, I am just learning.
Here is the piece of code I came up with when I wanted to group my query results:
btn.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View v) {
progressBar.setVisibility(View.VISIBLE);
tv_info.setText("Please wait ... ");
Query query = collectionRef.orderBy("timestamp", Query.Direction.DESCENDING);
query.get().addOnSuccessListener(new OnSuccessListener<QuerySnapshot>() {
#Override
public void onSuccess(QuerySnapshot queryDocumentSnapshots) {
String dataInfo = "";
int arrayLength = 1;
List <String> UsedNames = new ArrayList<String>();
String someName = "...";
String oneRow = "";
int orderNumber = -1;
int count = 0;
for (QueryDocumentSnapshot documentSnapshot : queryDocumentSnapshots) {
OneRecord oneRecord = documentSnapshot.toObject(OneRecord.class);
someName = oneRecord.getSomeName();
if (UsedNames.contains(someName)) {
// Log.i("test" , "Array Contains ");
} else {
orderNumber += 1;
UsedNames.add(someName);
}
}
List list = queryDocumentSnapshots.toObjects(OneRecord.class);
for (String someString : UsedNames) {
int counter = 0;
for (int i = 0; i < list.size(); i++) {
OneRecord oneRecord = (OneRecord) list.get(i);
String name = oneRecord.getName();
if (someString.equals(name)) {
counter += 1;
}
}
Log.i("test" , "Array: " + someString + " : " + counter);
count = count +1;
dataInfo = dataInfo + someString + " : " + counter + "\n";
}
Log.i("test" , "Used length: " + UsedNames.size());
progressBar.setVisibility(View.GONE);
tv_info.setText(dataInfo);
}
}).addOnFailureListener(new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
progressBar.setVisibility(View.GONE);
tv_info.setText("Could not query last records: " + e.getMessage());
}
});
}
});
Unfortunately I did not figure out how to sort them in DESCENDING or ASCENDING order

Related

Firestore managing auto increment ID [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
My application will have multiple users and they will be feeding complaints and getting back complaint ID's.
I have created a separate document which stores the latest incremental values, every time a complaint is saved, it firsts gets the maximum number from that document , increments it with + 1 and saves that complaint ID in new complaint document.
Users will be around 8, is this possible that this autoincrement number can be duplicated?
Just in case, attaching screenshot of console and copying my code
get_ticket_id(){
//1. check if the doc exists
var docRef = firebase.firestore().collection('complaints').doc("autoincrement");
docRef.get().then(async doc=> {
if (doc.exists) {
this.ticketnumber = doc.data().number + 1;
await firebase.firestore().collection("complaints").doc("autoincrement").update({
number: this.ticketnumber,
})
.then(async doc=> {
await this.CreateNewComplaint();
})
} else {
Am I following the best practice? Please note: I am not using increment option
const increment = firebase.firestore.FieldValue.increment(1);
number: increment
Is it safer to use increment ?
Edit 1
Use transactions as guided, I am trying to fetch the value of newIncId in a variable but it says, code unreachable on this code of line this.ticketnumber = newIncId; // code uncreable.
var docRef = firebase.firestore().collection('complaints').doc("autoincrement");
firebase.firestore().runTransaction(transaction=> {
return transaction.get(docRef).then(incDoc=> {
//if no value exist, assume it as 0 and increase to 1
var newIncId = (incDoc.data().number || 0) + 1;
transaction.update(docRef, { number: incDoc });
return newIncId;
this.ticketnumber = newIncId; // code uncreable
});
}).then(function(newIncId) {
...//some code
Edit 2
Giving this error:
Function Transaction.update() called with invalid data. Unsupported field value: a custom t object (found in field number)
Attaching screenshot of console
For events like these you should consider Firebase transactions which prevent unwanted behaviours like multiple concurrent overwrites/updates.
final DocumentReference sfDocRef = db.collection("complaints").document("autoincrement");
db.runTransaction(new Transaction.Function<Void>() {
#Override
public Void apply(Transaction transaction) throws FirebaseFirestoreException {
DocumentSnapshot snapshot = transaction.get(sfDocRef);
int newId = snapshot.getInt("number") + 1;
transaction.update(sfDocRef, "number", newPopulation);
// Success
return null;
}
}).addOnSuccessListener(new OnSuccessListener<Void>() {
#Override
public void onSuccess(Void aVoid) {
Log.d(TAG, "Transaction success!");
}
})
.addOnFailureListener(new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
Log.w(TAG, "Transaction failure.", e);
}
});
I've copied the code directly from the firebase documentation, so haven't properly tested this.
You could read more about transactions here: https://firebase.google.com/docs/firestore/manage-data/transactions#java
Updated to use transaction and get the new number:
var docRef = firebase.firestore().collection('complaints').doc("autoincrement");
db.runTransaction(function(transaction) {
return transaction.get(docRef).then(function(incDoc) {
//if no value exist, assume it as 0 and increase to 1
var newIncId = (incDoc.data().number || 0) + 1;
transaction.update(docRef, { number: newIncId });
return newIncId;
});
}).then(function(newIncId) {
//`newIncId` This is your new number incremented
//use newIncId here
this.ticketnumber = newIncId;
console.log("New autoincremented number ", newIncId);
}).catch(function(err) {
// Catch block to get any error
console.error(err);
});
See doc https://firebase.google.com/docs/firestore/manage-data/transactions#passing_information_out_of_transactions
Why you're not using number: firebase.firestore.FieldValue.increment(1);?
This use transactions internally. But be aware that firestore document have the write limit of 1/sec. You can use RTDB if you expect to write more than once.

I have a "Upload Record" PXAction to load records to grid and release these records

I have a custom PXbutton called UploadRecords, when I click this button I should populate the grid with records and release the records.
Release Action is pressed in the UploadRecords action delegate. The problem I get with this code is, the code here function properly for less records by release action but when passes thousands of records to release, it takes huge time(> 30 min.) and show the error like Execution timeout.
suggest me to avoid more execution time and release the records fastly.
namespace PX.Objects.AR
{
public class ARPriceWorksheetMaint_Extension : PXGraphExtension<ARPriceWorksheetMaint>
{
//public class string_R112 : Constant<string>
//{
// public string_R112()
// : base("4E5CCAFC-0957-4DB3-A4DA-2A24EA700047")
// {
// }
//}
public class string_R112 : Constant<string>
{
public string_R112()
: base("EA")
{
}
}
public PXSelectJoin<InventoryItem, InnerJoin<CSAnswers, On<InventoryItem.noteID, Equal<CSAnswers.refNoteID>>,
LeftJoin<INItemCost, On<InventoryItem.inventoryID, Equal<INItemCost.inventoryID>>>>,
Where<InventoryItem.salesUnit, Equal<string_R112>>> records;
public PXAction<ARPriceWorksheet> uploadRecord;
[PXUIField(DisplayName = "Upload Records", MapEnableRights = PXCacheRights.Select, MapViewRights = PXCacheRights.Select)]
[PXButton]
public IEnumerable UploadRecord(PXAdapter adapter)
{
using (PXTransactionScope ts = new PXTransactionScope())
{
foreach (PXResult<InventoryItem, CSAnswers, INItemCost> res in records.Select())
{
InventoryItem invItem = (InventoryItem)res;
INItemCost itemCost = (INItemCost)res;
CSAnswers csAnswer = (CSAnswers)res;
ARPriceWorksheetDetail gridDetail = new ARPriceWorksheetDetail();
gridDetail.PriceType = PriceTypeList.CustomerPriceClass;
gridDetail.PriceCode = csAnswer.AttributeID;
gridDetail.AlternateID = "";
gridDetail.InventoryID = invItem.InventoryID;
gridDetail.Description = invItem.Descr;
gridDetail.UOM = "EA";
gridDetail.SiteID = 6;
InventoryItemExt invExt = PXCache<InventoryItem>.GetExtension<InventoryItemExt>(invItem);
decimal y;
if (decimal.TryParse(csAnswer.Value, out y))
{
y = decimal.Parse(csAnswer.Value);
}
else
y = decimal.Parse(csAnswer.Value.Replace(" ", ""));
gridDetail.CurrentPrice = y; //(invExt.UsrMarketCost ?? 0m) * (Math.Round(y / 100, 2));
gridDetail.PendingPrice = y; // (invExt.UsrMarketCost ?? 0m)* (Math.Round( y/ 100, 2));
gridDetail.TaxID = null;
Base.Details.Update(gridDetail);
}
ts.Complete();
}
Base.Document.Current.Hold = false;
using (PXTransactionScope ts = new PXTransactionScope())
{
Base.Release.Press();
ts.Complete();
}
List<ARPriceWorksheet> lst = new List<ARPriceWorksheet>
{
Base.Document.Current
};
return lst;
}
protected void ARPriceWorksheet_RowSelected(PXCache cache, PXRowSelectedEventArgs e, PXRowSelected InvokeBaseHandler)
{
if (InvokeBaseHandler != null)
InvokeBaseHandler(cache, e);
var row = (ARPriceWorksheet)e.Row;
uploadRecord.SetEnabled(row.Status != SPWorksheetStatus.Released);
}
}
}
First, Do you need them all to be in a single transaction scope? This would revert all changes if there is an exception in any. If you need to have them all committed without any errors rather than each record, you would have to perform the updates this way.
I would suggest though moving your process to a custom processing screen. This way you can load the records, select one or many, and use the processing engine built into Acumatica to handle the process, rather than a single button click action. Here is an example: https://www.acumatica.com/blog/creating-custom-processing-screens-in-acumatica/
Based on the feedback that it must be all in a single transaction scope and thousands of records, I can only see two optimizations that may assist. First is increasing the Timeout as explained in this blog post. https://acumaticaclouderp.blogspot.com/2017/12/acumatica-snapshots-uploading-and.html
Next I would load all records into memory first and then loop through them with a ToList(). That might save you time as it should pull all records at once rather than once for each record.
going from
foreach (PXResult<InventoryItem, CSAnswers, INItemCost> res in records.Select())
to
var recordList = records.Select().ToList();
foreach (PXResult<InventoryItem, CSAnswers, INItemCost> res in recordList)

Dictionary and condition

I have been working on an assignment in which I have to upload some records from a file to Dictionary and manipulate.
Actually file have number of record with same invoice number and different tax values and I have to add all those values and make it only one invoice
what I'm trying to do
I'm passing values from a foreach loop to a function which check if the dictionary is empty so it will simply add first record and on second call it will check weather any record in dictionary have same invoice number so it will sum and update current tax value to one already added,
what I'm getting
when I pass 2nd value (and so on) the previous value of last entry in dictionary some how update itself with current value before even comparing which I don't want.
public jd_records jd = new jd_records();
Dictionary<int, jd_records> jdValues = new Dictionary<int, jd_records>();
//Calling values with loop while jd is a publicly declared object of class jd_records
foreach (DataRow dr in jddt.Rows)
{
//jd_records jdPass = new jd_records();
jd.supplierName = dr["Supplier"].ToString();
jd.supplierNTN = dr["Supplier NTN"].ToString();
jd.invoiceNo = dr["JDE Invoice Number"].ToString();
jd.invoiceDate = DateTime.Parse(dr["JDE Invoice Date"].ToString());
if (dr["Taxable Amount"].ToString().Equals(""))
{ jd.taxable = 0; }
else
{ jd.taxable = float.Parse(dr["Taxable Amount"].ToString()); }
if (dr["Tax To Pay"].ToString().Equals(""))
{ jd.tax = 0; }
else
{ jd.tax = float.Parse(dr["Tax To Pay"].ToString()); }
jdRecordCheck();
}
called function
public void jdRecordCheck()
{
if (jdValues.Count < 1)
{
jdValues.Add(0, jd);
}
else //previous record values (at key 0) changes to new jd value when come to this else part on execution
{
foreach (KeyValuePair<int,jd_records> jdVal in jdValues)
{
if ((jdVal.Value.supplierNTN.Equals(jd.supplierNTN)) && (jdVal.Value.invoiceNo.Equals(jd.invoiceNo)))
{
jdVal.Value.tax = jdVal.Value.tax + jd.tax;
jdVal.Value.taxable = jdVal.Value.taxable + jd.taxable;
jdValues[jdVal.Key] = jdVal.Value;
}
else
{
jdValues.Add(jdVal.Key + 1, jd);
}
}
}
}
I'll be very thankful if anyone helps.
Seems to me this would be much easier to do with Linq, which contains functions for grouping and summing.
List<jd_records> result = jddt.Rows
.GroupBy(jd => jd.invoiceNo)
.Select(jd => new jd_records
{
invoiceNo = jd.invoiceNo,
totalTax = jd.Sum(d => d.tax)
}).ToList();

Chargify how can get total transactions of between two dates?

Currently, i am getting only 20 transactions per pages and can be extends only 200 but i need all transaction between that dates no paging.
Or if it possible to get count of transactions.?
How can i archive?
Thanks
I would use the following to retrieve all transactions for a subscription:
bool isFinished = false;
int counter = 1;
var results = new Dictionary<int, ITransaction>();
while (!isFinished)
{
// Get results
var transactions = chargify.GetTransactionsForSubscription(activeSubscription.SubscriptionID, counter++, 20);
// Check condition
if (transactions.Count == 0) { isFinished = true; continue; }
// Merge results
transactions.ToList().ForEach(x => results.Add(x.Key, x.Value));
}
That should get all the transactions and merge them all into the single dictionary. :) If need to use dates, then just switch the data retrieval line to something like this:
var transactions = chargify.GetTransactionsForSubscription(activeSubscription.SubscriptionID, counter++, 20, null, int.MinValue, int.MinValue, DateTime.Today, DateTime.Now); (just switch to your dates).

Access Variable<int> in Custom Activity in WF4

I need to count how many times i iterate in a flowchart flow, but i need to be able to read and preferably write to the variable in a custom activity.
My current attempt is declaring the var in design view with scope of the entire Flowchart, default value 0 and incrementing using an Assign activity. But i cannot figure out how i can access the variable in a custom activity without resetting it.
My attempt to access the var is something like whats described in the answer here: Declare Variable<T> variable in a CodeActivity in windows workflow 4.0
Only i don't use a default value for the var when declaring though. Still it seems the var is not in any way related to the var i have defined in design view. I have also tried defining it in code only but then i cannot access it in for example a regular Assign activity.
So what can i do to be able to use the var as a "global" variable?
Thanks.
The most intuitive and perhaps correct way of doing it is to pass the variable that you're declaring on Flowchart level to inside your custom activity. Then you can do whatever you want with it's value and return it.
An example of a custom increment activity (this is how Assign activity works too):
public class IncrementActivity : CodeActivity<int>
{
[RequiredArgument]
public InArgument<int> CountVariable { get; set; }
protected override int Execute(CodeActivityContext context)
{
// Do whatever logic you want here
return CountVariable.Get(context) + 1;
}
}
Here it is a sample of usage using a Sequence (the same when using Flowchart):
var countVar = new Variable<int>("count");
var activity = new Sequence
{
Variables =
{
// declare counter variable at global scope
countVar
},
Activities =
{
new WriteLine { Text = new VisualBasicValue<string>(#"""Count: "" & count") },
new IncrementActivity { CountVariable = countVar, Result = countVar },
new WriteLine { Text = new VisualBasicValue<string>(#"""Count: "" & count") },
new IncrementActivity { CountVariable = countVar, Result = countVar },
new WriteLine { Text = new VisualBasicValue<string>(#"""Count: "" & count") },
new IncrementActivity { CountVariable = countVar, Result = countVar }
}
};
Output:
Count: 0
Count: 1
Count: 2
Note that is simpler through visual designer as you don't have to directly use VisualBasicValue<string> to build print string. Other than that, is exactly the same!

Resources