this is my format of data:
[{city:"Bhopal",id: 1},{city:"Bhopal",id: 2},{city:"Delhi",id: 3},{city:"Delhi",id:3}]
here i have Delhi repeated twice with same id.
now i need distinct count of city where id is distinct i.e like :
[key:"Bhopal",value:2, key:"Delhi",value:1]
where value is count
got the answer using Reductio and Crossfilter.
var payments = crossfilter([
{city: "Bhopal", id: 1},{city: "Bhopal", id: 2},{city: "Delhi", id: 3},{city: "Delhi", id: 3}
]);
var dim = payments.dimension(function(d) { return d.city; });
var group = dim.group();
var reducer = reductio()
.exception(function(d) { return d.id; })
.exceptionCount(true);
reducer(group);
console.log(group.top(Infinity));
output: [ { key: 'Bhopal', value: { exceptionCount: 2 },{ key: 'Delhi', value: { exceptionCount: 2 }]
Related
Consider a CosmosDB container with the following document model:
{
id: <string>,
userId: <string>, // partition key
data: <string>
}
I have a need to provide a query with N user ids and get the newest document for each one.
So for example, if I have this data in the container:
{ id: '1', userId: 'user1', data: 'a', _ts: 1 },
{ id: '2', userId: 'user1', data: 'b', _ts: 2 },
{ id: '3', userId: 'user2', data: 'c', _ts: 10 },
{ id: '4', userId: 'user2', data: 'd', _ts: 5 },
{ id: '5', userId: 'user3', data: 'e', _ts: 3 },
{ id: '6', userId: 'user3', data: 'f', _ts: 4 },
{ id: '7', userId: 'user4', data: 'g', _ts: 100 },
{ id: '8', userId: 'user4', data: 'h', _ts: 99 },
{ id: '9', userId: 'user5', data: 'i', _ts: 1 },
{ id: '10', userId: 'user5', data: 'j', _ts: 2 },
I want to do something like this:
-- This doesn't work
SELECT c.userId, (SELECT TOP 1 d.id, d.data WHERE d.userId = c.userId FROM d ORDER BY d._ts DESC) AS newest
WHERE c.userId IN ['user1', 'user2', 'user4', 'user5']
To get this result:
{ userId: 'user1', newest: { id: '2', data: 'b' } },
{ userId: 'user2', newest: { id: '3', data: 'c' } },
{ userId: 'user4', newest: { id: '7', data: 'g' } },
{ userId: 'user5', newest: { id: '10', data: 'j' } },
From what I can tell, JOIN in CosmosDB cannot be used to filter correlated documents. Is there still a way to accomplish this? I am open to using a stored procedure, but from what I can tell execution of a stored procedure can only occur on a specific partition given it's key. In my case, the primary grouping is the partition key.
I have considered a fan-out request approach, but I might be querying for 50 to 100 user ids at a time in the query. In that case it might be faster to just get all documents in each partition and when iterating only keep the newest -- but that's a large paged response to sift through.
My final thought is I could use ASB/EventGrid/Function and another dependent CosmosDB container to always clone the most recent updated document every time a document is updated, but it seems like overkill. Surely there is a way to construct a query to do what I want?
Thanks
I have an idea like
select c._ts from c where ARRAY_CONTAINS((select value max(c._ts) from c group by c.userId), c._ts)
But it can't get the result because select value max(c._ts) from c group by c.userId doesn't be recognized as an array, and if I use Array(select value max(c._ts) from c group by c.userId) instead, it returns all items.
So how about execute sql twice?
Get the timestamp array first :select value max(c._ts) from c where c.userId in ('user1','user2') group by c.userId
,
and then copy the result as the input to use array_contains function:
select c._ts,c.data from c where ARRAY_CONTAINS([1623306298,1623306259,1623306217], c._ts)
One way of doing this would be to use the following approach.
SELECT t.userid,
SUBSTRING(t.concat, 28,8000) AS data
FROM
(
SELECT c.userid,
MAX(CONCAT(TimestampToDateTime(c._ts*1000),c.data)) AS concat
FROM c
WHERE c.userid IN ('user1', 'user2')
GROUP BY c.userid
) AS t
which returns a result like
[
{
"userid": "user1",
"data": "b"
},
{
"userid": "user2",
"data": "d"
}
]
The derived table t returns results like the following...
[
{
"userid": "user2",
"concat": "2021-06-11T17:42:03.0000000Zd"
},
{
"userid": "user1",
"concat": "2021-06-11T17:41:41.0000000Zb"
}
]
The document with the highest _ts per user will have the lexicographically highest datetime prefix in the concatenated string and the ancillary data that is appended on behind it is extracted with SUBSTRING.
It should be able to use the index for the WHERE clause - but then will need to look at all documents for the given userids (so if there are many documents per user doing separate TOP 1 queries for each one would likely be much better)
Below is a simplistic version of the fire base database:
Players {
{
name: 'Messi',
goals: 90,
assists:2
id: 1
},
{
name: 'Beckham',
assists: 14,
goals: 22
id: 2
},
{
name: 'Rooney',
goals: 10,
assistst: 3
id: 3
},
{
name: 'Neymar',
goals: 14,
assists: 1
id: 4
}
}
The players are brought into the scope as such:
var ref = firebase.database().ref("players");
var ref3 = firebase.database().ref("users").child(uid).child("total");
$scope.players = $firebaseArray(ref);
$scope.selection = $firebaseArray(ref3);
Is there a way to index the player ids so that the following code will produce the selected players:
firebase.database().ref("players").orderByChild("id").equalTo($scope.selection);
You can retrieve data and limit to a set number of results. Check out the Limit Queries section in the docs:
https://firebase.google.com/docs/database/admin/retrieve-data
var ref = db.ref("dinosaurs");
ref.orderByChild("goals").limitToLast(2).on("child_added",
How one can update a list in dynamodb. consider the following example
Section:"A",
Group:[{Name:"Albert",
Age:24},
{Name:"Antony",
Age:25}],
GroupName:"Ply"
Now I need to change Alberts age from 24 to 26, considering Section as my primary partition key
var params = {
TableName: "Add_list",
Key:{
Section:"A"
},
"UpdateExpression" : "#Key = :value",
"ConditionExpression":"#Key1 = :value1",
"ExpressionAttributeNames" : {
"#Key" : "Group",
"#Key1":"Group[0].Name"
},
"ExpressionAttributeValues" : {
":value" : [{Name:"Albert",Age:26}],
":value1" : "Albert"
}
};
I've got a big set of data loaded into crossfilter for a dc.js project I'm working on. Of course a number of my columns have repeated values in them and I'd like to be able to find the unique list of values in one column that correspond to the repeated values in another. The sample data below probably illustrates the point more clearly.
var data = [
{ state: "WA", city: "Seattle", data: "a" },
{ state: "WA", city: "Seattle", data: "b" },
{ state: "WA", city: "Tacoma", data: "c" },
{ state: "OR", city: "Portland", data: "d" },
{ state: "OR", city: "Bend", data: "e" },
{ state: "OR", city: "Bend", data: "f" }
];
I'd like to be able to filter on a particular state and then find the unique list of cities for that state. So, if the input was "WA", I'd like get back a two element array containing "Seattle" and "Tacoma". The code below actually does exactly that (and also provides the counts, though I really don't care about those) but having to create a second crossfilter object feels very clumsy to me. I also don't know about the performance since I'll end up having to iterate through this several times, once for each state.
var Ndx = crossfilter(data);
var stateDim = Ndx.dimension(function (d) { return d.state; });
var cityDim = Ndx.dimension(function (d) { return d.city; });
var stateFilter = stateDim.filter("WA");
var stateRows = stateFilter.top(Infinity);
// It seems like there should be a better way than this.
var cityNdx = crossfilter(stateRows);
var cityDim2 = cityNdx.dimension(function (d) { return d.city; });
var cites = cityDim2.group().top(Infinity);
cites.forEach(function(d) {
console.log("City: " + d.key + ", Count: " + d.value);
});
/* --------------------------- *\
Log output:
City: Seattle, Count: 2
City: Tacoma, Count: 1
\* --------------------------- */
It seems like the should be a way to get to this kind of result with some filtering, grouping, or reducing strategy, but after spending way too much time trying, I haven't been able to come up with one. All the examples I've seen that use multiple dimensions produce aggregates, but that's not what I need. I need values. Is there a better way to go about this?
I'd use a custom reduce function to keep an array of all city values that have appeared for a given state. Something like the following (completely untested - sorry) should work:
var Ndx = crossfilter(data);
var stateDim = Ndx.dimension(function (d) { return d.state; });
var stateGroup = stateDim.group().reduce(
function(p, v) {
p.count++;
if(p.uniques.indexOf(v.city) === -1) p.uniques.push(v.city);
},
function(p, v) {
p.count--;
// Note: uniques are not filtered. You need to use a map and keep
// count of uniques to have uniques that match your current filter
},
function() {
return { count: 0, uniques: [] };
}
);
stateGroup.top(Infinity).forEach( function(g) {
console.log("State " + g.key + " has count " + g.value.count);
console.log("Unique cities in " + g.key + ":");
g.value.uniques.forEach(function (c) {
console.log(c);
});
});
How to avoid getting a exception from jQgrid when the row count is zero and filters are applied. my code looks like this
function filterGrid(grid, siteId, buildingId, cityId, selectedType) {
var today = new Date();
var dd = today.getDate();
var mm = today.getMonth() + 1; //January is 0!
var yyyy = today.getFullYear();
if (dd < 10) { dd = '0' + dd } if (mm < 10) { mm = '0' + mm } today = mm + '/' + dd + '/' + yyyy;
if (cityId == -1) {
grid.setGridParam({ search: false });
}
else {
var filter = { groupOp: "AND", rules: [{ field: "CityID", op: "eq", data: cityId}] };
if (siteId >= 0)
filter = { groupOp: "AND", rules: [{ field: "CityID", op: "eq", data: cityId }, { field: "SiteID", op: "eq", data: siteId}] }; // , { field: "BuildingID", op: "eq", data: buildingId}
if (buildingId >= 0)
filter = { groupOp: "AND", rules: [{ field: "CityID", op: "eq", data: cityId }, { field: "SiteID", op: "eq", data: siteId }, { field: "BuildingID", op: "eq", data: buildingId}] };
if (selectedType == "Outstanding Books")
filter = { groupOp: "AND", rules: [{ field: "CityID", op: "eq", data: cityId }, { field: "SiteID", op: "eq", data: siteId }, { field: "BuildingID", op: "eq", data: buildingId }, { field: "IsReturnDateNull", op: 'eq', data: true }, { field: "DueDate", op: 'gt', data: today}] };
//For filtering
grid.setGridParam({ search: true, postData: { filters: JSON.stringify(filter)} });
//For searching
//grid.setGridParam({ search: true, postData: { searchOper: "eq", searchField: "Status", searchString: selectedVal} });
}
grid.trigger("reloadGrid");
}
The Code that i have written works perfectly fine BUT if the datasource has no records to bind with the grid then i get an exception, I thought of getting the row count into a hidden field on the server side and on the client side if the row count is zero in the hidden field then skip searching, but since the datasource is getting assigned to the jqgrid during an asynchronous call back hence value in the hidden field is not getting updated(always get empty string in hiddenfield on client side) so can any body help me how to avoid getting a null exception from the jqgrid when there is no data bound to it.
I finally found an answer to this question. I used the event onsearching of the jqgrid and on the server side i get the row count from a hidden field that contains the current row and if the row count is 0 then i cancel the search operation.
protected void grdBookTransaction_Searching(object sender, Trirand.Web.UI.WebControls.JQGridSearchEventArgs e)
{
if (string.IsNullOrEmpty(GridCount.Value) || GridCount.Value == "0")
{
e.Cancel = true;
}
}