A Meteor server code tries to group documents by the month. So that the count of all the documents in each month is given.
The documents have createdAt = Date.now(); property and meteorhacks:aggregate
is installed. Any Suggestions? thx
What I considered:
1) Transform the collection adding new property `MMYYYY: 6-digit-number' and group by that.
Your consideration is a good one if you plan to aggregate often. It would certainly be faster if you wrote a year and month string in the doc when you create it. Keep in mind that you will have to remember to keep those strings up to date if you modify the corresponding date field.
If you won't be aggregating very often, then it's probably not worth the effort. Just convert your epoch time values to new Date objects and leverage the Date aggregation operators in your aggregation pipeline. Here is an example.
var Metrics = new Mongo.Collection('metrics');
Metrics.aggregate([
{$project: {
createdAtDate: { $add: [new Date(0), "$createdAt"] }
}},
{$project : {
year : {$year : "$createdAtDate"},
month : {$month : "$createdAtDate"}
}},
{$group : {
_id : {year : "$year", month : "$month"},
count : {$sum : 1}
}}
]);
If you go with a date string (where the document field called "createdAtString" is MMYYYY) then your pipeline would look like this instead.
var Metrics = new Mongo.Collection('metrics');
Metrics.aggregate([
{$group : {
_id : "$createdAtString",
count : {$sum : 1}
}}
]);
Related
I'm new to DynamoDB and I'm trying to query a table from javascript using the Dynamoose library. I have a table with a primary partition key of type String called "id" which is basically a long string with a user id. I have a second column in the table called "attributes" which is a DynamoDB map and is used to store arbitrary user attributes (I can't change the schema as this is how a predefined persistence adapter works and I'm stuck working with it for convenience).
This is an example of a record in the table:
Item{2}
attributes Map{2}
10 Number: 2
11 Number: 4
12 Number: 6
13 Number: 8
id String: YVVVNIL5CB5WXITFTV3JFUBO2IP2C33BY
The numeric fields, such as the "12" field, in the Map can be interpreted as "week10", "week11","week12" and "week13" and the numeric values 2,4,6 and 8 are the number of times the application was launched that week.
What I need to do is get all user ids of the records that have more than 4 launches in a specific week (eg week 12) and I also need to get the list of user ids with a sum of 20 launches in a range of four weeks (eg. from week 10 to 13).
With Dynamoose I have to use the following model:
dynamoose.model(
DYNAMO_DB_TABLE_NAME,
{id: String, attributes: Map},
{useDocumentTypes: true, saveUnknown: true}
);
(to match the table structure generated by the persistence adapter I'm using).
I assume I will need to do DynamoDB "scan" to achieve this rather than a "query" and I tried this to get started and get a records where week 12 equals 6 to no avail (I get an empty set as result):
const filter = {
FilterExpression: 'contains(#attributes, :val)',
ExpressionAttributeNames: {
'#attributes': 'attributes',
},
ExpressionAttributeValues: {
':val': {'12': 6},
},
};
model.scan(filter).all().exec(function (err, result, lastKey) {
console.log('query result: '+ JSON.stringify(result));
});
If you don't know Dynamoose but can help with solving this via the AWS SDK tu run a DynamoDB scan directly that might also be helpful for me.
Thanks!!
Try the following.
const filter = {
FilterExpression: '#attributes.#12 = :val',
ExpressionAttributeNames: {
'#attributes': 'attributes',
'#12': '12'
},
ExpressionAttributeValues: {
':val': 6,
},
};
Sounds like what you are really trying to do is filter the items where attributes.12 = 6. Which is what the query above will do.
Contains can't be used for objects or arrays.
I want to write data in dataframe to dynamodb table
item = {}
for row in datasource_archived_df_join_repartition.rdd.collect():
item['x'] = row.x
item['y'] = row.y
client.put_item( TableName='tryfail',
Item=item)
but im gettin this error
Invalid type for parameter Item.x, value: 478.2, type: '<'type 'float''>', valid types: '<'type 'dict''>'
Invalid type for parameter Item.y, value: 696- 18C 12, type: '<'type 'unicode''>', valid types: '<'type 'dict''>'
Old question, but it still comes up high in a search and hasn't been answered properly, so here we go.
When putting an item in a DynamoDB table it must be a dictionary in a particular nested form that indicates to the database engine the data type of the value for each attribute. The form looks like below. The way to think of this is that an AttributeValue is not a bare variable value but a combination of that value and its type. For example, an AttributeValue for the AlbumTitle attribute below is the dict {'S': 'Somewhat Famous'} where the 'S' indicates a string type.
response = client.put_item(
TableName='Music',
Item={
'AlbumTitle': { # <-------------- Attribute
'S': 'Somewhat Famous', # <-- Attribute Value with type string ('S')
},
'Artist': {
'S': 'No One You Know',
},
'SongTitle': {
'S': 'Call Me Today',
},
'Year': {
'N': '2021' # <----------- Note that numeric values are supplied as strings
}
}
)
In your case (assuming x and y are numbers) you might want something like this:
for row in datasource_archived_df_join_repartition.rdd.collect():
item = {
'x': {'N': str(row.x)},
'y': {'N': str(row.y)}
}
client.put_item( TableName='tryfail', Item=item)
Two things to note here: first, each item corresponds to a row, so if you are putting items in a loop you must instantiate a new one with each iteration. Second, regarding the conversion of the numeric x and y into strings, the DynamoDB docs explain that the reason the AttributeValue dict requires this is "to maximize compatibility across languages and libraries. However, DynamoDB treats them as number type attributes for mathematical operations." For fuller documentation on the type system for DynamoDB take a look at this or read the Boto3 doc here since you are using Python.
The error message is indicating you are using the wrong type, it looks like you need to be using a dictionary when assigning values to item['x'] and item[y]. e.g.
item['x'] = {'value': row.x}
item['y'] = {'value': row.y}
I am new to meteor and mongoDB and have been searching for an answer to this question for some time without any luck.
I have multiple documents in MongoDB similar to the one below:
{
"_id" : ObjectId("5abac4ea0c31d26804421371"),
"Points" : [
{
"Value" : 6.869752766626993,
"Time" : 1522284528946
},
{
"Value" : 3.9014587731230477,
"Time" : 1522284543946
},
{
"Value" : 1.2336926618519772,
"Time" : 1522284558946
},
{
"Value" : 6.504837583667155,
"Time" : 1522284573946
},
{
"Value" : 9.824138227740864,
"Time" : 1522284588946
},
{
"Value" : 9.707480757899235,
"Time" : 1522284603946
},
{
"Value" : 4.6122167850338105,
"Time" : 1522284618946
}
]
}
How can I implement a query in meteor that returns an array containing all the Points from all documents with 'Time' field greater than certain value?
As Jankapunkt has pointed out in his comment, it might be a lot easier and better if you created a new collection Points where each document includes only Value and Time attributes. The given example would then become seven separate documents rather than a single array.
It does nevertheless happen, that we want to query documents according to some inner values, e.g. attributes in objects in arrays.
Taken from the mongodb documentation on querying embedded documents, we can just use dot notation for this.
If you do not know the index position of the document nested in the array, concatenate the name of the array field, with a dot (.) and the name of the field in the nested document.
Such as for your question (assuming Points to be the name of your collection):
db.points.find( { 'Points.Time': { $gte: 123412341234 } } )
Which looks almost identical in Meteor:
Points.find({ 'Points.Time': { $gte: 123412341234 } })
I'm having a bit of trouble with a Firebase query, mainly due to the size of the dataset I am querying.
What I would like to achieve is:
Find all tshirts where brandStartsWith = 'A' and salesRank is between 1 and 100
I've started to pad this out, but I am running into an issue whereby I can't seem to get the data due to having over 300,000 records within t-shirts.
If call it within React when the page loads, after a while I get the following error in console:
Uncaught RangeError: Invalid string length
Here is the code I am using to get me started, but I'm not sure where to go. Looking at the solutions on this question it seems I need to download the data per my query below, and then sort it on the client side. Something I cant seem to do
firebase.database().ref('tshirts')
.orderByChild('brandStartsWith')
.equalTo('A')
.once('value', function (snapshot) {
console.log(snapshot.val())
})
You're going to need to create a combined key as you can only do one where clause at a time.
{
"tShirts" : {
"brandStartsWith" : 'A',
"salesRank" : 5
"brandStartsWith_salesRank" = 'A_00005' //pad for as many sales ranks as you have
}, {
"brandStartsWith" : 'B',
"salesRank" : 108
"brandStartsWith_salesRank" = 'B_00108' //pad for as many sales ranks as you have
}, {
"brandStartsWith" : 'C',
"salesRank" : 52
"brandStartsWith_salesRank" = 'C_00052' //pad for as many sales ranks as you have
}
}
This will allow you to do this query:
firebase.database().ref('tshirts')
.orderByChild('brandStartsWith_salesRank')
.startAt('A_00001')
.endAt('A_00100')
.once('value', function (snapshot) {
console.log(snapshot.val())
})
Don't forget to update your rules to .index brandStartsWith_salesRank
I have to compare a user entered date, "Dt" (in mm/dd/yyyy format) with the date in RavenDB - "ReleaseDate" (time stamp like "/Date(1187668800000)/"). For this I am using the following code which almost gets the job done, but I need little help to finalize loose ends...
How can I compare the two dates so I can get the query to run successfully.
public ActionResult Calculation(DateTime? Dt)
{
var store = new DocumentStore { Url = "http://localhost:80" };
store.Initialize();
var CalcModel = new CalcViewModel();
using (var session = store.OpenSession())
{
//Converting user entered date dt in mm/dd/yyyy format to total
//milliseconds - So that later I can compare this value to RavenDB
//time stamp date format (older versions)
DateTime d1 = new DateTime(1970, 1, 1);
DateTime d2 = Dt.Value.ToUniversalTime();
TimeSpan ts = new TimeSpan(d2.Ticks - d1.Ticks);
double tmillisecs = ts.TotalMilliseconds; //Not yet using this value.
CalcModel.MoviesByDate = session.Query<Movies>()
.Where(x => x.ReleaseDate.Ticks == ts.Ticks)
.Count();
// this is where I need to compare two dates - ts.ticks gives the
// required value of date (1187668800000) multiplied by 10000.
}
return View(CalcModel);
}
Right now, when I debug I know what value ts.ticks is showing... and its like I said above in the code comments, the required value multiplied by 10000. But I have no clue at run time , what the value in x.ReleaseDate is or x.ReleaseDate.Ticks is.. am I doing this correctly. Thanks for the help.
Umm... I think you seriously misunderstand how SQL dates work, and how it applies to .NET. The whole point about dates is that they're stored in a numeric format, not a text one. So when you have a DateTime object, it's not stored as the text date, it's stored as a numeric type that you can convert to any format you want.
Because the .net provider converts database native datetime objects to DateTime objects, you can just compare them natively. ie:
DateTime d1 = new DateTime(1970, 1, 1);
CalcModel.MoviesByDate = session.Query<Movies>()
.Where(x => x.ReleaseDates.Date == d1.Date)
.Count();
Regardless of how RavenDB stores the dates internally, when the DateTime object is materialized in the query, it will be in native .NET format.