I am new to DynamoDB. I wish to create a table which using DeviceID as the hash key, Timestamp as my range key and some data.
{ DeviceID: 123, Timestamp: "2016-11-11T17:21:07.5272333Z", X: 12, Y: 35 }
In SQL, we can use datetime type for Timestamp, but in DynamoDB there is none.
What data type should I use? String? Number?
For the chosen data type, what kind of timestamp format should I write in? ISO format (e.g: 2016-11-11T17:21:07.5272333Z) or epoch time (e.g: 1478943038816)?
I need to search through the table through a range of time, e.g: 1/1/2015 10:00:00am until 31/12/2016 11:00:00pm
The String data type should be used for Date or Timestamp.
You can use the String data type to represent a date or a timestamp.
One way to do this is by using ISO 8601 strings, as shown in these
examples:
2016-02-15
2015-12-21T17:42:34Z
20150311T122706Z
DynamoDB Data type for Date or Timestamp
Yes, the Range queries are supported when the date is stored as String. The BETWEEN can be used on FilterExpresssion. I have got the items in the result using the below filter expressions.
FilterExpression without time:-
FilterExpression : 'createdate between :val1 and :val2',
ExpressionAttributeValues : {
':hkey' : year_val,
':rkey' : title,
":val1" : "2010-01-01",
":val2" : "2010-12-31"
}
FilterExpression with time:-
FilterExpression : 'createdate between :val1 and :val2',
ExpressionAttributeValues : {
':hkey' : year_val,
':rkey' : title,
":val1" : "2010-01-01T00:00:00",
":val2" : "2010-12-31T00:00:00"
}
Database Values:-
Format 1 - with timezone:
{"Item":{"createdate":{"S":"2010-12-21T17:42:34+00:00"},"title":{"S":"The Big New Movie 2010"},"yearkey":{"N":"2010"},"info":{"M":{"rating":{"N":"0"},"plot":{"S":"Nothing happens at all."}}}}}
Format 2 - without timezone:-
{"Item":{"createdate":{"S":"2010-12-21T17:42:34Z"},"title":{"S":"The Big New Movie 2010"},"yearkey":{"N":"2010"},"info":{"M":{"rating":{"N":"0"},"plot":{"S":"Nothing happens at all."}}}}}
Data type depends on your requirements.
You may use String using ISO format or Number using epoch format.
The advantage of ISO format (String) is human readability however DynamoDB does not support Time To Live (TTL) for this format. All filters work such as 'between' and 'range' as explained by notionquest.
Time To Live (TTL) for DynamoDB allows you to define when items in a table expire so that they can be automatically deleted from the database.
The advantage of using epoch format (Number) is that you can use the TTL feature and all filters.
TLDR;
Epoch format (Number type) - Can use Time To Live
ISO format (String type) - Cannot use Time To Live but is more human readable
for me to be able to filter out results when sending a query request, I used epoch format for DateTime, it is more efficient than using string.
imagine these scenarios: last 31 days, last 24 hours, ... again all is possible using the string format since it also has begins_with operator(please check 3rd example in below link on AWS doc) but numeric values are much more efficient in terms of performance while sorting(comparing) & calculation.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.KeyConditionExpressions
it is easy to convert date-time to epoch format
Javascript:
var date = new Date();
var epoch = date.getTime();
// converting back to date-time
var initial_date = new Date(epoch);
C#
var date = DateTime.UtcNow;
var epoch = new DateTimeOffset(date).ToUnixTimeSeconds();
// converting back to date-time
var initial_date = DateTimeOffset.FromUnixTimeSeconds(epoch);
Python
import time
epoch = time.time()
# converting back to date-time
initial_date = time.gmtime(epoch )
The Number data type OR the String data type
can be used for Date or Timestamp - not just String as the Accepted Answer on this Question incorrectly singles out while ignoring Number.
You can use the number data type to represent a date or a timestamp. One way to do this is by using epoch time—the number of seconds since 00:00:00 UTC on 1 January 1970. For example, the epoch time 1437136300 represents 12:31:40 PM UTC on 17 July 2015.
For more information, see http://en.wikipedia.org/wiki/Unix_time.
...
You can use the String data type to represent a date or a timestamp. One way to do this is by using ISO 8601 strings, as shown in these examples:
2016-02-15
2015-12-21T17:42:34Z
20150311T122706Z
For more information, see http://en.wikipedia.org/wiki/ISO_8601.
DynamoDB Data type for Date or Timestamp
Related
I have my data loaded on Neo4j instance where, for each node, I recorded the temporal information in a property key (.time) following the format:
YYYY-MM-DD
example: time: 1937-01-01
These are all strings, that I would like to convert into datetime as to use them in Neo4j Bloom and various time-based queries. I tried to use the following formula (as well as various variations of it):
MATCH (p:Image)
WHERE p.time IS NOT NULL
SET p.time = datetime({ epochMillis: apoc.date.parse(p.time, 's', 'yyyy-MM-dd HH:mm:ss') })
without success. I always get the error
Failed to invoke function `apoc.date.parse`: Caused by: java.text.ParseException: Unparseable date: "1891-01-01"
Any idea what I am doing wrong and how to transform date as string to datetime ?
I read some previous posts on the subject but I couldn't find a satisfying answer...
Your data format and the format you specified in the apoc function don't match. Also, you might wanna parse milliseconds, rather than seconds, since you are using epochMillis field.
Try this:
MATCH (p:Image)
WHERE p.time IS NOT NULL AND p.time <> ""
SET p.time = datetime({ epochMillis: apoc.date.parse(p.time + " 00:00:00", 'ms', 'yyyy-MM-dd HH:mm:ss') })
Ok, then for your error you will have to use your previous format, and convert your date into your format. The error basically meant that millisecond component is not present.
You can use function date() if you have only date part without time. You can call date('2015-07-21') directly.
https://neo4j.com/docs/cypher-manual/current/functions/temporal/#functions-date-create-string
How to convert datetime in data loaded in Neo4j to timestamp?
Example datetime format: 2020-04-07T12:39:38.027Z
Please assist.
Neo4's temporal instants (like datetime) have a special variable, epochMillis, that provides the equivalent epoch time.
For example:
RETURN datetime('2020-04-07T12:39:38.027Z').epochMillis
returns
1586263178027
You can convert a DateTime to a string by first converting it to a timestamp with epochMillis and passing that to the apoc.date.toISO8601 function.
WITH datetime() as dt
RETURN apoc.date.toISO8601(dt.epochMillis, "ms") AS iso8601
returns
"2022-04-30T18:56:53.145Z"
I have a date column in my mysql db which stores
1999-03-30
as the date.
Using webservices when i extract the value by following code
DOB = Convert.ToDateTime(reader["DOB"]),
The output on the screen is
/Date(922732200000)/
How to get 1999-03-30 and what is this value /Date(922732200000)/...
The easiest way to handle dates in mysql is to use UNIX TIMESTAMP to store and retrieve dates.
For example:
The query for inserting date should be:
string query="insert into myTable(name,date) values ("StackOverflow",unix_timestamp('yyyy-mm-dd HH:mm:ss')";
eg. If you want to enter the current datetime, this query can be re-Written as:
string query="insert into myTable(name,date) values ("StackOverflow",unix_timestamp(current_timestamp)";
and when you retrive the data from your table, the query should be:
string queryToretrivedata="select name, from_unixtime(date) as DOB from myTable";
While the MySqlreader reads the data, the statement in your function should be:
DateTime DOB= Convert.ToDateTime(reader["DOB"].ToString());
Also, if you want to convert this dateTime into string you can use
string DateInString==DOB.ToString("yyyy-MM-dd HH:mm:ss"); //For 24 hour Format
string DateInString==DOB.ToString("yyyy-MM-dd hh:mm:ss tt"); //For 12 Hour format
Im using SQLite and trying to get my date into a blackberry datetime field.
in the DB its stored as string in format:
yyyy-MM-dd HH:mm
e.g. : 2012-02-01 15:45
But the field wants it as a Long.
date - The initial date value for this field. This is the number of milliseconds since midnight on January 1, 1970. To create an empty date field, set this parameter to Long.MIN_VALUE. This method will then remove the date value from this field, setting it to null.
And then convert it back again.
Check following code:
// conversion - string to long
long dateLong = HttpDateParser.parse("2012-04-17 16:09");
// conversion - long to string
Date dateObject = new Date(dateLong);
String dateStr = (new SimpleDateFormat("yyyy-MM-dd HH:mm")).format(dateObject);
Using vs2008 I have a SQL Server database attached to my web app. I want to use a computed time column in the database, along the lines of :
timenow is 1 column
hoursleft is another column
timeend would be another column.
I want timeend to = timenow + hoursleft.
Is it possible to do that, and if so what would the formula be that I would enter into the computed column field, and, what datatype would the columns be, timenow and timeend I would expect to be time(7) and hoursleft an int. But is that correct? THanks for any help.
TimeNow is datatype datetime, populated with function GetUtcDate().
HoursLeft is an int, populated as required.
TimeEnd is a computed column of type datetime, computed as DateAdd(hh, HoursLeft, TimeNow).
You'd use this to calculate timeend from the 2 existing columns hoursleft and timenow
CAST(DATEADD(hour, hoursleft, timenow) AS time(7))
DATEADD return type is
The return data type is the data type of the date argument
So the explicit CAST probably isn't needed if timenow is time(7) as you mentioned
DATEADD(hour, hoursleft, timenow)