I have my data loaded on Neo4j instance where, for each node, I recorded the temporal information in a property key (.time) following the format:
YYYY-MM-DD
example: time: 1937-01-01
These are all strings, that I would like to convert into datetime as to use them in Neo4j Bloom and various time-based queries. I tried to use the following formula (as well as various variations of it):
MATCH (p:Image)
WHERE p.time IS NOT NULL
SET p.time = datetime({ epochMillis: apoc.date.parse(p.time, 's', 'yyyy-MM-dd HH:mm:ss') })
without success. I always get the error
Failed to invoke function `apoc.date.parse`: Caused by: java.text.ParseException: Unparseable date: "1891-01-01"
Any idea what I am doing wrong and how to transform date as string to datetime ?
I read some previous posts on the subject but I couldn't find a satisfying answer...
Your data format and the format you specified in the apoc function don't match. Also, you might wanna parse milliseconds, rather than seconds, since you are using epochMillis field.
Try this:
MATCH (p:Image)
WHERE p.time IS NOT NULL AND p.time <> ""
SET p.time = datetime({ epochMillis: apoc.date.parse(p.time + " 00:00:00", 'ms', 'yyyy-MM-dd HH:mm:ss') })
Ok, then for your error you will have to use your previous format, and convert your date into your format. The error basically meant that millisecond component is not present.
You can use function date() if you have only date part without time. You can call date('2015-07-21') directly.
https://neo4j.com/docs/cypher-manual/current/functions/temporal/#functions-date-create-string
Related
In my ADF pipeline I am trying to convert an output from my lookup activity to be in YYYY-MM-DD hh:mm:ss date format within the source query of a copy activity. The current output from my lookup activity is in YYYY-MM-DDThh:mm:ss format and I need to remove the 'T'.
I have tried using the dynamic content and formatDateTime functions but am having problems with the syntax. I am also using an SQL query to retrieve only the relevant data. The below is what I am
using as an input in the dynamic content query. I am able to get this to work, but I need to change '03/15/2018 12:00:00' to refer to the output of my lookup activity named LookupNewWaterMarkActivity.
SELECT *
FROM tableName
WHERE updatedDate >
'#{formatDateTime('03/15/2018 12:00:00', 'yyyy-MM-dd HH:mm:ss')}'
I have tried the below, but get the following error message:
'cannot fit package::output:any & { count, value } into the function parameter string. (6)'
SELECT *
FROM tableName
WHERE updatedDate >
'#{formatDateTime(activity('LookupNewWaterMarkActivity').output, 'yyyy-MM-dd HH:mm:ss')}'
Does anyone know how I can format the output of my activity within an SQL query any other way?
I am getting the below error when running the below code.
#concat('SELECT * FROM tableName WHERE sys_updated_on_value > ''',formatDateTime(activity('LookupNewWaterMarkActivity').output.value[0].sys_updated_on_value, 'yyyy-MM-dd HH:mm:ss'),'''')
Error code: FailToResolveParametersInExploratoryController
Details
The parameters and expression cannot be resolved for schema operations.
Error Message: { "message": "ErrorCode=InvalidTemplate,
ErrorMessage=The expression 'concat('SELECT * FROM tableName WHERE
sys_updated_on_value >
''',formatDateTime(activity('LookupNewWaterMarkActivity').output.value[0].sys_updated_on_value,
'yyyy-MM-dd HH:mm:ss'),'''')\n\n' cannot be evaluated because property
'value' doesn't exist, available properties are 'value[0]'.." }
Use the lookup activity output (activity('Lookup1').output.value[0].columnName) value in your expression with column name as shown below to refer the lookup activity output in later activities.
Lookup activity output:
Copy activity:
Expression:
#concat('SELECT * FROM tb2 WHERE date1 > ''',formatDateTime(activity('Lookup1').output.value[0].date1, 'yyyy-MM-dd HH:mm:ss'),'''')
Update:
If you have enables firstRow property in your lookup, use the below expression:
#concat('SELECT * FROM tb2 WHERE date1 > ''',formatDateTime(activity('Lookup1').output.firstRow.date1, 'yyyy-MM-dd HH:mm:ss'),'''')
I am new to DynamoDB. I wish to create a table which using DeviceID as the hash key, Timestamp as my range key and some data.
{ DeviceID: 123, Timestamp: "2016-11-11T17:21:07.5272333Z", X: 12, Y: 35 }
In SQL, we can use datetime type for Timestamp, but in DynamoDB there is none.
What data type should I use? String? Number?
For the chosen data type, what kind of timestamp format should I write in? ISO format (e.g: 2016-11-11T17:21:07.5272333Z) or epoch time (e.g: 1478943038816)?
I need to search through the table through a range of time, e.g: 1/1/2015 10:00:00am until 31/12/2016 11:00:00pm
The String data type should be used for Date or Timestamp.
You can use the String data type to represent a date or a timestamp.
One way to do this is by using ISO 8601 strings, as shown in these
examples:
2016-02-15
2015-12-21T17:42:34Z
20150311T122706Z
DynamoDB Data type for Date or Timestamp
Yes, the Range queries are supported when the date is stored as String. The BETWEEN can be used on FilterExpresssion. I have got the items in the result using the below filter expressions.
FilterExpression without time:-
FilterExpression : 'createdate between :val1 and :val2',
ExpressionAttributeValues : {
':hkey' : year_val,
':rkey' : title,
":val1" : "2010-01-01",
":val2" : "2010-12-31"
}
FilterExpression with time:-
FilterExpression : 'createdate between :val1 and :val2',
ExpressionAttributeValues : {
':hkey' : year_val,
':rkey' : title,
":val1" : "2010-01-01T00:00:00",
":val2" : "2010-12-31T00:00:00"
}
Database Values:-
Format 1 - with timezone:
{"Item":{"createdate":{"S":"2010-12-21T17:42:34+00:00"},"title":{"S":"The Big New Movie 2010"},"yearkey":{"N":"2010"},"info":{"M":{"rating":{"N":"0"},"plot":{"S":"Nothing happens at all."}}}}}
Format 2 - without timezone:-
{"Item":{"createdate":{"S":"2010-12-21T17:42:34Z"},"title":{"S":"The Big New Movie 2010"},"yearkey":{"N":"2010"},"info":{"M":{"rating":{"N":"0"},"plot":{"S":"Nothing happens at all."}}}}}
Data type depends on your requirements.
You may use String using ISO format or Number using epoch format.
The advantage of ISO format (String) is human readability however DynamoDB does not support Time To Live (TTL) for this format. All filters work such as 'between' and 'range' as explained by notionquest.
Time To Live (TTL) for DynamoDB allows you to define when items in a table expire so that they can be automatically deleted from the database.
The advantage of using epoch format (Number) is that you can use the TTL feature and all filters.
TLDR;
Epoch format (Number type) - Can use Time To Live
ISO format (String type) - Cannot use Time To Live but is more human readable
for me to be able to filter out results when sending a query request, I used epoch format for DateTime, it is more efficient than using string.
imagine these scenarios: last 31 days, last 24 hours, ... again all is possible using the string format since it also has begins_with operator(please check 3rd example in below link on AWS doc) but numeric values are much more efficient in terms of performance while sorting(comparing) & calculation.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.KeyConditionExpressions
it is easy to convert date-time to epoch format
Javascript:
var date = new Date();
var epoch = date.getTime();
// converting back to date-time
var initial_date = new Date(epoch);
C#
var date = DateTime.UtcNow;
var epoch = new DateTimeOffset(date).ToUnixTimeSeconds();
// converting back to date-time
var initial_date = DateTimeOffset.FromUnixTimeSeconds(epoch);
Python
import time
epoch = time.time()
# converting back to date-time
initial_date = time.gmtime(epoch )
The Number data type OR the String data type
can be used for Date or Timestamp - not just String as the Accepted Answer on this Question incorrectly singles out while ignoring Number.
You can use the number data type to represent a date or a timestamp. One way to do this is by using epoch time—the number of seconds since 00:00:00 UTC on 1 January 1970. For example, the epoch time 1437136300 represents 12:31:40 PM UTC on 17 July 2015.
For more information, see http://en.wikipedia.org/wiki/Unix_time.
...
You can use the String data type to represent a date or a timestamp. One way to do this is by using ISO 8601 strings, as shown in these examples:
2016-02-15
2015-12-21T17:42:34Z
20150311T122706Z
For more information, see http://en.wikipedia.org/wiki/ISO_8601.
DynamoDB Data type for Date or Timestamp
I have an Oracle table which has a date column ( say its name is start_date) that has stored date as UTC date. I have another column that stores a timezone (like 'America/Los_Angeles'). My requirement is I need to display the date column with timestamp corresponding to the timezone stored in the timezone column.
I initially wrote a function that accepts utc_date and the timezone and returns the date as below:
return utc_date + (SUBSTR (TZ_OFFSET (timezone), 1, 1) || '1')
* TO_DSINTERVAL (
'0 '
|| SUBSTR (TZ_OFFSET (timezone), 2, 5)
|| ':00');
but I realized a flaw. It calculates offset based on current time. So it now returns -00 08:00:00.000000 for Los_Angeles. But if the date stored in the utc_date was a date when daylight was enforced, the tz_offset value is not valid anymore. Can someone provide me some pointers how can I approach this problem?
I found a solution to my problem. Instead of relying on TZ_OFFSET, I decided to do the following
return cast(from_tz(cast(utc_date as timestamp),'UTC') at time zone timezone as date);
This is returning me the desired date. If anyone see a flaw let me know
I'm currently working with a client that has a VB.NET web application that was developed internally. They've got everything storing to an Access database which they cannot alter or change for their own reasons. I'm not familiar with any of these technologies, so I'm hoping you may have a solution.
The client has a date field that they are only capturing mm/yyyy or blank. They need this information to save to a datetime field in the database. I'm trying to work up a statement that will automatically take the date entered and convert from mm/yyyy to mm/01/yyyy if the date is provided, or 01/01/1970 if the field was left blank. Can anyone assist?
If we are talking about MS Access functions DateSerial is what you are looking for. The basic syntax is below. If the stored value is text you will need to use the Mid function to parse the text into the year and month and you can use use a hard coded 1 for the day.
DateSerial ( year, month, day )
This function can be used in a select or update. Additional logic will be required to provide a default value for the blank result. Typically in Access this type of logic is done with an IIF.
You can use a combination of the IIf,IsNull and CDate functions, like so:
IIf(IsNull([YourDateFField]),#1/1/1970#,CDate([YourDateFField]))
This tests if your field is null and if yes it returns 1/1/1970, if no it will convert your date string to an actual date (e.g. CDate("04/2014") will return 4/1/2014)
This (MSAccess/VBA) function will do what you are asking. If you pass-in a string like mm/yyyy, it will return a datetime like mm/01/yyyy. However, if the string does not fit that pattern (or equiv), the function will return a date time of 1/1/1970, like you asked.
'in MSAccess:
Public Function mmyyyyToDate(mmyyyy As String) As Datetime
If IsDate(Replace(mmyyyy, "/", "/01/")) Then
Return CDate(Replace(mmyyyy, "/", "/01/"))
Else
Return #1/1/1970#
End If
End Function
It would be more efficient to run it in MSAccess, but if you want to run it in VB.net instead, the syntax is different:
'in VB.NET
Public Function mmyyyyToDate(mmyyyy As Object, Optional defaultDate As DateTime = "1/1/1970") As DateTime
Dim re As DateTime
If Convert.IsDbNull(mmyyyy)
return defaultDate
ElseIf DateTime.TryParse(Replace(mmyyyy, "/", "/01/"), re) Then
Return re
Else
Return defaultDate
End If
End Function
Example of running it:
'MSAccess query syntax
INSERT INTO NewDateTable (NewDateColumn)
SELECT mmyyyyToDate(oldColumn) FROM OldTable
If you can't add a new function to the MSAccess DB, you could turn this function into an inline statement (by using an IIF), but it looks pretty ugly:
'MSAccess query syntax
INSERT INTO NewDateTable (NewDateColumn)
SELECT IIF(IsDate(Replace(oldColumn, "/", "/01/")), Replace(mmyyyy, "/", "/01/"), #1/1/1970#)
FROM OldTable
In my SQL database, I have a column formatted as DateTime and when I retrieve data from that column in ASP.NET, I catch it on the Date variable, than pass the value to textbox:
Dim Y As Date = dt.Rows(0)("SCH_DATE")
txtSchedDate.Text = Y.Date.ToString
but when I debug my website, the txtSchedDate.Text still gives me the full DateTime value:
7/17/2013 12:00:00 AM
is it possible to eliminate the time value here and just return the date?
Have you tried using something like
txtSchedDate.Text = Y.Date.ToString("MM/dd/yyyy")
or which ever format you wish to display.
Have a look at
DateTime.ToString Method (String)
Converts the value of the current DateTime object to its equivalent
string representation using the specified format.
Custom Date and Time Format Strings
Standard Date and Time Format Strings
Convert.ToDateTime(dt.Rows(0)("SCH_DATE")).ToString("M/d/yyy")
you can get date by txtSchedDate.Text = Y.Date.ToShortDateString()
Besides answers above, you can try converting it in SQL server
SELECT CONVERT(varchar(15), GETDATE(), 11)
Keep in mind after converting it's VARCHAR(15) instead of DATETIME.
Once you have a Date object, you can get the constituent pieces if you wish as well, like this:
Dim Y As Date = dt.Rows(0)("SCH_DATE")
txtSchedDate.Text = Y.Date.Year & "-" & Y.Date.Month & "-" & Y.Date.Day
Or you can use the custom and standard date and time format strings mentioned by others.