In my ADF pipeline I am trying to convert an output from my lookup activity to be in YYYY-MM-DD hh:mm:ss date format within the source query of a copy activity. The current output from my lookup activity is in YYYY-MM-DDThh:mm:ss format and I need to remove the 'T'.
I have tried using the dynamic content and formatDateTime functions but am having problems with the syntax. I am also using an SQL query to retrieve only the relevant data. The below is what I am
using as an input in the dynamic content query. I am able to get this to work, but I need to change '03/15/2018 12:00:00' to refer to the output of my lookup activity named LookupNewWaterMarkActivity.
SELECT *
FROM tableName
WHERE updatedDate >
'#{formatDateTime('03/15/2018 12:00:00', 'yyyy-MM-dd HH:mm:ss')}'
I have tried the below, but get the following error message:
'cannot fit package::output:any & { count, value } into the function parameter string. (6)'
SELECT *
FROM tableName
WHERE updatedDate >
'#{formatDateTime(activity('LookupNewWaterMarkActivity').output, 'yyyy-MM-dd HH:mm:ss')}'
Does anyone know how I can format the output of my activity within an SQL query any other way?
I am getting the below error when running the below code.
#concat('SELECT * FROM tableName WHERE sys_updated_on_value > ''',formatDateTime(activity('LookupNewWaterMarkActivity').output.value[0].sys_updated_on_value, 'yyyy-MM-dd HH:mm:ss'),'''')
Error code: FailToResolveParametersInExploratoryController
Details
The parameters and expression cannot be resolved for schema operations.
Error Message: { "message": "ErrorCode=InvalidTemplate,
ErrorMessage=The expression 'concat('SELECT * FROM tableName WHERE
sys_updated_on_value >
''',formatDateTime(activity('LookupNewWaterMarkActivity').output.value[0].sys_updated_on_value,
'yyyy-MM-dd HH:mm:ss'),'''')\n\n' cannot be evaluated because property
'value' doesn't exist, available properties are 'value[0]'.." }
Use the lookup activity output (activity('Lookup1').output.value[0].columnName) value in your expression with column name as shown below to refer the lookup activity output in later activities.
Lookup activity output:
Copy activity:
Expression:
#concat('SELECT * FROM tb2 WHERE date1 > ''',formatDateTime(activity('Lookup1').output.value[0].date1, 'yyyy-MM-dd HH:mm:ss'),'''')
Update:
If you have enables firstRow property in your lookup, use the below expression:
#concat('SELECT * FROM tb2 WHERE date1 > ''',formatDateTime(activity('Lookup1').output.firstRow.date1, 'yyyy-MM-dd HH:mm:ss'),'''')
Related
I have my data loaded on Neo4j instance where, for each node, I recorded the temporal information in a property key (.time) following the format:
YYYY-MM-DD
example: time: 1937-01-01
These are all strings, that I would like to convert into datetime as to use them in Neo4j Bloom and various time-based queries. I tried to use the following formula (as well as various variations of it):
MATCH (p:Image)
WHERE p.time IS NOT NULL
SET p.time = datetime({ epochMillis: apoc.date.parse(p.time, 's', 'yyyy-MM-dd HH:mm:ss') })
without success. I always get the error
Failed to invoke function `apoc.date.parse`: Caused by: java.text.ParseException: Unparseable date: "1891-01-01"
Any idea what I am doing wrong and how to transform date as string to datetime ?
I read some previous posts on the subject but I couldn't find a satisfying answer...
Your data format and the format you specified in the apoc function don't match. Also, you might wanna parse milliseconds, rather than seconds, since you are using epochMillis field.
Try this:
MATCH (p:Image)
WHERE p.time IS NOT NULL AND p.time <> ""
SET p.time = datetime({ epochMillis: apoc.date.parse(p.time + " 00:00:00", 'ms', 'yyyy-MM-dd HH:mm:ss') })
Ok, then for your error you will have to use your previous format, and convert your date into your format. The error basically meant that millisecond component is not present.
You can use function date() if you have only date part without time. You can call date('2015-07-21') directly.
https://neo4j.com/docs/cypher-manual/current/functions/temporal/#functions-date-create-string
I have value of column
datecolumn -> 2021-04-22T00:00:00.000
I want select using condition where like below:
SELECT * FROM 'tblDOISOAT' where datecolumn = strftime('%Y-%m-%d %H:%M:%S','2021-04-22 00:00:00')
I not get được value and i me met error 'Error: near "=": syntax error'. please help me
You have a typo, but your code works.
--------
dt
--------
2021-04-22T00:00:00.000
2021-04-23T01:00:00.000
--------
select strftime('%Y-%m-%dT%H:%M:%S.000', '2021-04-22 00:00:00') as 'datecode'
returns 2021-04-22T00:00:00.000
The query:
select dt from 't' where dt= strftime('%Y-%m-%dT%H:%M:%S.000', '2021-04-22 00:00:00')
returns the expected result: 2021-04-22T00:00:00.000
You forgot to include the correct formatting including T and .000 at the end.
remember that SQLite does not have a datetime data structure by default. You can read about SQLite data types here
If you are storing date/times as ISO-8601 text, you don't need to call strftime() to make comparisons. Just compare the 2 values:
select * from t1 where datecol = '2021-04-22T00:00:00.000';
The whole point of ISO-8601 strings is to make sure that date/time values compare lexicographically the same as temporally.
Trying to preform a date range query on a datasource example:
query.where = 'TransactionDate BETWEEN: StartDate AND EndDate';
This is what I get:
Unexpected input at ': StartDate AND EndDate'.
Error: Unexpected input at ': StartDate AND EndDate'. at datasources.
I was assuming this would work similarly to a MySQL query:
WHERE TransactionDate BETWEEN "2012-03-15" AND "2012-03-31";
In order to use real SQL query you need to go with Calculated SQL model. With query.where = ... you are setting App Maker's Query Builder expression that supports limited set of operations. I think your Query Builder expression will look similar to this:
TransactionDate >= :StartDate AND TransactionDate <= :EndDate
Context is Teradata SQL Assistant
Successfully created the following table:
CREATE VOLATILE TABLE RSN_WEEKLY_TMP, NO LOG
(
EXPLICIT_DATE DATE FORMAT 'MM/DD/YYYY'
)
PRIMARY INDEX (EXPLICIT_DATE)
ON COMMIT PRESERVE ROWS;
1) The following INSERT works successfully:
INSERT INTO JOCOOPER.RSN_WEEKLY_TMP (EXPLICIT_DATE) VALUES (CURRENT_DATE);
2) The following INSERT does not work and returns with Error:INSERT Failed [26665] Invalid date.
INSERT INTO JOCOOPER.RSN_WEEKLY_TMP (EXPLICIT_DATE) VALUES (02/02/2016);
3) However, if I use a string 'date value' and CAST it as a Date it works.
INSERT INTO JOCOOPER.RSN_WEEKLY_TMP (EXPLICIT_DATE) VALUES (CAST('02/03/2016' AS DATE FORMAT 'MM/DD/YYYY') );
I need to know how to make example #2 work? Please Advise?
02/02/2016 is an INTEGER calculation, dividing 2 by 2 by 2016, results in zero, of course this is not a valid date.
CAST('02/03/2016' AS DATE FORMAT 'MM/DD/YYYY') works because it tells the parser how to convert the string to a DATE.
The only recommended (and the shortest) way is a Standard SQL DATE literal:
DATE '2016-02-03'
You never need to think about formats because there's only one: YYYY-MM-DD
Actually, this format works too
insert into table_name (datecol) select '2015/12/31';
In your example:
CREATE VOLATILE TABLE RSN_WEEKLY_TMP, NO LOG
(
--EXPLICIT_DATE DATE FORMAT 'MM/DD/YYYY'
EXPLICIT_DATE DATE
)
PRIMARY INDEX (EXPLICIT_DATE)
ON COMMIT PRESERVE ROWS;
INSERT INTO JOCOOPER.RSN_WEEKLY_TMP (EXPLICIT_DATE) select '2016/02/02';
I'm trying to find all the results between two dates in SQLite.
select log_ID
from Message
where Timestamp between DATE('2014-04-17 03:27:08','YYYY-MM-DD HH:MI:SS')
and DATE('2014-04-18 03:27:08','YYYY-MM-DD HH:MI:SS');
This query is executing successfully but it gives no results.
I've tried using DATE and TO_DATE functions as well.
Schema:
CREATE TABLE Message(
Log_ID INTEGER PRIMARY KEY,
Session_ID TEXT NOT NULL,
Timestamp DATETIME NOT NULL,
Admill_Msg TEXT, Logcat_Msg TEXT);
Sample tuples:
155|admil.out.txt|2014-04-17 03:26:48.730000||PID:926 TID:926 TAG:I/Zygote LOG:Preloading resources...
156|admil.out.txt|2014-04-17 03:26:48.730000||PID:926 TID:926 TAG:W/Resources LOG:Preloaded drawable resource #0x1080096 (android:drawable/toast_frame) that varies with configuration!!
157|admil.out.txt|2014-04-17 03:26:48.740000||PID:926 TID:926 TAG:W/Resources LOG:Preloaded drawable resource #0x1080105 (android:drawable/btn_check_on_pressed_holo_light) that varies with configuration!!
You are using the date function wrong; there is not format parameter.
strftime could do such formatting, but that is not necessary because the timestamps are simply strings.
Just compare the strings directly:
select log_ID
from Message
where Timestamp between '2014-04-17 03:27:08'
and '2014-04-18 03:27:08';