TPT load data type error - teradata

I'm trying to run a TPT load, but I am getting an error with my date datatype. Can someone help me with the right syntax?
DEFINE JOB MOVE_DATA
DESCRIPTION 'MOVE DATA'
(DEFINE SCHEMA SAMPLE_SCHEMA DESCRIPTION 'SAMPLE SCHEMA'
(
CUST DECIMAL(15,0) ,
START_DATE DATE FORMAT 'YYYY-MM-DD'
);
Error: TPT_INFRA: At "DATE" missing { PERIOD_ ANSIDATE_ BIGINT_ BLOB_
BYTE_ BYTEINT_ CHARACTER_ CHAR_ CHARACTERS_ CHARS_ CLOB_ DEC_ DECIMAL_
FLOAT_ GRAPHIC_ INT_ INTDATE_ INTEGER_ INTERVAL_ LONG_ NUMBER_
NUMERIC_ SMALLINT_ TIME_ TIMESTAMP_ VARBYTE_ VARCHAR_ VARDATE_
VARGRAPHIC_ XML_ } in Rule: Column Data Type

DATE is not a supported field type for your SCHEMA in a TPT. You can use VARCHAR(10). For Example:
START_DATE VARCHAR(10)
Or if you need to do a change of format in the TPT ETL, you can use VARDATE(). For example:
START_DATE VARDATE(10) FORMATIN ('YYYY-MM-DD') FORMATOUT ('MM/DD/YYYY')

Related

Neo4j: Converting string to datetime

I have my data loaded on Neo4j instance where, for each node, I recorded the temporal information in a property key (.time) following the format:
YYYY-MM-DD
example: time: 1937-01-01
These are all strings, that I would like to convert into datetime as to use them in Neo4j Bloom and various time-based queries. I tried to use the following formula (as well as various variations of it):
MATCH (p:Image)
WHERE p.time IS NOT NULL
SET p.time = datetime({ epochMillis: apoc.date.parse(p.time, 's', 'yyyy-MM-dd HH:mm:ss') })
without success. I always get the error
Failed to invoke function `apoc.date.parse`: Caused by: java.text.ParseException: Unparseable date: "1891-01-01"
Any idea what I am doing wrong and how to transform date as string to datetime ?
I read some previous posts on the subject but I couldn't find a satisfying answer...
Your data format and the format you specified in the apoc function don't match. Also, you might wanna parse milliseconds, rather than seconds, since you are using epochMillis field.
Try this:
MATCH (p:Image)
WHERE p.time IS NOT NULL AND p.time <> ""
SET p.time = datetime({ epochMillis: apoc.date.parse(p.time + " 00:00:00", 'ms', 'yyyy-MM-dd HH:mm:ss') })
Ok, then for your error you will have to use your previous format, and convert your date into your format. The error basically meant that millisecond component is not present.
You can use function date() if you have only date part without time. You can call date('2015-07-21') directly.
https://neo4j.com/docs/cypher-manual/current/functions/temporal/#functions-date-create-string

How to format an activity output as YYYY-MM-DD hh:mm:ss in Azure data factory

In my ADF pipeline I am trying to convert an output from my lookup activity to be in YYYY-MM-DD hh:mm:ss date format within the source query of a copy activity. The current output from my lookup activity is in YYYY-MM-DDThh:mm:ss format and I need to remove the 'T'.
I have tried using the dynamic content and formatDateTime functions but am having problems with the syntax. I am also using an SQL query to retrieve only the relevant data. The below is what I am
using as an input in the dynamic content query. I am able to get this to work, but I need to change '03/15/2018 12:00:00' to refer to the output of my lookup activity named LookupNewWaterMarkActivity.
SELECT *
FROM tableName
WHERE updatedDate >
'#{formatDateTime('03/15/2018 12:00:00', 'yyyy-MM-dd HH:mm:ss')}'
I have tried the below, but get the following error message:
'cannot fit package::output:any & { count, value } into the function parameter string. (6)'
SELECT *
FROM tableName
WHERE updatedDate >
'#{formatDateTime(activity('LookupNewWaterMarkActivity').output, 'yyyy-MM-dd HH:mm:ss')}'
Does anyone know how I can format the output of my activity within an SQL query any other way?
I am getting the below error when running the below code.
#concat('SELECT * FROM tableName WHERE sys_updated_on_value > ''',formatDateTime(activity('LookupNewWaterMarkActivity').output.value[0].sys_updated_on_value, 'yyyy-MM-dd HH:mm:ss'),'''')
Error code: FailToResolveParametersInExploratoryController
Details
The parameters and expression cannot be resolved for schema operations.
Error Message: { "message": "ErrorCode=InvalidTemplate,
ErrorMessage=The expression 'concat('SELECT * FROM tableName WHERE
sys_updated_on_value >
''',formatDateTime(activity('LookupNewWaterMarkActivity').output.value[0].sys_updated_on_value,
'yyyy-MM-dd HH:mm:ss'),'''')\n\n' cannot be evaluated because property
'value' doesn't exist, available properties are 'value[0]'.." }
Use the lookup activity output (activity('Lookup1').output.value[0].columnName) value in your expression with column name as shown below to refer the lookup activity output in later activities.
Lookup activity output:
Copy activity:
Expression:
#concat('SELECT * FROM tb2 WHERE date1 > ''',formatDateTime(activity('Lookup1').output.value[0].date1, 'yyyy-MM-dd HH:mm:ss'),'''')
Update:
If you have enables firstRow property in your lookup, use the below expression:
#concat('SELECT * FROM tb2 WHERE date1 > ''',formatDateTime(activity('Lookup1').output.firstRow.date1, 'yyyy-MM-dd HH:mm:ss'),'''')

Issue with Timestamp field in Fload

I am facing an issue with the fastload where my Timestamp fields are getting rejected to error table.
Below is the value of timestamp(6) field in my flat file.
23-06-2016 11:51:21.000000 23-06-2016 11:51:21.000000
Below is my code:
SET RECORD VARTEXT "ยก";
DEFINE
TRANSACTION_SOURCE_TYPE_ID (VARCHAR(54))
,TRANSACTION_SOURCE_TYPE_CODE (VARCHAR(20))
,TRANSACTION_SOURCE_TYPE_DESC (VARCHAR(110))
,EFFECTIVE_START_DATE (VARCHAR(54))
,EFFECTIVE_END_DATE (VARCHAR(54))
,COUNTRY_CODE (VARCHAR(13))
,SOURCE_SYSTEM_ID (VARCHAR(54))
,DW_LOAD_TIMESTAMP (VARCHAR(76))
,DW_UPD_LOAD_TIMESTAMP (VARCHAR(76))
,FORCE_SKEW_KEY (VARCHAR(51))
FILE=?INPUT_FILE;
SHOW;
INSERT INTO ?DWSBKPDB.TRANSACTION_TMP
(
TRANSACTION_SOURCE_TYPE_ID
,TRANSACTION_SOURCE_TYPE_CODE
,TRANSACTION_SOURCE_TYPE_DESC
,EFFECTIVE_START_DATE
,EFFECTIVE_END_DATE
,COUNTRY_CODE
,SOURCE_SYSTEM_ID
,DW_LOAD_TIMESTAMP
,DW_UPD_LOAD_TIMESTAMP
,FORCE_SKEW_KEY
)
VALUES
(
:TRANSACTION_SOURCE_TYPE_ID
,:TRANSACTION_SOURCE_TYPE_CODE
,:TRANSACTION_SOURCE_TYPE_DESC
,:EFFECTIVE_START_DATE (DATE, FORMAT 'YYYY-MM-DD')
,:EFFECTIVE_END_DATE (DATE, FORMAT 'YYYY-MM-DD')
,:COUNTRY_CODE
,:SOURCE_SYSTEM_ID
,:DW_LOAD_TIMESTAMP (TIMESTAMP, FORMAT 'YYYY-MM-DDBHH:MI:SS.S(6)')
,:DW_UPD_LOAD_TIMESTAMP (TIMESTAMP, FORMAT 'YYYY-MM-DDBHH:MI:SS.S(6)')
,:FORCE_SKEW_KEY
);
DW_LOAD_TIMESTAMP is creating problem here.
Any idea as to why this is happening.
Regards,
Amit
Your data obviously doesn't match the FORMAT.
:DW_LOAD_TIMESTAMP(TIMESTAMP, FORMAT 'DD-MM-YYYYBHH:MI:SS.S(6)')

Teradata SQL Assistant Date Inserts

Context is Teradata SQL Assistant
Successfully created the following table:
CREATE VOLATILE TABLE RSN_WEEKLY_TMP, NO LOG
(
EXPLICIT_DATE DATE FORMAT 'MM/DD/YYYY'
)
PRIMARY INDEX (EXPLICIT_DATE)
ON COMMIT PRESERVE ROWS;
1) The following INSERT works successfully:
INSERT INTO JOCOOPER.RSN_WEEKLY_TMP (EXPLICIT_DATE) VALUES (CURRENT_DATE);
2) The following INSERT does not work and returns with Error:INSERT Failed [26665] Invalid date.
INSERT INTO JOCOOPER.RSN_WEEKLY_TMP (EXPLICIT_DATE) VALUES (02/02/2016);
3) However, if I use a string 'date value' and CAST it as a Date it works.
INSERT INTO JOCOOPER.RSN_WEEKLY_TMP (EXPLICIT_DATE) VALUES (CAST('02/03/2016' AS DATE FORMAT 'MM/DD/YYYY') );
I need to know how to make example #2 work? Please Advise?
02/02/2016 is an INTEGER calculation, dividing 2 by 2 by 2016, results in zero, of course this is not a valid date.
CAST('02/03/2016' AS DATE FORMAT 'MM/DD/YYYY') works because it tells the parser how to convert the string to a DATE.
The only recommended (and the shortest) way is a Standard SQL DATE literal:
DATE '2016-02-03'
You never need to think about formats because there's only one: YYYY-MM-DD
Actually, this format works too
insert into table_name (datecol) select '2015/12/31';
In your example:
CREATE VOLATILE TABLE RSN_WEEKLY_TMP, NO LOG
(
--EXPLICIT_DATE DATE FORMAT 'MM/DD/YYYY'
EXPLICIT_DATE DATE
)
PRIMARY INDEX (EXPLICIT_DATE)
ON COMMIT PRESERVE ROWS;
INSERT INTO JOCOOPER.RSN_WEEKLY_TMP (EXPLICIT_DATE) select '2016/02/02';

how to pass date array as xml to store procedure

Bcz when u parse that string into date that will not parse properly.
for that u have to send date string as per sql server date format
ALTER PROCEDURE [dbo].[SprSelectScheduleForReAutoAllocate] --'<AADates><AADate AADateValue="2011-04-27 00:00:00.000" /><AADate AADateValue="2011-04-28 00:00:00.000" /></AADates>'
(
#Datexml xml
)
AS
BEGIN
EXEC sp_xml_preparedocument #xmlDoc output ,#DateXml
SELECT AADateValue
FROM OPENXML(#xmlDoc,'AADates/AADate',1)
WITH
(
AADateValue Datetime
)
xmlDocuments
EXEC sp_xml_removedocument #xmlDoc
End
SQL Server reads date times values according to the ISO 8601 standard in the following two formats:
YYYY-MM-DDThh:mm:ss[.mmm]
YYYYMMDDThh:mm:ss[.mmm]
Format you input like this and SQL Server will never throw an error due to incompatible culture or language. This is the culture/language agnostic way to pass the value if you cannot pass it as an actualy datetime value.

Resources