How to suppress Teradata warnings with CAST - teradata

Question

Let's say I need to CAST(birth_date AS DATE FORMAT 'MM/DD/YYYY')
If the birth_date field contains nulls or invalid characters it throws and untranslatable character error
Of course I can use the regex, otranslate but all that overcomplicates the sql
Is there any way to suppress all these errors ? CAST if you can, otherwise make it null?

Answer 1

The burden of checking whether the data fits into the data type you wish to store it must reside somewhere. You could use CASE {regular expression matching} THEN CAST() ELSE NULL END which may be the cleanest way to address the data quality validation in your SQL.
Otherwise, pre-process your data file to replace bad data with a token you can replace with NULL in your SQL. You can consider doing this in PowerShell, UNIX shell scripting, or perhaps a third-party tool (e.g. address cleansing/formatting, etc.).

Answer 2

Since there is no built in way to say CAST(<field> as <datatype) IGNORING ERRORS AS <alias> you could use a TPT script instead.
In TPT APPLY you can have an INSERT statement route errors into two different Error tables.
Something like the following would get you close. This is something that you would run after your dirty date table is loaded to get them into a clean date table.
DEFINE JOB DATA_insert_Example
(
DEFINE OPERATOR data_insert_Example
TYPE UPDATE
SCHEMA *
ATTRIBUTES
(
VARCHAR UserName,
VARCHAR UserPassword,
VARCHAR LogTable,
VARCHAR TargetTable,
INTEGER BufferSize,
INTEGER ErrorLimit = 5,
INTEGER MaxSessions = 4,
INTEGER MinSessions = 1,
INTEGER TenacityHours,
INTEGER TenacitySleep,
VARCHAR AccountID,
VARCHAR AmpCheck,
VARCHAR DeleteTask,
VARCHAR ErrorTable1 = '<yourdatabase>.<yourcleantable>'||'_ET',
VARCHAR ErrorTable2 = '<yourdatabase>.<yourcleantable>'||'_UV',
VARCHAR NotifyExit,
VARCHAR NotifyExitIsDLL,
VARCHAR NotifyLevel,
VARCHAR NotifyMethod,
VARCHAR NotifyString,
VARCHAR PauseAcq,
VARCHAR PrivateLogName,
VARCHAR TdpId,
VARCHAR TraceLevel,
VARCHAR WorkingDatabase = <yourdatabase>,
VARCHAR WorkTable = '<yourdatabase>.<yourcleantable>'||'_Work'
);
DEFINE SCHEMA data_insert_schema
(
field1 VARCHAR(20),
field2 VARCHAR(20),
field3 VARCHAR(20),
field4 VARCHAR(20)
);
DEFINE OPERATOR data_insert_export
TYPE EXPORT
SCHEMA W_0_s_DATA_esuh
ATTRIBUTES
(
VARCHAR UserName,
VARCHAR UserPassword,
STEP UPS
(
APPLY
(
'INSERT INTO <yourdatabase>.<yourcleantable>
field1,
field2,
field3,
field4
VALUES (
:field1,
:field2,
:field3,
:field4,
)';
)
TO OPERATOR
(
data_insert_Example[1]
ATTRIBUTES
(
UserName = <yourusername>,
UserPassword = <yourpass>,
LogTable = <yourdatabase>.<yourcleantable> ||'_LOG',
TargetTable = <yourdatabase>.<yourcleantable> ,
TdpId = <yourserverip/address>
)
)
SELECT * FROM OPERATOR
(
data_insert_export[1]
ATTRIBUTES
(
UserName = <yourusername>,
UserPassword = <yourpassword>,
SelectStmt = 'SELECT field1,field2,field3,field4 FROM <yourdatabase>.<yourtable> ;',
TdpId = '<yourserverip/address>'
)
);
);
)
Obviously, though, this is quite a bit more overkill than a simple REGEX. RegEx feels way overwhelming when you first start using it, but I think it's a completely reasonable usecase for checking dates stored as string literals before trying to convert them into their proper data type.
Overall, it sounds like you have garbage for data, so I totally get the frustration. Unfortunately for garbage data there is no magic bullet. You'll need some decent ETL between the garbage and your clean output.