Apache Nifi Expression Language - toDate formatting - datetime

I am trying to format a date string using the Apache Nifi expression language and the Replace Text processor(regex). Given a date string
date_str : "2018-12-05T11:44:39.717+01:00",
I wish to convert this to:
correct_mod_date_str: "2018-12-05 10:44:39.717",
(notice how the date is converted to UTC, and character 'T' replaced by a space.)
To do this, I am currently using:
toDate("yyyy-MM-dd'T'HH:mm:ss.SSSSSSXXX"):format("yyyy-MM-dd HH:mm:ss.SSS", '+00:00')
and this works perfectly.
However, when the date string has 6 digits in ms, rather than 3, things break:
another_date_str: "2018-12-05T11:44:39.717456+01:00"
is converted to:
incorrect_mod_date_str: "2018-12-05 10:56:36.456"
It seems the first 3 digits in the ms precision interferes with the conversion.
Appreciate inputs to resolve this - what am I missing?
Regards

seems that's a limitation in java.
according to java documentation there is no support of more then 3 milliseconds digits.
https://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html
the simplest way is to remove extra digits like this:
attr:replaceAll('(\.\d{3})\d*','$1'):toDate("yyyy-MM-dd'T'HH:mm:ss.SSSXXX"):format("yyyy-MM-dd HH:mm:ss.SSS", '+00:00')

I ran into a similar issue with date time encoded in ISO 8601. The problem is, that the digits after the second are defined as fragment of a second, not milliseconds.
See answer to related topic

Related

ISO datetime with timezone issue

I am just printing the ISO datetime with timezone as per the below documentation
http://support.sas.com/documentation/cdl/en/lrdict/64316/HTML/default/viewer.htm#a003169814.htm
This is my code
TimeZone tz = TimeZone.getTimeZone("UTC");
DateFormat df = new SimpleDateFormat("yyyy-mm-dd'T'hh:mm:ss.nnnnnn+|-hh:mm");
df.setTimeZone(tz);
dateTimeWithTimeZone = df.format(new Date());
However i am getting this exception
Illegal pattern character 'n'
I cant use this format directly in Java ?
java.time
dateTimeWithTimeZone = Instant.now().toString();
System.out.println(dateTimeWithTimeZone);
When I ran this snippet just now, I got this output:
2019-03-18T22:28:13.549319Z
It’s not clear from the page you link to, but it’s an ISO 8601 string in UTC, so should be all that you need. I am taking advantage of the fact that the classes of java.time produce ISO 8601 output from their toString methods. The linked page does show the format with hyphens, T and colons (2008-09-15T15:53:00+05:00), it shows another example with decimals on the seconds (15:53:00.322348) and a third one with Z meaning UTC (20080915T155300Z), so I would expect that the combination of all three of these would be OK too.
The format you used in the quesiton seems to try to get the offset as +00:00 rather than Z. If this is a requirement, it’s only a little bit more complicated. We are using an explicit formatter to control the variations within ISO 8601:
DateTimeFormatter iso8601Formatter
= DateTimeFormatter.ofPattern("uuuu-MM-dd'T'HH:mm:ss.SSSSSSxxx");
dateTimeWithTimeZone = OffsetDateTime.now(ZoneOffset.UTC).format(iso8601Formatter);
System.out.println(dateTimeWithTimeZone);
2019-03-18T22:28:13.729711+00:00
What went wrong in your code?
You tried to use the formatting symbols from your source with SimpleDateFormat. First, you should never, and especially not in Java 8 or later, want to use SimpleDateFormat. That class is notoriously troublesome and long outdated. Second, some of its format pattern letters agree with the symbols from your source, some of them don’t, so you cannot just use the symvol string from there. Instead you need to read the documentation and find the correct format pattern letters to use for year, month, etc. And be aware that they are case sensitive: MM and mm are different.
Link
Oracle Tutorial: Date Time
explaining how to use java.time.

Format hours, minutes and seconds with moment.js

I get this value from my backend service: 171054. It represents hh:mm:ss. But when I use the formatting options from the docs it gives me back 00:00:00.
Things I've tried:
moment('171054').format('hh-mm-ss')
moment('171054').format('HH-mm-ss')
moment('171054').format('HH-MM-SS')
You are confusing format option with parsing option. Since your input string is not in ISO 8601 format, you have to specify format when parsing.
Here a working example for your use case:
var mom = moment('171054', 'HHmmss');
console.log(mom.format());
console.log(mom.format('HH:mm:ss'));
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.15.1/moment.min.js"></script>
I am not sure if momentjs can read that as date since there is no identifier. I would suggest changing the value for example you have 171054, get each 2 digits since you sure that this is represent as hh:mm:ss then add identifier between then like ":" or "-" then try us momentjs formatting again.

Where do I find the reference for TO_DATE format string

I need to convert a string formatted as MM/DD/YYYY HH:MI plus AM/PM, but can't find a complete reference to the format string to find how to specify the AM/PM part.
I would certainly appreciate information on how to do this, but would appreciate a link to a good source of documentation for this even more.
:EDIT
SELECT top 1
v.CalendarDateTime
,TO_TIMESTAMP(v.CalendarDateTime,'MM/DD/YYYY HH:MIAM') as CalendarDateTimeTS
--,CAST(TO_TIMESTAMP(v.CalendarDateTime,'MM/DD/YYYY HH:MIAM') AS TIMESTAMP(0) FORMAT 'MM/DD/YYYYBHH:MIBT') AS CalendarDateTimeTS2
12/03/2015 03:00AM 12/3/2015 03:00:00.000000
The commented out line produces a "DateTime field overflow" error.
You probably want TO_TIMESTAMP instead of TO_DATE.
The only bad thing about the Oracle function is the resulting datatype of TIMESTAMP(6) which can't be changed:
TO_TIMESTAMP('12/03/2015 03:00AM', 'MM/DD/YYYY HH:MIAM')
Using Teradata's FORMAT you can specify the timestamp precision, but it's less flexible than Oracle's, the string must match the format exactly:
CAST('12/03/2015 03:00AM' AS TIMESTAMP(0) FORMAT 'MM/DD/YYYYbHH:MIT')
On the Teradata site you'll find the (slow) online docu, e.g. TO_DATE formats or Teradata FORMATs. Of course you should download the full documentation CD for your release.
Please tell us at least which programming language are you using.
Normally it would be something like "MM/DD/YYYY HH:MI a" but we need to know first you language.

ASP.NET "String was not recognized as a valid DateTime."

First off, I realize there's a million pages discussing this already. I have looked at least a hundred of them but cannot seem to make this work. My date and time is presented as a string, compiled from javascript to grab client's local time. It is formatted like this: 7/11/2015 8:34 PM.
I currently have:
Dim datetimeformated = DateTime.ParseExact(lblDateTime.Text, "MM/dd/yyyy HH:mm tt", CultureInfo.InvariantCulture)
I have tried many different variants, but this should be correct I think, yet it does not work. Any help is greatly appreciated. TIA
The correct format for your case is: M/dd/yyyy h:mm tt, and perhaps even, M/d/yyyy h:mm tt, if you can have the day of the month as a single digit.
Explanation: Why your format string didn't work.
MM: means that you must always have 2 digits for the month, clearly not the case in your example.
dd: again, means that you must always have 2 digits for the day of the month. Is that the case? Adjust the parameter if needed.
HH: This actually means that you are expecting the hour value as 2-digits using the 24-hour clock (00-23), which is clearly wrong on both accounts. You can have a single digit, and you are not using the 24-hour clock, because you are using the AM/PM designator.
Relevant documentation link: Custom Date and Time Format Strings.

DateTime issue on different platforms (.NET 2.0)

On a 32 bit OS, with an Intel processor,
DateTime e.g. 2/17/2009 12:00:00 AM
Notice that it is: mm/DD//yyyy
On a 64 bit OS, with an AMD processor,
DateTime e.g. 17-02-2009 00:00:00
Now when I try to parse the 1st format, it throws an error on the 2nd platform.
That means - DateTime.Parse("2/17/2009 12:00:00 AM") - throws an error - cannot convert.
whereas, on the same platform,
DateTime.Parse("17/2/2009 12:00:00 AM") works! That means DD/MM is fine, MM/DD is not.
What is causing this? The 64-bit OS? The processor?
How do I get rid of the problem?
DateTimes themselves don't have formats. You parse them or format them into strings. (It's like numbers - integers aren't stored in hex or decimal, they're just integers. You can format them in hex or decimal, but the value itself is just a number.)
The format will depend on the culture of the operating system (or more accurately, the culture of the thread, which is typically the same as the operating system one).
Personally I like to explicitly set the format I use for either parsing or formatting, unless I'm actually displaying the string to the user and know that the culture is appropriate already.
Check your "Date and time formats" in the "Region and Language" control panel.
Also, if you want DateTime to generate a specific format, don't just call plain ToString(), but pass it a parameter indicating the format you want. Similarly, if you know the format of the date you are asking it to parse, call TryParseExact(), and tell it the format you are providing.
See also MSDN's Standard Date and Time Format Strings

Resources