I have a flowfile attribute which is a UTC datetime in the format of yyyy-MM-dd HH:mm:ss.SSS
I need to convert this to a unix timestamp.
How can this be done? I know its possible to convert Unix to the above format using Jolt:
"time": "${time:format('yyyy-MM-dd HH:mm:ss.SSS')}"
however, im not sure how to do this in reverse?
Working with attributes in this way uses the NiFi Expression Langauge (not Jolt).
See the docs here https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html
${time:format('yyyy-MM-dd HH:mm:ss.SSS')}
Uses Expression Language to format the time attribute to the given SimpleDateFormat string.
${time:toNumber()}
Uses Expression Language to convert the given Date object to Epoch Millis.
https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#tonumber
Related
I followed the tutorial https://www.influxdata.com/blog/json-to-influxdb-with-telegraf-and-starlark/ to get started with Telegraf and Starlark.
I get data in JSON format, but following tutorial and treating the input as String and then parsing using Starlark.
My problem is, I need to use the datetime stamp that is part of the JSON input. I am using the below code and it works if I set it to some hard coded Unix epoch time
new_metric = Metric("mymetric")
new_metric.time =1615702479866917911
But how do I convert a date that I have in DD-Mon-YYYY H:M:S format (e.g.12-Mar-2021 15:30:00 ) to Unix epoch format in Starlark script. Since, one cannot import any python libraries in Starlark script, not sure how can I accomplish this conversion.
metric.time = time.parse_time("12-Mar-2021 15:30:00", format="02-Jan-2006 15:04:05").unix
And if you need to set a timezone (the above will parse to UTC), you can do so by adding an argument to the end, like so:
metric.time = time.parse_time("12-Mar-2021 15:30:00", format="02-Jan-2006 15:04:05", location="US/Eastern").unix
And from a regular time object, you can grab the unix time like this:
metric.time = time.now().unix
Im trying to convert a U.K. input date (dd-MM-yyyy) to format (yyyy-MM-dd)
I tried
"#formatDateTime('15-03-2019','yyyy-MM-dd')" ==> Error
but got error:
'In function 'convertTimeZone', the value provided
for date time string '15-03-2019' was not valid. The datetime
string must match ISO 8601 format.'
How do I go about converting this input date? The input format is (dd-MM-yyyy) and cannot be changed.
I can easily convert from (MM-dd-yyyy) as shown below, but im not able to convert from (dd-MM-yyyy)
"#formatDateTime('03-15-2019','yyyy-MM-dd')" ==> OK
Date and time functions provided by azure logic app cannot recognize the timestamp in dd-MM-yyyy format.
After my research, there is no existing function that can directly solve this problem, but you can use substring and concat to deal with this problem.
The workflow of the logic app looks like this:
The expression of the formatDataTime:
formatDateTime(concat(substring(<your-date-string>,6,4),'-',substring(<your-date-string>,3,2),'-',substring(<your-date-string>,0,2)),'yyyy-MM-dd')
Can't format the below date using moment.js, the below statement returns Invalid Date
moment('20171206T062406927Z').format('D-MMM-YYYY');
Please help me on this.
You need to tell moment which format your date string is in:
moment('20171206T062406927Z', 'YYYYMMDD[T]HHmmssSSSZ', true).format('D-MMM-YYYY');
Edit: updated as per #VincenzoC comment to ensure the timestamp is parsed in UTC
Also fix: use HH for 24-hour format (not hh), and pass a third true parameter to ensure the timestamp is parsed in strict mode.
I get this value from my backend service: 171054. It represents hh:mm:ss. But when I use the formatting options from the docs it gives me back 00:00:00.
Things I've tried:
moment('171054').format('hh-mm-ss')
moment('171054').format('HH-mm-ss')
moment('171054').format('HH-MM-SS')
You are confusing format option with parsing option. Since your input string is not in ISO 8601 format, you have to specify format when parsing.
Here a working example for your use case:
var mom = moment('171054', 'HHmmss');
console.log(mom.format());
console.log(mom.format('HH:mm:ss'));
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.15.1/moment.min.js"></script>
I am not sure if momentjs can read that as date since there is no identifier. I would suggest changing the value for example you have 171054, get each 2 digits since you sure that this is represent as hh:mm:ss then add identifier between then like ":" or "-" then try us momentjs formatting again.
On a 32 bit OS, with an Intel processor,
DateTime e.g. 2/17/2009 12:00:00 AM
Notice that it is: mm/DD//yyyy
On a 64 bit OS, with an AMD processor,
DateTime e.g. 17-02-2009 00:00:00
Now when I try to parse the 1st format, it throws an error on the 2nd platform.
That means - DateTime.Parse("2/17/2009 12:00:00 AM") - throws an error - cannot convert.
whereas, on the same platform,
DateTime.Parse("17/2/2009 12:00:00 AM") works! That means DD/MM is fine, MM/DD is not.
What is causing this? The 64-bit OS? The processor?
How do I get rid of the problem?
DateTimes themselves don't have formats. You parse them or format them into strings. (It's like numbers - integers aren't stored in hex or decimal, they're just integers. You can format them in hex or decimal, but the value itself is just a number.)
The format will depend on the culture of the operating system (or more accurately, the culture of the thread, which is typically the same as the operating system one).
Personally I like to explicitly set the format I use for either parsing or formatting, unless I'm actually displaying the string to the user and know that the culture is appropriate already.
Check your "Date and time formats" in the "Region and Language" control panel.
Also, if you want DateTime to generate a specific format, don't just call plain ToString(), but pass it a parameter indicating the format you want. Similarly, if you know the format of the date you are asking it to parse, call TryParseExact(), and tell it the format you are providing.
See also MSDN's Standard Date and Time Format Strings