I have a SQL DATETIME field DT that I want to feed into Kafka using Avro. DT is in local time but has no time zone; that's stored in another column TZ as an offset from UTC in minutes. It seems that an Avro timestamp-millis would be appropriate for DT, but I'm confused about conversion, given the lacking time zone. From looking at the connector source code, I think it will just default to UTC.
So that raises the questions:
Will all visibly similar datetimes convert to the same number of milliseconds since the/an epoch, regardless of their nominal time zone? For example, does (2018-01-01T00:00Z).to_timestamp_ms() == (2018-01-01T00:00).to_timestamp_ms() == (2018-01-01T00:00+05).to_timestamp_ms()?
More importantly, is it possible to convert to true UTC after ingestion by subtracting the time zone offset TZ from the datetime field DT (which is now in milliseconds since some epoch)? In other words, if the connector incorrectly assumes UTC, and falsely interprets the datetime as UTC, can the true datetime be recovered after that by subtracting the offset?
Details on transformation steps
I think the order of operations is something like this on the Connector (largely out of my control):
tz = read_field_as_int('tz')
dt = read_field_as_string('dt')
parsed_datetime = datetime(dt, timezone=UTC)
message = {
'dt': parsed_datetime.to_timestamp_ms(),
'tz': tz
}
producer.produce(message)
And then later, in the consumer, maybe this would work?
ms_per_min = 60 * 1000
message = consumer.poll()
true_timestamp = message['dt'] - message['tz'] * ms_per_min
true_dt = datetime.from_timestamp(true_timestamp, timezone=UTC)
Related
I am trying to format current timestamp into two different time zone, But ZonedDateTime format is returning same output.
DateTimeFormatter outputFormat = new DateTimeFormatterBuilder().appendPattern("ddMMMyyyy HH:mm:ss").toFormatter();
System.out.println(ZonedDateTime.of(LocalDateTime.now(), ZoneId.of("America/Phoenix")).format(outputFormat));
System.out.println(ZonedDateTime.of(LocalDateTime.now(), ZoneId.of("Asia/Calcutta")).format(outputFormat));
above code snipped is returning same time stamp as
16Apr2022 13:28:19
16Apr2022 13:28:19
Can some one help me understand but its returning same String. What is correct way to format time in different time zone, using Java 8 Date Time APIs.
I have a date in this format "2020-12-16T02:48:00" that came from the server. How can I convert this into local date and time? I tried some code but couldn't succeed.
Below is the attempt that I had made in angular after receiving date from the server.
response.data.map(date=>{
var centralDate = moment( date).zone("-06:00");
date = moment(centralDate).local().format('YYYY-MM-DD hh:mm:ss');
})
If indeed the value is in UTC (as per the title of your question), and it looks like "2020-12-16T02:48:00", and you want to convert it to local time, then you should do the following:
moment.utc(date).local().format('YYYY-MM-DD HH:mm:ss');
That does the following:
Parses the input in terms of UTC
Converts it to local time
Formats it as a string in the given format
Note also that you had hh in your original format. That is for hours in a 12-hour time format and thus you shouldn't use it without also using either A or a to indicate AM/PM or am/pm. Otherwise HH is for hours in a 24-hour time format.
If your issue is that the timezone doesn't change you can resolve using utcOffset (https://momentjscom.readthedocs.io/en/latest/moment/03-manipulating/09-utc-offset/) in this way:
response.data.map(date=>{
date = moment( date).utcOffset(-360);
})
Where 360 is the conversion fo the hours in minutes
var d= new Date();
d = new Date(d+ "Z")
I am not an expert in angular but I guess the trouble in your date is the word âTâ. May be using string removal function you can remove the word âTâ and then it becomes a proper date time value?
This question already has answers here:
How to make a timezone aware datetime object
(15 answers)
Closed 6 years ago.
I've got a datetime which has no timezone information. I'm now getting the timezone info and would like to add the timezone into the existed datetime instance, how can I do?
d = datetime.datetime.now()
tz = pytz.timezone('Asia/Taipei')
How to add the timezone info tz into datetime a
Use tz.localize(d) to localize the instance. From the documentation:
The first is to use the localize() method provided by the pytz library. This is used to localize a naive datetime (datetime with no timezone information):
>>> loc_dt = eastern.localize(datetime(2002, 10, 27, 6, 0, 0))
>>> print(loc_dt.strftime(fmt))
2002-10-27 06:00:00 EST-0500
If you don't use tz.localize(), but use datetime.replace(), chances are that a historical offset is used instead; tz.localize() will pick the right offset in effect for the given date. The US Eastern timezone DST start and end dates have changed over time, for example.
When you try to localize a datetime value that is ambiguous because it straddles the transition period from summer to winter time or vice-versa, the timezone will be consulted to see if the resulting datetime object should have .dst() return True or False. You can override the default for the timezone with the is_dst keyword argument for .localize():
dt = tz.localize(naive, is_dst=True)
or even switch off the choice altogether by setting is_dst=None. In that case, or in the rare cases there is no default set for a timezone, an ambiguous datetime value would lead to a AmbiguousTimeError exception being raised. The is_dst flag is only consulted for datetime values that are ambiguous and is ignored otherwise.
To go back the other way, turn a timezone-aware object back to a naive object, use .replace(tzinfo=None):
naivedt = awaredt.replace(tzinfo=None)
If you know that your original datetime was "measured" in the time zone you are trying to add to it, you could (but probably shouldn't) use replace rather than localize.
# d = datetime.datetime.now()
# tz = pytz.timezone('Asia/Taipei')
d = d.replace(tzinfo=tz)
I can imagine 2 times when this might make sense (the second one happened to me):
Your server locale is set to the incorrect time zone and you are trying to correct a datetime instance by making it aware of this incorrect timezone (and presumably later localizing it to the "correct" time zone so the values of now() match up to other times you are comparing it to (your watch, perhaps)
You want to "tag" a time instance (NOT a datetime) with a time zone (tzinfo) attribute so that attribute can be used later to form a full datetime instance.
In ballerina time packages createTime function, it asks for zoneID as a function parameter, when I provide a custom zoneID, program keeps running without any output. Is there a way to find those predefined time zoneIDs with relevant timezones?
The underlying implementation of ballerina time package is Java and the zone ID processing rules are similar to the Java rules. Basically rules are as follows.
If the zone ID equals 'Z', the result is UTC. If the zone ID equals
'GMT', 'UTC' or 'UT', it is equivalent to UTC.
If the zone ID starts with '+' or '-', the ID is parsed as an offset.
Offset can be specified in one of the following ways.
+h,
+hh,
+hh:mm,
-hh:mm,
+hhmm,
-hhmm,
+hh:mm:ss,
-hh:mm:ss,
+hhmmss,
-hhmmss
Also zone id can be a region-based zone ID. The format is '{area}/{city}' eg: "America/Panama". The zones are based on IANA Time Zone Database (TZDB) supplied data.
Ex:
Using UTC
time:Timezone zoneValue = {zoneId:"Z"};
time:Time time = new(1456876583555, zoneValue);
Using offset
time:Timezone zoneValue = {zoneId:"-05:00"};
time:Time time = new(1456876583555, zoneValue);
Using region-based zone ID
time:Timezone zoneValue = {zoneId:"America/Panama"};
time:Time time = new(1456876583555, zoneValue);
We hope to improve this API in a future version of Ballerina with proper representation for the zone ids etc.
You can refer to the Ballerina date time example for more details.
I'm using Joi package to validate a timestamp field but How can I set a max() value on it, I want the input timestamp to be less than current time stamp
var schema = Joi.object().keys({
t: Joi.date().timestamp('unix').max(moment().unix()),
})
but I get the error that:
child "t" fails because ["t" must be less than or equal to "Sun Jan 18
1970 07:35:17 GMT+0330 (IRST)"]
I'm sure that the moment().unix() returns the current timestamp, but here it is casted to string.
It seems that max() and min() functions can do the trick but they only work if the threshold is specified in milliseconds.
t: Joi.date().timestamp('unix')
.max(moment().unix() * 1000)
.min(moment().subtract('42', 'weeks').unix() * 1000),
It doesn't look like Joi.date().max() accepts unix timestamps properly despite being able to specify in your schema that a unix timestamp is expected for incoming values.
If you need to use the current date in your schema you can pass the string 'now' instead of using the date. Or just make sure you enter the current date in format that .max() expects. I tried this using milliseconds and it seems to work as expected. I think Joi is using the default Date constructor under the hood to construct dates to compare which expects milliseconds.
var schema = Joi.object().keys({
t: Joi.date().timestamp('unix').max(moment().unix() * 1000)
});
From the docs on date.max()
Notes: 'now' can be passed in lieu of date so as to always compare relatively to the current date, allowing to explicitly ensure a date is either in the past or in the future.