I am storing one timestamp column in Bigquery table. The source is of CET timestamp. But bigquery is storing it by default in UTC.
How to store it as CET timestamp? otherwise while fetching if also I am doing the timestamp conversion it is manipulating the timestamp value. that I do not want.
Or else I want to convert the value to UTC while fetching but for that I have to make Bigquery understood that the original column is in CET format. How to resolve this?
Related
I can't figure out how to handle the timezone efficiently with dynamodb get operation.
Say my partition primary key and range key (sk) are below, where each record is aggregated data for the whole day done by the user. 2022-09-13 00:00:00 to 2022-09-13 23:23:59
{
pk: 'userId',
sk: '2022-09-13'
}
What is the best approach to store the dates as UTC and fetch based on the client's timezone?
In the current behavior of my program, I'm saving the range key (sk) as UTC date. But if the client's local timezone is +8:00, and when he performs a save operation at 2022-09-14 00:01:00, it will still be stored in the 2022-09-13.
Just store a timestamp (date + time) in UTC instead of the date.
If the client needs data based on the local timezone, you calculate the timestamp of the day start and end in that timezone, shift it to UTC and then do a between query using the utc-start and utc-end.
I have an external data source automatically inserting data into my BigQuery table, this data source includes a timestamp field which does not have a timezone connected to it, however, I know this timestamp is in the Europe/Amsterdam timezone.
The problem here is that when this timestamp is inserted into BigQuery, BigQuery automatically defaults the timestamp to UTC, which it is not. And in my specific case, I want to convert this timestamp to UTC. However because BigQuery already defaulted the timestamp to UTC (while it is actually Europe/Amsterdam), I cannot easily convert it to the actual UTC timezone.
Is there any way to convert this timestamp, which BigQuery thinks is already UTC, to the actual UTC timezone within a query? I can't just give it a -02:00 offset due to Daylight Savings coming into play which changes this offset from 2 hours to only 1 hour depending on the time of year.
Any help would be appreciated, I have been kind of stuck on this :)
An example of the timestamp in BigQuery would be 2022-09-30 01:23:45 UTC
There is probably a better way but this should work
with
input as (select timestamp("2022-09-30 01:23:45 UTC") as ts)
select
ts,
timestamp(replace(cast(ts as string), '+00', " Europe/Amsterdam")) updated_ts
from input
ts
updated_ts
2022-09-30 01:23:45 UTC
2022-09-29 23:23:45 UTC
I am creating OpenApi Doc for an API.
Having a model object field with DateTime info with timezone.
Example: "2021-12-30T10:00:00+00:00"
The expected input date should be in UTC.
If I use the format: "date-time" then the corresponding generated example is "2021-12-30T10:00:00.123z" where timezone information is missing.
Please suggest the right format to use in the OpenApi spec so that I can get the desired DateTime with a timezone value.
"2021-12-30T10:00:00.123z" ends with a 'z'.
This means this DateTime represents UTC time.
z stands for Zulu Timezone
UTC is sometimes known as "Zulu time"
I have already calculated UTC (the datetime is set to be equal to the current UTC time) for a datetime property on my entity. I write it to CosmosDB, and I see in the DataExplorer that Microsoft has changed the datetime, doing a time conversion that puts it into the ALMT (Almaty, Kazakhstan) timezone. Why?
Does anyone know of any settings that can be turned off to prevent this from happening?
As far as the TableEntity TimeStamp property, it is set to be +00:00, using my datetime property.
Once it has been written to CosmosDB, it's been changed all the way from 09:43 PM July 10th to 03:43 AM July 11th, or ALMT timezone.
Do this to your DateTime properties before writing them to CosmosDB, or it will convert them from whatever your local timezone (still not sure how exactly / based on what it infers that) to Utc (and if you've created the DateTime with DateTime.UtcNow, then the resulting datetime after CosmosDB converts it will be wrong):
DateTime UTCDateTime = DateTime.SpecifyKind(DateTime.UtcNow, DateTimeKind.Utc);
After you set up your date time properties in your entity as such, no further date conversions will occur.
An API returns a timestamp as UNIX timestamp at UTC and I would like to know if this timestamp was more than x seconds ago. As expected, this works fine with os.time() - x > timestamp in UTC, but blows up in other timezones.
Unfortunately I can't find a good way solve this in lua.
os.date helpfully has the ! prefix (e.g. os.date("!%H:%M:%S")) to return time at UTC, but it seems that despite the documentation stating it supports all strftime options, this does not support the %s option. I have heard people mention that this is caused by Lua compile time options for a similar issue, but changing these is not possible as the interpreter is provided by the user.
You can use
os.time(os.date("!*t"))
to get the current UNIX epoch.
Ok, so you want the UTC time. Keep in mind that os.time actually knows nothing about timezones, so for example:
os.time(os.date("!*t"))
Will get UTC time and populate table struct.
Will convert table struct according to current timezone to unix timestamp.
So you actually would get your UNIX_TIME - TIMEZONE_OFFSET. If you are in GMT+5 you will get timestamp at UTC-5.
The correct way to do time conversion in lua is:
os.time() -- get current epoch value
os.time{ ... } -- get epoch value for local date/time values
os.date("*t"),os.date("%format") -- get your local date/time
os.date("!*t") or os.date("!%format") -- get UTC date/time
os.date("*t", timestamp),os.date("%format", timestamp) -- get your local date/time for given timestamp
os.date("!*t", timestamp) or os.date("!%format", timestamp) -- get UTC date/time for given timestamp
Kudos to Mons at https://gist.github.com/ichramm/5674287.
If you really need to convert any UTC date to timestamp, there's a good description on how to do this in this question: Convert a string date to a timestamp
os.time() gives you the unix timestamp. The timestamp is seconds since 00:00:00 UTC on 1 January 1970, so it's the same across timezones.
For example, run this code:
print('timestamp', os.time())
print('local hour', os.date("*t").hour)
print('utc hour', os.date("!*t").hour)
Presumably, your local and utc hour are different. Also run it in an online repl. The server's local and utc hour are the same, but both your and the server's timestamp are about the same.