I have a tab delimited file where each record has a timestamp field in 12-hour format:
mm/dd/yyyy hh:mm:ss [AM|PM].
I need to quickly convert these fields to 24-hour time:
mm/dd/yyyy HH:mm:ss.
What would be the best way to do this? I'm running on a Windows platform, but I have access to sed, awk, perl, python, and tcl in addition to the usual Windows tools.
Using Perl and hand-crafted regexes instead of facilities like strptime:
#!/bin/perl -w
while (<>)
{
# for date times that don't use leading zeroes, use this regex instead:
# (?:\d{1,2}/\d{1,2}/\d{4} )(\d{1,2})(?::\d\d:\d\d) (AM|PM)
while (m%(?:\d\d/\d\d/\d{4} )(\d\d)(?::\d\d:\d\d) (AM|PM)%)
{
my $hh = $1;
$hh -= 12 if ($2 eq 'AM' && $hh == 12);
$hh += 12 if ($2 eq 'PM' && $hh != 12);
$hh = sprintf "%02d", $hh;
# for date times that don't use leading zeroes, use this regex instead:
# (\d{1,2}/\d{1,2}/\d{4} )(\d{1,2})(:\d\d:\d\d) (?:AM|PM)
s%(\d\d/\d\d/\d{4} )(\d\d)(:\d\d:\d\d) (?:AM|PM)%$1$hh$3%;
}
print;
}
That's very fussy - but also converts possibly multiple timestamps per line.
Note that the transformation for AM/PM to 24-hour is not trivial.
12:01 AM --> 00:01
12:01 PM --> 12:01
01:30 AM --> 01:30
01:30 PM --> 13:30
Now tested:
perl ampm-24hr.pl <<!
12/24/2005 12:01:00 AM
09/22/1999 12:00:00 PM
12/12/2005 01:15:00 PM
01/01/2009 01:56:45 AM
12/30/2009 10:00:00 PM
12/30/2009 10:00:00 AM
!
12/24/2005 00:01:00
09/22/1999 12:00:00
12/12/2005 13:15:00
01/01/2009 01:56:45
12/30/2009 22:00:00
12/30/2009 10:00:00
Added:
In What is a Simple Way to Convert Between an AM/PM Time and 24 hour Time in JavaScript, an alternative algorithm is provided for the conversion:
$hh = ($1 % 12) + (($2 eq 'AM') ? 0 : 12);
Just one test...probably neater.
It is a 1-line thing in python:
time.strftime('%H:%M:%S', time.strptime(x, '%I:%M %p'))
Example:
>>> time.strftime('%H:%M:%S', time.strptime('08:01 AM', '%I:%M %p'))
'08:01:00'
>>> time.strftime('%H:%M:%S', time.strptime('12:01 AM', '%I:%M %p'))
'00:01:00'
Use Pythons datetime module someway like this:
import datetime
infile = open('input.txt')
outfile = open('output.txt', 'w')
for line in infile.readlines():
d = datetime.strptime(line, "input format string")
outfile.write(d.strftime("output format string")
Untested code with no error checking. Also it reads the entire input file in memory before starting.
(I know there is plenty of room for improvements like with statement...I make this a community wiki entry if anyone likes to add something)
To just convert the hour field, in python:
def to12(hour24):
return (hour24 % 12) if (hour24 % 12) > 0 else 12
def IsPM(hour24):
return hour24 > 11
def to24(hour12, isPm):
return (hour12 % 12) + (12 if isPm else 0)
def IsPmString(pm):
return "PM" if pm else "AM"
def TestTo12():
for x in range(24):
print x, to12(x), IsPmString(IsPM(x))
def TestTo24():
for pm in [False, True]:
print 12, IsPmString(pm), to24(12, pm)
for x in range(1, 12):
print x, IsPmString(pm), to24(x, pm)
This might be too simple thinking, but why not import it into excel, select the entire column and change the date format, then re-export as a tab delimited file? (I didn't test this, but it somehow sounds logical to me :)
Here i have converted 24 Hour system to 12 Hour system.
Try to use this method for your problem.
DateFormat fmt = new SimpleDateFormat("yyyyMMddHHssmm");
try {
Date date =fmt.parse("20090310232344");
System.out.println(date.toString());
fmt = new SimpleDateFormat("dd-MMMM-yyyy hh:mm:ss a ");
String dateInString = fmt.format(date);
System.out.println(dateInString);
} catch (Exception e) {
System.out.println(e.getMessage());
}
RESULT:
Tue Mar 10 23:44:23 IST 2009
10-March-2009 11:44:23 PM
In Python: Converting 12hr time to 24hr time
import re
time1=input().strip().split(':')
m=re.search('(..)(..)',time1[2])
sec=m.group(1)
tz=m.group(2)
if(tz='PM'):
time[0]=int(time1[0])+12
if(time1[0]=24):
time1[0]-=12
time[2]=sec
else:
if(int(time1[0])=12):
time1[0]-=12
time[2]=sec
print(time1[0]+':'+time1[1]+':'+time1[2])
Since you have multiple languages, I'll suggest the following algorithm.
1 Check the timestamp for the existence of the "PM" string.
2a If PM does not exist, simply convert the timestamp to the datetime object and proceed.
2b If PM does exist, convert the timestamp to the datetime object, add 12 hours, and proceed.
Related
In the azure data explorer documentation, there is a lot of supported formats but not the one that i am looking for.
What I need is to format the datetime like "yyyy-MM-dd HH" to set the minutes and seconds as 0
Input datetime
2020-04-21T17:44:27.6825985Z
expected results
2020-04-21 17:00:00
you can use bin() to round down to the hour, and if you still need to remove the datetime parts lower than seconds, you can use substring() (or format_datetime()). e.g.:
print d = datetime(2020-04-21T17:44:27.6825985Z)
| extend h = bin(d, 1h)
| extend h2 = substring(h, 0, 19)
hmm, if you always just want the rest to be 0, can you just use string concatenation?
let d = datetime(2020-04-21T17:44:27.6825985Z);
print strcat(format_datetime(d, "yyyy-MM-dd HH"), ":00:00")
the above code will give you the result of
2020-04-21 17:00:00
Edit: Apologies, the sample data frame is a little off. Below is the corrected sample dataframe I'm trying to convert:
Timestamp (CST)
12/8/2018 05:23 PM
11/29/2018 10:20 PM
I tried the following code based on recommendation below but got null values returned.
df = df.withColumn('Timestamp (CST)_2', from_unixtime(unix_timestamp(col(('Timestamp (CST)')), "yyyy/MM/dd hh:mm:ss aa"), "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"))
df = df.withColumn("Timestamp (CST)_3", F.to_timestamp(F.col("Timestamp (CST)_2")))
--------------------------------------------------------------------------------
I have a field called "Timestamp (CST)" that is a string. It is in Central Standard Time.
Timestamp (CST)
2018-11-21T5:28:56 PM
2018-11-21T5:29:16 PM
How do I create a new column that takes "Timestamp (CST)" and change it to UTC and convert it to a datetime with the time stamp on the 24 hour clock?
Below is my desired table and I would like the datatype to be timestamp:
Timestamp (CST)_2
2018-11-21T17:28:56.000Z
2018-11-21T17:29:16.000Z
I tried the following code but all the results came back null:
df = df.withColumn("Timestamp (CST)_2", to_timestamp("Timestamp (CST)", "yyyy/MM/dd h:mm p"))
Firstly, import from_unixtime, unix_timestamp and col using
from pyspark.sql.functions import from_unixtime, unix_timestamp, col
Then, reconstructing your scenario in a DataFrame df_time
>>> cols = ['Timestamp (CST)']
>>> vals = [
... ('2018-11-21T5:28:56 PM',),
... ('2018-11-21T5:29:16 PM',)]
>>> df_time = spark.createDataFrame(vals, cols)
>>> df_time.show(2, False)
+---------------------+
|Timestamp (CST) |
+---------------------+
|2018-11-21T5:28:56 PM|
|2018-11-21T5:29:16 PM|
+---------------------+
Then, my approach would be
>>> df_time_twenfour = df_time.withColumn('Timestamp (CST)', \
... from_unixtime(unix_timestamp(col(('Timestamp (CST)')), "yyyy-MM-dd'T'hh:mm:ss aa"), "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"))
>>> df_time_twenfour.show(2, False)
+------------------------+
|Timestamp (CST) |
+------------------------+
|2018-11-21T17:28:56.000Z|
|2018-11-21T17:29:16.000Z|
+------------------------+
Notes
If you want time to be in 24-Hour format then, you would use HH instead of hh.
Since, you have a PM, you use aa in yyyy-MM-dd'T'hh:mm:ss aa to specify PM.
Your, input string has T in it so, you have to specify it as above format.
the option aa as mentioned in #pyy4917's answer might give legacy errors. To fix it, replace aa with a.
The full code as below:
df_time_twenfour = df_time.withColumn('Timestamp (CST)', \ ...
from_unixtime(unix_timestamp(col(('Timestamp (CST)')), \...
"yyyy-MM-dd'T'hh:mm:ss a"), "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"))
I have users entering a date and a time zone (e.g. "America/Los Angeles") for that date and I'd like to convert that to UTC but to do that I need the utc offset for the time on that date.
I can easily convert a date to the offset for the time zone if I already know the UTC date but I need the other way around...
The utc offset can change depending on the date due to daylight saving so I need a way to enter a date and a timezone and get back the offset from UTC using that.
Knowing the most recent switch from PST to PDT On march 11 at 2AM I tried using
var tzOffset = moment.tz("3/11/2018 3:00 AM", "America/Los_Angeles").utcOffset();
document.write('utc offset is : ' + tzOffset + '<br/>') ;
but that gives 480 when the correct answer is 420
I can get the correct answer 420 if I use parseZone like so:
var tzOffset2 = moment.parseZone("3/11/2018 3:00 AM -07:00").utcOffset();
document.write('utc offset2 is : ' + tzOffset2 + '<br/>') ;
but that means I need to already know the -7 offset that I'm trying to find...
So how do I find the utcOffset for a specific date/time like "3/11/2018 3:00 AM" and timezone like "America/Los_Angeles"? Thanks
Your input is not in a ISO 8601 or RFC 2822 format recognized by moment(String), so you have to specify the format as second parameter using moment(String, String) (please note that, as docs states: The moment.tz constructor takes all the same arguments as the moment constructor, but uses the last argument as a time zone identifier.)
Your code could be like the following:
var tzOffset = moment.tz("3/11/2018 3:00 AM", "D/M/YYYY h:mm A", "America/Los_Angeles").utcOffset();
document.write('utc offset is : ' + tzOffset + '<br/>') ;
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.22.1/moment.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment-timezone/0.5.14/moment-timezone-with-data-2012-2022.min.js"></script>
As a beginner, creating timestamps or formatted dates ended up being a little more of a challenge than I would have expected. What are some basic examples for reference?
Ultimately you want to review the datetime documentation and become familiar with the formatting variables, but here are some examples to get you started:
import datetime
print('Timestamp: {:%Y-%m-%d %H:%M:%S}'.format(datetime.datetime.now()))
print('Timestamp: {:%Y-%b-%d %H:%M:%S}'.format(datetime.datetime.now()))
print('Date now: %s' % datetime.datetime.now())
print('Date today: %s' % datetime.date.today())
today = datetime.date.today()
print("Today's date is {:%b, %d %Y}".format(today))
schedule = '{:%b, %d %Y}'.format(today) + ' - 6 PM to 10 PM Pacific'
schedule2 = '{:%B, %d %Y}'.format(today) + ' - 1 PM to 6 PM Central'
print('Maintenance: %s' % schedule)
print('Maintenance: %s' % schedule2)
The output:
Timestamp: 2014-10-18 21:31:12
Timestamp: 2014-Oct-18 21:31:12
Date now: 2014-10-18 21:31:12.318340
Date today: 2014-10-18
Today's date is Oct, 18 2014
Maintenance: Oct, 18 2014 - 6 PM to 10 PM Pacific
Maintenance: October, 18 2014 - 1 PM to 6 PM Central
Reference link: https://docs.python.org/3.4/library/datetime.html#strftime-strptime-behavior
>>> import time
>>> print(time.strftime('%a %H:%M:%S'))
Mon 06:23:14
from datetime import datetime
dt = datetime.now() # for date and time
ts = datetime.timestamp(dt) # for timestamp
print("Date and time is:", dt)
print("Timestamp is:", ts)
You might want to check string to datetime operations for formatting.
from datetime import datetime
datetime_str = '09/19/18 13:55:26'
datetime_object = datetime.strptime(datetime_str, '%m/%d/%y %H:%M:%S')
print(type(datetime_object))
print(datetime_object) # printed in default format
Output:
<class 'datetime.datetime'>
2018-09-19 13:55:26
I'm trying to create a DateTime object with a specific UTC timestamp in PowerShell. What's the simplest way to do this?
I tried:
Get-Date
-Format (Get-Culture).DateTimeFormat.UniversalSortableDateTimePattern
-Date "1970-01-01 00:00:00Z"
but I get this output:
1969-12-31 19:00:00Z
It's a few hours off. Where's my lapse in understanding?
The DateTime object itself is being created with the proper UTC time. But when PowerShell prints it out it converts it to my local culture and time zone, thus the difference.
Proof:
$UtcTime = Get-Date -Date "1970-01-01 00:00:00Z"
$UtcTime.ToUniversalTime()
(get-date).ToUniversalTime().ToString("yyyyMMddTHHmmssfffffffZ")
$utctime = New-Object DateTime 1970, 1, 1, 0, 0, 0, ([DateTimeKind]::Utc)
If you print out $utctime, then you get:
1. januar 1970 00:00:00
Also, $utctime.Kind is correctly set to Utc.
$time = [DateTime]::UtcNow | get-date -Format "yyyy-MM-ddTHH:mm:ssZ"
This appears to also work
You can use the SpecifyKind method:
PS C:\IT\s3> $timestamp
Wednesday, July 18, 2018 7:57:14 PM
PS C:\IT\s3> $timestamp.kind
Unspecified
PS C:\IT\s3> $utctimestamp = [DateTime]::SpecifyKind($timestamp,[DateTimeKind]::Utc)
PS C:\IT\s3> $utctimestamp
Wednesday, July 18, 2018 7:57:14 PM
PS C:\IT\s3> $utctimestamp.kind
Utc
This is how it works in .NET, right? PowerShell just calls the ToUniversalTime method. From http://msdn.microsoft.com/en-us/library/system.datetime.touniversaltime.aspx
The Coordinated Universal Time (UTC) is equal to the local time minus the
UTC offset. For more information about the UTC offset, see TimeZone.GetUtcOffset.
The conversion also takes into account the daylight saving time rule that applies
to the time represented by the current DateTime object.