Convert 12 hr format Date time to 24 hour in pig? - datetime

02/07/2016 12:43:23.324 PM
mm/dd/yyyy hh:mm:ss.SSS AM/PM -current format 12 hr
Please help to convert this to 24 hour format in pig...
File in HDFS
02/07/2016 12:43:23.324 PM
03/08/2016 08:12:15.123 AM
Commands in pig :
date_data = LOAD 'hdfs path' as (date: chararray);
todate_data = foreach date_data generate ToDate(date,'yyyy/MM/dd HH:mm:ss.SSS');
dump todate_data;
Gives the following Exception.
java.lang.IllegalArgumentException: Invalid format: "02/07/2016 12:43:23.324 PM" is malformed at "16 12:43:23.324 PM"

You will have to specify the format of the input.For example, your data is in 'MM/dd/yyyy hh:mm:ss.SSS aa' format.So use the below script.
date_data = LOAD 'hdfs path' as (date: chararray);
todate_data = foreach date_data generate ToDate(date,'MM/dd/yyyy hh:mm:ss.SSS aa');
dump todate_data;
Below is a working example.The input 4 dates are in 'dd/MM/yyyy hh:mm:ss.SSS aa' format.
INPUT
30/06/2016 02:43:23.324 PM
01/12/2016 12:43:23.324 AM
21/08/2016 06:43:23.324 PM
13/07/2016 12:43:23.324 AM
SCRIPT
A = LOAD 'test4.txt' AS (create_dt:chararray);
B = FOREACH A GENERATE ToDate(create_dt,'dd/MM/yyyy hh:mm:ss.SSS aa') AS create_dt;
DUMP B;
OUTPUT

Related

Convert DateTime to AM/PM and restrict to 12 hours format in ASP.NET

I want to convert and restrict the DateTime to 12 hours format.
Right now the output is: 02-01-2022 18:30 PM
But I want to convert into: 02/01/2022 06:30 PM +00:00.
DateTime.Now.ToString("MM/dd/yyyy hh:mm tt")
Add tt at last for AM/PM
string CustomDateFormat = DateTime.Now.ToString("MM/dd/yyyy hh:mm tt +00:00");
or
string CustomDateFormat = DateTime.Now.ToString("MM/dd/yyyy hh:mm tt") + "+00:00";
where hh is for 12 hours format and tt for AM/PM as in this documentation.

parse strange date format

I'm working on a Parser which Parses log files from a game so I can do analysis on auctions made within the game, however the date format that's being written by the logger seems to be causing problems as the format seems to be custom written for the logger, an example datetime stamp looks like: [Wed Nov 23 23:26:10 2016] I try to Parse it with:
func (r *AuctionReader) extractSaleInformation(line string) {
fmt.Println("Extracting information from: ", line)
// Format mask for output
layout := "DD-MM-YYYY hh:mm:ss"
// Replace the square brackets so we're just left with the date-time string
date := strings.TrimSpace(strings.Replace((strings.Split(line, "]")[0]), "[", "", -1))
fmt.Println(time.Parse(date, layout))
}
When I attempt to Parse the above date-time string I get the following error:
0001-01-01 00:00:00 +0000 UTC parsing time "DD-MM-YYYY hh:mm:ss" as "Wed Nov 23 23:26:10 2016": cannot parse "DD-MM-YYYY hh:mm:ss" as "Wed Nov "
How am I able to get the parser to recognise this seemingly custom format, I will be saving this data to Mongo so I don't want to store the auction time as a string as I want to query the timestamps individually.
Golang handle all date formatting in a unique way - it uses the reference time Mon Jan 2 15:04:05 MST 2006 (01/02 03:04:05PM '06 -0700) to show the pattern with which to format/parse a given time/string.
So, to read the format "Wed Nov 23 23:26:10 2016" you would put the reference date into that format: "Mon Jan 2 15:04:05 2006", and then do:
t, _ := time.Parse("Mon Jan 2 15:04:05 2006", "Wed Nov 23 23:26:10 2016")
Then, to output it in the given format, if you wanted the format DD-MM-YYYY hh:mm:ss, you would put the reference time into that format: 02-01-2006 15:04:05, and then do:
t.Format("02-01-2006 15:04:05")
https://play.golang.org/p/VO5413Z7-z
So basically, the main change is
// Format mask for output
layout := "DD-MM-YYYY hh:mm:ss"
should be
// Format mask for output
layout := "02-01-2006 15:04:05"
and
time.Parse(date, layout)
should be
time.Parse(layout, date)

Convert Twitter Time into a Pig DateTime object

I am working on twitter data and have a field: user_created_at that looks like Thu Jun 11 16:41:35 +0000 2015.
I am not sure what the type of the field is since I got the fields using elephant bird. To covert it into datetime type, I did:
ToDate(user_created_at, 'yyyy.MM.dd') as user_created_at
but it failed with an error:
ERROR 0: Exception while executing [POUserFunc (Name: POUserFunc(org.apache.pig.builtin.ToDate2ARGS)[datetime] - scope-148 Operator Key: scope-148) children: null at []]: java.lang.IllegalArgumentException: Invalid format: "Thu Jun 11 16:41:35 +0000 2015".
What is wrong? I am using Pig version 0.15. Appreciate any help. Thanks!
Match datetime format with input datetime string. Something like this.
ToDate(user_created_at, 'EEE MMM dd HH:mm:ss Z yyyy')

How can I create basic timestamps or dates? (Python 3.4)

As a beginner, creating timestamps or formatted dates ended up being a little more of a challenge than I would have expected. What are some basic examples for reference?
Ultimately you want to review the datetime documentation and become familiar with the formatting variables, but here are some examples to get you started:
import datetime
print('Timestamp: {:%Y-%m-%d %H:%M:%S}'.format(datetime.datetime.now()))
print('Timestamp: {:%Y-%b-%d %H:%M:%S}'.format(datetime.datetime.now()))
print('Date now: %s' % datetime.datetime.now())
print('Date today: %s' % datetime.date.today())
today = datetime.date.today()
print("Today's date is {:%b, %d %Y}".format(today))
schedule = '{:%b, %d %Y}'.format(today) + ' - 6 PM to 10 PM Pacific'
schedule2 = '{:%B, %d %Y}'.format(today) + ' - 1 PM to 6 PM Central'
print('Maintenance: %s' % schedule)
print('Maintenance: %s' % schedule2)
The output:
Timestamp: 2014-10-18 21:31:12
Timestamp: 2014-Oct-18 21:31:12
Date now: 2014-10-18 21:31:12.318340
Date today: 2014-10-18
Today's date is Oct, 18 2014
Maintenance: Oct, 18 2014 - 6 PM to 10 PM Pacific
Maintenance: October, 18 2014 - 1 PM to 6 PM Central
Reference link: https://docs.python.org/3.4/library/datetime.html#strftime-strptime-behavior
>>> import time
>>> print(time.strftime('%a %H:%M:%S'))
Mon 06:23:14
from datetime import datetime
dt = datetime.now() # for date and time
ts = datetime.timestamp(dt) # for timestamp
print("Date and time is:", dt)
print("Timestamp is:", ts)
You might want to check string to datetime operations for formatting.
from datetime import datetime
datetime_str = '09/19/18 13:55:26'
datetime_object = datetime.strptime(datetime_str, '%m/%d/%y %H:%M:%S')
print(type(datetime_object))
print(datetime_object) # printed in default format
Output:
<class 'datetime.datetime'>
2018-09-19 13:55:26

Convert 12-hour date/time to 24-hour date/time

I have a tab delimited file where each record has a timestamp field in 12-hour format:
mm/dd/yyyy hh:mm:ss [AM|PM].
I need to quickly convert these fields to 24-hour time:
mm/dd/yyyy HH:mm:ss.
What would be the best way to do this? I'm running on a Windows platform, but I have access to sed, awk, perl, python, and tcl in addition to the usual Windows tools.
Using Perl and hand-crafted regexes instead of facilities like strptime:
#!/bin/perl -w
while (<>)
{
# for date times that don't use leading zeroes, use this regex instead:
# (?:\d{1,2}/\d{1,2}/\d{4} )(\d{1,2})(?::\d\d:\d\d) (AM|PM)
while (m%(?:\d\d/\d\d/\d{4} )(\d\d)(?::\d\d:\d\d) (AM|PM)%)
{
my $hh = $1;
$hh -= 12 if ($2 eq 'AM' && $hh == 12);
$hh += 12 if ($2 eq 'PM' && $hh != 12);
$hh = sprintf "%02d", $hh;
# for date times that don't use leading zeroes, use this regex instead:
# (\d{1,2}/\d{1,2}/\d{4} )(\d{1,2})(:\d\d:\d\d) (?:AM|PM)
s%(\d\d/\d\d/\d{4} )(\d\d)(:\d\d:\d\d) (?:AM|PM)%$1$hh$3%;
}
print;
}
That's very fussy - but also converts possibly multiple timestamps per line.
Note that the transformation for AM/PM to 24-hour is not trivial.
12:01 AM --> 00:01
12:01 PM --> 12:01
01:30 AM --> 01:30
01:30 PM --> 13:30
Now tested:
perl ampm-24hr.pl <<!
12/24/2005 12:01:00 AM
09/22/1999 12:00:00 PM
12/12/2005 01:15:00 PM
01/01/2009 01:56:45 AM
12/30/2009 10:00:00 PM
12/30/2009 10:00:00 AM
!
12/24/2005 00:01:00
09/22/1999 12:00:00
12/12/2005 13:15:00
01/01/2009 01:56:45
12/30/2009 22:00:00
12/30/2009 10:00:00
Added:
In What is a Simple Way to Convert Between an AM/PM Time and 24 hour Time in JavaScript, an alternative algorithm is provided for the conversion:
$hh = ($1 % 12) + (($2 eq 'AM') ? 0 : 12);
Just one test...probably neater.
It is a 1-line thing in python:
time.strftime('%H:%M:%S', time.strptime(x, '%I:%M %p'))
Example:
>>> time.strftime('%H:%M:%S', time.strptime('08:01 AM', '%I:%M %p'))
'08:01:00'
>>> time.strftime('%H:%M:%S', time.strptime('12:01 AM', '%I:%M %p'))
'00:01:00'
Use Pythons datetime module someway like this:
import datetime
infile = open('input.txt')
outfile = open('output.txt', 'w')
for line in infile.readlines():
d = datetime.strptime(line, "input format string")
outfile.write(d.strftime("output format string")
Untested code with no error checking. Also it reads the entire input file in memory before starting.
(I know there is plenty of room for improvements like with statement...I make this a community wiki entry if anyone likes to add something)
To just convert the hour field, in python:
def to12(hour24):
return (hour24 % 12) if (hour24 % 12) > 0 else 12
def IsPM(hour24):
return hour24 > 11
def to24(hour12, isPm):
return (hour12 % 12) + (12 if isPm else 0)
def IsPmString(pm):
return "PM" if pm else "AM"
def TestTo12():
for x in range(24):
print x, to12(x), IsPmString(IsPM(x))
def TestTo24():
for pm in [False, True]:
print 12, IsPmString(pm), to24(12, pm)
for x in range(1, 12):
print x, IsPmString(pm), to24(x, pm)
This might be too simple thinking, but why not import it into excel, select the entire column and change the date format, then re-export as a tab delimited file? (I didn't test this, but it somehow sounds logical to me :)
Here i have converted 24 Hour system to 12 Hour system.
Try to use this method for your problem.
DateFormat fmt = new SimpleDateFormat("yyyyMMddHHssmm");
try {
Date date =fmt.parse("20090310232344");
System.out.println(date.toString());
fmt = new SimpleDateFormat("dd-MMMM-yyyy hh:mm:ss a ");
String dateInString = fmt.format(date);
System.out.println(dateInString);
} catch (Exception e) {
System.out.println(e.getMessage());
}
RESULT:
Tue Mar 10 23:44:23 IST 2009
10-March-2009 11:44:23 PM
In Python: Converting 12hr time to 24hr time
import re
time1=input().strip().split(':')
m=re.search('(..)(..)',time1[2])
sec=m.group(1)
tz=m.group(2)
if(tz='PM'):
time[0]=int(time1[0])+12
if(time1[0]=24):
time1[0]-=12
time[2]=sec
else:
if(int(time1[0])=12):
time1[0]-=12
time[2]=sec
print(time1[0]+':'+time1[1]+':'+time1[2])
Since you have multiple languages, I'll suggest the following algorithm.
1 Check the timestamp for the existence of the "PM" string.
2a If PM does not exist, simply convert the timestamp to the datetime object and proceed.
2b If PM does exist, convert the timestamp to the datetime object, add 12 hours, and proceed.

Resources