I am working on twitter data and have a field: user_created_at that looks like Thu Jun 11 16:41:35 +0000 2015.
I am not sure what the type of the field is since I got the fields using elephant bird. To covert it into datetime type, I did:
ToDate(user_created_at, 'yyyy.MM.dd') as user_created_at
but it failed with an error:
ERROR 0: Exception while executing [POUserFunc (Name: POUserFunc(org.apache.pig.builtin.ToDate2ARGS)[datetime] - scope-148 Operator Key: scope-148) children: null at []]: java.lang.IllegalArgumentException: Invalid format: "Thu Jun 11 16:41:35 +0000 2015".
What is wrong? I am using Pig version 0.15. Appreciate any help. Thanks!
Match datetime format with input datetime string. Something like this.
ToDate(user_created_at, 'EEE MMM dd HH:mm:ss Z yyyy')
Related
I have an date string as : Wed Aug 30 2017 00:00:00 GMT+0530 (IST) and I want to convert it into like this: 2017-8-30
Now I am doing this:
moment($scope.date.selectedDate).format('YYYY-M-DD') and it is giving the right time but throws a warning as :
moment construction falls back to js date
As the input is JS date so you need to pass input format as well. This can be done by:
moment('Wed Aug 30 2017 00:00:00 GMT+0530', 'ddd MMM DD YYYY HH:mm:ss GMT+-HH:mm').format('YYYY-M-DD');
https://jsfiddle.net/o01ktajp/1/
Relative to the warning you can refer to this post Deprecation warning: moment construction falls back to js Date.
The easiest solution would be to pass the date string in the ISO format.
As for the date, if you simply want to display the date in the UI with that format you can use the 'date' angular filter: https://docs.angularjs.org/api/ng/filter/date.
In your case you could use it like this:
$scope.date.selectedDate | date: 'YYYY-M-DD'
Br,
You can do:
var d = new Date('Wed Aug 30 2017 00:00:00 GMT+0530');
var formated = moment(d).format('YYYY-M-DD');
I have a dataset that looks like this:
datetime count
18:28:20.602 UTC DEC 08 2016 1
20:42:32.017 UTC DEC 08 2016 5
15:33:40.691 UTC DEC 08 2016 1
17:11:54.008 UTC DEC 08 2016 3
20:28:57.861 UTC DEC 08 2016 0
.
.
.
.
The datetime column is in the string format. I'm having difficulty in converting it to a timestamp.
How do I write a Impala/Hive query so that I get the data between '18:28:00.000 UTC DEC 08 2016' to '18:33:00.000 UTC DEC 08 2016'
With Hive:
cast(from_unixtime(unix_timestamp(SHITTY_FORMAT, 'HH:mm:ss.SSS zzz MMM dd yyyy'), 'yyyy-MM-dd HH:mm:ss.SSS') as Timestamp)
...will translate your shitty String format into a UNIX timestamp, then into String standard format (in local timezone because that's the Hive convention), then into a Timestamp.
There is no easier way, unfortunately. And you may have some edge cases because of the 1h overlap in summer/winter times.
Source: the Hive documentation, of course...
With Impala (which does not support the zzz format modifier):
cast(from_unixtime(unix_timestamp(regexp_replace(SHITTY_FORMAT, ' UTC ', ' '), 'HH:mm:ss.SSS MMM dd yyyy'), 'yyyy-MM-dd HH:mm:ss.SSS') as Timestamp)
...will translate your shitty String format into a UNIX timestamp, assuming that all your inputs are in UTC, then into String standard format (in UTC timezone because that's the Impala convention), then into a Timestamp.
I'm working on a Parser which Parses log files from a game so I can do analysis on auctions made within the game, however the date format that's being written by the logger seems to be causing problems as the format seems to be custom written for the logger, an example datetime stamp looks like: [Wed Nov 23 23:26:10 2016] I try to Parse it with:
func (r *AuctionReader) extractSaleInformation(line string) {
fmt.Println("Extracting information from: ", line)
// Format mask for output
layout := "DD-MM-YYYY hh:mm:ss"
// Replace the square brackets so we're just left with the date-time string
date := strings.TrimSpace(strings.Replace((strings.Split(line, "]")[0]), "[", "", -1))
fmt.Println(time.Parse(date, layout))
}
When I attempt to Parse the above date-time string I get the following error:
0001-01-01 00:00:00 +0000 UTC parsing time "DD-MM-YYYY hh:mm:ss" as "Wed Nov 23 23:26:10 2016": cannot parse "DD-MM-YYYY hh:mm:ss" as "Wed Nov "
How am I able to get the parser to recognise this seemingly custom format, I will be saving this data to Mongo so I don't want to store the auction time as a string as I want to query the timestamps individually.
Golang handle all date formatting in a unique way - it uses the reference time Mon Jan 2 15:04:05 MST 2006 (01/02 03:04:05PM '06 -0700) to show the pattern with which to format/parse a given time/string.
So, to read the format "Wed Nov 23 23:26:10 2016" you would put the reference date into that format: "Mon Jan 2 15:04:05 2006", and then do:
t, _ := time.Parse("Mon Jan 2 15:04:05 2006", "Wed Nov 23 23:26:10 2016")
Then, to output it in the given format, if you wanted the format DD-MM-YYYY hh:mm:ss, you would put the reference time into that format: 02-01-2006 15:04:05, and then do:
t.Format("02-01-2006 15:04:05")
https://play.golang.org/p/VO5413Z7-z
So basically, the main change is
// Format mask for output
layout := "DD-MM-YYYY hh:mm:ss"
should be
// Format mask for output
layout := "02-01-2006 15:04:05"
and
time.Parse(date, layout)
should be
time.Parse(layout, date)
I can't figure out what I am doing wrong here.
Passing in a string to moment, with the format and calling .toDate().
toDate() ends up returning a time that is off by 1 hour
moment("2015-11-19T18:34:00-07:00", "YYYY-MM-DDTHH:mm:ssZ").toDate()
> Thu Nov 19 2015 17:34:00 GMT-0800 (PST)
The time should be 18:34, not 17:34. The timezone is showing -08, when it should be showing -07
Actually in my Flex Application DataGrid column getting one value like Sat Sep 1 00:00:00 GMT+0530 2012
how to convert this String as Date like 2012/09/01
i'm doing converting String Date but it giving null value...hear is my sample code
var startDS:String = saveDataGrid.selectedItem.startDate;
var sDate:Date=DateField.stringToDate(startDs,"YYYY/MM/DD");
Alert.show(sDate+"--===--"+startDS);//Alert giving null--===--Sat Sep 1 00:00:00 GMT+0530 2012
please help...
var startDS:String = "10022008";
var sDate:Date=DateField.stringToDate(startDS,"MMDDYYYY");
Alert.show(sDate+"--===--"+startDS);
You might also need to have a glance at http://livedocs.adobe.com/flex/3/html/help.html?content=formatters_4.html
output will be, "Thu Oct 2 00:00:00 GMT+0530 2008--===--10022008"