I'm working on a Parser which Parses log files from a game so I can do analysis on auctions made within the game, however the date format that's being written by the logger seems to be causing problems as the format seems to be custom written for the logger, an example datetime stamp looks like: [Wed Nov 23 23:26:10 2016] I try to Parse it with:
func (r *AuctionReader) extractSaleInformation(line string) {
fmt.Println("Extracting information from: ", line)
// Format mask for output
layout := "DD-MM-YYYY hh:mm:ss"
// Replace the square brackets so we're just left with the date-time string
date := strings.TrimSpace(strings.Replace((strings.Split(line, "]")[0]), "[", "", -1))
fmt.Println(time.Parse(date, layout))
}
When I attempt to Parse the above date-time string I get the following error:
0001-01-01 00:00:00 +0000 UTC parsing time "DD-MM-YYYY hh:mm:ss" as "Wed Nov 23 23:26:10 2016": cannot parse "DD-MM-YYYY hh:mm:ss" as "Wed Nov "
How am I able to get the parser to recognise this seemingly custom format, I will be saving this data to Mongo so I don't want to store the auction time as a string as I want to query the timestamps individually.
Golang handle all date formatting in a unique way - it uses the reference time Mon Jan 2 15:04:05 MST 2006 (01/02 03:04:05PM '06 -0700) to show the pattern with which to format/parse a given time/string.
So, to read the format "Wed Nov 23 23:26:10 2016" you would put the reference date into that format: "Mon Jan 2 15:04:05 2006", and then do:
t, _ := time.Parse("Mon Jan 2 15:04:05 2006", "Wed Nov 23 23:26:10 2016")
Then, to output it in the given format, if you wanted the format DD-MM-YYYY hh:mm:ss, you would put the reference time into that format: 02-01-2006 15:04:05, and then do:
t.Format("02-01-2006 15:04:05")
https://play.golang.org/p/VO5413Z7-z
So basically, the main change is
// Format mask for output
layout := "DD-MM-YYYY hh:mm:ss"
should be
// Format mask for output
layout := "02-01-2006 15:04:05"
and
time.Parse(date, layout)
should be
time.Parse(layout, date)
Related
I have an UTC time and a time offset in seconds, and need to return the corresponding Go time value.
It is trivial to instantiate the UTC time value by using the time.Unix() function. But to set the Zone, I need to determine the time.Location.
How can I find the time.Location when knowing the UTC time and time offset ?
Without an actual entry to lookup in the time zone database, you can't know the true location for the time. If you want to work with just an offset, you can create a fixed location using time.FixedZone
edt := time.FixedZone("EDT", -60*60*4)
t, _ := time.ParseInLocation("02 Jan 06 15:04", "15 Sep 17 14:55", edt)
fmt.Println(t)
// 2017-09-15 14:55:00 -0400 EDT
You can opt to specify a non-existent zone name, or none at all, as long as the output format you use doesn't require one.
minus4 := time.FixedZone("", -60*60*4)
t, _ = time.ParseInLocation("02 Jan 06 15:04", "15 Sep 17 14:55", minus4)
fmt.Println(t.Format(time.RFC3339))
// 2017-09-15T14:55:00-04:00
I have an date string as : Wed Aug 30 2017 00:00:00 GMT+0530 (IST) and I want to convert it into like this: 2017-8-30
Now I am doing this:
moment($scope.date.selectedDate).format('YYYY-M-DD') and it is giving the right time but throws a warning as :
moment construction falls back to js date
As the input is JS date so you need to pass input format as well. This can be done by:
moment('Wed Aug 30 2017 00:00:00 GMT+0530', 'ddd MMM DD YYYY HH:mm:ss GMT+-HH:mm').format('YYYY-M-DD');
https://jsfiddle.net/o01ktajp/1/
Relative to the warning you can refer to this post Deprecation warning: moment construction falls back to js Date.
The easiest solution would be to pass the date string in the ISO format.
As for the date, if you simply want to display the date in the UI with that format you can use the 'date' angular filter: https://docs.angularjs.org/api/ng/filter/date.
In your case you could use it like this:
$scope.date.selectedDate | date: 'YYYY-M-DD'
Br,
You can do:
var d = new Date('Wed Aug 30 2017 00:00:00 GMT+0530');
var formated = moment(d).format('YYYY-M-DD');
I have a dataset that looks like this:
datetime count
18:28:20.602 UTC DEC 08 2016 1
20:42:32.017 UTC DEC 08 2016 5
15:33:40.691 UTC DEC 08 2016 1
17:11:54.008 UTC DEC 08 2016 3
20:28:57.861 UTC DEC 08 2016 0
.
.
.
.
The datetime column is in the string format. I'm having difficulty in converting it to a timestamp.
How do I write a Impala/Hive query so that I get the data between '18:28:00.000 UTC DEC 08 2016' to '18:33:00.000 UTC DEC 08 2016'
With Hive:
cast(from_unixtime(unix_timestamp(SHITTY_FORMAT, 'HH:mm:ss.SSS zzz MMM dd yyyy'), 'yyyy-MM-dd HH:mm:ss.SSS') as Timestamp)
...will translate your shitty String format into a UNIX timestamp, then into String standard format (in local timezone because that's the Hive convention), then into a Timestamp.
There is no easier way, unfortunately. And you may have some edge cases because of the 1h overlap in summer/winter times.
Source: the Hive documentation, of course...
With Impala (which does not support the zzz format modifier):
cast(from_unixtime(unix_timestamp(regexp_replace(SHITTY_FORMAT, ' UTC ', ' '), 'HH:mm:ss.SSS MMM dd yyyy'), 'yyyy-MM-dd HH:mm:ss.SSS') as Timestamp)
...will translate your shitty String format into a UNIX timestamp, assuming that all your inputs are in UTC, then into String standard format (in UTC timezone because that's the Impala convention), then into a Timestamp.
I am working on twitter data and have a field: user_created_at that looks like Thu Jun 11 16:41:35 +0000 2015.
I am not sure what the type of the field is since I got the fields using elephant bird. To covert it into datetime type, I did:
ToDate(user_created_at, 'yyyy.MM.dd') as user_created_at
but it failed with an error:
ERROR 0: Exception while executing [POUserFunc (Name: POUserFunc(org.apache.pig.builtin.ToDate2ARGS)[datetime] - scope-148 Operator Key: scope-148) children: null at []]: java.lang.IllegalArgumentException: Invalid format: "Thu Jun 11 16:41:35 +0000 2015".
What is wrong? I am using Pig version 0.15. Appreciate any help. Thanks!
Match datetime format with input datetime string. Something like this.
ToDate(user_created_at, 'EEE MMM dd HH:mm:ss Z yyyy')
I have this datetime, or something that looks like it.
2014-11-17 23:02:03 +0000 UTC
I want to convert this to a time object and I've been unable to produce any output from time.Parse apart from:
0001-01-01 00:00:00 +0000 UTC
I've tried these layouts:
time.RFC3339
0001-01-01 00:00:00 0000 UTC
2016-10-10
time.UnixDate
And a few more - none have worked.
This is how I'm calling parse :
updatedAt, err := time.Parse(time.UnixDate, updatedAtVar)
How do I create a time object from a string?
Most likely you used a wrong layout, and you didn't check the returned error.
The layout must be this date/time, in the format your input time is:
Mon Jan 2 15:04:05 -0700 MST 2006
See this working code:
layout := "2006-01-02 15:04:05 -0700 MST"
t, err := time.Parse(layout, "2014-11-17 23:02:03 +0000 UTC")
fmt.Println(t, err)
Output (try it on the Go Playground):
2014-11-17 23:02:03 +0000 UTC <nil>
EDIT:
In your question you included a + sign in your input time (as part of the zone offset), but you have error with times of other formats.
Time.String() uses the following format string:
"2006-01-02 15:04:05.999999999 -0700 MST"
So either use this to parse the times, or use Time.Format() to produce your string representations where you can specify the layout, so you can use the same layout to parse the time strings.
2nd round:
You're including your time strings in URLs. The + sign is a special character in URL encoding: it denotes the space. So the + gets converted to space (and so it vanishes from your time string). Use proper URL encoding! Check out the net/url package, and this example.
Didn't see this yet but for those that don't know the formats, time has the formats builtin as constants. so you can reference them when parsing or formating.
time.Parse(time.RFC3339, <your time.Time object here>)
<time.Time object>.Format(time.RFC3339) //or other type of formats
Here they are for reference
ANSIC = "Mon Jan _2 15:04:05 2006"
UnixDate = "Mon Jan _2 15:04:05 MST 2006"
RubyDate = "Mon Jan 02 15:04:05 -0700 2006"
RFC822 = "02 Jan 06 15:04 MST"
RFC822Z = "02 Jan 06 15:04 -0700" // RFC822 with numeric zone
RFC850 = "Monday, 02-Jan-06 15:04:05 MST"
RFC1123 = "Mon, 02 Jan 2006 15:04:05 MST"
RFC1123Z = "Mon, 02 Jan 2006 15:04:05 -0700" // RFC1123 with numeric zone
RFC3339 = "2006-01-02T15:04:05Z07:00"
RFC3339Nano = "2006-01-02T15:04:05.999999999Z07:00"
Kitchen = "3:04PM"
You are likely using the wrong layout. As explained in time.Parse, you need to specify a layout that helps Go to understand how the date passed as input is formatted.
There are predefined layouts (like the ones you were using), but none matches your input. Hence you need to define a custom layout.
A layout uses the following date as reference:
Mon Jan 2 15:04:05 MST 2006
The layout is nothing else that a representation of that date, that matches the representation of your input:
t, err := time.Parse("2006-01-02 15:04:05 -0700 MST", "2014-11-17 23:02:03 +0000 UTC")
Also remember to check err for errors. It's likely your attempts returned an error, but you didn't check it.
package main
import (
"fmt"
"log"
"time"
)
func main() {
t, err := time.Parse("2006-01-02 15:04:05 -0700 UTC", "2014-11-17 23:02:03 +0000 UTC")
if err != nil {
log.Fatalln(err)
}
fmt.Println(t)
}