I'm creating Unix bash script that parses web-server log file and inserts this data into database. So I need to convert timestamp which has format "05/Oct/2010:07:38:40 +0400" into "YYYY-mm-dd".
I've tried to use /bin/date -d, but it does not accept given format. I could not find a way to specify input date format for this tool. Is is possible or i should consider alternative solution?
input_date="05/Oct/2010:07:38:40 +0400"
better_date=`echo $input_date | sed -E 's|(..)/(...)/(.{4}).*|\1 \2 \3|'`
date -I -d "$better_date"
You need:
date +%F
Also, this seems to work:
date -d "05 Oct 2010 07:38:40 +0400" +%F
You need to use spaces as separators rather than / or :.
Related
I have requirement in unix. I have a file which has the end time of a workflow like the below:
End Time:[Thu oct 05:12:12:12 2017]
I have to convert this to mm-dd-yyy hh24:mi:ss and write to another file.
Please let us know how can i do that?
Thanks,
Amit
Something like
grep input_filename -e "End Time:" | sed -E 's/End Time:\[\w{3}\s(\w{3})\s([0-9]{1,2}):([0-9]{1,2}):([0-9]{1,2}):([0-9]{1,2})\s([0-9]{4}).*$/\1-\2-\6 \3:\4:\5/' | sed 's/jan/01/ig; s/feb/02/ig; s/mar/03/ig; s/apr/04/ig; s/may/05/ig; s/jun/06/ig; s/jul/07/ig; s/aug/08/ig; s/sep/09/ig; s/oct/10/ig; s/nov/11/ig; s/dec/12/ig' > output_filename
This is broken into three commands. First is grep to extract the line with the time you need. Second is sed to rearrange the date into the output format, but note that the month is still a string. The third command is another sed call and this replaces the month name with a number.
Caveats:
the above regex are rather strict and will fail on simple deviations
no need to split the regex stuff into two sed invocations, this could be handled in one
or for that matter, the whole process could be more gracefully handled with python, perl, etc
aman#gmail.com,"08OCT2012"
abc#gmail.com,"11JUL2012"
def#gmail.com,"16DEC2010"
abc#gmail.com,"16MAR2011"
aman#gmail.com,"21APR2011"
abc#apple.com,"12DEC2010"
xyz#fb.com,"06MAR2011"
Want to sort above csv using unix sort command 1st by email address and then by date
I have tried something like
sort -k1 -k212 -k23M -k26 file.csv
But it didn't work out. Anybody has any idea how to sort this csv?
You may need sort -t, to indicate the delimiter is a comma ,.
Then, something like this should work:
sort -t, -k1 -k2 file.csv
Anyway, to sort by date you should firstly do some -> to UNIX stamp conversion in your date field.
You can't. Always use ISO8601 in tabular data because it is the only true format which can be lexically sorted (big endian).
I'm having a little trouble getting the output of diff to write to file. I have a new and old version of a .strings file and I want to be able to write the diff between these two files to a .strings.diff file.
Here's where I am right now:
diff -u -a -B $PROJECT_DIR/new/Localizable.strings $PROJECT_DIR/old/Localizable.strings >> $PROJECT_DIR/diff/Localizable.strings.diff
fgrep + $PROJECT_DIR/diff/Localizable.strings.diff > $PROJECT_DIR/diff/Localizable.txt
The result of the diff command writes to Localizable.strings.diff without any issues, but Localizable.strings.diff appears to be a binary file. Is there any way to output the diff to a UTF-8 encoded file instead?
Note that I'm trying to just get the additions using fgrep in my second command. If there's an easier way to do this, please let me know.
Thanks,
Sean
First, you probably need to identify the encoding of the Localizable.strings files. This might be done in a manner described by How to find encoding of a file in Unix via script(s), for example.
Then probably you need to convert the Localizable.strings file to UTF-8 with a tool like iconv using commands something like:
iconv -f x -t UTF-8 $PROJECT_DIR/new/Localizable.strings >Localizable.strings.new.utf8
iconv -f x -t UTF-8 $PROJECT_DIR/old/Localizable.strings >Localizable.strings.old.utf8
Where x is the actual encoding in a form recognized by iconv. You can use iconv --list to show all the encodings it knows about.
Then, you probably need to diff without having to use -a.
diff -u -B Localizable.strings.old.utf8 Localizable.strings.new.utf8 >Localizable.strings.diff.utf8
I have a dat file with some data in it seperated by '|' character.
I want to read each line and take out 5th and 6th column data( date format) from here and then validate the date using unix. I am new to unix . Please let me know how to do it.
Try using an awk script (http://www.manpagez.com/man/1/awk/) for parsing files. Something like
awk -F\| '{print $5, $6}' test.dat
This can then be extended to perform date validation, depending on what validation needs are. For example - http://unix.derkeiler.com/Newsgroups/comp.unix.shell/2003-08/1340.html seems to perform a reasonable amount of validation.
I'm writing an autoconf script that needs the current UTC offset. There's no obvious way to get this out of the date program. Is there any straightforward way to get this from a command-line utility, or should I write a test that gets the information and somehow captures it?
Try this, and see whether it works for you:
date +%z
For others doing ISO8601, you might pick some variant of:
date +%Y%m%dT%H%M%S%z # 20140809T092143-0700
date -u +%Y%m%dT%H%M%S%z # 20140809T162143+0000
date -u +%Y%m%dT%H%M%SZ # 20140809T162143Z
I like those because the lack of punctuation supports universal use. Note that the capital Z is 'hard-coded' for UTC - using %Z will put UTC or the other named timezone. If you prefer punctuation:
date +%Y-%m-%dT%H:%M:%S%z # 2014-08-09T09:21:43-0700
date +%Y-%m-%dT%H:%M:%S%:z # 2014-08-09T09:21:43-07:00 - NOT ALL SYSTEMS
date -u +%Y-%m-%dT%H:%M:%S%z # 2014-08-09T16:21:43+0000
date -u +%Y-%m-%dT%H:%M:%S%:z # 2014-08-09T16:21:43+00:00 - NOT ALL SYSTEMS
date -u +%Y-%m-%dT%H:%M:%SZ # 2014-08-09T16:21:43Z
Consult man strftime as supported formats vary. For instance, some systems support inserting colons into the offset using %:z, %::z, or %:::z - only two of my five systems do (Debian, Ubuntu do, but Mac, BusyBox, QNX do not).
And I often go back to https://en.wikipedia.org/wiki/ISO_8601 for reference.
Yes, date can do this:
[tomalak#lolphin:~] date -R
Mon, 02 May 2011 17:37:45 +0100
Or, more specifically:
[tomalak#lolphin:~] date -R | awk '{print $6}'
+0100
[tomalak#lolphin:~] date +%z
+0100
Reading date --help is very useful.