How to convert datestring to local time? - jq

2021-08-04T22:55:12+0000
I want to convert the above string to local time.
But fromdateiso8601 does not work on this format. What is the best way to convert this kind of string to local time?
EDIT: I tried the following but the part of 23:55:12 does not change when the timezone is changed. I'd expect that this should be changed as TZ is changed.
$ TZ=America/New_York jq '.x | gsub("[+]0000"; "Z") | fromdateiso8601| gmtime | strftime("%Y-%m-%dT%H:%M:%S%Z")' <<< '{"x": "2021-08-04T22:55:12+0000" }'
"2021-08-04T23:55:12EST"
$ TZ=Europe/Madrid jq '.x | gsub("[+]0000"; "Z") | fromdateiso8601| gmtime | strftime("%Y-%m-%dT%H:%M:%S%Z")' <<< '{"x": "2021-08-04T22:55:12+0000" }'
"2021-08-04T23:55:12CET"

The first problem here is that jq's handling of timezones (TZ) is buggy; the second is that jq's built-ins do not recognize timezone offsets.
Unfortunately, there is to my knowledge not much that can easily be
done about jq's TZ-related bugginess other than using gojq, the Go
implementation of jq, instead.
Fortunately, time offsets can be handled quite easily, e.g. using the datetime_to_seconds filter as defined below.
So, for the time being, the following solution to the stated problem assumes the use of gojq rather than stedolan/jq. It has two main steps:
Use the generic filter datetime_to_seconds to convert the timestamp with an offset to "seconds since the epoch";
Use strflocaltime, which recognizes the environment variable TZ.
The solution is embedded in a bash script to facilitate a comparison between different versions of jq and gojq.
bash script
#!/bin/bash
# Syntax: go TZ
function go {
TZ="$1" $jq -Rr '
# Convert a timestamp with a possibly empty timezone offset to seconds since the Epoch.
# Input should be a string of the form yyyy-mm-ddThh:mm:ss or yyyy-mm-ddThh:mm:ss<OFFSET>
# where <OFFSET> is Z, or has the form [-+]hh:mm or [-+]hhmm
# If no timezone offset is explicitly given, it is taken to be Z.
def datetime_to_seconds:
if test("[-+]")
then
sub("(?<s>[-+])(?<d1>[0-9]{2})(?<d2>[0-9]{2})$"; "\(.s)\(.d1):\(.d2)")
| capture("(?<datetime>^.*T[0-9:]+)(?<s>[-+])(?<hh>[0-9]+):?(?<mm>[0-9]*)")
| (.datetime +"Z" | fromdateiso8601) as $seconds
| (if .s == "+" then -1 else 1 end) as $plusminus
| (.mm | if . == "" then 0 else . end) as $mm
| ([.hh,$mm] | map(tonumber) |.[0] *= 60 | add * 60 * $plusminus) as $offset
| ($seconds + $offset)
else . + (if test("Z") then "" else "Z" end) | fromdateiso8601
end;
datetime_to_seconds
| strflocaltime("%Y-%m-%dT%H:%M:%S %Z")
'
}
for jq in jq-1.6 jqMaster gojq ; do
echo $jq is $($jq --version)
done
echo
for TZ in America/New_York Europe/Madrid ;do
for jq in jq-1.6 jqMaster gojq ; do
for time in 2021-08-04T22:55:12+0000 ; do
echo $jq $TZ $time
echo $time | go $TZ
echo
done
done
done
Output
Here is the output with some "#" annotations.
jq-1.6 is jq-master-2e01ff1
jqMaster is jq-1.6-129-g80052e5-dirty
gojq is gojq 0.12.4 (rev: 244f9f7/go1.16.4)
jq-1.6 America/New_York 2021-08-04T22:55:12+0000
2021-08-04T19:55:12 EST # wrong
jqMaster America/New_York 2021-08-04T22:55:12+0000
2021-08-04T18:55:12 EST # wrong
gojq America/New_York 2021-08-04T22:55:12+0000
2021-08-04T18:55:12 EDT # correct
jq-1.6 Europe/Madrid 2021-08-04T22:55:12+0000
2021-08-05T01:55:12 CET # wrong
jqMaster Europe/Madrid 2021-08-04T22:55:12+0000
2021-08-05T00:55:12 CET # wrong
gojq Europe/Madrid 2021-08-04T22:55:12+0000
2021-08-05T00:55:12 CEST # correct

Related

jq: convert milliseconds to date

I need to convert seconds to date:
My input documents are:
{"_id":"ae53d3ec-8fc3-44fc-a7eb-f2f32da4eaae","birthDate":{"value":{"$date":{"$numberLong":"-469929600000"}}}}
{"_id":"ef92c3e4-d5d7-4b81-8a0b-1ab1eac10331","birthDate":{"value":{"$date":{"$numberLong":"-854755200000"}}}}
I need to get:
{"_id":"ae53d3ec-8fc3-44fc-a7eb-f2f32da4eaae","birthDate":"1955-02-10"}
{"_id":"ef92c3e4-d5d7-4b81-8a0b-1ab1eac10331","birthDate":"1942-12-01"}
Any ideas?
Grab the source value, chop off the last three digits to turn the number into seconds and call todate to convert it into a date string (GMT).
jq -c '.birthDate |= (.value."$date"."$numberLong"[:-3] | tonumber | todate[:10])'
{"_id":"ae53d3ec-8fc3-44fc-a7eb-f2f32da4eaae","birthDate":"1955-02-10"}
{"_id":"ef92c3e4-d5d7-4b81-8a0b-1ab1eac10331","birthDate":"1942-12-01"}
Demo
.birthDate |= (
.value."$date"."$numberLong" |
tonumber |
. / 1000 |
strftime("%F")
)
With -c, this produces the desired output exactly.
{"_id":"ae53d3ec-8fc3-44fc-a7eb-f2f32da4eaae","birthDate":"1955-02-10"}
{"_id":"ef92c3e4-d5d7-4b81-8a0b-1ab1eac10331","birthDate":"1942-12-01"}
Demo on jqplay

File pulling with yesterday's date in UNIX based on file name [duplicate]

I've got a shell script which does the following to store the current day's date in a variable 'dt':
date "+%a %d/%m/%Y" | read dt
echo ${dt}
How would i go about getting yesterdays date into a variable?
Basically what i'm trying to achieve is to use grep to pull all of yesterday's lines from a log file, since each line in the log contains the date in "Mon 01/02/2010" format.
Thanks a lot
dt=$(date --date yesterday "+%a %d/%m/%Y")
echo $dt
On Linux, you can use
date -d "-1 days" +"%a %d/%m/%Y"
You can use GNU date command as shown below
Getting Date In the Past
To get yesterday and earlier day in the past use string day ago:
date --date='yesterday'
date --date='1 day ago'
date --date='10 day ago'
date --date='10 week ago'
date --date='10 month ago'
date --date='10 year ago'
Getting Date In the Future
To get tomorrow and day after tomorrow (tomorrow+N) use day word to get date in the future as follows:
date --date='tomorrow'
date --date='1 day'
date --date='10 day'
date --date='10 week'
date --date='10 month'
date --date='10 year'
If you have Perl available (and your date doesn't have nice features like yesterday), you can use:
pax> date
Thu Aug 18 19:29:49 XYZ 2010
pax> dt=$(perl -e 'use POSIX;print strftime "%d/%m/%Y%",localtime time-86400;')
pax> echo $dt
17/08/2010
If you are on a Mac or BSD or something else without the --date option, you can use:
date -r `expr \`date +%s\` - 86400` '+%a %d/%m/%Y'
Update: or perhaps...
date -r $((`date +%s` - 86400)) '+%a %d/%m/%Y'
I have shell script in Linux and following code worked for me:
#!/bin/bash
yesterday=`TZ=EST+24 date +%Y%m%d` # Yesterday is a variable
mkdir $yesterday # creates a directory with YYYYMMDD format
You have atleast 2 options
Use perl:
perl -e '#T=localtime(time-86400);printf("%02d/%02d/%02d",$T[4]+1,$T[3],$T[5]+1900)'
Install GNU date (it's in the sh_utils package if I remember correctly)
date --date yesterday "+%a %d/%m/%Y" | read dt
echo ${dt}
Not sure if this works, but you might be able to use a negative timezone. If you use a timezone that's 24 hours before your current timezone than you can simply use date.
Try the following method:
dt=`case "$OSTYPE" in darwin*) date -v-1d "+%s"; ;; *) date -d "1 days ago" "+%s"; esac`
echo $dt
It works on both Linux and OSX.
Here is a ksh script to calculate the previous date of the first argument, tested on Solaris 10.
#!/bin/ksh
sep=""
today=$(date '+%Y%m%d')
today=${1:-today}
ty=`echo $today|cut -b1-4` # today year
tm=`echo $today|cut -b5-6` # today month
td=`echo $today|cut -b7-8` # today day
yy=0 # yesterday year
ym=0 # yesterday month
yd=0 # yesterday day
if [ td -gt 1 ];
then
# today is not first of month
let yy=ty # same year
let ym=tm # same month
let yd=td-1 # previous day
else
# today is first of month
if [ tm -gt 1 ];
then
# today is not first of year
let yy=ty # same year
let ym=tm-1 # previous month
if [ ym -eq 1 -o ym -eq 3 -o ym -eq 5 -o ym -eq 7 -o ym -eq 8 -o ym - eq 10 -o ym -eq 12 ];
then
let yd=31
fi
if [ ym -eq 4 -o ym -eq 6 -o ym -eq 9 -o ym -eq 11 ];
then
let yd=30
fi
if [ ym -eq 2 ];
then
# shit... :)
if [ ty%4 -eq 0 ];
then
if [ ty%100 -eq 0 ];
then
if [ ty%400 -eq 0 ];
then
#echo divisible by 4, by 100, by 400
leap=1
else
#echo divisible by 4, by 100, not by 400
leap=0
fi
else
#echo divisible by 4, not by 100
leap=1
fi
else
#echo not divisible by 4
leap=0 # not divisible by four
fi
let yd=28+leap
fi
else
# today is first of year
# yesterday was 31-12-yy
let yy=ty-1 # previous year
let ym=12
let yd=31
fi
fi
printf "%4d${sep}%02d${sep}%02d\n" $yy $ym $yd
Tests
bin$ for date in 20110902 20110901 20110812 20110801 20110301 20100301 20080301 21000301 20000301 20000101 ; do yesterday $date; done
20110901
20110831
20110811
20110731
20110228
20100228
20080229
21000228
20000229
19991231
Thanks for the help everyone, but since i'm on HP-UX (after all: the more you pay, the less features you get...) i've had to resort to perl:
perl -e '#T=localtime(time-86400);printf("%02d/%02d/%04d",$T[3],$T[4]+1,$T[5]+1900)' | read dt
If your HP-UX installation has Tcl installed, you might find it's date arithmetic very readable (unfortunately the Tcl shell does not have a nice "-e" option like perl):
dt=$(echo 'puts [clock format [clock scan yesterday] -format "%a %d/%m/%Y"]' | tclsh)
echo "yesterday was $dt"
This will handle all the daylight savings bother.
If you don't have a version of date that supports --yesterday and you don't want to use perl, you can use this handy ksh script of mine. By default, it returns yesterday's date, but you can feed it a number and it tells you the date that many days in the past. It starts to slow down a bit if you're looking far in the past. 100,000 days ago it was 1/30/1738, though my system took 28 seconds to figure that out.
#! /bin/ksh -p
t=`date +%j`
ago=$1
ago=${ago:=1} # in days
y=`date +%Y`
function build_year {
set -A j X $( for m in 01 02 03 04 05 06 07 08 09 10 11 12
{
cal $m $y | sed -e '1,2d' -e 's/^/ /' -e "s/ \([0-9]\)/ $m\/\1/g"
} )
yeardays=$(( ${#j[*]} - 1 ))
}
build_year
until [ $ago -lt $t ]
do
(( y=y-1 ))
build_year
(( ago = ago - t ))
t=$yeardays
done
print ${j[$(( t - ago ))]}/$y
ksh93:
dt=${ printf "%(%a %d/%m/%Y)T" yesterday; }
or:
dt=$(printf "%(%a %d/%m/%Y)T" yesterday)
The first one runs in the same process, the second one in a subshell.
For Hp-UX only below command worked for me:
TZ=aaa24 date +%Y%m%d
you can use it as :
ydate=`TZ=aaa24 date +%Y%m%d`
echo $ydate
If you have access to python, this is a helper that will get the yyyy-mm-dd date value for any arbitrary n days ago:
function get_n_days_ago {
local days=$1
python -c "import datetime; print (datetime.date.today() - datetime.timedelta(${days})).isoformat()"
}
# today is 2014-08-24
$ get_n_days_ago 1
2014-08-23
$ get_n_days_ago 2
2014-08-22
$var=$TZ;
TZ=$TZ+24;
date;
TZ=$var;
Will get you yesterday in AIX and set back the TZ variable back to original
Though all good answers, unfortunately none of them worked for me. So I had to write something old school. ( I was on a bare minimal Linux OS )
$ date -d #$( echo $(( $(date +%s)-$((60*60*24)) )) )
You can combine this with date's usual formatting. Eg.
$ date -d #$( echo $(( $(date +%s)-$((60*60*24)) )) ) +%Y-%m-%d
Explanation :
Take date input in terms of epoc seconds ( the -d option ), from which you would have subtracted one day equivalent seconds. This will give the date precisely one day back.

How can I find the current date minus seven days in Unix?

I am trying to find the date that was seven days before today.
CURRENT_DT=`date +"%F %T"`
diff=$CURRENT_DT-7
echo $diff
I am trying stuff like the above to find the 7 days less than from current date. Could anyone help me out please?
GNU date will to the math for you:
date --date "7 days ago"
Other version will require you to covert the current date into seconds since the UNIX epoch first, manually subtract 7 days' worth of seconds, and convert that back into the desired form. Consult the documentation for your version of date for details on how to convert to and from Unix timestamps. Here's an example using GNU date again:
x=$(date +%s)
x=$((x - 7 * 24 * 60 * 60))
date --date #$x
Here is a simple Perl script which (unlike the other examples) works with Unix:
perl -e 'use POSIX qw(ctime); printf "%s", ctime(time - (7 * 24 * 60 * 60));'
(Tested with Solaris 10, and a token Linux system, of course - with the caveat that Perl is not necessarily part of one's configuration, merely very likely).
Adding this one for shells on OSX:
date -v-7d
> Tue Apr 3 15:16:31 EDT 2018
date
> Tue Apr 10 15:16:33 EDT 2018
Need that formated?
date -v-7d +%Y-%m-%d
> 2018-04-03
Ksh's printf can do time calculation:
$ printf '%(%Y-%m-%d)T\n'
2015-04-07
$ printf '%(%Y-%m-%d)T\n' '7 days ago'
2015-03-31
$
I haven't used unix in a while but I found this in one of my scripts
echo `date +%s`-604800 | bc
DATE=$(date --date "7 days ago" | awk '{print$1,$2,$3}')
echo "$DATE"
if [ -z "$(grep -i "$DATE" test.log)" ]; then
exit 1
fi
sed -i "1,/$DATE/d" test.log

Convert a collection of datetime to yyyy-mm-dd

I have a collection of Dates in a text file.
Ex. file.txt
2011-01-01
2011-02-14
2011-02-21
2011-03-17
2011-09-11
2011-11-11
I have a function called Important-Dates which holds a collection of [datetime] dates
Ex. Function holds these type of dates
November-11-15 12:00:00 AM
November-11-14 12:00:00 AM
November-11-13 12:00:00 AM
February-14-15 12:00:00 AM
I want to convert the function dates into the format yyyy-mm-dd and append to the file.txt file
So far, I have this:
$Dates = Important-Dates
if ( -not (test-path 'file.txt' -pathtype leaf))
{
"{0:yyyy-MM-dd}" -f $Dates | Set-Content 'file.txt'
exit 1
}
else {
"{0:yyyy-MM-dd}" -f #($_.$Dates) | Add-Content 'file.txt'
exit 1
}
All I get as a result to the File.txt file is the first date in the Important-Dates list
"November-11-15 12:00:00 AM" - which does convert to 2015-11-11
I want all dates though. What I am doing wrong?
Providing a collection to -f will only use the first element in the collection. You want to process each date object in the collection.
To process each object in the collection you can use foreach-object:
$Dates | ForEach-Object -Process { "{0:yyyy-MM-dd}" -f $_ } | set-content 'file.txt'
}
...or more concisely:
$Dates | % { "{0:yyyy-MM-dd}" -f $_ } | Set-Content 'file.txt'
DateTime objects can be formatted via their ToString() method:
$Dates | % { $_.ToString('yyyy-MM-dd') } | Set-Content 'file.txt'

In a unix shell, how to get yesterday's date into a variable?

I've got a shell script which does the following to store the current day's date in a variable 'dt':
date "+%a %d/%m/%Y" | read dt
echo ${dt}
How would i go about getting yesterdays date into a variable?
Basically what i'm trying to achieve is to use grep to pull all of yesterday's lines from a log file, since each line in the log contains the date in "Mon 01/02/2010" format.
Thanks a lot
dt=$(date --date yesterday "+%a %d/%m/%Y")
echo $dt
On Linux, you can use
date -d "-1 days" +"%a %d/%m/%Y"
You can use GNU date command as shown below
Getting Date In the Past
To get yesterday and earlier day in the past use string day ago:
date --date='yesterday'
date --date='1 day ago'
date --date='10 day ago'
date --date='10 week ago'
date --date='10 month ago'
date --date='10 year ago'
Getting Date In the Future
To get tomorrow and day after tomorrow (tomorrow+N) use day word to get date in the future as follows:
date --date='tomorrow'
date --date='1 day'
date --date='10 day'
date --date='10 week'
date --date='10 month'
date --date='10 year'
If you have Perl available (and your date doesn't have nice features like yesterday), you can use:
pax> date
Thu Aug 18 19:29:49 XYZ 2010
pax> dt=$(perl -e 'use POSIX;print strftime "%d/%m/%Y%",localtime time-86400;')
pax> echo $dt
17/08/2010
If you are on a Mac or BSD or something else without the --date option, you can use:
date -r `expr \`date +%s\` - 86400` '+%a %d/%m/%Y'
Update: or perhaps...
date -r $((`date +%s` - 86400)) '+%a %d/%m/%Y'
I have shell script in Linux and following code worked for me:
#!/bin/bash
yesterday=`TZ=EST+24 date +%Y%m%d` # Yesterday is a variable
mkdir $yesterday # creates a directory with YYYYMMDD format
You have atleast 2 options
Use perl:
perl -e '#T=localtime(time-86400);printf("%02d/%02d/%02d",$T[4]+1,$T[3],$T[5]+1900)'
Install GNU date (it's in the sh_utils package if I remember correctly)
date --date yesterday "+%a %d/%m/%Y" | read dt
echo ${dt}
Not sure if this works, but you might be able to use a negative timezone. If you use a timezone that's 24 hours before your current timezone than you can simply use date.
Try the following method:
dt=`case "$OSTYPE" in darwin*) date -v-1d "+%s"; ;; *) date -d "1 days ago" "+%s"; esac`
echo $dt
It works on both Linux and OSX.
Here is a ksh script to calculate the previous date of the first argument, tested on Solaris 10.
#!/bin/ksh
sep=""
today=$(date '+%Y%m%d')
today=${1:-today}
ty=`echo $today|cut -b1-4` # today year
tm=`echo $today|cut -b5-6` # today month
td=`echo $today|cut -b7-8` # today day
yy=0 # yesterday year
ym=0 # yesterday month
yd=0 # yesterday day
if [ td -gt 1 ];
then
# today is not first of month
let yy=ty # same year
let ym=tm # same month
let yd=td-1 # previous day
else
# today is first of month
if [ tm -gt 1 ];
then
# today is not first of year
let yy=ty # same year
let ym=tm-1 # previous month
if [ ym -eq 1 -o ym -eq 3 -o ym -eq 5 -o ym -eq 7 -o ym -eq 8 -o ym - eq 10 -o ym -eq 12 ];
then
let yd=31
fi
if [ ym -eq 4 -o ym -eq 6 -o ym -eq 9 -o ym -eq 11 ];
then
let yd=30
fi
if [ ym -eq 2 ];
then
# shit... :)
if [ ty%4 -eq 0 ];
then
if [ ty%100 -eq 0 ];
then
if [ ty%400 -eq 0 ];
then
#echo divisible by 4, by 100, by 400
leap=1
else
#echo divisible by 4, by 100, not by 400
leap=0
fi
else
#echo divisible by 4, not by 100
leap=1
fi
else
#echo not divisible by 4
leap=0 # not divisible by four
fi
let yd=28+leap
fi
else
# today is first of year
# yesterday was 31-12-yy
let yy=ty-1 # previous year
let ym=12
let yd=31
fi
fi
printf "%4d${sep}%02d${sep}%02d\n" $yy $ym $yd
Tests
bin$ for date in 20110902 20110901 20110812 20110801 20110301 20100301 20080301 21000301 20000301 20000101 ; do yesterday $date; done
20110901
20110831
20110811
20110731
20110228
20100228
20080229
21000228
20000229
19991231
Thanks for the help everyone, but since i'm on HP-UX (after all: the more you pay, the less features you get...) i've had to resort to perl:
perl -e '#T=localtime(time-86400);printf("%02d/%02d/%04d",$T[3],$T[4]+1,$T[5]+1900)' | read dt
If your HP-UX installation has Tcl installed, you might find it's date arithmetic very readable (unfortunately the Tcl shell does not have a nice "-e" option like perl):
dt=$(echo 'puts [clock format [clock scan yesterday] -format "%a %d/%m/%Y"]' | tclsh)
echo "yesterday was $dt"
This will handle all the daylight savings bother.
If you don't have a version of date that supports --yesterday and you don't want to use perl, you can use this handy ksh script of mine. By default, it returns yesterday's date, but you can feed it a number and it tells you the date that many days in the past. It starts to slow down a bit if you're looking far in the past. 100,000 days ago it was 1/30/1738, though my system took 28 seconds to figure that out.
#! /bin/ksh -p
t=`date +%j`
ago=$1
ago=${ago:=1} # in days
y=`date +%Y`
function build_year {
set -A j X $( for m in 01 02 03 04 05 06 07 08 09 10 11 12
{
cal $m $y | sed -e '1,2d' -e 's/^/ /' -e "s/ \([0-9]\)/ $m\/\1/g"
} )
yeardays=$(( ${#j[*]} - 1 ))
}
build_year
until [ $ago -lt $t ]
do
(( y=y-1 ))
build_year
(( ago = ago - t ))
t=$yeardays
done
print ${j[$(( t - ago ))]}/$y
ksh93:
dt=${ printf "%(%a %d/%m/%Y)T" yesterday; }
or:
dt=$(printf "%(%a %d/%m/%Y)T" yesterday)
The first one runs in the same process, the second one in a subshell.
For Hp-UX only below command worked for me:
TZ=aaa24 date +%Y%m%d
you can use it as :
ydate=`TZ=aaa24 date +%Y%m%d`
echo $ydate
If you have access to python, this is a helper that will get the yyyy-mm-dd date value for any arbitrary n days ago:
function get_n_days_ago {
local days=$1
python -c "import datetime; print (datetime.date.today() - datetime.timedelta(${days})).isoformat()"
}
# today is 2014-08-24
$ get_n_days_ago 1
2014-08-23
$ get_n_days_ago 2
2014-08-22
$var=$TZ;
TZ=$TZ+24;
date;
TZ=$var;
Will get you yesterday in AIX and set back the TZ variable back to original
Though all good answers, unfortunately none of them worked for me. So I had to write something old school. ( I was on a bare minimal Linux OS )
$ date -d #$( echo $(( $(date +%s)-$((60*60*24)) )) )
You can combine this with date's usual formatting. Eg.
$ date -d #$( echo $(( $(date +%s)-$((60*60*24)) )) ) +%Y-%m-%d
Explanation :
Take date input in terms of epoc seconds ( the -d option ), from which you would have subtracted one day equivalent seconds. This will give the date precisely one day back.

Resources