I am trying to fetch lines which are within a specified date range. I have tried many online solutions they did not work
Below is the log file I have
Nov 21 03:31:28 Sample Log test
Nov 21 03:32:01 Sample Log test
Nov 21 03:33:01 Sample Log test
Nov 21 03:34:01 Sample Log test
Nov 21 03:35:02 Sample Log test
Nov 21 03:36:01 Sample Log test
Nov 21 03:37:01 Sample Log test
Nov 21 03:38:01 Sample Log test
Nov 21 03:39:01 Sample Log test
Nov 21 03:39:01 Sample Log test
Nov 21 03:39:01 Sample Log test
Nov 21 03:40:01 Sample Log test
Nov 21 03:40:01 Sample Log test
Nov 21 03:40:29 Sample Log test
Nov 21 03:40:29 Sample Log test
Nov 21 03:41:01 Sample Log test
Nov 21 03:41:22 Sample Log test
Nov 21 03:41:22 Sample Log test
Nov 21 03:41:43 Sample Log test
Nov 21 03:41:43 Sample Log test
Nov 21 03:42:01 Sample Log test
Awk command I am using is
-bash-4.2$ b="03:31:28"
-bash-4.2$ e="03:41:00"
-bash-4.2$ awk -v "b=$b" -v "e=$e" -F ',' '$1 >= b && $1 <= e' ~/test
-bash-4.2$
It is not returning output
Using awk
$ awk -v b=03:31:28 -v e=03:41:00 '$3 >= b && $3 <= e' input_file
Nov 21 03:31:28 Sample Log test
Nov 21 03:32:01 Sample Log test
Nov 21 03:33:01 Sample Log test
Nov 21 03:34:01 Sample Log test
Nov 21 03:35:02 Sample Log test
Nov 21 03:36:01 Sample Log test
Nov 21 03:37:01 Sample Log test
Nov 21 03:38:01 Sample Log test
Nov 21 03:39:01 Sample Log test
Nov 21 03:39:01 Sample Log test
Nov 21 03:39:01 Sample Log test
Nov 21 03:40:01 Sample Log test
Nov 21 03:40:01 Sample Log test
Nov 21 03:40:29 Sample Log test
Nov 21 03:40:29 Sample Log test
Related
I've been trying to set up an OpenVPN server on my Linux recently but I continuously get the same error every time I try to connect to my server.
My settings are like this:
proto tcp
port 443
resolv-retry infinite
nobind
user nobody
group nogroup
cipher AES-256-CBC
auth SHA256
script-security 2
up /etc/openvpn/update-systemd-resolved
down /etc/openvpn/update-systemd-resolved
down-pre
dhcp-option DOMAIN-ROUTE .
I have checked the settings on my server and local computer a million times and all of them are the same. Still don't know what I have to do about it. Thanks in advance! :*
Sat Nov 27 23:45:11 2021 OpenVPN 2.4.7 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Jul 19 2021
Sat Nov 27 23:45:11 2021 library versions: OpenSSL 1.1.1f 31 Mar 2020, LZO 2.10
Sat Nov 27 23:45:11 2021 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Sat Nov 27 23:45:11 2021 TCP/UDP: Preserving recently used remote address: [AF_INET]myserverip:443
Sat Nov 27 23:45:11 2021 Socket Buffers: R=[131072->131072] S=[16384->16384]
Sat Nov 27 23:45:11 2021 Attempting to establish TCP connection with [AF_INET]myserverip:443 [nonblock]
Sat Nov 27 23:45:12 2021 TCP connection established with [AF_INET]myserverip:443
Sat Nov 27 23:45:12 2021 TCP_CLIENT link local: (not bound)
Sat Nov 27 23:45:12 2021 TCP_CLIENT link remote: [AF_INET]myserverip:443
Sat Nov 27 23:45:12 2021 NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
Sat Nov 27 23:45:12 2021 Connection reset, restarting [0]
Sat Nov 27 23:45:12 2021 SIGUSR1[soft,connection-reset] received, process restarting
Sat Nov 27 23:45:12 2021 Restart pause, 5 second(s)
This question already has answers here:
How to loop through dates using Bash?
(10 answers)
Closed 1 year ago.
How can I write a shell scripts that calls another script for each day
i.e currently have
#!/bin/sh
./update.sh 2020 02 01
./update.sh 2020 02 02
./update.sh 2020 02 03
but I want to just specify start date (2020 02 01) and let is run update.sh for every day upto current date, but don't know how to manipulate date in shell script.
I made a stab at it, but rather messy, would prefer if it could process date itself.
#!/bin/bash
for j in {4..9}
do
for k in {1..9}
do
echo "update.sh" 2020 0$j 0$k
./update.sh 2020 0$j 0$k
done
done
for j in {10..12}
do
for k in {10..31}
do
echo "update.sh" 2020 $j $k
./update.sh 2020 $j $k
done
done
for j in {1..9}
do
for k in {1..9}
do
echo "update.sh" 2021 0$j 0$k
./update.sh 2021 0$j 0$k
done
done
for j in {1..9}
do
for k in {10..31}
do
echo "update.sh" 2021 0$j $k
./update.sh 2021 0$j $k
done
done
You can use date to convert your input dates into seconds in order to compare. Also use date to add one day.
#!/bin/bash
start_date=$(date -I -d "$1") # Input in format yyyy-mm-dd
end_date=$(date -I) # Today in format yyyy-mm-dd
echo "Start: $start_date"
echo "Today: $end_date"
d=$start_date # In case you want start_date for later?
end_d=$(date -d "$end_date" +%s) # End date in seconds
while [ $(date -d "$d" +%s) -le $end_d ]; do # Check dates in seconds
# Replace `echo` in the below with your command/script
echo ${d//-/ } # Output the date but replace - with [space]
d=$(date -I -d "$d + 1 day") # Next day
done
In this example, I use echo but replace this with the path to your update.sh.
Sample output:
[user#server:~]$ ./dateloop.sh 2021-08-29
Start: 2021-08-29
End : 2021-09-20
2021 08 29
2021 08 30
2021 08 31
2021 09 01
2021 09 02
2021 09 03
2021 09 04
2021 09 05
2021 09 06
2021 09 07
2021 09 08
2021 09 09
2021 09 10
2021 09 11
2021 09 12
2021 09 13
2021 09 14
2021 09 15
2021 09 16
2021 09 17
2021 09 18
2021 09 19
2021 09 20
I would like to achieve the following task. Using a linear mixed model, I would like to check whether "Month" (see dat table) has a significant effect on the "Response" variable. As for some of the tanks, data comes from different months, I included it as a random factor in my model. Please note, that sampling the same tank in different months does not change the "Response" variable. For some tank-month combinations there are multiple records, as we are included the compartment of the tank that was sampled (e.g. NW =north west).
Here the data:
print(dat)
Tank Month ID Response
1 AEW1 Jul AEW01SOBFJul2008 1.80522937
2 AEW10 Jul AEW10NWBFJul2008 2.13374401
3 AEW10 Jul AEW10NWBFJul2008 2.13374401
4 AEW11 Jun AEW11SWBFJun2008 1.65010205
5 AEW14 Jun AEW14SWBFJun2008 1.75459326
6 AEW15 Jun AEW15SOBFJun2008 2.82200903
7 AEW15 Jun AEW15SOBFJun2008 2.82200903
8 AEW18 Jul AEW18SOBFJul2008 0.39349330
9 AEW19 Jul AEW19NWBFJul2008 0.65886661
10 AEW20 Jul AEW20NWBFJul2008 1.07838018
11 AEW24 Jun AEW24NOBFJun2008 2.56677635
12 AEW27 Jul AEW27SWBFJul2008 2.64019328
13 AEW27 Jul AEW27SWBFJul2008 2.64019328
14 AEW29 Jul AEW29SOBFJul2008 2.06251217
15 AEW30 Jul AEW30NWBFJul2008 1.17010646
16 AEW31 Jun AEW31SWBFJun2008 2.25518873
17 AEW32 Jun AEW32SOBFJun2008 2.38707614
18 AEW33 Jun AEW33SOBFJun2008 2.30498448
19 AEW33 Jun AEW33SOBFJun2008 2.30498448
20 AEW36 Jul AEW36NOBFJul2008 1.92368247
21 AEW37 Jun AEW37NOBFJun2008 0.99387013
22 AEW39 Jul AEW39NOBFJul2008 1.24163732
23 AEW4 Jul AEW04SWBFJul2008 1.56327732
24 AEW42 Jun AEW42SWBFJun2008 1.26012579
25 AEW44 Jun AEW44SWBFJun2008 0.75985267
26 AEW48 Aug AEW48SOBFAug2008 1.57920494
27 AEW50 Jul AEW50NOBFJul2008 0.90052629
28 AEW8 Jul AEW08NOBFJul2008 0.00000000
29 AEW8 Jul AEW08NOBFJul2008 0.00000000
30 AEW9 Jul AEW09NOBFJul2008 0.48529647
31 HEW10 Jun HEW10SWBFJun2008 0.06412823
32 HEW10 Aug HEW10SOBFAug2008 0.06412823
33 HEW12 Jul HEW12NOBFJul2008 0.00000000
34 HEW13 Aug HEW13NWBFAug2008 2.24515850
35 HEW13 Jul HEW13SOBFJul2008 2.24515850
36 HEW13 Jul HEW13NOBFJul2008 2.24515850
37 HEW13 Jun HEW13SOBFJun2008 2.24515850
38 HEW13 Jun HEW13NWBFJun2008 2.24515850
39 HEW14 Jul HEW14SOBFJul2008 1.64783184
40 HEW18 Jun HEW18NWBFJun2008 1.32435721
41 HEW18 Jun HEW18NWBFJun2008 1.32435721
42 HEW19 Jul HEW19SWBFJul2008 1.01761003
43 HEW19 Jul HEW19SWBFJul2008 1.01761003
44 HEW22 Aug HEW22SWBFAug2008 0.63861037
45 HEW23 Jun HEW23SWBFJun2008 1.38472769
46 HEW23 Jun HEW23NWBFJun2008 1.38472769
47 HEW28 Jun HEW28NOBFJun2008 1.44377199
48 HEW3 Jun HEW03SWBFJun2008 2.19793633
49 HEW3 Jul HEW03SWBFJul2008 2.19793633
50 HEW30 Aug HEW30NWBFAug2008 0.76260579
51 HEW31 Jul HEW31SWBFJul2008 1.07879539
52 HEW35 Jun HEW35NWBFJun2008 0.86098152
53 HEW35 Jun HEW35NWBFJun2008 0.86098152
54 HEW36 Aug HEW36SOBFAug2008 0.36533352
55 HEW39 Jun HEW39SOBFJun2008 0.09283168
56 HEW4 Jun HEW04SWBFJun2008 1.89046783
57 HEW41 Aug HEW41NWBFAug2008 0.31996275
58 HEW41 Aug HEW41NWBFAug2008 0.31996275
59 HEW41 Jul HEW41NWBFJul2008 0.31996275
60 HEW41 Jul HEW41NWBFJul2008 0.31996275
61 HEW42 Jul HEW42NWBFJul2008 0.53998250
62 HEW43 Jun HEW43SWBFJun2008 1.85594061
63 HEW43 Jun HEW43SWBFJun2008 1.85594061
64 HEW44 Jun HEW44SOBFJun2008 1.79972095
65 HEW44 Jun HEW44SOBFJun2008 1.79972095
66 HEW49 Jun HEW49SWBFJun2008 1.25229249
67 HEW5 Aug HEW05SWBFAug2008 0.95559764
68 HEW50 Jun HEW50NWBFJun2008 0.42309531
69 HEW50 Jun HEW50NWBFJun2008 0.42309531
70 HEW7 Jul HEW07NWBFJul2008 0.69484213
71 HEW7 Jun HEW07NWBFJun2008 0.69484213
72 HEW8 Jul HEW08SWBFJul2008 1.15617440
73 SEW1 Aug SEW01NWBFAug2008 1.90030109
74 SEW1 Sep SEW01SWBFSep2008 1.90030109
75 SEW11 Aug SEW11NWBFAug2008 2.11940912
76 SEW12 Aug SEW12SOBFAug2008 2.29658624
77 SEW12 Jul SEW12SOBFJul2008 2.29658624
78 SEW17 Aug SEW17NOBFAug2008 1.49277937
79 SEW17 Jul SEW17NOBFJul2008 1.49277937
80 SEW17 Sep SEW17NOBFSep2008 1.49277937
81 SEW17 Aug SEW17SOBFAug2008 1.49277937
82 SEW18 Aug SEW18SOBFAug2008 1.70247509
83 SEW19 Aug SEW19SOBFAug2008 2.11617036
84 SEW20 Jul SEW20SWBFJul2008 1.87718089
85 SEW20 Jul SEW20SOBFJul2008 1.87718089
86 SEW22 Aug SEW22NOBFAug2008 0.77473833
87 SEW23 Aug SEW23NWBFAug2008 0.96183454
88 SEW23 Aug SEW23NOBFAug2008 0.96183454
89 SEW24 Jul SEW24SWBFJul2008 0.64090368
90 SEW24 Jul SEW24NWBFJul2008 0.64090368
91 SEW29 Jul SEW29SOBFJul2008 1.54699664
92 SEW29 Aug SEW29SWBFAug2008 1.54699664
93 SEW29 Aug SEW29SOBFAug2008 1.54699664
94 SEW34 Aug SEW34NWBFAug2008 1.79425003
95 SEW36 Jul SEW36SOBFJul2008 1.20337761
96 SEW4 Aug SEW04SWBFAug2008 1.59611963
97 SEW40 Sep SEW40SOBFSep2008 1.36486039
98 SEW40 Aug SEW40SWBFAug2008 1.36486039
99 SEW43 Sep SEW43SOBFSep2008 1.03169382
100 SEW44 Aug SEW44SWBFAug2008 0.79705660
101 SEW45 Jul SEW45NWBFJul2008 0.34130398
102 SEW46 Aug SEW46SOBFAug2008 0.20690386
103 SEW47 Aug SEW47SWBFAug2008 0.01564703
104 SEW47 Sep SEW47SWBFSep2008 0.01564703
105 SEW48 Aug SEW48SWBFAug2008 0.46745254
106 SEW5 Aug SEW05SWBFAug2008 0.68900435
107 SEW50 Aug SEW50NWBFAug2008 1.10731406
108 SEW7 Aug SEW07SWBFAug2008 0.08552432
109 SEW8 Jul SEW08NWBFJul2008 0.18731374
The model I generated so far is: Mod1 <- lmer(Response ~ Month + (1|Tank), data=dat)
Again, I included "Tank" because we sampled some tanks in several months but that does not change the response variable. Consequently, the response variable is fixed for each tank. Nevertheless, multiple data points originate from the same tank and I tried to account for that by including it as a random factor.
Fitting Mod1 results in the following message:
Warning messages:
1: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
Model failed to converge with max|grad| = 0.306567 (tol = 0.002, component 1)
2: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
Model is nearly unidentifiable: very large eigenvalue
- Rescale variables?
The question is now, whether the model is overly complex and whether I can drop "Tank" as a random factor, as measuring tanks repeatedly did not have an effect on the response variable.
Thus, the question is, would a simple linear model Mod1 <- lm(Response ~ Month, data =dat) be valid? And if not, how can I solve the 2 convergence issues.
Any help is very much appreciated! :)
I am running the following R codes in Rstudio with the aim to convert a wide data frame (called 'merged') into a long one.
> merged
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2017 (A) 5980 5341 5890 5596 5753 5470 5589 5545 5749 5938 5844 5356
2017 (P) 5762 5275 5733 5411 5406 4954 5464 5536 5805 5819 5903 5630
I'm after the following output:
Description Month RN
2017 (A) Jan 5980
2017 (P) Jan 5762
2017 (A) Feb 5341
2017 (P) Feb 5275
... ... ...
I have tried the following (but with no success):
library(reshape2)
merged_long <- melt(data=merged,
id.vars="Description",
variable.name="Month",
value.name="RN")
I'm getting the following error message:
Error: id variables not found in data: Description
What am I doing wrong?
As noted by #Sotos in the comments, data in the rownames of the merged data set is required to uniquely identify an observation in the melted data set. To include the rownames in the melted data set, add the following to your code.
merged$Description <- rownames(merged)
Then your original code should produce the expected result.
library(reshape2)
merged_long <- melt(data=merged,
id.vars="Description",
variable.name="Month",
value.name="RN")
It's easiest to just use melt(as.matrix(...)) given the nature of your data. Omit the as.matrix part if your data is already a matrix, obviously.
melt(as.matrix(mydf))
You can use setNames to rename the columns at the same time:
setNames(melt(as.matrix(mydf)), c("Description", "Month", "RN"))
# Description Month RN
# 1 2017 (A) Jan 5980
# 2 2017 (P) Jan 5762
# 3 2017 (A) Feb 5341
# .........................
# .........................
# 23 2017 (A) Dec 5356
# 24 2017 (P) Dec 5630
I know that if I put this in my Makefile
a: b c
Then run make -j2 a then it will run b and c in parallel, but what if I always want b and c to run in parallel, even if the user doesn't include -j >= 2?
In my scenario, b and c are actually daemons that never complete, so I need them to run in parallel. I don't want them to run in the background either.
It's an odd use of make, but if you want to do it with make,
given that b and c are .PHONY, you could do so like:
.PHONY: a b c
a:
$(MAKE) -j2 b c
b c:
while [ "1" ]; do echo "$#: $$(date)"; sleep 1; done
A rule:
a: b c
isn't to the purpose. It says that a depends on b (being done) and
a depends on c (being done), but you don't want that at all. You
just want a to denote the concurrent execution of the recipes for
b and c.
You get:
$ make b
while [ "1" ]; do echo "b: $(date)"; sleep 1; done
b: Wed Mar 9 18:29:55 GMT 2016
b: Wed Mar 9 18:29:56 GMT 2016
b: Wed Mar 9 18:29:57 GMT 2016
b: Wed Mar 9 18:29:58 GMT 2016
b: Wed Mar 9 18:29:59 GMT 2016
^CMakefile:7: recipe for target 'b' failed
make: *** [b] Interrupt
$ make c
while [ "1" ]; do echo "c: $(date)"; sleep 1; done
c: Wed Mar 9 18:30:38 GMT 2016
c: Wed Mar 9 18:30:39 GMT 2016
c: Wed Mar 9 18:30:40 GMT 2016
c: Wed Mar 9 18:30:41 GMT 2016
c: Wed Mar 9 18:30:42 GMT 2016
^CMakefile:7: recipe for target 'c' failed
make: *** [c] Interrupt
$ make a
make -j2 b c
make[1]: Entering directory '/home/imk/develop/so/parallel_make'
while [ "1" ]; do echo "b: $(date)"; sleep 1; done
while [ "1" ]; do echo "c: $(date)"; sleep 1; done
b: Wed Mar 9 18:31:13 GMT 2016
c: Wed Mar 9 18:31:13 GMT 2016
b: Wed Mar 9 18:31:14 GMT 2016
c: Wed Mar 9 18:31:14 GMT 2016
b: Wed Mar 9 18:31:15 GMT 2016
c: Wed Mar 9 18:31:15 GMT 2016
b: Wed Mar 9 18:31:16 GMT 2016
c: Wed Mar 9 18:31:16 GMT 2016
b: Wed Mar 9 18:31:17 GMT 2016
c: Wed Mar 9 18:31:17 GMT 2016
^CMakefile:7: recipe for target 'b' failed
make[1]: *** [b] Interrupt
Makefile:7: recipe for target 'c' failed
make[1]: *** [c] Interrupt
Makefile:4: recipe for target 'a' failed
make: *** [a] Interrupt