Related
I am trying to understand how autodependency is generated in makefiles in the given link, i cannot understand the following piece of code:
DEPDIR = .deps
df = $(DEPDIR)/$(*F)
SRCS = foo.c bar.c ...
%.o : %.c
#$(MAKEDEPEND); \
cp $(df).d $(df).P; \
sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < $(df).d >> $(df).P; \
rm -f $(df).d
$(COMPILE.c) -o $# $<
-include $(SRCS:%.c=$(DEPDIR)/%.P)
I got it from this link. I know it will generate dependency files but I am not able to understand what this line does:
sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < $(df).d >> $(df).P; \
Can somebody please explain me this code, so many wildcards give me butterflies, i am new to makefiles.
That is a number of distinct commands so break it up.
-e 's/#.*//'
Delete everything that starts with a # (Comments? Pre-processor directives?)
-e 's/^[^:]*: *//'
Delete everything up to a : on any that has : it.
-e 's/ *\\$$//'
Delete line-continuation slashes (and spaces before them) from the end of lines.
-e '/^$$/ d'
Delete any blank lines.
-e 's/$$/ :/'
Add : to the end of every line.
This adds explicit targets for every listed dependency file so that make "knows" how to build them to avoid the "No rule to make target" errors. The reasoning here is explained in your link in an earlier section.
Briefly, this creates a .P file with the original prerequisites list, then adds targets to it by taking each line, removing any existing target information and any line continuation (\) characters, then adding a target separator (:) to the end. This works with the values for MAKEDEPEND I suggest below; it’s possible you will need to modify the translation for other dependency generators you might use.
This is not meant to be an answer to your actual question, but since you said you were new to GNU make, I think spreading the words that a simpler way to handle auto dependencies exists won't do any harm.
Nowadays compilers like GCC or Clang can do this for you while compiling your code !
Simply pass them a preprocessor flag:
# Preprocessor flags
CPPFLAGS += -MMD
And include the generated files into the Makefile:
-include $(wildcard *.d)
And you're done.
You can learn more about the preprocessor options here for GCC, Clang simply mirror these options.
A relatively good example lies here too.
I have a file that contains the version name and version number. The contents of the first file looks as-
File1-
<Line contains the name of product1>
package_name0_9_8 >= 1.2.3x-4.5.6
package_name0_9_8-32bit >= 3.6.1g-3.5.1
package_name0_9_8-xx >= 6.3.2v-3.0.4
<Line contains the name of product2>
anotherpackage_name0_9_8 >= 3.5.6u-3.6.5
And,
File2.xml-
<package name="package_name0_9_8" version="1.2.3x-4.4.4"/>
<package name="package_name0_9_8-32bit" version="3.6.1g-3.4.0"/>
.
.
Is there a way to check the existance of package_name that is present in File1 with the package_name of File2 and check if the corresponding version of package_name in File1 with that of corresponding version of package_name of File2?
I am frank that I am pretty much weak in concatenating the 'grep' and 'awk' commands along with options to be used here. Please help out.
for a in $(sed -n '/>=/p' File1.txt | grep -o '^[^ ]*'); do for b in $(sed -n "/^$a /{s/.*>=\(.*\)$/\1/p}" File1.txt); do ((! $(grep -c "$a.*$b" File2.txt))) && (echo "$a $b" >> missing_pkgs.txt); done; done;
this is a quick one liner - you could print it out a bit prettier
the way this works is nested for loop that grabs both pieces separate into variables (you could do that with read and put them in on one loop if you want) and then just counts the occurences in the second file with grep and whenever there is a count of zero it will reverse the value making the test (()) turn true and echo the missing packages to the file missing_pkgs.txt
here is another quick one liner that does the same thing except more efficient with one loop and variables loaded via read
while read each; do read a b < <(echo $each) && ((! $(grep -c "$a.*$b" File2.txt))) && (echo "$a $b" >> missing_pkgs.txt); done < <(awk '/>=/{ print $1" "$3 }' File1.txt)
more simplified:
while read a b; do ((! $(grep -c "$a.*$b" File2.txt))) && (echo "$a $b" >> missing_pkgs.txt); done < <(awk '/>=/{ print $1" "$3 }' File1.txt)
sed -n 's².*²s#<package name="\\(&"/>#\\1 Present#p²;s/ *>= */\\)" *version="/p' File1 > /tmp/File1.sed
sed -n -f /tmp/File1.sed File2
rm /tmp/File1.sed
not in on instruction like awk could do, but do the job (posix version so --posix on GNU sed
you could change the output message that is the \\1 Present text where \\1 will the be the package name (with few modification, version could also be used)
It looks like you already got a much shorter solution in a format closer to what you desired. However, since I asked if a Python solution would work, and you said yes, check out the code here:
http://pastebin.com/F5LYrmea
(I haven't debugged it more than a little, but it seems to work on at least a little more than your example files. I released the code to the public domain. CC-BY-SA isn't a software license, according to the makers of CC; so, that's why I didn't post it here, as posting it here would give it that license. Plus, you get syntax highlighting specific to Python at the link provided.)
Basically, it's a lot of complicated text parsing. Not much of an algorithm to explain. It gets the contents of both files, strips out the packages, their versions and the operands (puts all those in a dictionary for use later), and loops through lines of the other file and compares versions; then it tells you which ones match and which ones don't.
When I have a diff, how can I colorize it so that it looks good?
I want it for the command line, so please no GUI solutions.
Man pages for diff suggest no solution for colorization from within itself. Please consider using colordiff. It's a wrapper around diff that produces the same output as diff, except that it augments the output using colored syntax highlighting to increase readability:
diff old new | colordiff
or just:
colordiff old new
Installation:
Ubuntu/Debian: sudo apt-get install colordiff
OS X: brew install colordiff or port install colordiff
Use Vim:
diff /path/to/a /path/to/b | vim -R -
Or better still, VimDiff (or vim -d, which is shorter to type) will show differences between two, three or four files side-by-side.
Examples:
vim -d /path/to/[ab]
vimdiff file1 file2 file3 file4
Actually there seems to be yet another option (which I only noticed recently, when running into the problem described above):
git diff --no-index <file1> <file2>
# output to console instead of opening a pager
git --no-pager diff --no-index <file1> <file2>
If you have Git around (which you already might be using anyway), then you will be able to use it for comparison, even if the files themselves are not under version control. If not enabled for you by default, then enabling color support here seems to be considerably easier than some of the previously mentioned workarounds.
diff --color option (added to GNU diffutils 3.4 in 2016-08-08)
This is the default diff implementation on most distributions, which will soon be getting it.
Ubuntu 18.04 (Bionic Beaver) has diffutils 3.6 and therefore has it.
On 3.5 it looks like this:
Tested with:
diff --color -u \
<(seq 6 | sed 's/$/ a/') \
<(seq 8 | grep -Ev '^(2|3)$' | sed 's/$/ a/')
Apparently added in commit c0fa19fe92da71404f809aafb5f51cfd99b1bee2 (Mar 2015).
Word-level diff
Like diff-highlight. It is not possible it seems, but there is a feature request: https://lists.gnu.org/archive/html/diffutils-devel/2017-01/msg00001.html
Related questions:
Using 'diff' (or anything else) to get character-level diff between text files
https://unix.stackexchange.com/questions/11128/diff-within-a-line
https://superuser.com/questions/496415/using-diff-on-a-long-one-line-file
ydiff does it though. See below.
ydiff side-by-side word level diff
https://github.com/ymattw/ydiff
Is this nirvana?
python3 -m pip install --user ydiff
diff -u a b | ydiff -s
Outcome:
If the lines are too narrow (default 80 columns), fit to the screen with:
diff -u a b | ydiff -w 0 -s
Contents of the test files:
a
1
2
3
4
5 the original line the original line the original line the original line
6
7
8
9
10
11
12
13
14
15 the original line the original line the original line the original line
16
17
18
19
20
b
1
2
3
4
5 the original line the original line the original line the original line
6
7
8
9
10
11
12
13
14
15 the original line the original line the original line the original line
16
17
18
19
20
ydiff Git integration
ydiff integrates with Git without any configuration required.
From inside a Git repository, instead of git diff, you can do just:
ydiff -s
and instead of git log:
ydiff -ls
See also: How can I get a side-by-side diff when I do "git diff"?
Tested on Ubuntu 16.04 (Xenial Xerus), Git 2.18.0, and ydiff 1.1.
And for those occasions when a yum install colordiff or an apt-get install colordiff is not an option due to some insane constraint beyond your immediate control, or you're just feeling crazy, you can reinvent the wheel with a line of sed:
sed 's/^-/\x1b[41m-/;s/^+/\x1b[42m+/;s/^#/\x1b[34m#/;s/$/\x1b[0m/'
Throw that in a shell script and pipe unified diff output through it.
It makes hunk markers blue and highlights new/old filenames and added/removed lines in green and red background, respectively.1 And it will make trailing space2 changes more readily apparent than colordiff can.
1 Incidentally, the reason for highlighting the filenames the same as the modified lines is that to correctly differentiate between the filenames and the modified lines requires properly parsing the diff format, which is not something to tackle with a regex. Highlighting them the same works "well enough" visually and makes the problem trivial. That said, there are some interesting subtleties.
2 But not trailing tabs. Apparently tabs don't get their background set, at least in my xterm. It does make tab vs. space changes stand out a bit though.
Coloured, word-level diff ouput
Here's what you can do with the the below script and diff-highlight:
#!/bin/sh -eu
# Use diff-highlight to show word-level differences
diff -U3 --minimal "$#" |
sed 's/^-/\x1b[1;31m-/;s/^+/\x1b[1;32m+/;s/^#/\x1b[1;34m#/;s/$/\x1b[0m/' |
diff-highlight
(Credit to #retracile's answer for the sed highlighting)
You can change the Subversion configuration to use colordiff:
~/.subversion/config.diff
### Set diff-cmd to the absolute path of your 'diff' program.
### This will override the compile-time default, which is to use
### Subversion's internal diff implementation.
-# diff-cmd = diff_program (diff, gdiff, etc.)
+diff-cmd = colordiff
via: https://gist.github.com/westonruter/846524
I use grc (Generic Colouriser), which allows you to colour the output of a number of commands including diff.
It is a Python script which can be wrapped around any command. So instead of invoking diff file1 file2, you would invoke grc diff file1 file2 to see colourised output. I have aliased diff to grc diff to make it easier.
Here is another solution that invokes sed to insert the appropriate ANSI escape sequences for colors to show the +, -, and # lines in red, green, and cyan, respectively.
diff -u old new | sed "s/^-/$(tput setaf 1)&/; s/^+/$(tput setaf 2)&/; s/^#/$(tput setaf 6)&/; s/$/$(tput sgr0)/"
Unlike the other solutions to this question, this solution does not spell out the ANSI escape sequences explicitly. Instead, it invokes the tput setaf and tput sgr0 commands to generate the ANSI escape sequences to set an appropriate color and reset terminal attributes, respectively.
To see the available colors for each argument to tput setaf, use this command:
for i in {0..255}; do tput setaf $i; printf %4d $i; done; tput sgr0; echo
Here is how the output looks:
Here is the evidence that the tput setaf and tput sgr0 commands generate the appropriate ANSI escape sequences:
$ tput setaf 1 | xxd -g1
00000000: 1b 5b 33 31 6d .[31m
$ tput setaf 2 | xxd -g1
00000000: 1b 5b 33 32 6d .[32m
$ tput setaf 6 | xxd -g1
00000000: 1b 5b 33 36 6d .[36m
$ tput sgr0 | xxd -g1
00000000: 1b 28 42 1b 5b 6d .(B.[m
Since wdiff accepts arguments specifying the string at the beginning and end of both insertions and deletions, you can use ANSI color sequences as those strings:
wdiff -n -w $'\033[30;41m' -x $'\033[0m' -y $'\033[30;42m' -z $'\033[0m' file1 file2
For example, this is the output of comparing two CSV files:
Example from 2.2 Actual examples of wdiff usage.
I would suggest you to give diff-so-fancy a try. I use it during my work and it sure seems great as of now. It comes packed with many options and it's really easy to configure your diffs the way you want.
You can install it by:
sudo npm install -g diff-so-fancy
or on Mac:
brew install diff-so-fancy
Afterwards, you can highlight your diffs like this:
diff -u file1 file2 | diff-so-fancy
Character-level color diff:
Install ccdiff
ccdiff -r /usr/share/dict/words /tmp/new-dict
No one has mentioned delta so far. It supports syntax colored diff view with syntax highlighting.
With the bat command:
diff file1 file2 | bat -l diff
On recent versions of Git on Ubuntu, you can enable diff-highlighting with:
sudo ln -s /usr/share/doc/git/contrib/diff-highlight/diff-highlight /usr/local/bin
sudo chmod a+x /usr/share/doc/git/contrib/diff-highlight/diff-highlight
And then adding this to your .gitconfig file:
[pager]
log = diff-highlight | less
show = diff-highlight | less
diff = diff-highlight | less
It's possible the script is located somewhere else in other distributions. You can use locate diff-highlight to find out where.
My favorite choice is vdiff <file1> <file2> function (I forgot from where I got it).
It will open two windows in Vim side-by-side, to see clearly see the difference between the two files.
vdiff () {
if [ "${#}" -ne 2 ] ; then
echo "vdiff requires two arguments"
echo " comparing dirs: vdiff dir_a dir_b"
echo " comparing files: vdiff file_a file_b"
return 1
fi
local left="${1}"
local right="${2}"
if [ -d "${left}" ] && [ -d "${right}" ]; then
vim +"DirDiff ${left} ${right}"
else
vim -d "${left}" "${right}"
fi
}
Put this script in your (.alias) or (.zshrc), and then call it using
vdiff <file1> <file2>.
Example
The results are:
diff --color=always file_a file_b | less
works for me
For me I found some solutions: it is a working solution
#echo off
Title a game for YouTube
explorer "https://thepythoncoding.blogspot.com/2020/11/how-to-echo-with-different-colors-in.html"
SETLOCAL EnableDelayedExpansion
for /F "tokens=1,2 delims=#" %%a in ('"prompt #$H#$E# & echo on & for %%b in (1) do rem"') do (
set "DEL=%%a"
)
echo say the name of the colors, don't read
call :ColorText 0a "blue"
call :ColorText 0C "green"
call :ColorText 0b "red"
echo(
call :ColorText 19 "yellow"
call :ColorText 2F "black"
call :ColorText 4e "white"
goto :Beginoffile
:ColorText
echo off
<nul set /p ".=%DEL%" > "%~2"
findstr /v /a:%1 /R "^$" "%~2" nul
del "%~2" > nul 2>&1
goto :eof
:Beginoffile
The version of diff in my cygwin has a number of advanced options which allow me to print out one difference per line.
Given two files one.txt and two.txt.
one.txt:
one
two
three
four
five
six
two.txt
one
two2
three
four
five5
six
And running diff in cygwin with the following options/parameters:
diff -y --suppress-common-lines one.txt two.txt
Gives an output of:
two |two2
five |five5
This is the type of format I'm after whereby one difference is printed out per line.
On my dev solaris box, the "-y" option is not supported, so I'm stuck with an output which looks like this:
2c2
< two
---
> two2
5c5
< five
---
> five5
Does anyone know of a way I can get an output of one difference per line on this solaris box? Maybe using a sed/awk one liner to massage the output from this more primitive diff output? (Please note, I am not able to install a more up-to-date diff version on this solaris box).
Thanks!
Use GNU diff.
http://www.gnu.org/software/diffutils/
You can build and install it into your local directory, no? If you have a home directory and a compiler and a make, you can build your own GNU diff.
I don't have Solaris, but I can't imagine it would be much more than this:
./configure --prefix=/home/bob
make
make install
No root privileges required.
comm -3 almost does what you want, but requires sorted input. It also will put them in separate lines by alphabetical order. Your example (once sorted) would show up as
five
five5
two
two2
If solaris diff won't do what you want, then nothing on a standard solaris box is liable to do so either, which means introducing code from elsewhere, either your own or someone else's. As GNU diff does what you want, just use that.
Example output:
# ~/config_diff.sh postfix_DIST/master.cf postfix/master.cf
postfix_DIST/master.cf: -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o smtpd_tls_wrappermode=yes -o smtp_fallback_relay=
postfix/master.cf: -o cleanup_service_name=cleanup_sasl -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o cleanup_service_name=cleanup_sasl -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o smtp_fallback_relay=
postfix_DIST/master.cf:smtp inet n - - - - smtpd smtp unix - - - - - smtp
postfix/master.cf:smtp inet n - - - - smtpd smtp unix - - - - - smtp
Sadly, it cannot currently handle several same configurations variables ... it counts them and will think the the files differ.
All the answers given above and below are perfect but just typing a command and getting a result wont help you in solving similar problems in future.
Here a link which explains how diff works. once you go through the link, you can the problem yourself
Here is a link. https://www.youtube.com/watch?v=5_dyVrvbWjc
#! /bin/bash
FILES="$#"
COLUMN=1
for variable in $( awk -F: '{print $X}' X=${COLUMN} ${FILES} | sort -u ) ; do
NUM_CONFIGS=$( for file in ${FILES} ; do
grep "^${variable}" ${file}
done | sort -u | wc -l )
if [ "${NUM_CONFIGS}" -ne 1 ] ; then
for file in ${FILES} ; do
echo ${file}:$( grep "^${variable}" ${file} )
done
echo
fi
done
I ended up writing a quick little script for this in Python, but I was wondering if there was a utility you could feed text into which would prepend each line with some text -- in my specific case, a timestamp. Ideally, the use would be something like:
cat somefile.txt | prepend-timestamp
(Before you answer sed, I tried this:
cat somefile.txt | sed "s/^/`date`/"
But that only evaluates the date command once when sed is executed, so the same timestamp is incorrectly prepended to each line.)
ts from moreutils will prepend a timestamp to every line of input you give it. You can format it using strftime too.
$ echo 'foo bar baz' | ts
Mar 21 18:07:28 foo bar baz
$ echo 'blah blah blah' | ts '%F %T'
2012-03-21 18:07:30 blah blah blah
$
To install it:
sudo apt-get install moreutils
Could try using awk:
<command> | awk '{ print strftime("%Y-%m-%d %H:%M:%S"), $0; fflush(); }'
You may need to make sure that <command> produces line buffered output, i.e. it flushes its output stream after each line; the timestamp awk adds will be the time that the end of the line appeared on its input pipe.
If awk shows errors, then try gawk instead.
annotate, available via that link or as annotate-output in the Debian devscripts package.
$ echo -e "a\nb\nc" > lines
$ annotate-output cat lines
17:00:47 I: Started cat lines
17:00:47 O: a
17:00:47 O: b
17:00:47 O: c
17:00:47 I: Finished with exitcode 0
Distilling the given answers to the simplest one possible:
unbuffer $COMMAND | ts
On Ubuntu, they come from the expect-dev and moreutils packages.
sudo apt-get install expect-dev moreutils
How about this?
cat somefile.txt | perl -pne 'print scalar(localtime()), " ";'
Judging from your desire to get live timestamps, maybe you want to do live updating on a log file or something? Maybe
tail -f /path/to/log | perl -pne 'print scalar(localtime()), " ";' > /path/to/log-with-timestamps
Kieron's answer is the best one so far. If you have problems because the first program is buffering its out you can use the unbuffer program:
unbuffer <command> | awk '{ print strftime("%Y-%m-%d %H:%M:%S"), $0; }'
It's installed by default on most linux systems. If you need to build it yourself it is part of the expect package
http://expect.nist.gov
Just gonna throw this out there: there are a pair of utilities in daemontools called tai64n and tai64nlocal that are made for prepending timestamps to log messages.
Example:
cat file | tai64n | tai64nlocal
Use the read(1) command to read one line at a time from standard input, then output the line prepended with the date in the format of your choosing using date(1).
$ cat timestamp
#!/bin/sh
while read line
do
echo `date` $line
done
$ cat somefile.txt | ./timestamp
I'm not an Unix guy, but I think you can use
gawk '{print strftime("%d/%m/%y",systime()) $0 }' < somefile.txt
#! /bin/sh
unbuffer "$#" | perl -e '
use Time::HiRes (gettimeofday);
while(<>) {
($s,$ms) = gettimeofday();
print $s . "." . $ms . " " . $_;
}'
$ cat somefile.txt | sed "s/^/`date`/"
you can do this (with gnu/sed):
$ some-command | sed "x;s/.*/date +%T/e;G;s/\n/ /g"
example:
$ { echo 'line1'; sleep 2; echo 'line2'; } | sed "x;s/.*/date +%T/e;G;s/\n/ /g"
20:24:22 line1
20:24:24 line2
of course, you can use other options of the program date. just replace date +%T with what you need.
Here's my awk solution (from a Windows/XP system with MKS Tools installed in the C:\bin directory). It is designed to add the current date and time in the form mm/dd hh:mm to the beginning of each line having fetched that timestamp from the system as each line is read. You could, of course, use the BEGIN pattern to fetch the timestamp once and add that timestamp to each record (all the same). I did this to tag a log file that was being generated to stdout with the timestamp at the time the log message was generated.
/"pattern"/ "C\:\\\\bin\\\\date '+%m/%d %R'" | getline timestamp;
print timestamp, $0;
where "pattern" is a string or regex (without the quotes) to be matched in the input line, and is optional if you wish to match all input lines.
This should work on Linux/UNIX systems as well, just get rid of the C\:\\bin\\ leaving the line
"date '+%m/%d %R'" | getline timestamp;
This, of course, assumes that the command "date" gets you to the standard Linux/UNIX date display/set command without specific path information (that is, your environment PATH variable is correctly configured).
Mixing some answers above from natevw and Frank Ch. Eigler.
It has milliseconds, performs better than calling a external date command each time and perl can be found in most of the servers.
tail -f log | perl -pne '
use Time::HiRes (gettimeofday);
use POSIX qw(strftime);
($s,$ms) = gettimeofday();
print strftime "%Y-%m-%dT%H:%M:%S+$ms ", gmtime($s);
'
Alternative version with flush and read in a loop:
tail -f log | perl -pne '
use Time::HiRes (gettimeofday); use POSIX qw(strftime);
$|=1;
while(<>) {
($s,$ms) = gettimeofday();
print strftime "%Y-%m-%dT%H:%M:%S+$ms $_", gmtime($s);
}'
caerwyn's answer can be run as a subroutine, which would prevent the new processes per line:
timestamp(){
while read line
do
echo `date` $line
done
}
echo testing 123 |timestamp
Disclaimer: the solution I am proposing is not a Unix built-in utility.
I faced a similar problem a few days ago. I did not like the syntax and limitations of the solutions above, so I quickly put together a program in Go to do the job for me.
You can check the tool here: preftime
There are prebuilt executables for Linux, MacOS, and Windows in the Releases section of the GitHub project.
The tool handles incomplete output lines and has (from my point of view) a more compact syntax.
<command> | preftime
It's not ideal, but I though I'd share it in case it helps someone.
The other answers mostly work, but have some drawbacks. In particular:
Many require installing a command not commonly found on linux systems, which may not be possible or convenient.
Since they use pipes, they don't put timestamps on stderr, and lose the exit status.
If you use multiple pipes for stderr and stdout, then some do not have atomic printing, leading to intermingled lines of output like [timestamp] [timestamp] stdout line \nstderr line
Buffering can cause problems, and unbuffer requires an extra dependency.
To solve (4), we can use stdbuf -i0 -o0 -e0 which is generally available on most linux systems (see How to make output of any shell command unbuffered?).
To solve (3), you just need to be careful to print the entire line at a time.
Bad: ruby -pe 'print Time.now.strftime(\"[%Y-%m-%d %H:%M:%S] \")' (Prints the timestamp, then prints the contents of $_.)
Good: ruby -pe '\$_ = Time.now.strftime(\"[%Y-%m-%d %H:%M:%S] \") + \$_' (Alters $_, then prints it.)
To solve (2), we need to use multiple pipes and save the exit status:
alias tslines-pipe="stdbuf -i0 -o0 ruby -pe '\$_ = Time.now.strftime(\"[%Y-%m-%d %H:%M:%S] \") + \$_'"
function tslines() (
stdbuf -o0 -e0 "$#" 2> >(tslines-pipe) > >(tslines-pipe)
status="$?"
exit $status
)
Then you can run a command with tslines some command --options.
This almost works, except sometimes one of the pipes takes slightly longer to exit and the tslines function has exited, so the next prompt has printed. For example, this command seems to print all the output after the prompt for the next line has appeared, which can be a bit confusing:
tslines bash -c '(for (( i=1; i<=20; i++ )); do echo stderr 1>&2; echo stdout; done)'
There needs to be some coordination method between the two pipe processes and the tslines function. There are presumably many ways to do this. One way I found is to have the pipes send some lines to a pipe that the main function can listen to, and only exit after it's received data from both pipe handlers. Putting that together:
alias tslines-pipe="stdbuf -i0 -o0 ruby -pe '\$_ = Time.now.strftime(\"[%Y-%m-%d %H:%M:%S] \") + \$_'"
function tslines() (
# Pick a random name for the pipe to prevent collisions.
pipe="/tmp/pipe-$RANDOM"
# Ensure the pipe gets deleted when the method exits.
trap "rm -f $pipe" EXIT
# Create the pipe. See https://www.linuxjournal.com/content/using-named-pipes-fifos-bash
mkfifo "$pipe"
# echo will block until the pipe is read.
stdbuf -o0 -e0 "$#" 2> >(tslines-pipe; echo "done" >> $pipe) > >(tslines-pipe; echo "done" >> $pipe)
status="$?"
# Wait until we've received data from both pipe commands before exiting.
linecount=0
while [[ $linecount -lt 2 ]]; do
read line
if [[ "$line" == "done" ]]; then
((linecount++))
fi
done < "$pipe"
exit $status
)
That synchronization mechanism feels a bit convoluted; hopefully there's a simpler way to do it.
doing it with date and tr and xargs on OSX:
alias predate="xargs -I{} sh -c 'date +\"%Y-%m-%d %H:%M:%S\" | tr \"\n\" \" \"; echo \"{}\"'"
<command> | predate
if you want milliseconds:
alias predate="xargs -I{} sh -c 'date +\"%Y-%m-%d %H:%M:%S.%3N\" | tr \"\n\" \" \"; echo \"{}\"'"
but note that on OSX, date doesn't give you the %N option, so you'll need to install gdate (brew install coreutils) and so finally arrive at this:
alias predate="xargs -I{} sh -c 'gdate +\"%Y-%m-%d %H:%M:%S.%3N\" | tr \"\n\" \" \"; echo \"{}\"'"
No need to specify all the parameters in strftime() unless you really want to customize the outputting format :
echo "abc 123 xyz\njan 765 feb" \
\
| gawk -Sbe 'BEGIN {_=strftime()" "} sub("^",_)'
Sat Apr 9 13:14:53 EDT 2022 abc 123 xyz
Sat Apr 9 13:14:53 EDT 2022 jan 765 feb
works the same if you have mawk 1.3.4. Even on awk-variants without the time features, a quick getline could emulate it :
echo "abc 123 xyz\njan 765 feb" \
\
| mawk2 'BEGIN { (__="date")|getline _;
close(__)
_=_" " } sub("^",_)'
Sat Apr 9 13:19:38 EDT 2022 abc 123 xyz
Sat Apr 9 13:19:38 EDT 2022 jan 765 feb
If you wanna skip all that getline and BEGIN { }, then something like this :
mawk2 'sub("^",_" ")' \_="$(date)"
If the value you are prepending is the same on every line, fire up emacs with the file, then:
Ctrl + <space>
at the beginning of the of the file (to mark that spot), then scroll down to the beginning of the last line (Alt + > will go to the end of file... which probably will involve the Shift key too, then Ctrl + a to go to the beginning of that line) and:
Ctrl + x r t
Which is the command to insert at the rectangle you just specified (a rectangle of 0 width).
2008-8-21 6:45PM <enter>
Or whatever you want to prepend... then you will see that text prepended to every line within the 0 width rectangle.
UPDATE: I just realized you don't want the SAME date, so this won't work... though you may be able to do this in emacs with a slightly more complicated custom macro, but still, this kind of rectangle editing is pretty nice to know about...