Toy Example:
prog.sh:
#!/usr/bin/bash
echo $*
makefile:
%.sh_log: %.sh
sh $< $(OPTARGS) > $# 2>&1
targ1: OPTARGS=hello world
targ1: prog.sh_log
targ2: OPTARGS=goodbye
targ2: targ1 prog.sh_log
When I make targ2, targ1 is executed as expected but since the result of that is prog.sh_log, the dependency for targ2 is satisfied and it does not fire. Since make is based on timestamps, how would one use the same program with different parameters in separate parts of the makefile. My real world program is a multistep program which takes the steps numbers as command line parameters. Due to changes in source systems, I have added other programs between steps.
If you don't want the command to be run only once, dont make it a target in the makefile:
targ1: OPTARGS=hello world
targ1:
sh prog.sh $(OPTARGS) > prog.sh.log 2>&1
echo > $#
targ2: OPTARGS=goodbye
targ2: targ1
sh prog.sh $(OPTARGS) > prog.sh.log 2>&1
echo > $#
I have a Bash shell script which takes three input files as an argument. I would like to package them all, so I can place that package on any UNIX machines and run it.
There is an old and ancient technique to include a tar file in a executable well known to the grey haired admins ;)
At first create a script ... put it into a file named script
#!/bin/bash
TAR_STARTS=`awk '/^__TARMAN BEGINS__/ { print NR + 1; exit 0; }' $0`
NAME_OF_SCRIPT=`pwd`/$0
tail +$TAR_STARTS $NAME_OF_SCRIPT | gunzip -c | tar -xvf -
# Insert commands to execute after untaring here .... with relative
# pathname (e.g. if there is directory "bin" with a file "executable"
# then you should insert a line bin/executable
exit
__TARMAN BEGINS__
No newline after the last __
Of course this script is derived from somewhere in the internet. It's not mine. I just cannot remember where for proper kudos.
Then create your tarfile and put it at the end of the file. This is the reason why it's nescessary that there is no newline after the __
$ cat script test.tar.gz > selfexploding.sh
Now you can just try it
$ bash ./selfexploding.sh
tar: blocksize = 9
x testtar, 0 bytes, 0 tape blocks
x testtar/test2, 1024 bytes, 2 tape blocks
x testtar/test1, 1024 bytes, 2 tape blocks
You could of course put the name of a script before the exit, that you create by unpack ... of course path must be relative to the pwd of the execution. Don't know if this works with AIX. At least with Solaris 11.3 it works. But as it only uses standard command. It should work everywhere. Besides of this you could of course create native packages for Solaris and AIX.
I am trying to understand how autodependency is generated in makefiles in the given link, i cannot understand the following piece of code:
DEPDIR = .deps
df = $(DEPDIR)/$(*F)
SRCS = foo.c bar.c ...
%.o : %.c
#$(MAKEDEPEND); \
cp $(df).d $(df).P; \
sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < $(df).d >> $(df).P; \
rm -f $(df).d
$(COMPILE.c) -o $# $<
-include $(SRCS:%.c=$(DEPDIR)/%.P)
I got it from this link. I know it will generate dependency files but I am not able to understand what this line does:
sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < $(df).d >> $(df).P; \
Can somebody please explain me this code, so many wildcards give me butterflies, i am new to makefiles.
That is a number of distinct commands so break it up.
-e 's/#.*//'
Delete everything that starts with a # (Comments? Pre-processor directives?)
-e 's/^[^:]*: *//'
Delete everything up to a : on any that has : it.
-e 's/ *\\$$//'
Delete line-continuation slashes (and spaces before them) from the end of lines.
-e '/^$$/ d'
Delete any blank lines.
-e 's/$$/ :/'
Add : to the end of every line.
This adds explicit targets for every listed dependency file so that make "knows" how to build them to avoid the "No rule to make target" errors. The reasoning here is explained in your link in an earlier section.
Briefly, this creates a .P file with the original prerequisites list, then adds targets to it by taking each line, removing any existing target information and any line continuation (\) characters, then adding a target separator (:) to the end. This works with the values for MAKEDEPEND I suggest below; it’s possible you will need to modify the translation for other dependency generators you might use.
This is not meant to be an answer to your actual question, but since you said you were new to GNU make, I think spreading the words that a simpler way to handle auto dependencies exists won't do any harm.
Nowadays compilers like GCC or Clang can do this for you while compiling your code !
Simply pass them a preprocessor flag:
# Preprocessor flags
CPPFLAGS += -MMD
And include the generated files into the Makefile:
-include $(wildcard *.d)
And you're done.
You can learn more about the preprocessor options here for GCC, Clang simply mirror these options.
A relatively good example lies here too.
When I have a diff, how can I colorize it so that it looks good?
I want it for the command line, so please no GUI solutions.
Man pages for diff suggest no solution for colorization from within itself. Please consider using colordiff. It's a wrapper around diff that produces the same output as diff, except that it augments the output using colored syntax highlighting to increase readability:
diff old new | colordiff
or just:
colordiff old new
Installation:
Ubuntu/Debian: sudo apt-get install colordiff
OS X: brew install colordiff or port install colordiff
Use Vim:
diff /path/to/a /path/to/b | vim -R -
Or better still, VimDiff (or vim -d, which is shorter to type) will show differences between two, three or four files side-by-side.
Examples:
vim -d /path/to/[ab]
vimdiff file1 file2 file3 file4
Actually there seems to be yet another option (which I only noticed recently, when running into the problem described above):
git diff --no-index <file1> <file2>
# output to console instead of opening a pager
git --no-pager diff --no-index <file1> <file2>
If you have Git around (which you already might be using anyway), then you will be able to use it for comparison, even if the files themselves are not under version control. If not enabled for you by default, then enabling color support here seems to be considerably easier than some of the previously mentioned workarounds.
diff --color option (added to GNU diffutils 3.4 in 2016-08-08)
This is the default diff implementation on most distributions, which will soon be getting it.
Ubuntu 18.04 (Bionic Beaver) has diffutils 3.6 and therefore has it.
On 3.5 it looks like this:
Tested with:
diff --color -u \
<(seq 6 | sed 's/$/ a/') \
<(seq 8 | grep -Ev '^(2|3)$' | sed 's/$/ a/')
Apparently added in commit c0fa19fe92da71404f809aafb5f51cfd99b1bee2 (Mar 2015).
Word-level diff
Like diff-highlight. It is not possible it seems, but there is a feature request: https://lists.gnu.org/archive/html/diffutils-devel/2017-01/msg00001.html
Related questions:
Using 'diff' (or anything else) to get character-level diff between text files
https://unix.stackexchange.com/questions/11128/diff-within-a-line
https://superuser.com/questions/496415/using-diff-on-a-long-one-line-file
ydiff does it though. See below.
ydiff side-by-side word level diff
https://github.com/ymattw/ydiff
Is this nirvana?
python3 -m pip install --user ydiff
diff -u a b | ydiff -s
Outcome:
If the lines are too narrow (default 80 columns), fit to the screen with:
diff -u a b | ydiff -w 0 -s
Contents of the test files:
a
1
2
3
4
5 the original line the original line the original line the original line
6
7
8
9
10
11
12
13
14
15 the original line the original line the original line the original line
16
17
18
19
20
b
1
2
3
4
5 the original line the original line the original line the original line
6
7
8
9
10
11
12
13
14
15 the original line the original line the original line the original line
16
17
18
19
20
ydiff Git integration
ydiff integrates with Git without any configuration required.
From inside a Git repository, instead of git diff, you can do just:
ydiff -s
and instead of git log:
ydiff -ls
See also: How can I get a side-by-side diff when I do "git diff"?
Tested on Ubuntu 16.04 (Xenial Xerus), Git 2.18.0, and ydiff 1.1.
And for those occasions when a yum install colordiff or an apt-get install colordiff is not an option due to some insane constraint beyond your immediate control, or you're just feeling crazy, you can reinvent the wheel with a line of sed:
sed 's/^-/\x1b[41m-/;s/^+/\x1b[42m+/;s/^#/\x1b[34m#/;s/$/\x1b[0m/'
Throw that in a shell script and pipe unified diff output through it.
It makes hunk markers blue and highlights new/old filenames and added/removed lines in green and red background, respectively.1 And it will make trailing space2 changes more readily apparent than colordiff can.
1 Incidentally, the reason for highlighting the filenames the same as the modified lines is that to correctly differentiate between the filenames and the modified lines requires properly parsing the diff format, which is not something to tackle with a regex. Highlighting them the same works "well enough" visually and makes the problem trivial. That said, there are some interesting subtleties.
2 But not trailing tabs. Apparently tabs don't get their background set, at least in my xterm. It does make tab vs. space changes stand out a bit though.
Coloured, word-level diff ouput
Here's what you can do with the the below script and diff-highlight:
#!/bin/sh -eu
# Use diff-highlight to show word-level differences
diff -U3 --minimal "$#" |
sed 's/^-/\x1b[1;31m-/;s/^+/\x1b[1;32m+/;s/^#/\x1b[1;34m#/;s/$/\x1b[0m/' |
diff-highlight
(Credit to #retracile's answer for the sed highlighting)
You can change the Subversion configuration to use colordiff:
~/.subversion/config.diff
### Set diff-cmd to the absolute path of your 'diff' program.
### This will override the compile-time default, which is to use
### Subversion's internal diff implementation.
-# diff-cmd = diff_program (diff, gdiff, etc.)
+diff-cmd = colordiff
via: https://gist.github.com/westonruter/846524
I use grc (Generic Colouriser), which allows you to colour the output of a number of commands including diff.
It is a Python script which can be wrapped around any command. So instead of invoking diff file1 file2, you would invoke grc diff file1 file2 to see colourised output. I have aliased diff to grc diff to make it easier.
Here is another solution that invokes sed to insert the appropriate ANSI escape sequences for colors to show the +, -, and # lines in red, green, and cyan, respectively.
diff -u old new | sed "s/^-/$(tput setaf 1)&/; s/^+/$(tput setaf 2)&/; s/^#/$(tput setaf 6)&/; s/$/$(tput sgr0)/"
Unlike the other solutions to this question, this solution does not spell out the ANSI escape sequences explicitly. Instead, it invokes the tput setaf and tput sgr0 commands to generate the ANSI escape sequences to set an appropriate color and reset terminal attributes, respectively.
To see the available colors for each argument to tput setaf, use this command:
for i in {0..255}; do tput setaf $i; printf %4d $i; done; tput sgr0; echo
Here is how the output looks:
Here is the evidence that the tput setaf and tput sgr0 commands generate the appropriate ANSI escape sequences:
$ tput setaf 1 | xxd -g1
00000000: 1b 5b 33 31 6d .[31m
$ tput setaf 2 | xxd -g1
00000000: 1b 5b 33 32 6d .[32m
$ tput setaf 6 | xxd -g1
00000000: 1b 5b 33 36 6d .[36m
$ tput sgr0 | xxd -g1
00000000: 1b 28 42 1b 5b 6d .(B.[m
Since wdiff accepts arguments specifying the string at the beginning and end of both insertions and deletions, you can use ANSI color sequences as those strings:
wdiff -n -w $'\033[30;41m' -x $'\033[0m' -y $'\033[30;42m' -z $'\033[0m' file1 file2
For example, this is the output of comparing two CSV files:
Example from 2.2 Actual examples of wdiff usage.
I would suggest you to give diff-so-fancy a try. I use it during my work and it sure seems great as of now. It comes packed with many options and it's really easy to configure your diffs the way you want.
You can install it by:
sudo npm install -g diff-so-fancy
or on Mac:
brew install diff-so-fancy
Afterwards, you can highlight your diffs like this:
diff -u file1 file2 | diff-so-fancy
Character-level color diff:
Install ccdiff
ccdiff -r /usr/share/dict/words /tmp/new-dict
No one has mentioned delta so far. It supports syntax colored diff view with syntax highlighting.
With the bat command:
diff file1 file2 | bat -l diff
On recent versions of Git on Ubuntu, you can enable diff-highlighting with:
sudo ln -s /usr/share/doc/git/contrib/diff-highlight/diff-highlight /usr/local/bin
sudo chmod a+x /usr/share/doc/git/contrib/diff-highlight/diff-highlight
And then adding this to your .gitconfig file:
[pager]
log = diff-highlight | less
show = diff-highlight | less
diff = diff-highlight | less
It's possible the script is located somewhere else in other distributions. You can use locate diff-highlight to find out where.
My favorite choice is vdiff <file1> <file2> function (I forgot from where I got it).
It will open two windows in Vim side-by-side, to see clearly see the difference between the two files.
vdiff () {
if [ "${#}" -ne 2 ] ; then
echo "vdiff requires two arguments"
echo " comparing dirs: vdiff dir_a dir_b"
echo " comparing files: vdiff file_a file_b"
return 1
fi
local left="${1}"
local right="${2}"
if [ -d "${left}" ] && [ -d "${right}" ]; then
vim +"DirDiff ${left} ${right}"
else
vim -d "${left}" "${right}"
fi
}
Put this script in your (.alias) or (.zshrc), and then call it using
vdiff <file1> <file2>.
Example
The results are:
diff --color=always file_a file_b | less
works for me
For me I found some solutions: it is a working solution
#echo off
Title a game for YouTube
explorer "https://thepythoncoding.blogspot.com/2020/11/how-to-echo-with-different-colors-in.html"
SETLOCAL EnableDelayedExpansion
for /F "tokens=1,2 delims=#" %%a in ('"prompt #$H#$E# & echo on & for %%b in (1) do rem"') do (
set "DEL=%%a"
)
echo say the name of the colors, don't read
call :ColorText 0a "blue"
call :ColorText 0C "green"
call :ColorText 0b "red"
echo(
call :ColorText 19 "yellow"
call :ColorText 2F "black"
call :ColorText 4e "white"
goto :Beginoffile
:ColorText
echo off
<nul set /p ".=%DEL%" > "%~2"
findstr /v /a:%1 /R "^$" "%~2" nul
del "%~2" > nul 2>&1
goto :eof
:Beginoffile
The version of diff in my cygwin has a number of advanced options which allow me to print out one difference per line.
Given two files one.txt and two.txt.
one.txt:
one
two
three
four
five
six
two.txt
one
two2
three
four
five5
six
And running diff in cygwin with the following options/parameters:
diff -y --suppress-common-lines one.txt two.txt
Gives an output of:
two |two2
five |five5
This is the type of format I'm after whereby one difference is printed out per line.
On my dev solaris box, the "-y" option is not supported, so I'm stuck with an output which looks like this:
2c2
< two
---
> two2
5c5
< five
---
> five5
Does anyone know of a way I can get an output of one difference per line on this solaris box? Maybe using a sed/awk one liner to massage the output from this more primitive diff output? (Please note, I am not able to install a more up-to-date diff version on this solaris box).
Thanks!
Use GNU diff.
http://www.gnu.org/software/diffutils/
You can build and install it into your local directory, no? If you have a home directory and a compiler and a make, you can build your own GNU diff.
I don't have Solaris, but I can't imagine it would be much more than this:
./configure --prefix=/home/bob
make
make install
No root privileges required.
comm -3 almost does what you want, but requires sorted input. It also will put them in separate lines by alphabetical order. Your example (once sorted) would show up as
five
five5
two
two2
If solaris diff won't do what you want, then nothing on a standard solaris box is liable to do so either, which means introducing code from elsewhere, either your own or someone else's. As GNU diff does what you want, just use that.
Example output:
# ~/config_diff.sh postfix_DIST/master.cf postfix/master.cf
postfix_DIST/master.cf: -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o smtpd_tls_wrappermode=yes -o smtp_fallback_relay=
postfix/master.cf: -o cleanup_service_name=cleanup_sasl -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o cleanup_service_name=cleanup_sasl -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o smtp_fallback_relay=
postfix_DIST/master.cf:smtp inet n - - - - smtpd smtp unix - - - - - smtp
postfix/master.cf:smtp inet n - - - - smtpd smtp unix - - - - - smtp
Sadly, it cannot currently handle several same configurations variables ... it counts them and will think the the files differ.
All the answers given above and below are perfect but just typing a command and getting a result wont help you in solving similar problems in future.
Here a link which explains how diff works. once you go through the link, you can the problem yourself
Here is a link. https://www.youtube.com/watch?v=5_dyVrvbWjc
#! /bin/bash
FILES="$#"
COLUMN=1
for variable in $( awk -F: '{print $X}' X=${COLUMN} ${FILES} | sort -u ) ; do
NUM_CONFIGS=$( for file in ${FILES} ; do
grep "^${variable}" ${file}
done | sort -u | wc -l )
if [ "${NUM_CONFIGS}" -ne 1 ] ; then
for file in ${FILES} ; do
echo ${file}:$( grep "^${variable}" ${file} )
done
echo
fi
done