I am trying to reverse engineer a Linux box to flash my own firmware, but to do that I need to patch a binary file. The patch is actually quite simple, I just need to alter one byte at a known offset. However, the Linux box doesn't have any programs like dd, sed, awk, etc. Not even telnet (I am communicating with it over serial). However, it does have sh. Is there a way to replace a byte at a known offset just using shell commands?
Thank you.
If you have head -c (the -c option is not specified by posix), printf and tail available, then this should work:
file=pathToYourFile
address=1234 # `address=1` changes the first byte
newByteOctal=567
{
head -c "$((address-1))" "$file"
printf "\\0$newByteOctal"
tail -c "+$((address+1))" "$file"
} > patchedFiled
If you don't have head -c, but the file is very small and does not (!) contain any null byte before the offset, then you can replace head -c with
printf "%.$((offset1based-1))s" "$(< "$file")"
my zsh shell has a quite long start up time and if I hold done the enter key the terminal prompt lags behind significantly. So I looked out for solutions and profiling options and found this github issue. Further done one comment suggests to run the following commands:
zsh -xv 2>&1 | ts -i "%.s" > zsh_startup.log
followed by
sort --field-separator=' ' -r -k1 zsh_startup.log> sorted.log
which gives me the following output:
2.106722 ESC[?1hESC=ESC[?1lESC>l
1.388755 ESC[?1hESC=ESC[?1lESC>l
1.185498 ESC[?1hESC=ESC[?1lESC>exit
1.153527 ESC[?1hESC=ESC[?1lESC>ls -la
0.065941 +powerline_precmd:1> PS1=$'%{\C-[[38;5;15m%}%{\C-[[48;5;31m%} ~ %{\C-[[48;5;237m%}%{\C-[[38;5;31m%}%{\C-[[38;5;250m%}%{\C-[[48;5;237m%} Private %{\C-[[48;5;237m%}%{\C-[[38;5;244m%}%{\C-[[38;5;254m%}%{\C-[[48;5;237m%} tmp %{\C-[[48;5;238m%}%{\C-[[38;5;237m%}%{\C-[[38;5;39m%}%{\C-[[48;5;238m%} 1 %{\C-[[48;5;236m%}%{\C-[[38;5;238m%}%{\C-[[38;5;15m%}%{\C-[[48;5;236m%} %# %{\C-[[0m%}%{\C-[[38;5;236m%}%{\C-[[0m%} '
0.052396 +powerline_precmd:1> PS1=$'%{\C-[[38;5;15m%}%{\C-[[48;5;31m%} ~ %{\C-[[48;5;237m%}%{\C-[[38;5;31m%}%{\C-[[38;5;250m%}%{\C-[[48;5;237m%} Private %{\C-[[48;5;237m%}%{\C-[[38;5;244m%}%{\C-[[38;5;254m%}%{\C-[[48;5;237m%} tmp %{\C-[[48;5;238m%}%{\C-[[38;5;237m%}%{\C-[[38;5;39m%}%{\C-[[48;5;238m%} 1 %{\C-[[48;5;236m%}%{\C-[[38;5;238m%}%{\C-[[38;5;15m%}%{\C-[[48;5;236m%} %# %{\C-[[0m%}%{\C-[[38;5;236m%}%{\C-[[0m%} '
...
It looks like the first four commands (or whatever the lines represent) are taking up the majority of the time. I'm having a hard time understanding the output and would like to have some guidance in interpreting the first lines - no need for solving the underlying problem of having a slow shell :) just the interpretation of the first four lines.
The file itself is much longer and I just included the first few lines.
Notes:
I'm using oh-my-zsh with the following plugins
plugins=(
z
git
kubectl
zsh-syntax-highlighting
zsh-autosuggestions
)
I configured l to be an alias for l -la in my .zshrc
this is what happens when I keep pressing the enter key:
(when there are no gaps between prompts I had already released the enter key again)
I am trying to get the target directory for modules in a multimodule project. The challenge I have is that SBT's logging makes it hard to consume in a script.
Here is what I have at the moment:
function sbt-target {
sbt -Dsbt.log.noformat=true "project $1" 'show target' |
tail -n1 |
cut -c8-
}
I think this is very hackish as it knows about the [INFO] prefix (the cut -c8-) of each output line from SBT and about the fact that SBT's last line is the output I need (the tail -n1).
More problematic is that each invocation of sbt-target takes almost 11 seconds so invoking it for each module for a large number of modules in this project dominates the time.
How do I get the target directory in a script?
I can't speak to SBT. In terms of bash best-practices, you might consider something more akin to the following:
sbt_target() {
# declare locals as such
local line version
# iterate through all lines; later lines overwrite variable set by prior ones
while read -r line; do
version=${line#"[INFO] "} # strip undesired prefix if present
done < <(sbt -Dsbt.log.noformat=true "project $1" 'show target')
# emit result to stdout
printf '%s\n' "$version"
}
Unlike the version relying on tail and cut, this does everything in-process within bash, and is thus more efficient (presuming that sbt's show target emits a relatively small amount of output).
The version of diff in my cygwin has a number of advanced options which allow me to print out one difference per line.
Given two files one.txt and two.txt.
one.txt:
one
two
three
four
five
six
two.txt
one
two2
three
four
five5
six
And running diff in cygwin with the following options/parameters:
diff -y --suppress-common-lines one.txt two.txt
Gives an output of:
two |two2
five |five5
This is the type of format I'm after whereby one difference is printed out per line.
On my dev solaris box, the "-y" option is not supported, so I'm stuck with an output which looks like this:
2c2
< two
---
> two2
5c5
< five
---
> five5
Does anyone know of a way I can get an output of one difference per line on this solaris box? Maybe using a sed/awk one liner to massage the output from this more primitive diff output? (Please note, I am not able to install a more up-to-date diff version on this solaris box).
Thanks!
Use GNU diff.
http://www.gnu.org/software/diffutils/
You can build and install it into your local directory, no? If you have a home directory and a compiler and a make, you can build your own GNU diff.
I don't have Solaris, but I can't imagine it would be much more than this:
./configure --prefix=/home/bob
make
make install
No root privileges required.
comm -3 almost does what you want, but requires sorted input. It also will put them in separate lines by alphabetical order. Your example (once sorted) would show up as
five
five5
two
two2
If solaris diff won't do what you want, then nothing on a standard solaris box is liable to do so either, which means introducing code from elsewhere, either your own or someone else's. As GNU diff does what you want, just use that.
Example output:
# ~/config_diff.sh postfix_DIST/master.cf postfix/master.cf
postfix_DIST/master.cf: -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o smtpd_tls_wrappermode=yes -o smtp_fallback_relay=
postfix/master.cf: -o cleanup_service_name=cleanup_sasl -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o cleanup_service_name=cleanup_sasl -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject -o smtp_fallback_relay=
postfix_DIST/master.cf:smtp inet n - - - - smtpd smtp unix - - - - - smtp
postfix/master.cf:smtp inet n - - - - smtpd smtp unix - - - - - smtp
Sadly, it cannot currently handle several same configurations variables ... it counts them and will think the the files differ.
All the answers given above and below are perfect but just typing a command and getting a result wont help you in solving similar problems in future.
Here a link which explains how diff works. once you go through the link, you can the problem yourself
Here is a link. https://www.youtube.com/watch?v=5_dyVrvbWjc
#! /bin/bash
FILES="$#"
COLUMN=1
for variable in $( awk -F: '{print $X}' X=${COLUMN} ${FILES} | sort -u ) ; do
NUM_CONFIGS=$( for file in ${FILES} ; do
grep "^${variable}" ${file}
done | sort -u | wc -l )
if [ "${NUM_CONFIGS}" -ne 1 ] ; then
for file in ${FILES} ; do
echo ${file}:$( grep "^${variable}" ${file} )
done
echo
fi
done
Is it possible when listing a directory to view numerical Unix permissions such as 644, rather than the symbolic output -rw-rw-r-- ?
Thanks.
it almost can ..
ls -l | awk '{k=0;for(i=0;i<=8;i++)k+=((substr($1,i+2,1)~/[rwx]/) \
*2^(8-i));if(k)printf("%0o ",k);print}'
Closest I can think of (keeping it simple enough) is stat, assuming you know which files you're looking for. If you don't, * can find most of them:
/usr/bin$ stat -c '%a %n' *
755 [
755 a2p
755 a2ps
755 aclocal
...
It handles sticky, suid and company out of the box:
$ stat -c '%a %n' /tmp /usr/bin/sudo
1777 /tmp
4755 /usr/bin/sudo
you can just use GNU find.
find . -printf "%m:%f\n"
You can use the following command
stat -c "%a %n" *
Also you can use any filename or directoryname instead of * to get a specific result.
On Mac, you can use
stat -f '%A %N' *
Use this to display the Unix numerical permission values (octal values) and file name.
stat -c '%a %n' *
Use this to display the Unix numerical permission values (octal values) and the folder's sgid and sticky bit, user name of the owner, group name, total size in bytes and file name.
stat -c '%a %A %U %G %s %n' *
Add %y if you need time of last modification in human-readable format. For more options see stat.
Better version using an Alias
Using an alias is a more efficient way to accomplish what you need and it also includes color. The following displays your results organized by group directories first, display in color, print sizes in human readable format (e.g., 1K 234M 2G) edit your ~/.bashrc and add an alias for your account or globally by editing /etc/profile.d/custom.sh
Typing cls displays your new LS command results.
alias cls="ls -lha --color=always -F --group-directories-first |awk '{k=0;s=0;for(i=0;i<=8;i++){;k+=((substr(\$1,i+2,1)~/[rwxst]/)*2^(8-i));};j=4;for(i=4;i<=10;i+=3){;s+=((substr(\$1,i,1)~/[stST]/)*j);j/=2;};if(k){;printf(\"%0o%0o \",s,k);};print;}'"
Folder Tree
While you are editing your bashrc or custom.sh include the following alias to see a graphical representation where typing lstree will display your current folder tree structure
alias lstree="ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'"
It would display:
|-scripts
|--mod_cache_disk
|--mod_cache_d
|---logs
|-run_win
|-scripts.tar.gz
#The MYYN
wow, nice awk! But what about suid, sgid and sticky bit?
You have to extend your filter with s and t, otherwise they will not count and you get the wrong result. To calculate the octal number for this special flags, the procedure is the same but the index is at 4 7 and 10. the possible flags for files with execute bit set are ---s--s--t amd for files with no execute bit set are ---S--S--T
ls -l | awk '{
k = 0
s = 0
for( i = 0; i <= 8; i++ )
{
k += ( ( substr( $1, i+2, 1 ) ~ /[rwxst]/ ) * 2 ^( 8 - i ) )
}
j = 4
for( i = 4; i <= 10; i += 3 )
{
s += ( ( substr( $1, i, 1 ) ~ /[stST]/ ) * j )
j/=2
}
if ( k )
{
printf( "%0o%0o ", s, k )
}
print
}'
For test:
touch blah
chmod 7444 blah
will result in:
7444 -r-Sr-Sr-T 1 cheko cheko 0 2009-12-05 01:03 blah
and
touch blah
chmod 7555 blah
will give:
7555 -r-sr-sr-t 1 cheko cheko 0 2009-12-05 01:03 blah
You don't use ls to get a file's permission information. You use the stat command. It will give you the numerical values you want. The "Unix Way" says that you should invent your own script using ls (or 'echo *') and stat and whatever else you like to give the information in the format you desire.
Building off of the chosen answer and the suggestion to use an alias, I converted it to a function so that passing a directory to list is possible.
# ls, with chmod-like permissions and more.
# #param $1 The directory to ls
function lls {
LLS_PATH=$1
ls -AHl $LLS_PATH | awk "{k=0;for(i=0;i<=8;i++)k+=((substr(\$1,i+2,1)~/[rwx]/) \
*2^(8-i));if(k)printf(\"%0o \",k);print}"
}
Solution
It is strange that still nobody mentioned the (quote) "modern replacement for ls" - an alternative and quite powerful tool exa.
You can easily achieve the desired output by using exa command along with the -l (which is equivalent to ls's -l) and the --octal-permissions options.
Example
Here is a simple example of listing the contents of a user's root directory (/) on a macOS machine using exa command and the --octal-permissions option:
exa -lh --octal-permissions /
Result:
Notice how besides the nice colorful output exa can also show the headers for each column thanks to the -h option (long form is --header).
Read man exa or the official online documentation for more information about how to customize the desired output according to your specific needs.
Considering the question specifies UNIX, not Linux, use of a stat binary is not necessary. The solution below works on a very old UNIX, though a shell other than sh (i.e. bash) was necessary. It is a derivation of glenn jackman's perl stat solution. It seems like an alternative worth exploring for conciseness.
$ alias lls='llsfn () { while test $# -gt 0; do perl -s -e \
'\''#fields = stat "$f"; printf "%04o\t", $fields[2] & 07777'\'' \
-- -f=$1; ls -ld $1; shift; done; unset -f llsf; }; llsfn'
$ lls /tmp /etc/resolv.conf
1777 drwxrwxrwt 7 sys sys 246272 Nov 5 15:10 /tmp
0644 -rw-r--r-- 1 bin bin 74 Sep 20 23:48 /etc/resolv.conf
The alias was developed using information in this answer
The whole answer is a modified version of a solution in this answer
After reading MANY answers here, following links provided in comments back to the original UNIX way of doing this, and while wanting to combined what was offered here, as well as the tips and tricks I've already learned, I came up with a new solution.
First off, I used to use this alias, to give me column headers:
alias l='echo "Dir Size|Perms|Link Count|Owner|Group|Size|Mod. Time|Name"; ls -haFl --time-style=long-iso --color=always --group-directories-first --format=long'
After combining this, with AWK, I first learned I had to alter the awk command, when using the "-s" option for ls, as this shows size in the first column, and you need to then read and parse the second (no longer first) column of data.
The ISSUE I found, was when you then provide input to " l " i.e., you are at a path, and type: l, fine, but what if you type "l" then a directory name? what if you want to list out everything in a subdirectory? This wasn't able to happen, as the alias did not handle input. I was able to handle this with a function. Then AWk broke, which I was able to handle with a sub-alias.
Combining the two worked perfectly.
Added to my .bashrc file
function _bestLS() {
echo 'MODE|Dir Size|Perms|Link Count|Owner|Group|Size|Mod. Time|Name';
if [ "$*" == '' ]; then
alias _awk4ls="awk '{k=0;s=0;for(i=0;i<=8;i++){;k+=((substr(\$2,i+2,1)~/[rwxst]/)*2^(8-i));};j=4;for(i=4;i<=10;i+=3){;s+=((substr(\$1,i,1)~/[stST]/)*j);j/=2;};if(k){;printf(\"%0o%0o \",s,k);};print;}'";
output=`ls -shaFl --time-style=long-iso --color=always -F --group-directories-first --format=long | _awk4ls`;
else
output=`ls -shaFl --time-style=long-iso --color=always -F --group-directories-first --format=long $* | _awk4ls`;
fi
echo "$output"
}
alias l="_bestLS"
Observe (please note, it is a lower case letter L, not a numeric 1, which look the same in this site's font):
>>l
>>l [SOME DIRECTORY]
P.S. Please excuse my very long (3 line) prompt (PS1)