How do I convert between decimal and binary? I'm working on a Solaris 10 platform
Decimal to Binary
4000000002 -> 100000000000000000000000000010
Binary to Decimal
100000000000000000000000000010 -> 4000000002
I used the following command in unix but it takes lot of time. I have 20 million records like this
For decimal to binary, set obase to 2:
echo 'obase=2;4000000002' | bc
For binary to decimal, set ibase to 2:
echo 'ibase=2;100000000000000000000000000010' | bc
If you are running bc once for each number that will be slow.
Can you not arrange for the data to be delivered to a file and input in one go?
Here's a simple illustration, starting with your numbers in the file called input.txt:
# To binary
$ ( echo 'obase=2;ibase=16;'; cat input.txt ) | bc | paste input.txt - > output.txt
# To hex
$ ( echo 'obase=16;ibase=2;'; cat input.txt ) | bc | paste input.txt - > output.txt
The results are written to the file output.txt.
The paste is included to produce a tab-spearated output result like
07 111
1A 11010
20 100000
2B 101011
35 110101
80 10000000
FF 11111111
showing input value versus output value.
If you just want the results you can omit the paste, e.g.:
$ ( echo 'obase=2;ibase=16;'; cat input.txt ) | bc > output.txt
Note that you probably have to set ibase as well as obase for the conversion to be correct.
gclswceap1d-mc48191-CRENG_DEV [/home/mc48191/scratch]
Related
I'm developing a script on which I have a hex string 31323334353637383930313233 and I want to transform it into ASCII. Desired output is 1234567890123.
I already have it working using:
echo "31323334353637383930313233" | xxd -r -p
or
echo "31323334353637383930313233" | perl -pe 's/(..)/chr(hex($1))/ge'
But the point is try to use the minimum possible requirements for the script. I want it working in suse, fedora, debian, ubuntu, arch, etc... It seems the xxd command is included in vim package. I'm wondering if there is a way to achieve this using only awk or any internal Linux tool which is going to be present by default in all Linux systems.
Found this script here:
#!/bin/bash
function hex2string () {
I=0
while [ $I -lt ${#1} ];
do
echo -en "\x"${1:$I:2}
let "I += 2"
done
}
hex2string "31323334353637383930313233"
echo
You may change the line hex2string "31323334353637383930313233" so that it takes the hex value from parameters, that is:
#!/bin/bash
function hex2string () {
I=0
while [ $I -lt ${#1} ];
do
echo -en "\x"${1:$I:2}
let "I += 2"
done
}
hex2string "$1"
echo
So when executed as:
./hexstring.sh 31323334353637383930313233
It will provide the desired ascii output.
NOTE: Can't test if it works in all Linux systems.
Using gawk, from HEX to ASCII
$ gawk '{
gsub(/../,"0x& ");
for(i=1;i<=NF;i++)
printf("%c", strtonum($i));
print ""
}' <<<"31323334353637383930313233"
1234567890123
Using any awk
$ cat hex2asc_anyawk.awk
BEGIN{
split("0 1 2 3 4 5 6 7 8 9 A B C D E F", d, / /)
for(i in d)Decimal[d[i]]=i-1
}
function hex2dec(hex, h,i,j,dec)
{
hex = toupper(hex);
i = length(hex);
while(i)
{
dec += Decimal[substr(hex,i,1)] * 16 ^ j++
i--
}
return dec;
}
{
gsub(/../,"& ");
for(i=1;i<=NF;i++)
printf("%d",hex2dec($i));
print ""
}
Execution
$ awk -f hex2asc_anyawk.awk <<<"31323334353637383930313233"
1234567890123
Explanation
Steps :
Get the decimal equivalent of hex from table.
Multiply every digit with 16 power of digit location.
Sum all the multipliers.
Example :
BEGIN{
# Here we created decimal conversion array, like above table
split("0 1 2 3 4 5 6 7 8 9 A B C D E F", d, / /)
for(i in d)Decimal[d[i]]=i-1
}
function hex2dec(hex, h,i,j,dec)
{
hex = toupper(hex); # uppercase conversion if any A,B,C,D,E,F
i = length(hex); # length of hex string
while(i)
{
# dec var where sum is stored
# substr(hex,i,1) gives 1 char from RHS
# multiply by 16 power of digit location
dec += Decimal[substr(hex,i,1)] * 16 ^ j++
i-- # decrement by 1
}
return dec;
}
{
# it modifies record
# suppose if given string is 31323334353637383930313233
# after gsub it becomes 31 32 33 34 35 36 37 38 39 30 31 32 33
# thus re-evaluate the fields
gsub(/../,"& ");
# loop through fields , NF gives no of fields
for(i=1;i<=NF;i++)
# convert from hex to decimal
# and print equivalent ASCII value
printf("%c",hex2dec($i));
# print newline char
print ""
}
Meaning of dec += Decimal[substr(hex,i,1)] * 16 ^ j++
dec += Decimal[substr(hex,i,1)] * 16 ^ j++
^ ^ ^
| | |
| | 2.Multiply every digit with 16 power of digit location.
| |
| 1.Gives decimal equivalent of hex
|
|
3. Sum all the multipliers
here's a special cheating trick for u - due to ingenuity of how they originally mapped decimal digits to bytes, their hex are all x3[0-9],
so therefore, if u already know they would decode out to digits and nothing else, here's a fast shortcut :
echo "31323334353637383930313233" |
mawk 'gsub("..","_&") + gsub("_3",_)^_'
1234567890123
if it's already URL-percent-encoded, then it's even simpler :
echo '%31%32%33%34%35%36%37%38%39%30%31%32%33' |
mawk 'gsub("%3",_)^_'
or
gawk ++NF FS='%3' OFS=
1234567890123
This specialized approach can handle hex of absolutely any arbitrary size, even for awks that don't have built-in support for bigints
TL;DR : don't "do math" when none is needed
Alternate (g)awk solution:
echo "31323334353637383930313233" | awk 'RT{printf "%c", strtonum("0x"RT)}' RS='[0-9]{2}'
How to show trailing zeros in integer or how to convert it to float in bc?
I know about scale, but it works only with floats:
$ echo "scale=3; 3/2" | bc
1.500
$ echo "scale=3; 1+1" | bc
2
I want to get result seems like 2.000. I guess it's not difficult to do it with sed, but I'm a novice at this.
Divide by 1 to convert to a number with fractional part:
$ echo "scale=3; 1+1/1" | bc
2.000
once a divide of any kind has been done, if that value ends up being part of the output, it will be printed with the specified number of digits.
They're not "floats" in bc -- they're "fixed-point" -- numbers with a fixed number of digits after the decimal point. Internally, they're just integers divided by a fixed power of 10 (set by the scale command)
echo "scale=3; $1+$2" | bc | sed 's/^[0-9]*$/&\.000/g'
It works fine for me:
$ echo "scale=3; 3/2" | bc | sed 's/^[0-9]*$/&\.000/g'
1.500
$ echo "scale=3; 3+2" | bc | sed 's/^[0-9]*$/&\.000/g'
5.000
How can I compare this 2 big files in unix.
I've already tried using 'grep -Fxvf file1.txt file2.txt | wc -l' but the output is 2,000,480 and when switching file1 and file2 the output is 1,999,999.
How can I get the output of '480' because that's what i am expecting.
I've also tried using diff/cmp commands but the output is too complicated.
I think you want an absolute value of a difference in line numbers in 2 files. You can achieve it easily with awk and get a decent result. You'd read numbers of lines in an array and later subtract the array values in the END block. For pure shell it'd have to get more complex. Imagine you get some test data generated (10 and 14 line files):
$ seq 1 10 > ten
$ seq 1 14 > fourteen
And then you do:
$ ( wc -l ten ; wc -l fourteen ) | awk '{ print $1}' | sort -rn | xargs -J % echo % - p | dc
The result:
4
But much better way would be do just do it in 3 lines (get word count for file1, then file2 and then subtract)
I'm running this on an AIX 6.1.
The intended purpose of this command is to display the following information in the following format:
GetUsedRAM:GetUsedSwap:CPU_0_System:CPU_0_User:…CPU_N_System:CPU_N_User
The command is composed of several sub commands:
echo `vmstat 1 2 | tr -s ' ' ':' | cut -d':' -f4,5,14-15 | tail -1 | sed 's/\([0-9]*:[0-9]*:\)\([0-9]*:[0-9]*\)/\1/'``mpstat -a 1 1 | tr -s ' ' '|' | head -8 | tail -4 | cut -d'|' -f 25,27 | awk -F "|" '{printf "%.0f:%.0f:",$2,$1}' | sed '$s/.$//'| sed -e "s/ \{1,\}$//"| awk '{int a[10];split($1, a,":");printf("%d:%d:%d:%d:%d:%d:%d:%d",a[0],a[1],a[2],a[3],a[4],a[5],a[6],a[7])}'`
Which I'll re format for clarity:
echo \
`vmstat 1 2 |
tr -s ' ' ':' |
cut -d':' -f4,5,14-15 |
tail -1 |
sed 's/\([0-9]*:[0-9]*:\)\([0-9]*:[0-9]*\)/\1/' \
` \
`mpstat -a 1 1 |
tr -s ' ' '|' |
head -8 |
tail -4 |
cut -d'|' -f 25,27 |
awk -F "|" '{printf "%.0f:%.0f:",$2,$1}' |
sed '$s/.$//' |
sed -e "s/ \{1,\}$//" |
awk '{int a[10];split($1, a,":");printf("%d:%d:%d:%d:%d:%d:%d:%d",a[0],a[1],a[2],a[3],a[4],a[5],a[6],a[7])}' \
`
I understand all of the tr, cut, head tail, and (roughly) vmstat/mpstat commands. The first sed is where I get lost, I've tried running the command in smaller segments and not quite sure why it seems to work as a whole but not when I truncate the command before the next tr.
I'm also not so sure on the awk command although I understand the premise vaguely, as a function allowing formatted output.
Similarly, I have a vague understanding of sed being a command allowing certain strings/characters being replaced in some file.
I'm not able to make out what this specific implementation in the above case is.
Could anyone provide some clarity or direction as to exactly what is happening at each sed and awk step within the context of the entire command?
Thanks for your help.
Simplification
This two simpler commands will get the exact same output:
# GetUsedRAM:GetUsedSwap:CPU_0_System:CPU_0_User:…CPU_N_System:CPU_N_User
# Select fields 4,5 of last line, and format with :
comm1=`vmstat 1 2 |
awk '$4~/[0-9]/{avm=$4;fre=$5} END{printf "%s:%s",avm,fre}'
`
# Select fields 27 (sy) and 25 (us) for four cpu, print as decimal.
comm2=`mpstat -A 1 1 |
awk -v firstline=6 -v cpus=4 '
BEGIN{start=firstline-1; end=firstline+cpus;}
NR>start && NR<end {printf( ":%d:%d", $27,$25)}'
`
echo "${comm1}${comm2}"
Description.
Description of original commands
The whole command is the concatenation of two commands.
The first command:
The output of the vmstat is shown in this link.
The columns 4 and 5 are 'avm' and 'fre'. The output in columns 14 and 15,
seem to be 'us' (user) and 'sy' (system). And I say seem as no output
from the user is available to confirm.
The first command
`vmstat 1 2 | # Execute the command vmstat.
tr -s ' ' ':' | # convert all spaces to colon (:).
cut -d':' -f4,5,14-15 | # select fields 4,5,14,and 15
tail -1 | # select last line.
sed 's/\([0-9]*:[0-9]*:\)\([0-9]*:[0-9]*\)/\1/' \ # See below.
`
The sed command selects inside braces all digits [0-9]* before a colon
repeated twice. And then again (without the last colon). That's the whole
string in two parts: « (dd:dd:)(dd:dd) » (d means digit).
And finally, it replaces such whole string by what was selected inside
the first braces /\1/.
All this complexity just removes fields 14 and 15 as selected by cut.
A simpler command with exactly the same output is:
Select fields 4,5 of last line, and format with (:).
`vmstat 1 2 | awk '
$4~/[0-9]/{avm=$4;fre=$5} END{printf "%s:%s:",avm,fre}'
`
The second command:
The output of mpstat -A is similar to this one from Linux.
And also similar to this AIX mpstat -d output.
However, the exact output of AIX 6.1 for mpstat -a (ALL) on the computer
used could have several variations. Anyway, guided by the intended final
output desired: CPU_0_System:CPU_0_User:…CPU_N_System:CPU_N_User.
It seems that the columns to be selected should be us (user) and sy
(sys) percent of time that used the cpu for all cpu in use,
which seem to be four on the computer measured.
The manual for AIX 6.1 mpstat is here.
It has a list of all the 40 columns that are presented when the option
-a ALL is used:
CPU min maj mpcs mpcr dev soft dec ph cs ics bound rq push
S3pull S3grd S0rd S1rd S2rd S3rd S4rd S5rd S3hrd S4hrd S5hrd
sysc us sy wa id pc %ec ilcs vlcs lcs %idon %bdon %istol %bstol %nsp
us and sy are listed as the fields 27 and 28, however the command presented
by the user selects fields number 25 and 27. Close but not the same. The
only way to confirm would be to receive the output of the command from the user.
For testing I will be using the output of mpstat 5 1 from here.
# mpstat 5 1
System configuration: lcpu=4 ent=1.0 mode=Uncapped
cpu min maj mpc int cs ics rq mig lpa sysc us sy wt id pc %ec lcs
0 4940 0 1 632 685 268 0 320 100 263924 42 55 0 4 0.57 35.1 277
1 990 0 3 1387 2234 805 0 684 100 130290 28 47 0 25 0.27 16.6 649
2 3943 0 2 531 663 223 0 389 100 276520 44 54 0 3 0.57 34.9 270
3 1298 0 2 1856 2742 846 0 752 100 82141 31 40 0 29 0.22 13.4 650
ALL 11171 0 8 4406 6324 2142 0 2145 100 752875 39 51 0 10 1.63 163.1 1846
The second command
`mpstat -A 1 1 | # execute command
tr -s ' ' '|' | # replace all spaces with (|).
head -8 | # select 8 first lines.
tail -4 | # select last four lines.
cut -d'|' -f 25,27 | # select fields 25 and 27
awk -F "|" '{printf "%.0f:%.0f:",$2,$1}' | # print the fields as integers.
sed '$s/.$//' | # on the last line ($), substitute the last character (.$) by nothing.
sed -e "s/ \{1,\}$//" | # remove trailing space(s).
awk '{
int a[10];
split($1, a,":");
printf("%d:%d:%d:%d:%d:%d:%d:%d",a[0],a[1],a[2],a[3],a[4],a[5],a[6],a[7])
}' \
`
About the int: For older versions of awk, calling a function without the parentheses is equivalent to call the function on $0. int is equivalent to int($0), which is not printed, nor used. The same happens to the value of a[10].
The split sets each value of the command in a[i]. Then, all values of a[i] are printed as decimals.
The equivalent, and way simpler is:
Command #2
`mpstat -A 1 1 |
awk -v firstline=6 -v cpus=4 '
BEGIN{start=firstline-1; end=firstline+cpus;}
NR>start && NR<end {printf( ":%d:%d", $27,$25)}'
`
i am trying to solve this bash script which reads an arithmetic expression from user and echoes it to the output screen with round up of 3 decimal places in the end.
sample input
5+50*3/20 + (19*2)/7
sample output
17.929
my code is
read x
echo "scale = 3; $x" | bc -l
when there is an input of
5+50*3/20 + (19*2)/7
**my output is **
17.928
which the machine wants it to be
17.929
and due to this i get the solution wrong. any idea ?
The key here is to be sure to use printf with the formatting spec of "%.3f" and printf will take care of doing the rounding as you wish, as long as "scale=4" for bc.
Here's a script that works:
echo -e "please enter math to calculate: \c"
read x
printf "%.3f\n" $(echo "scale=4;$x" | bc -l)
You can get an understanding of what is going on with the above solution, if you run this command at the commandline: echo "scale=4;5+50*3/20 + (19*2)/7" | bc the result will be 17.9285. When that result is provided to printf as an argument, the function takes into account the fourth decimal place and rounds up the value so that the formatted result displays with precisely three decimal places and with a value of 17.929.
Alternatively, this works, too without a pipe by redirecting the here document as input for bc, as follows which avoids creating a sub-shell:
echo -e "please enter math to calculate: \c"
read x
printf "%.3f\n" $(bc -l <<< "scale=4;$x")
You are not rounding the number, you are truncating it.
$ echo "5+50*3/20 + (19*2)/7" | bc -l
17.92857142857142857142
$ echo "scale = 3; 5+50*3/20 + (19*2)/7" | bc -l
17.928
The only way I know to round a number is using awk:
$ awk 'BEGIN { rounded = sprintf("%.3f", 5+50*3/20 + (19*2)/7); print rounded }'
17.929
So, in you example:
read x
awk 'BEGIN { rounded = sprintf("%.3f", $x; print rounded }'
I entirely agree with jherran that you are not rounding the number, you are truncating it. I would go on to say that scale is probably just not behaving at all the way you want it, possibly in a way that noone would want it to behave.
> x="5+50*3/20 + (19*2)/7"
> echo "$x" | bc -l
17.92857142857142857142
> echo "scale = 3; $x" | bc -l
17.928
Furthermore, because of the behaviour of scale, you are rounding each multiplication/division separately from the additions. Let me prove my point with some examples :
> echo "scale=0; 5/2" | bc -l
2
> echo "scale=0; 5/2 + 7/2" | bc -l
5
> echo "5/2 + 7/2" | bc -l
6.00000000000000000000
However scale without any operation doesn't work either. There is an ugly work-around :
> echo "scale=0; 5.5" | bc -l
5.5
> echo "scale=0; 5.5/1" | bc -l
5
So tow things come out of this.
If you want to use bc's scale, do it only for the final result already computed, and even then, beware.
Remember that rounding is the same as truncating a number + half of the desired precision.
Let us take the example of rounding to the nearest integer, if you add .5 to a number that should be rounded up, its integer part will take the next integer value and truncation will give the desired result. If that number should have been rounded down, then adding .5 will not change its integer value and truncation will yield the same result as when nothing was added.
Thus my solution follows :
> y=$(echo "$x" | bc -l)
> echo "scale=3; ($y+0.0005)/1" | bc -l # scale doesn't apply to the +, so we get the expected result
17.929
Again, note that the following doesn't work (as explained above), thus breaking it up in two operations is really needed :
> echo "scale=3; ($x+0.0005)/1" | bc -l
17.928