I'm using jq 1.4 and am confused about the following situation. I can calculate a number, but get an error when I try to construct an object with this number:
echo '{"aggregations":{"sent":{"value":25},"bounced":{"value":null},"incoming_act":{"value":25}}}' |
jq '.aggregations
| {"num_sent": .sent.value, "num_incoming_act": .incoming_act.value }
| .num_sent as $x
| .num_incoming_act as $y
| $y-$x as $d
| $d'
0
works fine. But
echo '{"aggregations":{"sent":{"value":25},"bounced":{"value":null},"incoming_act":{"value":25}}}' |
jq '.aggregations
| {"num_sent": .sent.value, "num_incoming_act": .incoming_act.value }
| .num_sent as $x
| .num_incoming_act as $y
| $y-$x as $d
| {diff: $d}'
jq: error: number and object cannot be subtracted
doesn't work. Same happens when I ask for objects in the last part:
echo '{"aggregations":{"sent":{"value":25},"bounced":{"value":null},"incoming_act":{"value":25}}}' |
jq '.aggregations
| {"num_sent": .sent.value, "num_incoming_act": .incoming_act.value }
| .num_sent as $x
| .num_incoming_act as $y
| $y-$x as $d
| objects'
jq: error: number and object cannot be subtracted
I love jq's pipe system. However, something seems to be going on here. What is the "0" that I get in the first example? It doesn't seem to be a normal number 0. This works again:
jq -n ' 0 as $x | {diff: $x} '
This
echo '{"aggregations":{"sent":{"value":25},"bounced":{"value":null},"incoming_act":{"value":12}}}' | jq '.aggregations | {"num_sent": .sent.value, "num_incoming_act": .incoming_act.value } | {diff:(.num_sent as $x | .num_incoming_act as $y | $y-$x as $d | $d)}'
Will Produce:
{
"diff": -13
}
Difference being here;
Previous: .num_sent as $x | .num_incoming_act as $y | $y-$x as $d | {diff: $d}'
Now: {diff:(.num_sent as $x | .num_incoming_act as $y | $y-$x as $d | $d)}'
You can probably see by visualising the difference, where jq is processing things.
In the examples where you get an error, write ($y-$x) as $d rather than just $y-$x as $d. The parentheses are sometimes necessary, and always advisable, when writing (COMPOUND INFIX EXPRESSION) as $variable.
Explanation:
The parser treats expressions of the form:
3-2 as $d | EXPR
as:
3-(2 as $d | EXPR)
This means that 3-2 as $d|$d is parsed as 3-(2 as $d|$d) which evaluates to 3-2. Notice, though, that in this case, $d itself has the value 2.
Related
I have been trying to understand jq, and the following piuzzle is giving me a headache: I can construct two expressions, A and B, which seem to produce the same output. And yet, when I surround them with [] array construction braces (as in [A] and [B]), they produce different output. In this case, the expressions are:
A := jq '. | add'
B := jq -s `.[] | add`
Concretely:
$ echo '[1,2] [3,4]' | jq '.'
[1,2]
[3,4]
$ echo '[1,2] [3,4]' | jq '. | add'
3
7
# Now surround with array construction and we get two values:
$ echo '[1,2] [3,4]' | jq '[. | add]'
[3]
[7]
$ echo '[1,2] [3,4]' | jq -s '.[]'
[1,2]
[3,4]
$ echo '[1,2] [3,4]' | jq -s '.[] | add'
3
7
# Now surround with array construction and we get only one value:
$ echo '[1,2] [3,4]' | jq -s '[.[] | add]'
[3,7]
What is going on here? Why is it that the B expression, which applies the --slurp setting but appears to produce identical intermediate output to the A expression, produces different output when surrounded with [] array construction brackets?
When jq is fed with a stream, just like [1,2] [3,4] with two inputs, it executes the filter independently for each. That's why jq '[. | add]' will produce two results; each addition will separately be wrapped into an array.
When jq is given the --slurp option, it combines the stream to an array, rendering it just one input. Therefore jq -s '[.[] | add]' will have one result only; the multiple additions will be caught by the array constructor, which is executed just once.
I have a very large file (around 70GB), and I want to list all strings that appear more than once in the whole file.
I can list all the matches when I specify which string to search in a file, but I want to list all strings that have more than one occurrence.
For example, assuming my file looks like this:
+------+------------------------------------------------------------------+----------------------------------+--+
| HHID | VAL_CD64 | VAL_CD32 | |
+------+------------------------------------------------------------------+----------------------------------+--+
| 203 | 8c5bfd9b6755ffcdb85dc52a701120e0876640b69b2df0a314dc9e7c2f8f58a5 | 373aeda34c0b4ab91a02ecf55af58e15 | |
| 7AB | f6c581becbac4ec1291dc4b9ce566334b1cb2c85e234e489e7fd5e1393bd8751 | 2c4f97a04f02db5a36a85f48dab39b5b | |
| 7AB | abad845107a699f5f99575f8ed43e0440d87a8fc7229c1a1db67793561f0f1c3 | 2111293e946703652070968b224875c9 | |
| 348 | 25c7cf022e6651394fa5876814a05b8e593d8c7f29846117b8718c3dd951e496 | 5c80a555fcda02d028fc60afa29c4a40 | |
| 348 | 67d9c0a4bb98900809bcfab1f50bef72b30886a7b48ff0e9eccf951ef06542f9 | 6c10cd11b805fa57d2ca36df91654576 | |
| 348 | 05f1e412e7765c4b54a9acfd70741af545564f6fdfe48b073bfd3114640f5e37 | 6040b29107adf1a41c4f5964e0ff6dcb | |
| 4D3 | 3e8da3d63c51434bcd368d6829c7cee490170afc32b5137be8e93e7d02315636 | 71a91c4768bd314f3c9dc74e9c7937e8 | |
+------+------------------------------------------------------------------+----------------------------------+--+
And I want to list only records which have HHID more than once, i.e, 7AB and 348.
Any idea how can I implement this?
awk to the rescue:
awk -F'[ |]+' '
$2 ~ /^[[:alnum:]]+$/ { count[$2]++ }
END {
for (hhid in count) {
if (count[hhid] >= 2) {
print hhid
}
}
}
' file
-F'[ |]+' sets the field separator.
$2 ~ /^[[:alnum:]]+$/ filters out the header and horizontal lines.
count[$2]++ increases the value at $2, the string we’re counting. On the first occurrence this initialises the value to 1. On the second occurrence it increases it to 2, and so on.
END is run after all lines have been processed.
for (hhid in count) iterates over the strings in count.
if (count[hhid] >= 2) skips any <2 counts.
print hhid prints the string.
I'm trying to adapt the following snippet:
echo '{"a":{"value":"b"}, "c":{"value":"d"}}' \
| jq -r '. as $in | keys[] | [$in[.].value | tostring + " 1"] | #tsv'
b 1
d 1
to output:
b 1
b 2
d 1
d 2
The following adaptation produces the desired output:
echo '{"a":{"value":"b"}, "c":{"value":"d"}}' |
jq -r '
def addindex(start;lessthan):
range(start;lessthan) as $i | "\(.) \($i)";
. as $in
| keys[]
| $in[.].value
| addindex(1;3)'
Note that keys emits the key names after they have been sorted, whereas keys_unsorted retains the ordering.
I am trying to execute the below
if [[ $1 == 'R' ]]
then
echo "Running the recovery steps..."
for i in 1 2 3 4 5 6
do
head -${i} cons.txt | tail -1 | read -r r${i}f1 r${i}f2 r${i}f3 r${i}f4 r${i}f5 r${i}f6 r${i}f7 r${i}f8 r${i}f9;
if (( ${Time} >= ${r${i}f1} && ${Time} < ${r${i}f2} ))
then
sed "s/$r$if3}/`echo $r$if3 | cut -c1-4`/;s/$r$if4/`echo $r$if4 | cut -c1-4`/;s/$r$if5/`echo $r$if5 | cut -c1-4`/;s/$r$if6/`echo $r$if6 | cut -c1-4`/;s/$r$if7/`echo $r$if7 | cut -c1-4`/;s/$r$if8/`echo $r$if8 | cut -c1-4`/;s/$r$if9/`echo $r$if9 | cut -c1-4`/" cons.txt > cons.txt.tmp && mv cons.txt.tmp cons.txt
fi
done
fi
but the inside if condition gives me error. I believe I am using wrong set of braces here but can't seem to figure out the correct way
trim.sh[6]: " ${Time} >= ${r${i}f1} && ${Time} < ${r${i}f2} ": 0403-011 The specified substitution is not valid for this command.
Parameter expansion is not recursive (or repeated, or inside-out) in < ${r${i}f2}, so this can't work.
You could use some convoluted code using eval to construct variable names before expansion, but that's a can of worms. What about simply unrolling the six element loop?
You are not able to do directly the variable inside the variable reference.
${r${i}f2}
You have to use the indirect reference. Try the below code it will work. Using eval we can done this.
if [[ $1 == 'R' ]]
then
echo "Running the recovery steps..."
for i in 1 2 3 4 5 6
do
head -${i} cons.txt | tail -1 | read -r r${i}f1 r${i}f2 r${i}f3 r${i}f4 r${i}f5 r${i}f6 r${i}f7 r${i}f8 r${i}f9;
eval var1=r${i}f1
eval var2=r${i}f2
eval val1=\$$var1
eval val2=\$$var2
if (( ${Time} >= $val1 && ${Time} < $val2 ))
then
sed "s/$r$if3}/`echo $r$if3 | cut -c1-4`/;s/$r$if4/`echo $r$if4 | cut -c1-4`/;s/$r$if5/`echo $r$if5 | cut -c1-4`/;s/$r$if6/`echo $r$if6 | cut -c1-4`/;s/$r$if7/`echo $r$if7 | cut -c1-4`/;s/$r$if8/`echo $r$if8 | cut -c1-4`/;s/$r$if9/`echo $r$if9 | cut -c1-4`/" cons.txt > cons.txt.tmp && mv cons.txt.tmp cons.txt
fi
done
fi
I need to do a rule with no constraints in a decision table.
i.e.:
rule ...
when
$p : Person()
then
$p.setCity("none");
end
I tried these:
| 1 | RuleTable example |
| 2 | CONDITION | ACTION |
| 3 | p:Person() | |
| 4 | name | p.setCity("$param"); |
| 5 | description | config person |
| 6 | | none |
But when I run application throws this exception:
person cannot be resolved
Exception in thread "main" java.lang.IllegalArgumentException: No se puede parsear base de conocimiento.
Probably it fails because you have no real condition in your table.
Try putting $param == $param as condition
Use condition like as shown in picture. It will generate DRL as:
rule "XYZ"
when
doc:Document()
then
doc.setX("Y");
end