Related
I have many JSON objects where I would like to transform some objects to reduce redundancy and limit some depth.
For example, given:
{
"foo": [
{
"foobarList": {
"foobar": 2
}
},
{
"raboofList": {
"raboof": [3, 5, 7]
}
}
],
"bazList": {
"baz": 11
},
"foobar": {
"barbazList": {
"barbaz": [13, 17, 19]
},
"foobazList": {
"foobaz": {
"barfooList": {
"barfoo": [23, 29]
}
}
}
}
}
... I'd like to "hoist" the values of all the fields that end in "List", e.g., "foobarList".
So, the above would be transformed to:
{
"foo": [
{
"foobar": 2
},
{
"raboof": [3, 5, 7]
}
],
"baz": 11,
"foobar": {
"barbaz": [13, 17, 19],
"foobaz": {
"barfoo": [23, 29]
}
}
}
I've tried various filters, but so far I can only identify the "List$" fields with:
jq '.. |iterables |to_entries[] |select((has("key")) and (.key|type == "string") and (.key|test("List$")) )'
How can this transformation be accomplished with jq?
Here's a jqplay.org to help getting started.
The following produces the answer you want, but may not meet your expectations in every respect:
def hoist:
with_entries(if (.key | endswith("List"))
and (.value | type == "object")
and (.value | keys_unsorted | length==1)
then (.value | keys_unsorted[0]) as $k
| .key = $k
| .value |= .[$k]
else . end);
walk(if type == "object" then hoist else . end)
If the input is "hoistable" structurally, you could also operate on its stream representation:
[fromstream(tostream | del(.[0][] | select(endswith("List")?)))] | add
Demo
I need to write this function in AST, preorder, but when I run my yacc file, it prints "Segmentatio fault(core dumped)". If you can please help me resolve my problem, because it as been a few days and I still do not understand what to do. I checked my syntax and it is working, but for some reason when I add mknode and printtree to it, it prints this message. Please help me.
void foo(int x, y, z; real f){
if (x>y) {
x = x + f;
}
else {
y = x + y + z;
x = f*2;
z = f;
}
This is my yacc file, including my function printtree and mknode.
%{
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct node{
char *token;
struct node *left;
struct node *right;
}node;
node *mknode(char *token, node *left, node *right);
void printtree(node *tree);
%}
%union
{
char *s;
struct node *node;
}
%token IF ELSE INT CHAR VOID REAL RETURN GUI
%left '*'
%left '+'
%token <s> NUM ID FUNC
%type <node> S start function func args args1 body if_st ret_st expr block ass calc
%type <s> type
%%
S: start {printtree($1);};
start: function {$$ = mknode("CODE",$1,NULL);};
function: func { $$ = mknode("FUNC",$1, NULL); };
func: type ID '(' args ')' '{' body '}' {$$ = mknode($2,NULL, mknode("ARGS", $4,mknode($1, NULL,$7)));};
type: INT {$$ = "INT";}
| CHAR {$$ = "CHAR";}
| VOID {$$ = "VOID";}
| REAL {$$ = "REAL";};
args: type args1 args {$$ = mknode($1,$2,$3);} | type args1 {$$ = mknode($1,$2,NULL);} ;
args1: ID {$$ = mknode($1,NULL,NULL);}
| ID ';' {$$ = mknode($1,NULL,NULL);}
| ID ',' args1 {$$ = mknode($1,NULL,$3);}
| { $$ = NULL; };
body: if_st {$$ = mknode("BODY", $1, NULL);}
| ret_st {$$ = mknode("BODY", $1, NULL);};
if_st: IF'(' expr ')' '{'block'}' ELSE '{'block'}' {$$ = mknode("IF-ELSE",mknode(NULL,$3,mknode(NULL,$6,$10)),NULL);}
| IF '(' expr ')' '{'block'}'{$$ = mknode("IF",$3,$6);} ;
expr: ID '<' ID {$$ = mknode("<",mknode($1,NULL,NULL),mknode($3,NULL,NULL));}
| ID '>' ID {$$ = mknode(">",mknode($1,NULL,NULL),mknode($3,NULL,NULL));}
| ID '=' ID {$$ = mknode("==",mknode($1,NULL,NULL),mknode($3,NULL,NULL));}
| ID '<' NUM {$$ = mknode("<",mknode($1,NULL,NULL),mknode($3,NULL,NULL));}
| ID '>' NUM {$$ = mknode(">",mknode($1,NULL,NULL),mknode($3,NULL,NULL));}
| ID '=' NUM {$$ = mknode("==",mknode($1,NULL,NULL),mknode($3,NULL,NULL));};
block: block ass {$$ = mknode(NULL,$1,$2);}
| ass {$$ = mknode(NULL,$1,NULL);};
ass: ID '=' calc ';'{$$ = mknode("=",mknode($1,NULL,NULL),mknode(NULL,$3,NULL));};
calc: ID '+' calc {$$ = mknode("+",mknode($1,NULL,NULL),mknode(NULL,$3,NULL));}
| ID '*' calc {$$ = mknode("*",mknode($1,NULL,NULL),mknode(NULL,$3,NULL));}
| NUM '+' calc {$$ = mknode("+",mknode($1,NULL,NULL),mknode(NULL,$3,NULL));}
| NUM '*' calc {$$ = mknode("*",mknode($1,NULL,NULL),mknode(NULL,$3,NULL));}
| NUM {$$ = mknode($1,NULL,NULL);}
| ID {$$ = mknode($1,NULL,NULL);};
ret_st: RETURN GUI calc GUI ';' { $$ = mknode("RET", $3, NULL); };
%%
#include "lex.yy.c"
int main()
{
return yyparse();
}
node *mknode(char *token,node *left,node *right)
{
node *newnode = (node*)malloc(sizeof(node));
char *newstr = (char*)malloc(sizeof*(token)+1);
strcpy(newstr,token);
newnode->left = left;
newnode->right = right;
newnode->token = newstr;
return newnode;
}
void printtree(node *tree)
{
printf("%s\n",tree->token);
if(tree->left)
printtree(tree->left);
if(tree->right)
printtree(tree->right);
}
int yyerror()
{
printf("ERROR\n");
return 0;
}
Most likely cause of the crash:
you call mknode in a couple of places (eg, the block rule) with NULL as the first argument, but mknode calls strcpy with this argument as the source string, so it will crash
Other problems:
you use sizeof(token) where token is a char * (getting the size of a pointer, not the length of the string. You need strlen(token). Better yet, use strdup(token) to do the malloc+strcpy all in one.
your grammar is inflexible, with almost-dupliacted rules and limited nesting. You're better off using fewer rules -- get rid of all the calc/expr stuff and just have
expr: expr '+' expr
| expr '*' expr
| expr '<' expr
| expr '>' expr
| expr '=' expr
| ID
| NUM
| '(' expr ')'
and set precedence of your operators appropriately. Similarly block and body should be combined into one non-terminal and a couple of rules.
The grammar, in a pinch, is as follows:
Phi ::= Phi_sub ( ("&&" | "||") Phi_sub )*
Phi_sub ::= "(" Phi ")" | ...
Psi ::= Psi_sub ( ("&&" | "||") Psi_sub )*
Psi_sub ::= "(" Psi ")" | ...
Xi ::= LOOKAHEAD( Phi ) Phi | LOOKAHEAD( Psi ) Psi
As you can see, an infinite lookahead would in general be required in the Xi production, because the parser needs to distinguish cases like:
((Phi_sub && Phi_sub) || Phi_sub) vs ((Psi_sub && Psi_sub) || Psi_sub)
i.e. an arbitrary amount of prefixing (.
I thought, that making the lookahead like above would work, but it doesn't. For example, Phi is chosen, even if Xi does not expand to Phi, but does to Psi. This can be easily checked on a certain stream S by calling Phi with the debugger just after the parsed decided, within Xi, to choose Phi, and is about to call Phi. The debugger in such a case shows a proper expansion to Psi, while allowing the parser just to call Phi as it wants would cause a parse exception.
The other way of testing it is swapping Phi and Psi:
Xi ::= LOOKAHEAD( Psi ) Psi | LOOKAHEAD( Phi ) Phi
This will make the parser parse the particular S correctly, and so it seems that simply the first branch within Xi is chosen, be it the valid one or not.
I guess I got some basic assumption wrong, but have no idea what can it be. Should the above work in general, if there are no additional factors, like an ignored inner lookahead?
Your assumptions are not wrong. What you are trying to do should work. And it should work for the reasons you think it should work.
Here is a complete example written in JavaCC.
void Start() : {} { Xi() <EOF> }
void Xi() : {} {
LOOKAHEAD( Phi() ) Phi() { System.out.println( "Phi" ) ; }
| LOOKAHEAD( Psi() ) Psi() { System.out.println( "Psi" ) ; }
}
void Phi() : {} { Phi_sub() ( ("&&" | "||") Phi_sub() )*}
void Phi_sub() : {} { "(" Phi() ")" | "Phi_sub" }
void Psi() : {} { Psi_sub() ( ("&&" | "||") Psi_sub() )* }
void Psi_sub() : {} { "(" Psi() ")" | "Psi_sub" }
And here is some sample output:
Input is : <<Phi_sub>>
Phi
Input is : <<Psi_sub>>
Psi
Input is : <<((Phi_sub && Phi_sub) || Phi_sub)>>
Phi
Input is : <<((Psi_sub && Psi_sub) || Psi_sub)>>
Psi
The problem you are having lies in something not shown in the question.
By the way, it's a bad idea to put a lookahead specification in front of every alternative.
void X() : {} { LOOKAHEAD(Y()) Y() | LOOKAHEAD(Z()) Z() }
is roughly equivalent to
void X() : {} { LOOKAHEAD(Y()) Y() | LOOKAHEAD(Z()) Z() | fail with a stupid error message }
For example, here is another run of the above grammar
Input is : <<((Psi_sub && Psi_sub) || Phi_sub)>>
NOK.
Encountered "" at line 1, column 1.
Was expecting one of:
After all lookahead has failed, the parser is left with an empty set of expectations!
If you change Xi to
void Xi() : {} {
LOOKAHEAD( Phi() ) Phi() { System.out.println( "Phi" ) ; }
| Psi() { System.out.println( "Psi" ) ; }
}
you get a slightly better error message
Input is : <<((Psi_sub && Psi_sub) || Phi_sub)>>
NOK.
Encountered " "Phi_sub" "Phi_sub "" at line 1, column 26.
Was expecting one of:
"(" ...
"Psi_sub" ...
You can also make a custom error message
void Xi() : {} {
LOOKAHEAD( Phi() ) Phi() { System.out.println( "Phi" ) ; }
| LOOKAHEAD( Psi() ) Psi() { System.out.println( "Psi" ) ; }
| { throw new ParseException( "Expected either a Phi or a Psi at line "
+ getToken(1).beginLine
+ ", column " + getToken(1).beginColumn + "." ) ;
}
}
I need to transform elements from an array to column index and return the value of $3 for each column index.
I don´t have access to gawk 4 so I cannot work with real multidimensional arrays.
Input
Name^Code^Count
Name1^0029^1
Name1^0038^1
Name1^0053^1
Name2^0013^3
Name2^0018^3
Name2^0023^5
Name2^0025^1
Name2^0029^1
Name2^0038^1
Name2^0053^1
Name3^0018^1
Name3^0060^1
Name4^0018^2
Name4^0025^5
Name5^0018^2
Name5^0025^1
Name5^0060^1
Desired output
Name^0013^0018^0023^0025^0029^0038^0053^0060
Name1^^^^^1^1^1^
Name2^3^3^5^1^1^1^1^
Name3^^1^^^^^^1
Name4^^2^^5^^^^
Name5^^^^1^^^^1
Any suggestions on how to tackle this task without using real multidimensional arrays?
The following solution uses GNU awk v3.2 features for sorting. This does not use multi-dimensional arrays. It only simulates one.
awk -F"^" '
NR>1{
map[$1,$2] = $3
name[$1]++
value[$2]++
}
END{
printf "Name"
n = asorti(value, v_s)
for(i=1; i<=n; i++) {
printf "%s%s", FS, v_s[i]
}
print ""
m = asorti(name, n_s)
for(i=1; i<=m; i++) {
printf "%s", n_s[i]
for(j=1; j<=n; j++) {
printf "%s%s", FS, map[n_s[i],v_s[j]]
}
print ""
}
}' file
Name^0013^0018^0023^0025^0029^0038^0053^0060
Name1^^^^^1^1^1^
Name2^3^3^5^1^1^1^1^
Name3^^1^^^^^^1
Name4^^2^^5^^^^
Name5^^2^^1^^^^1
This will work with any awk and will order the output of counts numerically while keeping the names in the order they occur in your input file:
$ cat tst.awk
BEGIN{FS="^"}
NR>1 {
if (!seenNames[$1]++) {
names[++numNames] = $1
}
if (!seenCodes[$2]++) {
# Insertion Sort - start at the end of the existing array and
# move everything greater than the current value down one slot
# leaving open the slot for the current value to be inserted between
# the last value smaller than it and the first value greater than it.
for (j=++numCodes;codes[j-1]>$2+0;j--) {
codes[j] = codes[j-1]
}
codes[j] = $2
}
count[$1,$2] = $3
}
END {
printf "%s", "Name"
for (j=1;j<=numCodes;j++) {
printf "%s%s",FS,codes[j]
}
print ""
for (i=1;i<=numNames;i++) {
printf "%s", names[i]
for (j=1;j<=numCodes;j++) {
printf "%s%s",FS,count[names[i],codes[j]]
}
print ""
}
}
...
$ awk -f tst.awk file
Name^0013^0018^0023^0025^0029^0038^0053^0060
Name1^^^^^1^1^1^
Name2^3^3^5^1^1^1^1^
Name3^^1^^^^^^1
Name4^^2^^5^^^^
Name5^^2^^1^^^^1
Since you only have two "dimensions", it is easy enough to use one array for each dimension and a joining array with a calculated column name. I didn't do the sorting of columns or rows, but the idea is pretty basic.
#!/usr/bin/awk -f
#
BEGIN { FS = "^" }
(NR == 1) {next}
{
rows[$1] = 1
columns[$2] = 1
join_table[$1 "-" $2] = $3
}
END {
printf "Name"
for (col_name in columns) {
printf "^%s", col_name
}
printf "\n"
for (row_name in rows) {
printf row_name
for (col_name in columns) {
printf "^%s", join_table[row_name "-" col_name]
}
printf "\n"
}
}
I have two maps in Groovy [a: 1, b: 2] and [b:1, c:3] and would like to create from them a third map [a: 1, b: 3, c: 3]. Is there a Groovy command that does that?
Edit: Notice that the values in the third map, are a sum of the values from the first two maps, if the keys are identical.
Thanks
Yet another solution would be:
def m1 = [ a:1, b:2 ]
def m2 = [ b:1, c:3 ]
def newMap = [m1,m2]*.keySet().flatten().unique().collectEntries {
[ (it): [m1,m2]*.get( it ).findAll().sum() ]
}
Taking epidemian's answer as inspiriation, you can also write a method to handle multiple maps
def m1 = [a: 1, b: 2]
def m2 = [b: 1, c: 3]
def combine( Map... m ) {
m.collectMany { it.entrySet() }.inject( [:] ) { result, e ->
result << [ (e.key):e.value + ( result[ e.key ] ?: 0 ) ]
}
}
def newMap = combine( m1, m2 )
This should work for any number of maps:
def maps = [[a: 1, b: 2], [b:1, c:3]]
def result = [:].withDefault{0}
maps.collectMany{ it.entrySet() }.each{ result[it.key] += it.value }
assert result == [a: 1, b: 3, c: 3]
The maps.collectMany{ it.entrySet() } expression returns a list of map entries, like [a=1, b=2, b=1, c=3], and then each of those is added to the result.
Another option, if you'd like to keep all the transformation into one expression and make it "more functional", is to first group the entries by key and then sum the values, but I think it's less readable:
def result = maps.collectMany{ it.entrySet() }
.groupBy{ it.key }
.collectEntries{[it.key, it.value.sum{ it.value }]}
The groupBy part returns a map of the form [a:[a=1], b:[b=2, b=1], c:[c=3]] and then the collectEntries transforms that map into another one that has the same kays but has the sum of the lists in the values instead.
The following example demonstrates adding two maps, to a third map m3:
Map m1 = [ a:1, b:2 ];
Map m2 = [ b:1, c:3 ];
Map m3 = [:];
m3 << m1
m3 << m2
A nice way is it use the spread operator:
def m1 = [ a:1, b:2 ]
def m2 = [ b:1, c:3 ]
def m3 = [ *:m1, *:m2 ]
println "m1 = $m1"
println "m2 = $m2"
println "m3 = $m3"
*Credit:http://mrhaki.blogspot.ch/2009/09/groovy-goodness-spread-operator.html
The code above does not work in groovysh but it is fine in the groovy console. (for version 1.8)
I don't think there's a ready-made method for this, maybe use something like:
def m1 = [a: 1, b: 2]
def m2 = [b: 1, c: 3]
def newMap = (m1.keySet() + m2.keySet()).inject([:]) {
acc, k -> acc[k] = (m1[k] ?: 0) + (m2[k] ?: 0); acc
}
This does it:
Map additionJoin( Map map1, Map map2 )
{
def result = [:];
result.putAll( map1 );
result.putAll( map2 );
result.each { key, value ->
if( map1[key] && map2[key] )
{
result[key] = map1[key] + map2[key]
}
}
return result;
}
def a = [a: 1, b: 2]
def b = [b:1,c:3]
def c = additionJoin( a, b )
println c
def merge(map1, map2) {
def add = { map, entry -> map << entry }
map2.inject(map1.inject([:], add), add)
}