Makefile: how to loop on data to set arguments within the script? - console

Given a working makefile which crop a world map to a specific country's bounding box.
# boxing:
INDIA_crop.tif: ETOPO1_Ice_g_geotiff.tif
gdal_translate -projwin 67.0 37.5 99.0 05.0 ETOPO1_Ice_g_geotiff.tif INDIA_crop.tif
# ulx uly lrx lry // W N E S
# unzip:
ETOPO1_Ice_g_geotiff.tif: ETOPO1.zip
unzip ETOPO1.zip
touch ETOPO1_Ice_g_geotiff.tif
# download:
ETOPO1.zip:
curl -o ETOPO1.zip 'http://www.ngdc.noaa.gov/mgg/global/relief/ETOPO1/data/ice_surface/grid_registered/georeferenced_tiff/ETOPO1_Ice_g_geotiff.zip'
clean:
rm `ls | grep -v 'zip' | grep -v 'Makefile'`
Given I currently have to change this makefile each time by hand editing the makefile to change:
1. the country name,
2. its North border geocoordinate,
3. its South border geocoordinate,
4. its East border geocoordinate,
5. its West border geocoordinate.
Given I also have a dataset for all countries such :
data = [
{ "W":-62.70; "S":-27.55;"E": -54.31; "N":-19.35; "item":"Paraguay" },
{ "W": 50.71; "S": 24.55;"E": 51.58; "N": 26.11; "item":"Qatar" },
{ "W": 20.22; "S": 43.69;"E": 29.61; "N": 48.22; "item":"Romania" },
{ "W": 19.64; "S": 41.15;"E":-169.92; "N": 81.25; "item":"Russia" },
{ "W": 29.00; "S": -2.93;"E": 30.80; "N": -1.14; "item":"Rwanda" },
{ "W": 34.62; "S": 16.33;"E": 55.64; "N": 32.15; "item":"Saudi Arabia"}
];
How to loop on each line of the data so to set parameters into my makefile ? So I output at once all the files COUNTRYNAME_crop.tif with the correct bounding boxes.

Assuming you're using GNU make, this seems to me like a perfect problem for autogenerated makefiles. After make reads in its makefiles it will test each one as if it were a target to see if it can be rebuilt. If so, and it is rebuilt, make will automatically re-exec itself. This is an extraordinarily powerful type of meta-programming. I would combine this with recursive variable naming.
1. Data: Let's assume your dataset is in dataset.out such :
[
{ "W":-62.70; "S":-27.55;"E": -54.31; "N":-19.35; "item":"Paraguay" },
{ "W": 50.71; "S": 24.55;"E": 51.58; "N": 26.11; "item":"Qatar" },
{ "W": 20.22; "S": 43.69;"E": 29.61; "N": 48.22; "item":"Romania" },
{ "W": 19.64; "S": 41.15;"E":-169.92; "N": 81.25; "item":"Russia" },
{ "W": 29.00; "S": -2.93;"E": 30.80; "N": -1.14; "item":"Rwanda" },
{ "W": 34.62; "S": 16.33;"E": 55.64; "N": 32.15; "item":"Saudi Arabia"}
];
2. Converter: Now you need to write the utility convert-to-makefile. I would write it in Perl myself but the new kids would probably choose Python. Whatever. Anyway, for each country, the output should be something like this:
COUNTRIES += <countryname>
<countryname>-NORTH := <north-coord>
<countryname>-SOUTH := <south-coord>
<countryname>-EAST := <east-coord>
<countryname>-WEST := <west-coord>
so that bounding.mk, after being generated, has one of those stanzas for each country.
3a. Makefile: Then, add this to the beginning of your makefile:
-include bounding.mk
3b. Then add this rule to the end of your makefile:
bounding.mk: dataset.out
convert-to-makefile $< > $#
3c. Then you can write your rules like this:
all: $(COUNTRIES:%=%_crop.tif)
%_crop.tif: ETOPO1_Ice_g_geotiff.tif
gdal_translate -projwin $($*-WEST) $($*-NORTH) $($*-EAST) $($*-SOUTH) $< $#
That should about do it!

Related

jq pipe operator in nested array

Using code below to pull data from a local json file.
The file is very large and is nested with objects and arrays. There are multiple objects in the .ratings[] that I would like to extract.
How can I use the pipe operator in the .ratings[] array so that I don't have to retype .ratings[] for each piece of data that I would like to pull?
jq -r '.players[] | [.firstName,.lastName,.tid,.pid,.ratings[].spd,.ratings[].jmp] | join(", ")'
You can enclose it in () to use the pipe sign:
.players[] | [.firstName, .lastName, .tid, .pid, (.ratings[] | .spd, .jmp)] | join(", ")
Try it online
You didn't specify the expected output, so it is not clear if your proposed solution gives you the output you want.
Given the following input:
{
"players": [
{
"firstName": "fname1",
"lastName": "lname1",
"tid": "tid1",
"pid": "pid1",
"ratings": [
{
"spd": "spd1-1",
"jmp": "jmp1-1"
}
]
},
{
"firstName": "fname2",
"lastName": "lname2",
"tid": "tid2",
"pid": "pid2",
"ratings": [
{
"spd": "spd2-1",
"jmp": "jmp2-1"
},
{
"spd": "spd2-2",
"jmp": "jmp2-2"
}
]
},
{
"firstName": "fname3",
"lastName": "lname3",
"tid": "tid3",
"pid": "pid3",
"ratings": [
{
"spd": "spd3-1",
"jmp": "jmp3-2"
},
{
"spd": "spd3-2",
"jmp": "jmp3-2"
},
{
"spd": "spd3-3",
"jmp": "jmp3-3"
}
]
}
]
}
Your solution and the answer from 0ston0 will give you 1 line per player, but a different number of columns per line:
.players[] | [.firstName,.lastName,.tid,.pid,(.ratings[]|.spd,.jmp)] | join(", ")
generates:
fname1, lname1, tid1, pid1, spd1-1, jmp1-1
fname2, lname2, tid2, pid2, spd2-1, jmp2-1, spd2-2, jmp2-2
fname3, lname3, tid3, pid3, spd3-1, jmp3-2, spd3-2, jmp3-2, spd3-3, jmp3-3
This might or might not be what want your result to look like.
A different solution will print one line per rating, but duplicate the players' names. Running:
.players[] | [.firstName,.lastName,.tid,.pid] + (.ratings[]|[.spd,.jmp]) | join(", ")
will result in:
fname1, lname1, tid1, pid1, spd1-1, jmp1-1
fname2, lname2, tid2, pid2, spd2-1, jmp2-1
fname2, lname2, tid2, pid2, spd2-2, jmp2-2
fname3, lname3, tid3, pid3, spd3-1, jmp3-2
fname3, lname3, tid3, pid3, spd3-2, jmp3-2
fname3, lname3, tid3, pid3, spd3-3, jmp3-3
Both solutions are valid for different use cases and depending on how you are going to subsequently process the data.

Json use Jq to parse, read and return true if enrty found

Apologies if this is basic but the doc for jq is not so good
i have this json:
{
"jsonrpc": "2.0",
"result": [{
"hostid": "10084",
"host": "Zabbix server",
"interfaces": [{
"interfaceid": "1",
"ip": "127.0.0.1"
}]
}, {
"hostid": "10336",
"host": "AUTO",
"interfaces": [{
"interfaceid": "4",
"ip": "1.2.3.4"
}]
}, {
"hostid": "10337",
"host": "AUTOSERVER",
"interfaces": [{
"interfaceid": "5",
"ip": "4.5.6.7"
}]
}, {
"hostid": "10348",
"host": "Server00001",
"interfaces": [{
"interfaceid": "16",
"ip": "4.5.6.7"
}]
}],
"id": 2
}
i need to find a way to use jq to find if "Server0001" exists in one of the hosts
i know i can use grep but i prefer using jq here, like select..
any help or ref toa good doc would be much appriciated
any (see the manual) can return a boolean value if a condition matches with at least one item.
jq 'any(.result[]; .host == "Server0001")'
false
Demo
jq 'any(.result[]; .host == "Server00001")'
true
Demo
You may also want to use the some parameters when invoking jq (see the manual): The --arg option, for instance, lets you add a variable which can be initialized from outside the filter string. And with the -e (or --exit-status) flag you can have jq set the exit status according to the filter's final result. Together, this enables you to use jq like this:
if jq --arg host "Server0001" -e 'any(.result[]; .host == $host)';
then
…
else
…
fi

JQ to filter only vaule of id

the following is the JSON data. need to get only of id key
{apps:[ {
"id": "/application1/4b693882-ffba-4c93-a0f2-cccafcb4d7dd",
"cmd": null,
"args": null,
"user": null,
"env": {},
"constraints": [
[
"hostname",
"GROUP_BY",
"5"
]
},
{
"id": "/application2/4b693882-ffba-4c93-a0f2-cccafcb4d7dd",
"cmd": null,
"args": null,
"user": null,
"env": {},
"constraints": [
[
"hostname",
"GROUP_BY",
"5"
]
]},
output expected is
/application1/4b693882-ffba-4c93-a0f2-cccafcb4d7dd
/application2/4b693882-ffba-4c93-a0f2-cccafcb4d7dd
Thanks in advance
After fixing the errors in your JSON, we can use the following jq filter to get the desired output:
.apps[] | .id
JqPlay Demo
Result jq -r '.apps[] | .id':
/application1/4b693882-ffba-4c93-a0f2-cccafcb4d7dd
/application2/4b693882-ffba-4c93-a0f2-cccafcb4d7dd
You can use map() to create an array from the properties of the objects. Try this:
let data = {apps:[{"id":"/application1/4b693882-ffba-4c93-a0f2-cccafcb4d7dd","cmd":null,"args":null,"user":null,"env":{},"constraints":["hostname","GROUP_BY","5"]},{"id":"/application2/4b693882-ffba-4c93-a0f2-cccafcb4d7dd","cmd":null,"args":null,"user":null,"env":{},"constraints":["hostname","GROUP_BY","5"]}]}
let ids = data.apps.map(o => o.id);
console.log(ids);
Note that I corrected the invalid brace/bracket combinations in the data structure you posted in the question. I assume this is just a typo in that example, otherwise there would be parsing errors in the console.

User keybindings - return 'letter/symbol' combination not command

I want to create a custom keybinding in sublime text 3 that doesn't return a command but returns the key combination used in R to define a variable like below.
variable <- variable_definition //for example
z1 <- seq(1,100)
In R 3.2.2 GUI mac OS X the keybinding:
"alt+-" returns " <- "
I have read the documentation for user keybindings but couldn't find something that I could use.
I have tried "print" and "echo" as below but they don't work.
[
{ "keys": ["alt+-"], "print": " <- "}
]
or
[
{ "keys": ["alt+-"], "echo": " <- "}
]
Some help would be much appreciated
In Sublime Text you run commands with arguments. If you want to insert something the command is insert and the argument is called characters. If you want to limit it to the language R you can add a context. Hence the keybinding:
[
{
"keys": ["alt+-"], "command": "insert", "args": {"characters": " <- "},
"context":
[
{ "key": "selector", "operator": "equal", "operand": "source.r" }
]
}
]
Aside: it could also be interesting for you to use snippets as keybindings.
[
{
"keys": ["alt+-"], "command": "insert_snippet", "args": {"contents": "${1:variable} <- ${0:definition}"}
}
]

Sublime Text 2 OS specific Run configuration?

I would like to create 4 build options for building Actionscript files. One for Building and one for Running in Linux, and the same for Windows.
Is it possible to do this?
Currently it seems 'windows' and 'linux' sections can only overwrite the default build option?
My current broken .sublime-build file is:
{
"selector": "source.actionscript",
"windows":
{
"cmd": [
"${packages}\\User\\Flash-build.bat",
"${project_path}",
"${file}"
],
"variants":
[
{
"name": "Run",
"cmd": [
"${packages}\\User\\Flash-run.bat",
"${file_path}",
"${file_base_name}"
]
}
],
},
"linux":
{
"cmd": [
"${packages}/User/Flash-build.sh",
"${packages}",
"${project_path}",
"${file_path}",
"${file_base_name}"
],
"variants":
[
{
"name": "Run",
"cmd": [
"${packages}/User/Flash-run.sh",
"${packages}",
"${project_path}",
"${file_path}",
"${file_base_name}"
]
}
]
}
}
Looking at the documentation, I can't see why your solution wouldn't work, however, I know that the following works for me.
Sublime preference files can be specified on a per-OS basis. You could create three separate files:
User/actionscript (OSX).sublime-build
User/actionscript (Linux).sublime-build
User/actionscript (Windows).sublime-build
Sublime will only pick up the one that is relevant to your OS.

Resources