The below is the code I use to give a certain set of users access to any indices that start with "custom-". But I want the users to be able to do whatever they want to ANY index (so "*"), EXCEPT for the system indices (which start with "."). Is there a way I can do this with the below code, perhaps with a regex? I tried "names": [ "a-zA-Z0-9" ]," but that didn't work.
curl -XPUT https://elasticsearch1:9200/_xpack/security/role/custom-users -u elastic -k -v -H "Content-Type: application/json" -d '
{
"indices": [{
"names": [ "custom-*" ],
"privileges": ["all"]
}],
"metadata": {
"version" : 1
}
}
'
This worked:
{ "names": ["#&~(^..+)"], "privileges": ["all"] }
Related
given this input
{
"properties": {
"hello": [
"saysay"
],
"test": [
"1"
]
},
"uri": "uri"
}
I've managed to check whether hello exists using this command
curl url | jq 'has("hello")'
Now how to check if the value of hello equals to saysay when it is a string variable ?
if hardcoded, below command return true which is expected
curl url | jq -r '.properties.hello[]=="saysay"'
when i try this one, I've got the following error : (23) Failed writing body
curl url | jq -r '.properties.$variable[]=="saysay"'
You can use --arg to set a jq variable from the command line.
field=hello
curl url | jq --arg property "$field" '.properties[$property][] == "saysay"'
I'm trying to scrape an Amazon page with browserless:
curl -X POST \
"https://chrome.browserless.io/content?token=<token>" \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d ' {
"url": "https://www.amazon.com/gp/your-account/order-details/?orderID=114-5444651-3149007",
"elements": [{
"selector": "a",
}],
"cookies": [
<many cookies>
],
}'
but I keep getting:
[{"message":"\"elements\" is not allowed","path":["elements"],"type":"object.unknown","context":{"child":"elements","label":"elements","value":[{"selector":"a","timeout":10000}],"key":"elements"}}]%
If I exclude the elements object, it works fine but returns the entire 6,000 lines of <html>.
(What I actually want is document.getElementsByClassName('shipment')[0].innerText)
When I try the examples (from the docs) they work fine.
Its because elements only available for /scrape API
You are using /content
According to https://developer.openstack.org/api-ref/container-infrastructure-management/#create-new-cluster all I would need to create the cluster is pass the parameters like:
curl --header "X-Auth-Token: blah" \
-X POST https://myopenstack:9511/v1/clusters -d name="Swarm-cluster-ansible" -d cluster_template_id="7402f9d3-4881-440f-8496-08d420935f58" -d node_count=2 -d keypair="k8s-gitlab-ci"
It is giving me:
{"errors": [{"status": 400, "code": "client", "links": [], "title": "Unknown argument: \"cluster_template_id, node_count, keypair, name\"", "detail": "Unknown argument: \"cluster_template_id, node_count, keypair, name\"", "request_id": ""}]}
If I try it this way:
curl --header "X-Auth-Token: blah" \
-X POST https://myopenstack:9511/v1/clusters -d cluster='{
"name":"swarm",
"master_count":1,
"discovery_url":null,
"cluster_template_id":"7402f9d3-4881-440f-8496-08d420935f58",
"node_count":1,
"keypair":"k8s-gitlab-ci",
"master_flavor_id":null,
"labels":{
},
"flavor_id":null
}'
{"errors": [{"status": 400, "code": "client", "links": [], "title": "Invalid input for field/attribute cluster", "detail": "Invalid input for field/attribute cluster. Value: '{\n \"name\":\"swarm\",\n \"master_count\":1,\n \"discovery_url\":null,\n \"cluster_template_id\":\"7402f9d3-4881-440f-8496-08d420935f58\",\n \"node_count\":1,\n \"keypair\":\"k8s-gitlab-ci\",\n \"master_flavor_id\":null,\n \"labels\":{\n },\n \"flavor_id\":null\n}'. unable to convert to Cluster. Error: __init__() takes exactly 1 argument (2 given)", "request_id": ""}]}
Any idea?
EDIT: I am able to do a GET and retrieve the list of existing clusters.
-H "Content-Type: application/json" is enough in this case so the body is interpreted as JSON.
Facing an issue with a simple requirement.
Make a CURL request and store in a variable
Parse JSON from variable using JQ and output into CSV file.
When I try to do it, I am getting extra leading space in the file from the second line for some reason. Is there a way to configure jq to not do this ?
Code :
output=`curl -v -k -H 'Content-Type: application/json' '<<URL>>' -d '<<PARAMS>>'`
csvOutput=`echo $output | jq --raw-output '.result .items[]|[.name,.key,.created,.updated]|#csv'`
echo $csvOutput >> out.csv
The following code (all in one line) does not have the issue. But I need the curl output in a variable to perform some checks ,so this option does not help.
echo "$(curl -v -k -H 'Content-Type: application/json' '<<URL>>' -d '<<PARAMS>>" | jq -r '.result .items[]|[.name,.key,.created,.updated]|#csv' >> out.csv
Strangely, when I read the json output (taken from curl response) from a file and then apply jq filters, I don't see the issue.
cat response.json | jq -r '.result .items[]|[.name,.key,.created,.updated]|#csv' > out.csv
response.json (formatted) :
{
"result": {
"total_items": 22,
"total_pages": 1,
"items_per_page": 1000,
"current_page": 1,
"items": [
{
"key": "1",
"name": "Name 1",
"created": "2016-12-20T08:51:13Z",
"updated": "2016-12-20T09:29:08Z"
},
{
"key": "2",
"name": "Name 2",
"created": "2016-12-20T08:51:13Z",
"updated": "2016-12-20T09:29:08Z"
},
{
"key": "3",
"name": "Name 3",
"created": "2016-12-20T08:51:13Z",
"updated": "2016-12-20T09:29:08Z"
}
]
},
"id": 1
}
When you write your result to your csv file your are using echo $csvOutput >> out.csv instead of :
echo "$csvOutput" >> out.csv
The double quotes will preserve newlines, see this post
jwt1=`echo -n '{"alg":"RS256","typ":"JWT"}' | openssl base64 -e`
jwt2=`echo -n '{\
"iss":"...#developer.gserviceaccount.com",\
"scope":"https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/datastore",\
"aud":"https://accounts.google.com/o/oauth2/token",\
"exp":'$(($(date +%s)+3600))',\
"iat":'$(date +%s)'}' | openssl base64 -e`
jwt3=`echo -n "$jwt1.$jwt2" | tr -d '\n' | tr -d '=' | tr '/+' '_-'`
jwt4=`echo -n "$jwt3" | openssl sha -sha256 -sign google.p12 | openssl base64 -e`
jwt5=`echo -n "$jwt4" | tr -d '\n' | tr -d '=' | tr '/+' '_-'`
curl -H "Content-type: application/x-www-form-urlencoded" -X POST "https://accounts.google.com/o/oauth2/token" -d \
"grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=$jwt3.$jwt5"
I receive a token successfully but when I use it I get permission denied?
When I copy the oauth2 token from https://developers.google.com/datastore/docs/apis/v1beta1/datasets/blindWrite#try-it
it works?
curl -X GET "https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=$1"
curl -X GET "https://www.googleapis.com/oauth2/v2/userinfo?access_token=$1"
curl -H "Content-type: application/json" -H "Authorization: Bearer $1" -X POST "https://www.googleapis.com/datastore/v1beta1/datasets/.../blindWrite" -d \
'{
"mutation": {
"upsert": [
{
"key": {
"path": [
{
"kind": "person",
"name": "gert"
}
]
}
}
]
}
}'
difference between the 2 tokens:
1) from jwt (permission denied)
{
"issued_to": "522156758812-u8hj8dhnk5br3vnpqqvuscievhbnl0gg.apps.googleusercontent.com",
"audience": "522156758812-u8hj8dhnk5br3vnpqqvuscievhbnl0gg.apps.googleusercontent.com",
"scope": "https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/datastore",
"expires_in": 3588,
"email": "522156758812-u8hj8dhnk5br3vnpqqvuscievhbnl0gg#developer.gserviceaccount.com",
"verified_email": true,
"access_type": "offline"
}
{
"email": "522156758812-u8hj8dhnk5br3vnpqqvuscievhbnl0gg#developer.gserviceaccount.com",
"verified_email": true
}
2) from https://developers.google.com/datastore/docs/apis/v1beta1/datasets/blindWrite#try-it (works)
{
"issued_to": "292824132082.apps.googleusercontent.com",
"audience": "292824132082.apps.googleusercontent.com",
"user_id": "116469479527388802962",
"scope": "https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/datastore https://www.googleapis.com/auth/plus.me",
"expires_in": 3568,
"email": "gert.cuykens#gmail.com",
"verified_email": true,
"access_type": "online"
}
{
"id": "116469479527388802962",
"email": "gert.cuykens#gmail.com",
"verified_email": true
}
What is wrong with my jwt received token?
How do I make jwt work also?
In order for service account to be properly configured with your Cloud Datastore instance you have to create them using the Cloud Console as described in the documentation.
Alternatively if you really want to use the service account you created using the [Google APIs console][3], you can do the following:
Go to cloud.google.com/console
Click on your project id
Click on APIs
Make sure Google Cloud Datastore API is ON
Click on the gear symbol (⚙) on the top right
Click on Teams
Click Add member
Add your service account as a Viewer