AppSync BatchResolver AssumeRole Error - amazon-dynamodb

I’m trying to use the new DynamoDB BatchResolvers to write to two DynamoDB table in an AppSync resolver (currently using a Lambda function to do this). However, I’m getting the following permission error when looking at the CloudWatch logs:
“User: arn:aws:sts::111111111111:assumed-role/appsync-datasource-ddb-xxxxxx-TABLE-ONE/APPSYNC_ASSUME_ROLE is not authorized to perform: dynamodb:BatchWriteItem on resource: arn:aws:dynamodb:us-east-1:111111111111:table/TABLE-TWO (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException;
I’m using TABLE-ONE as my data source in my resolver.
I added the "dynamodb:BatchWriteItem" and "dynamodb:BatchGetItem" to TABLE-ONE’s permission:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"dynamodb:BatchGetItem",
"dynamodb:BatchWriteItem",
"dynamodb:PutItem",
"dynamodb:DeleteItem",
"dynamodb:GetItem",
"dynamodb:Scan",
"dynamodb:Query",
"dynamodb:UpdateItem"
],
"Resource": [
"arn:aws:dynamodb:us-east-1:111111111111:table/TABLE-ONE",
"arn:aws:dynamodb:us-east-1:111111111111:table/TABLE-ONE/*",
"arn:aws:dynamodb:us-east-1:111111111111:table/TABLE-TWO",
"arn:aws:dynamodb:us-east-1:111111111111:table/TABLE-TWO/*"
]
}
]
}
I have another resolver that uses the BatchGetItem operation and was getting null values in my response - changing the table’s policy access level fixed the null values:
However, checking the box for BatchWriteItem doesn’t seem to solve the issue either adding the permissions to the data source table’s policy.
I also tested my resolver test feature in AppSync, the evaluated request and response are working as intended.
Where else could I set the permissions for a BatchWriteItem operation between two tables? It seems like it's invoking the user's assumed-role instead of the table's role - can I 'force' it to use the table's role?

It is using the role that you have configured for the table in the AppSync console. Note that that particular role, should have appsync as a trusted entity.
Or if you use the new role tick box when creating the data source in the console, it should take care of it.

variable "dynamodb_table" {
description = "Name of DynamoDB table"
type = any
}
# value of var
dynamodb_table = {
"dyn-notification-inbox" = {
type = "AMAZON_DYNAMODB"
table = data.aws_dynamodb_table.dyntable
}
"dyn-notification-count" = {
type = "AMAZON_DYNAMODB"
table = data.aws_dynamodb_table.dyntable2
}
}
locals {
roles_arns = {
dynamodb = var.dynamodb_table
kms = var.kms_keys
}
}
data "aws_iam_policy_document" "invoke_dynamodb_document" {
statement {
actions = [
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:DeleteItem",
"dynamodb:UpdateItem",
"dynamodb:Query"
]
# dynamic dynamodb table
# for dynamodb table v.table.arn and v.table.arn/*
resources = flatten([
for k, v in local.roles_arns.dynamodb : [
v.table.arn,
"${v.table.arn}/*"
]
])
}
}
# make policy
resource "aws_iam_policy" "iam_invoke_dynamodb" {
name = "policy-${var.name}"
policy = data.aws_iam_policy_document.invoke_dynamodb_document.json
}
# attach role
resource "aws_iam_role_policy_attachment" "invoke_dynamodb" {
role = aws_iam_role.iam_appsync_role.name
policy_arn = aws_iam_policy.iam_invoke_dynamodb.arn
}
Result:
resources: [
'arn:aws:dynamodb:eu-west-2:xxxxxxxx:table/my-table',
'arn:aws:dynamodb:eu-west-2:xxxxxxxx:table/my-table/*'
]

Related

How can I filter a subscription using a custom resolver

I am working on a messaging app using AWS AppSync.
I have the following message type...
type Message
#model
#auth(
rules: [
{ allow: groups, groups: ["externalUser"], operations: [] }
]
) {
id: ID!
channelId: ID!
senderId: ID!
channel: Channel #connection(fields: ["channelId"])
createdAt: AWSDateTime!
text: String
}
And I have a subscription onCreatemessage. I need to filter the results to only channels that the user is in. So I get a list of channels from a permissions table and add the following to my response mapping template.
$extensions.setSubscriptionFilter({
"filterGroup": [
{
"filters" : [
{
"fieldName" : "channelId",
"operator" : "in",
"value" : $context.result.channelIds
}
]
}
]
})
$util.toJson($messageResult)
And it works great. But if a user is in more than 5 channels, I get the following error.
{
"message": "Connection failed: {"errors":[{"message":"subscription exceeds maximum value limit 5 for operator `in`.","errorCode":400}]}"
}
I am new to vtl. So my question is, how can I break up that filter in to multiple or'd filters?
According to Creating enhanced subscription filters, "multiple rules in a filter are evaluated using AND logic, while multiple filters in a filter group are evaluated using OR logic".
Therefore, as I understand it, you just need to split $context.result.channelIds into groups of 5 and add an object to the filters array for each group.
Here is a VTL template that will do this for you:
#set($filters = [])
#foreach($channelId in $context.result.channelIds)
#set($group = $foreach.index / 5)
#if($filters.size() < $group + 1)
$util.qr($filters.add({
"fieldName" : "channelId",
"operator" : "in",
"value" : []
}
))
#end
$util.qr($filters.get($group).value.add($channelId))
#end
$extensions.setSubscriptionFilter({
"filterGroup": [
{
"filters" : $filters
}
]
})
You can see this template running here: https://mappingtool.dev/app/appsync/042769cd78b0e928db31212f5ee6aa17
(Note: The Mapping Tool errors on line 15 are a result of the $filters array being dynamically populated. You can safely ignore them.)
Do you want to add server-side filter for GraphQL Subscriptions?
If so, Now, Amplify is supported for server-side filter for Subscriptions.
After you checking below blog, you may sense it.
https://aws.amazon.com/blogs/mobile/announcing-server-side-filters-for-real-time-graphql-subscriptions-with-aws-amplify/

Retrieving values from Firebase database?

Here is my structure of realtime database in firebase
{
"student1" : {
"name" : "somename",
"skillset" : [
"cpp",
"c",
"java"
],
other properties
},
"student2" : {
"name" : "somename",
"skillset" : [
"javascript",
"c",
"python"
],
other properties
},
"student3" : {
"name" : "somename",
"skillset" : [
"cpp",
"java"
],
other properties
},
"student4" : {
"name" : "somename",
"skillset" : [
"java",
"kotlin"
],
other properties
} }
I want to retrieve all the students having some specific set of all skills
e.g. skills = ["cpp","java"]
then answer should be ["student1","student3"]
Your current structure allows you to easily determine the skills for a user. It does however not make it easy to determine the users for a skill. To allow that, you'll need to add a reverse index, looking something like:
skills: {
java: {
student1: true,
student3: true,
student4: true
},
kotlin: {
student4: true
}
...
}
With the above you can look up the user IDs for a skill, and from there look up each user. For more on this, see my answer here: Firebase query if child of child contains a value
But this still won't allow you to query for users by multiple skills. To allow that, you'll have to add skill combinations to the new data structure. For example with the above skills, there is one user who knows both kotlin and Java:
skills: {
java: {
student1: true,
student3: true,
student4: true
},
java_kotlin: {
student4: true
}
kotlin: {
student4: true
}
...
}
While this leads to extra data, it performs quite well in practice, since you can always directly access the data that you need (so there's no real database query needed).
That is not possible under this structure, as firebase only support filtering by one child.
In your case, you would need to get all the data and filter by code

AppSync BatchDeleteItem not executes properly

I'm working on a React Native application with AppSync, and following is my schema to the problem:
type JoineeDeletedConnection {
items: [Joinee]
nextToken: String
}
type Mutation {
deleteJoinee(ids: [ID!]): [Joinee]
}
In 'request mapping template' to resolver to deleteJoinee, I have following (following the tutorial from https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-dynamodb-batch.html):
#set($ids = [])
#foreach($id in ${ctx.args.ids})
#set($map = {})
$util.qr($map.put("id", $util.dynamodb.toString($id)))
$util.qr($ids.add($map))
#end
{
"version" : "2018-05-29",
"operation" : "BatchDeleteItem",
"tables" : {
"JoineesTable": $util.toJson($ids)
}
}
..and in 'response mapping template' to the resolver,
$util.toJson($ctx.result.data.JoineesTable)
The problem is, when I ran the query, I got empty result and nothing deleted to database as well:
// calling the query
mutation DeleteJoinee {
deleteJoinee(ids: ["xxxx", "xxxx"])
{
id
}
}
// returns
{
"data": {
"deleteJoinee": [
null
]
}
}
I finally able to solve this puzzle, thanks to the answer mentioned here to point me to some direction.
Although, I noticed that JoineesTable does have trusted entity/role to the IAM 'Roles' section, yet it wasn't working for some reason. Looking into this more, I noticed that the existing policy had following actions as default:
"Action": [
"dynamodb:DeleteItem",
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:Query",
"dynamodb:Scan",
"dynamodb:UpdateItem"
]
Once I added following two more actions to the list, things have started working:
"dynamodb:BatchWriteItem",
"dynamodb:BatchGetItem"
Thanks to #Vasileios Lekakis and #Ionut Trestian on this appSync quest )

Can't create cloudsql role for Service Account via api

I have been trying to use the api to create service accounts in GCP.
To create a service account I send the following post request:
base_url = f"https://iam.googleapis.com/v1/projects/{project}/serviceAccounts"
auth = f"?access_token={access_token}"
data = {"accountId": name}
# Create a service Account
r = requests.post(base_url + auth, json=data)
this returns a 200 and creates a service account:
Then, this is the code that I use to create the specific roles:
sa = f"{name}#dotmudus-service.iam.gserviceaccount.com"
sa_url = base_url + f'/{sa}:setIamPolicy' + auth
data = {"policy":
{"bindings": [
{
"role": roles,
"members":
[
f"serviceAccount:{sa}"
]
}
]}
}
If roles is set to one of roles/viewer, roles/editor or roles/owner this approach does work.
However, if I want to use, specifically roles/cloudsql.viewer The api tells me that this option is not supported.
Here are the roles.
https://cloud.google.com/iam/docs/understanding-roles
I don't want to give this service account full viewer rights to my project, it's against the principle of least privilege.
How can I set specific roles from the api?
EDIT:
here is the response using the resource manager api: with roles/cloudsql.admin as the role
POST https://cloudresourcemanager.googleapis.com/v1/projects/{project}:setIamPolicy?key={YOUR_API_KEY}
{
"policy": {
"bindings": [
{
"members": [
"serviceAccount:sa#{project}.iam.gserviceaccount.com"
],
"role": "roles/cloudsql.viewer"
}
]
}
}
{
"error": {
"code": 400,
"message": "Request contains an invalid argument.",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.cloudresourcemanager.projects.v1beta1.ProjectIamPolicyError",
"type": "SOLO_REQUIRE_TOS_ACCEPTOR",
"role": "roles/owner"
}
]
}
}
With the code provided it appears that you are appending to the first base_url which is not the correct context to modify project roles.
This will try to place the appended path to: https://iam.googleapis.com/v1/projects/{project}/serviceAccount
The POST path for adding roles needs to be: https://cloudresourcemanager.googleapis.com/v1/projects/{project]:setIamPolicy
If you remove /serviceAccounts from the base_url and it should work.
Edited response to add more information due to your edit
OK, I see the issue here, sorry but I had to set up a new project to test this.
cloudresourcemanager.projects.setIamPolicy needs to replace the entire policy. It appears that you can add constraints to what you change but that you have to submit a complete policy in json for the project.
Note that gcloud has a --log-http option that will help you dig through some of these issues. If you run
gcloud projects add-iam-policy-binding $PROJECT --member serviceAccount:$NAME --role roles/cloudsql.viewer --log-http
It will show you how it pulls the existing existing policy, appends the new role and adds it.
I would recommend using the example code provided here to make these changes if you don't want to use gcloud or the console to add the role to the user as this could impact the entire project.
Hopefully they improve the API for this need.

Google Cloud Datastore runQuery returning 412 "no matching index found"

** UPDATE **
Thanks to Alfred Fuller for pointing out that I need to create a manual index for this query.
Unfortunately, using the JSON API, from a .NET application, there does not appear to be an officially supported way of doing so. In fact, there does not officially appear to be a way to do this at all from an app outside of App Engine, which is strange since the Cloud Datastore API was designed to allow access to the Datastore outside of App Engine.
The closest hack I could find was to POST the index definition using RPC to http://appengine.google.com/api/datastore/index/add. Can someone give me the raw spec for how to do this exactly (i.e. URL parameters, what exactly should the body look like, etc), perhaps using Fiddler to inspect the call made by appcfg.cmd?
** ORIGINAL QUESTION **
According to the docs, "a query can combine equality (EQUAL) filters for different properties, along with one or more inequality filters on a single property".
However, this query fails:
{
"query": {
"kinds": [
{
"name": "CodeProse.Pogo.Tests.TestPerson"
}
],
"filter": {
"compositeFilter": {
"operator": "and",
"filters": [
{
"propertyFilter": {
"operator": "equal",
"property": {
"name": "DepartmentCode"
},
"value": {
"integerValue": "123"
}
}
},
{
"propertyFilter": {
"operator": "greaterThan",
"property": {
"name": "HourlyRate"
},
"value": {
"doubleValue": 50
}
}
},
{
"propertyFilter": {
"operator": "lessThan",
"property": {
"name": "HourlyRate"
},
"value": {
"doubleValue": 100
}
}
}
]
}
}
}
}
with the following response:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "FAILED_PRECONDITION",
"message": "no matching index found.",
"locationType": "header",
"location": "If-Match"
}
],
"code": 412,
"message": "no matching index found."
}
}
The JSON API does not yet support local index generation, but we've documented a process that you can follow to generate the xml definition of the index at https://developers.google.com/datastore/docs/tools/indexconfig#Datastore_Manual_index_configuration
Please give this a shot and let us know if it doesn't work.
This is a temporary solution that we hope to replace with automatic local index generation as soon as we can.
The error "no matching index found." indicates that an index needs to be added for the query to work. See the auto index generation documentation.
In this case you need an index with the properties DepartmentCode and HourlyRate (in that order).
For gcloud-node I fixed it with those 3 links:
https://github.com/GoogleCloudPlatform/gcloud-node/issues/369
https://github.com/GoogleCloudPlatform/gcloud-node/blob/master/system-test/data/index.yaml
and most important link:
https://cloud.google.com/appengine/docs/python/config/indexconfig#Python_About_index_yaml to write your index.yaml file
As explained in the last link, an index is what allows complex queries to run faster by storing the result set of the queries in an index. When you get no matching index found it means that you tried to run a complex query involving order or filter. So to make your query work, you need to create your index on the google datastore indexes by creating a config file manually to define your indexes that represent the query you are trying to run. Here is how you fix:
create an index.yaml file in a folder named for example indexes in your app directory by following the directives for the python conf file: https://cloud.google.com/appengine/docs/python/config/indexconfig#Python_About_index_yaml or get inspiration from the gcloud-node tests in https://github.com/GoogleCloudPlatform/gcloud-node/blob/master/system-test/data/index.yaml
create the indexes from the config file with this command:
gcloud preview datastore create-indexes indexes/index.yaml
see https://cloud.google.com/sdk/gcloud/reference/preview/datastore/create-indexes
wait for the indexes to serve on your developer console in Cloud Datastore/Indexes, the interface should display "serving" once the index is built
once it is serving your query should work
For example for this query:
var q = ds.createQuery('project')
.filter('tags =', category)
.order('-date');
index.yaml looks like:
indexes:
- kind: project
ancestor: no
properties:
- name: tags
- name: date
direction: desc
Try not to order the result. After removing orderby(), it worked for me.

Resources