I am testing out publishing to SNS using the AWS Privatelink feature. While at that, I found that I am able to create multiple endpoints to the SNS service (com.amazonaws.us-west-1.sns). Is there a valid use case for having multiple endpoints to the SNS service ?
An endpoint is a VPC level construct and therefore not usable outside the VPC it was created in. I am assuming in your question you are looking for use cases for having multiple endpoints to the same AWS service in the same VPC.
Creation of an endpoint requires several input parameters, several of which are listed under Creating an Interface Endpoint at Interface VPC Endpoints (AWS PrivateLink). Alternatively, these input parameters can be best experienced on the endpoint creation form in the AWS Console itself.
It is to allow for varying combinations of these input parameters, to build or accommodate fine-grained security policies and segmentation practices, that multiple endpoints make sense.
For instance, you may have two different applications in the same VPC, best given access to the SNS service with actions and topics specific to their unique requirements. This example policy from the documentation at Creating an Amazon VPC Endpoint Policy for Amazon SNS should be able to highlight the possibilities:
{
"Statement": [{
"Action": ["sns:Publish"],
"Effect": "Allow",
"Resource": "arn:aws:sns:us-east-2:123456789012:MyTopic",
"Principal": "arn:aws:iam:123456789012us-east:user/MyUser"
}]
}
Alternatively, your applications may require association with different Security Groups (which is one of the input parameters to creating an endpoint.) In which case, dedicated endpoints to those segments of the network, optionally with other customisations such as that of the associated policy, would provide the desired levels of segmentation. The segmentation itself may be required to actively prevent accidental application misconfiguration from negatively impacting the business, or to contain threats if they were to arise in a particular application.
Related
this question has been on my mind for several months now. In very simple terms, pretty much any SaaS that operates B2B will be required to isolate user-generated files (PDFs, Images, Documents) between tenants. If a user from "Company A" uploads a PDF to the SaaS, only members of "Company A" should be able to read/see this file. Seems pretty basic.
However, I can't seem to find any information on how to design a system like this on a high, architectural level. Without going into the specifics of a cloud provider, my understanding is that IAM rules are usually meant for members of the organization that provides the service (so the company selling the SaaS), not for their end users. Another option, pre-signed URLs, seem like they are not truly secure, since sharing the link will grant access to anyone, even if not part of "Company A".
I know of a single service that can authorize storage bucket access down to an individual object, and that is Google's Firestore (Storage Rules). How do they do it?!? Is there a reverse proxy that checks for valid auth and looks up the access that user has been granted before "tunneling back" the response?
TLDR: On a system design level, how does Firebase's Storage Rules enable object-level authorization for objects? Is it a reverse proxy to a private bucket that auths the user, checks permission to the requested file and sends back the data via the proxy? This seems like powerful proxies are needed to serve all incoming requests.
Firebase servers do not use a reverse proxy. They serve objects directly from the storage bucket after evaluating the conditions you've set in the Storage Rules.
You really shouldn't have to worry about using proxies at all. If you have on-premises data you need to access consider using a VPN behind the application. However object-level storage rules are the better choice as you have more control and security down to the object level.
However, I can't seem to find any information on how to design a system like this on a high, architectural level. Without going into the specifics of a cloud provider, my understanding is that IAM rules are usually meant for members of the organization that provides the service (so the company selling the SaaS), not for their end users. Another option, pre-signed URLs, seem like they are not truly secure, since sharing the link will grant access to anyone, even if not part of "Company A".
A common approach is to not use the AWS root user.
For more basic apps, one would just
Create an IAM group using only the built-in policy AmazonS3FullAccess (or limiting even more to just the bucket)
Create a user and adds the user to that group.
That's the user one uses to access the S3 bucket for all of the existing users in that app (which means your guess is partially correct).
Also, generally speaking, to access the file, what one does is to use an S3 URL along with AWSAccessKeyId, Signature and Expires. So users can't access it without the extra parameters or after the expiration time.
When it comes to multitenant applications, it's also possible to
Create an IAM role for each tenant.
Create a bucket policy to allow your app Tenants to access it.
Ensure that each file uploaded to S3 has an object key with your app Tenant ID prefix.
So, this way you can ensure that only users with that specific IAM role can access objects in the bucket that have a specific prefix.
On a system design level, how does Firebase's Storage Rules enable object-level authorization for objects?
First, one has to understand we're dealing with Object storage (as opposed to Block or File storage). Check here for a Storage systems overview.
In that same article, Alex Xu mentions that
Most public cloud service providers have an object storage offering, such as AWS S3, Google block storage, and Azure blob storage.
In addition to placing the offerings in the same level of storage type, Xu also has an article where he explains how an S3-like storage system work.
I've been using Cloud Functions for a while and it's been great so far - though, it seems like there's no builtin way to set limits on how often the function is invoked.
I've set the max # instances to a reasonable number, but for the # invocations, Firebase doesn't really provide a way to set this. Would using a Node package that limits or slows down requests, when combined with the limited max instances be sufficient to slow down attacks if they happen?
Also know Cloud Endpoints exist - I'm pretty new to OpenAPI and it seems like something that should just be integrated with Functions at an additional cost... but wondering if that would be a good solution too.
Pretty new to all this so appreciate any help!
If you use only Google Cloud services (I don't know the other cloud provider offers to solve your issue, or even existing framework for this), you can limit the unwanted access at different layer
Firtly, Google Front End (GFE) protects all Google resources (Gmail, Maps, Cloud, Your cloud functions,...) especially against layer 3 and layer 4 common DDoS attacks. In addition, this layer is in charge of the TLS communication establishment, and will also discard the bad connexions.
Activate the "private mode". This mode forbid the unauthenticated request. With this feature, Google Front End will check if
A id_token is present in the request header
If the token is valid (correct signature, not expired)
If the identity of the token is authorized to access to the resource.
-> Only the valid request reach your service and you will pay only for that. All the bad traffic is processed by Google and "paid" by Google.
Use a Load balancer with Cloud Armor activated. You can also customize your WAF policies if you need them. Use it in front of your Cloud Functions thanks to the serverless NEG feature
If you use API Keys, you can use Cloud Endpoint (or API Gateway, a managed version of Cloud Endpoint) where you can enforce rate limit per API keys. I wrote an article on this (Cloud Endpoinr + ESPv2)
I have an API that is hosted on GCP (no matter if its Compute Eninge, App Engine or any other) and I want to give access to it for selected applications that run outside of my company. I do not need such sophisticated service like Apigee (it is too expensive also), I just want to distinguish applications that call my API and do some individual restrictions for each (like which endpoint they can call).
As I have done some research I think that Cloud Endpoints + API Keys fits my requirements. I could create API Key for each client (API Key is sent in query, so I suppose I can get it in my backend?), but I am a bit confused if it is "the right/recommended" way. As stated here https://cloud.google.com/endpoints/docs/openapi/when-why-api-key#api_keys_provide_project_authorization it should help me to identify project, or rather for my case application, which is communicating with my API, but it feels like its more dedicated for apps that run within my organizations because it can have access to Google services Apis (like I should not give API Key to someone else outside, although I can even restrict IP addresess which can use it).
Would be glad to hear if someone had a smiliar goal and if solved it this way.
I am looking in to how to use API gateway in my project. My current API gateway pattern idea is to create a layer of composite APIs calling multiple APIs, and expose the composite APIs through the API gateway.
Researching elsewhere, I've found that it is not recommended to create composite APIs if using an API gateway, but rather that composition should happen in the API gateway itself. But if I need to do the composition in API gateway, doesn't that mean that I need to expose some domain logic there, because would I need to construct a meaningful request contract and possibly a sequence of calling APIs? Personally, I don't feel comfortable doing that.
Is there other way to do this without exposing some logic?
From what I research, it is not recommended to create composite API if using API gateway but composition should happen in the API gateway itself.
could you refer where did you find the statement?
IMHO the service composition is not task of the api gateway. It's task of n integration layer (you don't need to have a separate esb product or service, under integration lyer you can understand any services or service layer implementing the capabilities, such as composition, transformation, etc)
You did not specify any api product or service, it depends on the product if it is capable to execute any logic or not (some do, some some don't). Even if the api gateway product is capable of executing any logic, indeed I don't recommend it either (you will have to fight with maintenance, error and state handling,..)
But if I need to do the composition in API gateway, doesn't it means that I need to expose some domain logic there
(personally, I don't feel comfortable doing that) because I need to construct a meaningful request contract and possibly sequence of calling APIs?
what do you mean by composition in API gateway? I assume under the term executing some logic. Then the logic is hidden from the api clients.
Or could it be layer of composite API sounds better?
Indeed.
Generally try to treat an api gateway as a smart proxy (doing authorization, throttling, client facing api store,..) but the exposed backend services should be already exposed as they are
I am trying stream data from the AWS API Gateway to DynamoDB in AWS (directly, without something like lambda). I have looked for several tutorials, such as [1] and [2], who describe exactly this scenario. All of these assume that the right policies and roles are in place. Normally, I play and hack around till I get a working proof of concept, after which I rebuild a proper model, but with access rights I want to make sure I understand what I am doing. For [2], I also found a stack overflow question at [3] from somebody with the same problem that got solved, but not sure exactly how. I also looked at [4], describing API Gateway with Lambda.
Here is my guess:
Create a policy that allows calling from the API Gateway.
"AmazonAPIGatewayInvokeFullAccess" fits the name, but might not
be necessary + overkill with too much access
Create a policy that allows access to dynamoDB.
Here, "AmazonDynamoDBFullAccess" might be appropriate, even
though it might be overkill (too much access), and might only work
from the Management Console
Create a role that has those two policies attached.
Here, I run into the trouble that when I click create role, and
select AWS service, I can not find the correct "service that will use
this role" that has the policies I described above behind it. For
example, when clicking dynamoDB, I get the following "use-cases", none of which seem to relate to the dynamoDB full access policy:
Amazon DynamoDB Accelerator (DAX) - DynamoDB access
DynamoDB - Global Tables
DynamoDB Accelerator (DAX) - Cluster management
My main question is: How do I set the right minimal set of roles and policies to connect AWS API Gateway to DynamoDB (read and write), as described in [1]?
[1] https://sanderknape.com/2017/10/creating-a-serverless-api-using-aws-api-gateway-and-dynamodb/
[2] https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-as-a-proxy-for-dynamodb/
[3] API Gateway does not have permission to assume the provided role DynamoDB
[4] https://docs.aws.amazon.com/apigateway/latest/developerguide/permissions.html
What you need to do is create an IAM Service Role that allows API Gateway to assume this role. You can easily do this through the UI. When you create a new role, the "Service Role" is selected by default and below the "Choose the service that will use this role" header, you can select API Gateway.
A role is a container of permissions that can be assumed by a certain entity (in our case, an API Gateway API resource). Your role needs "permissions" for the role to have any use. You add this permissions by adding policies to your role. This is explained more in depth here: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html
Be sure to read the AWS Service Role part. You say that you need to "Create a policy that allows calling from the API Gateway" but this is incorrect: you need to create a role that can be assumed by API Gateway.
In your case, you'll want specific DynamoDB permissions for your role. Following the least-privilege principle as you mention, you should only add the specific actions for the specific DynamoDB table. The list of possible permissions can be found here: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/api-permissions-reference.html
Let's say you only want API Gateway to get items from a specific table. Your policy might look something like this then:
{
"Effect": "Allow",
"Action": "dynamodb:GetItem",
"Resource": "arn:aws:dynamodb:eu-west-1:[aws_account_id]:table/[table_name]"
}
Hope this helps!
This recent new tutorial by ankana likhita sri priya includes starting off in high detail/screenshots of IAM (Policy, Role, etc.): https://medium.com/#likhita507/using-api-gateway-to-get-data-from-dynamo-db-using-without-using-aws-lambda-e51434a4f5a0