Is it possible to make Riak CS apply ACL with public access by default upon new bucket or file in some bucket created? I mean I wanna put files for example using simply
s3cmd put file.jpg s3://my-bucket
And I'd like to have file.jpg in my-bucket to be public accessible.
Do you mean that "objects are anonymously readable" by "public
access"? I will continue with the assumption that it is true.
Because ACL is per bucket or per object, bucket policy will be more
suitable for the use case. After creating the bucket my-bucket, one
can set particular bucket policy through PUT Bucket Policy API [1].
Example policy JSON to allow public access to the bucket is like this:
{
"Version": "2008-10-17",
"Id": "Policy1355283297687",
"Statement": [
{
"Sid": "Stmt1355283289",
"Action": [
"s3:GetObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::my-bucket/*",
"Principal": { "AWS": ["*"] }
}
]
}
Then you can PUT it with proper URL as API doc [1] by any means,
a simple way is to use s3cmd as
s3cmd setpolicy </path/to/above/json/as/file> s3://my-bucket
Then each object written under the bucket can be accessed by any user
including anonymous one.
Unfortunately there is no way to apply such bucket policy at creating
bucket but, I hope, it's not difficult to write wrapper script to
create bucket and apply policy to it.
[1] http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTpolicy.html
Related
The amplify docks here says that we can configure a lambda function as a dynamodb trigger by running **amplify add function** and selecting the "Lambda Trigger" option, but when I run the "amplify add api" (selected Python as runtime language) I am not getting the lambda trigger option, I'm only getting the "Serverless function" and "lambda layer" options.
Please help me to resolve this issue to access the feature.
docs snapshot - showing 4 options
my CLI snapshot - showing only 2 options
I know it works for nodejs runtime lambda, but I want this option for Python Lambda as well.
Just followed these steps with amplify CLI version 4.50.2.
To create a lambda function that is triggered by changes to a DynamoDB table, you can use the following command line actions, which are walked-through inside of the CLI after entering the below command:
amplify add function
Select which capability you want to add:
❯ Lambda function (serverless function)
Provide an AWS Lambda function name:
<YourFunctionsName>
Choose the runtime that you want to use:
> NodeJS # IMPORTANT: Must be NodeJS as of now, you can change this later by manually editing ...-cloudformation-template.json file inside function directory
Choose the function template you want to use
> Lambda Trigger
What event source do you want to associate with the lambda trigger
> Amazon DynamoDB Stream
Choose a DynamoDB event source option
>Use API category graphql #model backend DynamoDB table(s) in the current Amplify project
Choose the graphql #model(s)
<Select any models (using spacebar) you want to trigger the function after editing>
Do you want to trigger advanced settings
Y # IMPORTANT: If you are using a dynamodb event source based on a table defined by graphql schema, you will need to give this function read access to the api resource that contains the graphql schema that defines the table that drives the event
Do you want to access other resources in this project from your Lambda function?
y # See above, select your api that contains the data model and make sure that the function has at least read access.
After this, the other options (layer, call scheduling) are up to you.
After creating the function via the above CLI options, you can change the "Runtime" field inside the -cloudformation-template.json file inside function directory, eg if you want a python lambda function change the runtime to "python3.8". You will also need to create a file called index.py inside your function's directory which has a handler(event, context) function. See example below:
import json
def handler(event, context):
print("Triggered via DynamoDB")
print(event)
return json.dumps({'status_code': 200, "message": "Received from DynamoDB"})
After making these edits, you can run amplify push and, if you open your fxn in the management console online, it should show an attached dynamoDB stream.
Doesn't appear to be available anymore in the CLI codebase - see Supported-service.json deleted and replaced by supported-services.ts
https://github.com/aws-amplify/amplify-cli/commit/607ae21287941805f44ea8a9b78dd12d16d71f85#diff-a0fd8c5607fd81977cb4745b9af3af2c6649ded748991bf9968a7d782b000c6b
https://github.com/aws-amplify/amplify-cli/commits/4e974007d95c894ab4108a2dff8d5996e7e3ce25/packages/amplify-category-function/src/provider-utils/supported-services.ts
Select nodejs and you will be able to view lambda trigger
just add the following to {YOUR_FUNCTION_NAME}-cloudformation-template.json, remember to replace (YOUR_TABLE_NAME) to your table name.
"LambdaTriggerPolicyPurchase": {
"DependsOn": [
"LambdaExecutionRole"
],
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "amplify-lambda-execution-policy-Purchase",
"Roles": [
{
"Ref": "LambdaExecutionRole"
}
],
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"dynamodb:DescribeStream",
"dynamodb:GetRecords",
"dynamodb:GetShardIterator",
"dynamodb:ListStreams"
],
"Resource": {
"Fn::ImportValue": {
"Fn::Sub": "${apilanguageGraphQLAPIIdOutput}:GetAtt:(YOUR_TABLE_NAME):StreamArn"
}
}
}
]
}
}
},
"LambdaEventSourceMappingPurchase": {
"Type": "AWS::Lambda::EventSourceMapping",
"DependsOn": [
"LambdaTriggerPolicyPurchase",
"LambdaExecutionRole"
],
"Properties": {
"BatchSize": 100,
"Enabled": true,
"EventSourceArn": {
"Fn::ImportValue": {
"Fn::Sub": "${apilanguageGraphQLAPIIdOutput}:GetAtt:(YOUR_TABLE_NAME):StreamArn"
}
},
"FunctionName": {
"Fn::GetAtt": [
"LambdaFunction",
"Arn"
]
},
"StartingPosition": "LATEST"
}
},
i got them by creating a dummy function using the template that shows up after you choose nodejs and checking compare its -cloudformation-template.json with my own function
I can't update IAM policy in my AI Platform Notebook.
I created a new AI Platform Notebooks instance:
gcloud beta notebooks instances create nb1 \
--vm-image-project=deeplearning-platform-release \
--vm-image-family=tf-latest-cpu \
--machine-type=n1-standard-4 \
--location=us-west1-b
When I try to apply a new IAM policy I get an Error:
gcloud beta notebooks instances set-iam-policy nb1 --location=us-west1-b notebooks.policy
ERROR: (gcloud.beta.notebooks.instances.set-iam-policy) INTERNAL: An
internal error has occurred (506011f7-b62e-4308-9bde-10b97dd7b99c)
My policy looks like this:
{
"bindings": [
{
"members": [
"user:myuser#gmail.com",
],
"role": "roles/notebooks.admin"
}
],
"etag": "BwWlgdvxWT0=",
"version": 1
}
when I do a
gcloud beta notebooks instances get-iam-policy nb1 --location=us-west1-b --format=json
I get:
ACAB
As there is no policy set.
Please take a look at etag field:
An etag is returned in the response to getIamPolicy, and systems are expected to put that etag in the request to setIamPolicy to ensure that their change will be applied to the same version of the policy.
From documentation here
string (bytes format)
etag is used for optimistic concurrency control as a way to help prevent simultaneous updates of a policy from overwriting each other. It is strongly suggested that systems make use of the etag in the read-modify-write cycle to perform policy updates in order to avoid race conditions: An etag is returned in the response to getIamPolicy, and systems are expected to put that etag in the request to setIamPolicy to ensure that their change will be applied to the same version of the policy.
Important: If you use IAM Conditions, you must include the etag field whenever you call setIamPolicy. If you omit this field, then IAM allows you to overwrite a version 3 policy with a version 1 policy, and all of the conditions in the version 3 policy are lost.
A base64-encoded string.
You can just easily change your policy etag to ACAB, which is the default one.
{
"bindings": [
{
"members": [
"user:myuser#gmail.com",
],
"role": "roles/notebooks.admin"
}
],
"etag": "ACAB",
"version": 1
}
or you can use add-iam-policy-binding command, to create a new policy, then you can extract the etag using get-iam-policy and update your JSON file with it, finally run the set-iam-policy
You may also use this format:
{
"policy": {
"bindings": [
{
"members": [
"user:myuser#gmail.com"
],
"role": "roles/notebooks.admin"
}
],
"etag": "ACAB",
"version": 1
}
}
I have been trying to use the api to create service accounts in GCP.
To create a service account I send the following post request:
base_url = f"https://iam.googleapis.com/v1/projects/{project}/serviceAccounts"
auth = f"?access_token={access_token}"
data = {"accountId": name}
# Create a service Account
r = requests.post(base_url + auth, json=data)
this returns a 200 and creates a service account:
Then, this is the code that I use to create the specific roles:
sa = f"{name}#dotmudus-service.iam.gserviceaccount.com"
sa_url = base_url + f'/{sa}:setIamPolicy' + auth
data = {"policy":
{"bindings": [
{
"role": roles,
"members":
[
f"serviceAccount:{sa}"
]
}
]}
}
If roles is set to one of roles/viewer, roles/editor or roles/owner this approach does work.
However, if I want to use, specifically roles/cloudsql.viewer The api tells me that this option is not supported.
Here are the roles.
https://cloud.google.com/iam/docs/understanding-roles
I don't want to give this service account full viewer rights to my project, it's against the principle of least privilege.
How can I set specific roles from the api?
EDIT:
here is the response using the resource manager api: with roles/cloudsql.admin as the role
POST https://cloudresourcemanager.googleapis.com/v1/projects/{project}:setIamPolicy?key={YOUR_API_KEY}
{
"policy": {
"bindings": [
{
"members": [
"serviceAccount:sa#{project}.iam.gserviceaccount.com"
],
"role": "roles/cloudsql.viewer"
}
]
}
}
{
"error": {
"code": 400,
"message": "Request contains an invalid argument.",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.cloudresourcemanager.projects.v1beta1.ProjectIamPolicyError",
"type": "SOLO_REQUIRE_TOS_ACCEPTOR",
"role": "roles/owner"
}
]
}
}
With the code provided it appears that you are appending to the first base_url which is not the correct context to modify project roles.
This will try to place the appended path to: https://iam.googleapis.com/v1/projects/{project}/serviceAccount
The POST path for adding roles needs to be: https://cloudresourcemanager.googleapis.com/v1/projects/{project]:setIamPolicy
If you remove /serviceAccounts from the base_url and it should work.
Edited response to add more information due to your edit
OK, I see the issue here, sorry but I had to set up a new project to test this.
cloudresourcemanager.projects.setIamPolicy needs to replace the entire policy. It appears that you can add constraints to what you change but that you have to submit a complete policy in json for the project.
Note that gcloud has a --log-http option that will help you dig through some of these issues. If you run
gcloud projects add-iam-policy-binding $PROJECT --member serviceAccount:$NAME --role roles/cloudsql.viewer --log-http
It will show you how it pulls the existing existing policy, appends the new role and adds it.
I would recommend using the example code provided here to make these changes if you don't want to use gcloud or the console to add the role to the user as this could impact the entire project.
Hopefully they improve the API for this need.
I'm trying to use Cloud Functions for Firebase to serve content and I'd like to detect whether a user is logged in or not. I've set up a rewrite in my firebase.json that looks like this:
{
"database": {
"rules": "database.rules.json"
},
"hosting": {
"public": "public",
"rewrites": [{
"source": "**",
"function": "getProfile"
}]
}
}
This works fine and I'm serving appropriate content based on the path that's being requested. However, because I'm not doing anything on the client side (i.e. I'm using rewrites rather than client-side redirects), I'm missing the opportunity to get the current user from a client-side script.
Is there some way I can use a header or a property of the request object so that I can serve different content to logged in vs. non-logged in users in my server-side rewrites scenario?
Firebase Hosting passes along any cookie named __session when it calls a Cloud Function. An easy way to do this is to simply listen for ID tokens in your web app and set the cookie appropriately:
firebase.auth().onIdTokenChanged(user => {
if (user) {
user.getIdToken().then(token => {
document.cookie = `__session=${token};max-age=3600`;
});
} else {
document.cookie = '__session=;max-age=0';
}
});
Then, in your Cloud Function, you can parse the ID token out of the cookie and verify it using code like in this sample.
I'm trying to publish a data factory solution with this ADF DataLakeAnalyticsU-SQL pipeline activity following the azure step by step doc (https://learn.microsoft.com/en-us/azure/data-factory/data-factory-usql-activity).
{
"type": "DataLakeAnalyticsU-SQL",
"typeProperties": {
"scriptPath": "\\scripts\\111_risk_index.usql",
"scriptLinkedService": "PremiumAzureDataLakeStoreLinkedService",
"degreeOfParallelism": 3,
"priority": 100,
"parameters": {
"in": "/DF_INPUT/Consodata_Prelios_consegna_230617.txt",
"out": "/DF_OUTPUT/111_Analytics.txt"
}
},
"inputs": [
{
"name": "PremiumDataLakeStoreLocation"
}
],
"outputs": [
{
"name": "PremiumDataLakeStoreLocation"
}
],
"policy": {
"timeout": "06:00:00",
"concurrency": 1,
"executionPriorityOrder": "NewestFirst",
"retry": 1
},
"scheduler": {
"frequency": "Minute",
"interval": 15
},
"name": "ConsodataFilesProcessing",
"linkedServiceName": "PremiumAzureDataLakeAnalyticsLinkedService"
}
During publishing got this error:
25/07/2017 18:51:59- Publishing Project 'Premium.DataFactory'....
25/07/2017 18:51:59- Validating 6 json files
25/07/2017 18:52:15- Publishing Project 'Premium.DataFactory' to Data
Factory 'premium-df'
25/07/2017 18:52:15- Value cannot be null.
Parameter name: value
Trying to figure up what could be wrong with the project it came up that the issues reside into the activity options "typeProperties" as shown above, specifically for scriptPath and scriptLinkedService attributes. The doc says:
scriptPath: Path to folder that contains the U-SQL script. Name of the file
is case-sensitive.
scriptLinkedService: Linked service that links the storage that contains the
script to the data factory
Publishing the project without them (using hard-coded script) it will complete successfully. The problem is that I can't either figure out what exactly put into them. I tried with several combinations paths. The only thing I know is that the script file must be referenced locally into the solution as a dependency.
The script linked service needs to be Blob Storage, not Data Lake Storage.
Ignore the publishing error, its misleading.
Have a linked service in your solution to an Azure Storage Account, referred to in the 'scriptLinkedService' attribute. Then in the 'scriptPath' attribute reference the blob container + path.
For example:
"typeProperties": {
"scriptPath": "datafactorysupportingfiles/CreateDimensions - Daily.usql",
"scriptLinkedService": "BlobStore",
"degreeOfParallelism": 2,
"priority": 7
},
Hope this helps.
Ps. Double check for case sensitivity on attribute names. It can also throw unhelpful errors.