I am trying to authenticate Google Datastore c# SDK in a k8 pod running in google cloud.
I could not find any way to inject the account.json file in to DatastoreDb or DatastoreClient beside using GOOGLE_APPLICATION_CREDENTIALS environment variable.
Using GOOGLE_APPLICATION_CREDENTIALS environment variable is problematic since i do not want to leave the account file exposed.
According to the documentations in: https://googleapis.github.io/google-cloud-dotnet/docs/Google.Cloud.Datastore.V1/index.html
When running on Google Cloud Platform, no action needs to be taken to
authenticate.
But that does not seem to work.
A push in the right direction will be appreciated (:
I'd suggest using a K8s secret to store the service account key and then mounting it in the pod at run time. See below:
Create a service account for the desired application.
Generate and encode a service account key: just generate a .json key for the newly created service account from the previous step and then encode it using base64 -w 0 key-file-name. This is important: K8S expects the secret's content to be Base64 encoded.
Create the K8s secret manifest file (see content below) and then apply it.
apiVersion: v1
kind: Secret
metadata:
name: your-service-sa-key-k8s-secret
type: Opaque
data:
sa_json: previously_generated_base64_encoding
Mount the secret.
volumes:
- name: service-account-credentials-volume
secret:
secretName: your-service-sa-key-k8s-secret
items:
- key: sa_json
path: secrets/sa_credentials.json
Now all you have to do is set the GOOGLE_APPLICATION_CRENDENTIALS to be secrets/sa_credentials.json.
Hope this helps. Sorry for the formatting (on a hurry).
This is how it can be done:
var credential =
GoogleCredential.FromFile(#"/path/to/google.credentials.json").CreateScoped(DatastoreClient.DefaultScopes);
var channel = new Grpc.Core.Channel(DatastoreClient.DefaultEndpoint.ToString(), credential.ToChannelCredentials());
DatastoreClient client = DatastoreClient.Create(channel, settings:
DatastoreSettings.GetDefault());
DatastoreDb db = DatastoreDb.Create(YOUR_PROJECT_ID, client:client);
// Do Datastore stuff...
// Shutdown the channel when it is no longer required.
await channel.ShutdownAsync();
Taken from: https://github.com/googleapis/google-cloud-dotnet/blob/master/apis/Google.Cloud.Datastore.V1/Google.Cloud.Datastore.V1/DatastoreClient.cs
Related
I have the following policies:
path "/kv/dev/*" {
capabilities = ["read","list", "update"]
}
path "/kv/data/dev/*" {
capabilities = ["read","list", "update"]
}
Using the CLI I and able to use the following command to get the secrets:
vault kv get -mount=kv dev/db
And it outputs the secrets correctly. The issue occurs when using the the UI
-With the input of dev/db I get Ember Data Request POST /v1/sys/capabilities-self returned a 400 Payload (application/json) [object Object]
-With the input of /data/dev/db I get undefined is not an object (evaluating 'n.data')
Any advice on how to access the secrets using the UI ?
I think I get the state you are looking for. Let me share with you what i did:
First I specified in my terminal what I need in terms of my Vault:
export VAULT_TOKEN='the token I use to authenticate myself in the UI'
export VAULT_ADDR='my vault address'
Login myself in the same way i will do in the UI:
vault login -method=token token=$VAULT_TOKEN
Creating policy
vault policy write my-policy - << EOF
path "/kv/dev/*" {
capabilities = ["read","list", "update"]
}
path "/kv/data/dev/*" {
capabilities = ["read","list", "update"]
}
EOF
Enabling secrets engine for specific path. As you can see in this StackOverflow question
vault secrets enable -path=kv kv
Inserting and reading secret:
vault kv put kv/dev/db value=yes
vault kv get -mount=kv dev/db
After all of this steps I can see the secret in:
VAULT_ADDR/ui/vault/secrets/kv/show/dev/db
So, if VAULT_ADDR was http://127.0.0.1:8200 the full path in the browser will be:
http://127.0.0.1:8200/ui/vault/secrets/kv/show/dev/db
When running amplify push -y in the CLI, my project errors with this message:
["Index: 0 State: {\"deploy\":\"waitingForDeployment\"} Message: Resource is not in the state stackUpdateComplete"]
How do I resolve this error?
The "Resource is not in the state stackUpdateComplete" is the message that comes from the root CloudFormation stack associated with the Amplify App ID. The Amplify CLI is just surfacing the error message that comes from the update stack operation. This indicates that the Amplify's CloudFormation stack may have been still be in progress or stuck.
Solution 1 – “deployment-state.json”:
To fix this issue, go to the S3 bucket containing project settings and deleted the “deployment-state.json” file in root folder as this file holds the app deployment states. The bucket should end with, or contain the word “deployment”.
Solution 2 – “Requested resource not found”:
Check the status of the CloudFormation stack and see if you can notice that the stack failed because of a “Requested resource not found” error indicating that the DynamoDB table “tableID” was missing and confirm that you have deleted it (possibly accidentally). Manually create the above DynamoDB table and retry to push again.
Solution 3A - “#auth directive with 'apiKey':
If you recieve an error stating that “#auth directive with 'apiKey' provider found, but the project has no API Key authentication provider configured”. This error appears when you define a public authorisation in your GraphQL schema without specifying a provider. The public authorization specifies that everyone will be allowed to access the API, behind the scenes the API will be protected with an API Key. To be able to use the public API you must have API Key configured.
The #auth directive allows the override of the default provider for a given authorization mode. To fix the issue specify “IAM” as the provider which allows to use an "Unauthenticated Role" from Cognito Identity Pools for public access instead of an API Key.
Below is the sample code for public authorisation rule:
type Todo #model #auth(rules: [{ allow: public, provider: iam, operations: [create, read, update, delete] }]) {
id: ID!
name: String!
description: String
}
After making the above changes, you can run “amplify update api” and add a IAM auth provider, the CLI generated scoped down IAM policies for the "UnAuthenticated" role automatically.
Solution 3B - Parameters: [AuthCognitoUserPoolId] must have values:
Another issue could occur here, where the default authorization type is API Key when you run the command “amplify add api” without specifying the API type. To fix this issue, follow these steps:
Deleted the the API
Recreate a new one by specifying the “Amazon Cognito user pool” as the authorization mode
Add IAM as an additional authorization type
Re-enable #auth directive in the newly created API Schema
Run “amplify push”
Documentation:
Public Authorisation
Troubleshoot CloudFormation stack issues in my AWS Amplify project
Cannot for the life of me get Weaviate to launch with OIDC authentication - I have tried every combination under the sun and the pod falls into a CrashLoopBackOff.
I can successfully deploy with the defaults (anon access = true and no admin list); but as soon as OIDC is added, no luck.
Is there something wrong with my config? I add the config as per the documentation to the values.yaml before continuing on with the instructions.
I should note, not at all familiar with Kubernetes and using this as a learning experience.
Thanks so much for any help
YAML (from azure portal):
data:
conf.yaml: |-
---
authentication:
anonymous_access:
enabled: false
oidc:
client_id: client-id
enabled: true
issuer: https://issuerURL/authorize/
skip_client_id_check: false
username_claim: email
authorization:
admin_list:
enabled: true
users:
- user#user.com
query_defaults:
limit: 100
debug: false
For the issuer URL with Azure you will want it to be of the format https://login.microsoftonline.com/xxx-xxx-xxx-xxx/v2.0. You can find this via Azure > App Registrations > Endpoints > OpenID Metadata document (without the .well-known/openid-configuration suffix.
As of Weaviate version 1.15.3, the Weaviate console and python client do not work with Azure but there is an issue to fix this planned for 1.16.
This is the code snippet in my main.tf file:
provider "github" {
token = var.github_token_ssm
owner = var.owner
}
data "github_repository" "github" {
full_name = var.repository_name
}
The github token is stored in AWS secretsmanager parameter.
If the value of the token is hardcoded github token, then it works fine.
If the value of the token is a AWS secretsmanager parameter (eg. arn:aws:secretsmanager:us-east-1:xxxxxxxxxxxx:secret:xxxx-Github-t0UOOD:xxxxxx), it is not working.
I don't want to hardcode github token in the code. How can I use secretsmanager parameter for token above?
As far as I know, Terraform not supporting aws Secret Manager (but you can use the vault to store secrets).
you can also deploy it with TF_VAR variable and ENV Var
export TF_VAR_db_username=admin TF_VAR_db_password=adifferentpassword
You can also run a script that will pull the secret from aws and store it in EnvVar.
just remember to secure your state file (the password will exist in clear text)
I'm trying to save encrypted data to DynamoDB with the help of the "Amazon DynamoDB Encryption Client for Java".
I also use the Serverless framework to deploy my application (some Lamdba functions using DynamoDB). The AWS Lamdba functions are written in Kotlin.
I have this in my serverless.yml in the iamRoleStatements section (under the provider section):
- Effect: “Allow”
Action:
- “kms:GenerateDataKey”
Resource: “*”
I thought that it should be enough but I'm getting this error when I'm trying to save encrypted data to DynamoDB:
com.amazonaws.services.kms.model.AWSKMSException: User: arn:aws:sts::120102300450:assumed-role/appname-username-eu-west-1-lambdaRole/appname-username-functionname is not authorized to perform: kms:GenerateDataKey on resource: arn:aws:kms:eu-west-1:120102300450:key/12d3f45c-6fff-0007-b123-5bfe5678e012 (Service: AWSKMS; Status Code: 400; Error Code: AccessDeniedException; Request ID: ...)
(the alphanumeric IDs are obfuscated here to not disclose real data)
I also tried to add several other KMS permissions but it didn't help:
- Effect: "Allow"
Action:
- "kms:GenerateDataKey"
- "kms:GenerateDataKeyWithoutPlaintext"
- "kms:CreateAlias"
- "kms:CreateKey"
- "kms:Decrypt"
- "kms:Encrypt"
- "kms:EnableKey"
- "kms:UpdateAlias"
Resource: "*"
So, the whole purpose of the "Amazon DynamoDB Encryption Client for Java" cannot be achieved in this case.
What is missing here?
I've found what was the problem. I used the default KMS key, and I had to use a custom key. Then it worked.