Error getting keys from Azure Storage Account with listkeys(...) method with Bicep syntax - azure-resource-manager

I have a Bicep template to create an Azure Storage Account
#description('the name of the storage account')
param name string
#description('the alias of the storage account')
param shortName string
#description('tags')
param tags object
#description('the name of the key vault resource where place output secrets')
param keyVaultName string
resource storageAccount 'Microsoft.Storage/storageAccounts#2022-09-01' = {
name: name
location: resourceGroup().location
sku: {
name: 'Standard_LRS'
tier: 'Standard'
}
kind: 'StorageV2'
tags: union(tags, {
type: 'storage-account'
})
}
Then, I need to get the keys
var keys = listkeys(storageAccount.id, storageAccount.apiVersion)
output keyObject object = keys[0]
output KeyValue string = keys[0].value
But everytime that I runs the template, I receive these errors:
{
"code": "DeploymentOutputEvaluationFailed",
"message": "Unable to evaluate template outputs: 'keyObject,keyValue'. Please see error details and deployment operations. Please see https://aka.ms/arm-common-errors for usage details.",
"details": [
{
"code": "DeploymentOutputEvaluationFailed",
"target": "keyObject",
"message": "The template output 'keyObject' is not valid: The language expression property '0' can't be evaluated, property name must be a string.."
},
{
"code": "DeploymentOutputEvaluationFailed",
"target": "keyValue",
"message": "The template output 'keyValue' is not valid: The language expression property '0' can't be evaluated, property name must be a string.."
}
]
}
The purpose of get keys is to save it into Azure Key Vault by using KeyValue var from previous step
resource keyVault 'Microsoft.KeyVault/vaults#2022-07-01' existing = {
name: keyVaultName
}
resource secret 'Microsoft.KeyVault/vaults/secrets#2022-07-01' = {
parent: keyVault
name: secretName
properties: {
value: KeyValue
contentType: 'plain/text'
}
}
So..
What's wrong with listKeys(...) method?

By following this tweet https://twitter.com/adotfrank/status/1341084692100108288?s=46&t=sWx0hvS0sS47llWLlbWZTw I found an alternative method to get keys.
Just referencing to a storage account object and use the method listKeys()
resource storageAccount 'Microsoft.Storage/storageAccounts#2022-09-01' = {
name: name
location: resourceGroup().location
sku: {
name: 'Standard_LRS'
tier: 'Standard'
}
kind: 'StorageV2'
tags: union(tags, {
type: 'storage-account'
})
}
var storageAccountKeys = storageAccount.listKeys()
Then, I can access to primary or secondary key with storageAccountKeys.keys[0].value
This fix solve my issue.

Related

ARM resource group deployment showing modification for new deployments eventhough there are no chnages

I am using below Bicep file for Azure role assignments . So here I have a Azuredevops pipeline which wil build the bicepfile to arm template and from pipeline variables, the paramaters.json file will be getting updated.
main.bicep
targetScope = 'resourceGroup'
#description('Principal type of the assignee.')
#allowed([
'Device'
'ForeignGroup'
'Group'
'ServicePrincipal'
'User'
])
param principalType string
#description('the id for the role defintion, to define what permission should be assigned')
param RoleDefinitionId string
#description('the id of the principal that would get the permission')
param principalId string
#description('the role deffinition is collected')
resource roleDefinition 'Microsoft.Authorization/roleDefinitions#2018-01-01-preview' existing = {
scope: resourceGroup()
name: RoleDefinitionId
}
resource RoleAssignment 'Microsoft.Authorization/roleAssignments#2020-10-01-preview' = {
name: guid(resourceGroup().id, RoleDefinitionId, principalId)
properties: {
roleDefinitionId: roleDefinition.id
principalId: principalId
principalType: principalType
}
}
paramters.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"principalType": {
"value": "#{principalType}#"
},
"RoleDefinitionId": {
"value": "#{RoleDefinitionId}#"
},
"principalId": {
"value": "#{principalId}#"
}
}
}
pipeline build task for creation deployment.
'az deployment group create --resource-group $(resourceGroup) --template-file $(System.DefaultWorkingDirectory)/template/main.json --parameters $(System.DefaultWorkingDirectory)/template/parameters.json'
When I triggered the pipeline firstime, i got output summary as below.
The deployment will update the following scope:
Scope: /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/XXXXXXXXXXXXXXXXXXX-rg
+ Microsoft.Authorization/roleAssignments/xxxxxxxxxxxxxxxx [2020-10-01-preview]
apiVersion: "2020-10-01-preview"
id: "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/XXXXXXXXXXXXXXXXXXX-rg/providers/Microsoft.Authorization/roleAssignments/xxxxxxxxxxxxxxxxx"
name: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
properties.principalId: "xxxxxxxxxxxxx"
properties.roleDefinitionId: "/subscriptions/XXXXXXXXXXXXXXXXXXXXX/resourceGroups/XXXXXXXXXXXXXXXXXXX-rg/providers/Microsoft.Authorization/roleDefinitions/xxxxxxxxxxxxxxxxxxxxxxx"
type: "Microsoft.Authorization/roleAssignments"
And after that, if I retrigger the pipeline again without any change to the templates. Its showing as 1 to modify, but expected that the output will show as "no change". Because we havenet made any changes to the resource either from pipeline side or manually.
Scope: /subscriptions/xxxxxxxxxxxxxxxxxx/resourceGroups/xxxxxxxxxxxxxxxx-rg
~ Microsoft.Authorization/roleAssignments/xxxxxxxxxxxxxxxxxxxxxxx [2020-10-01-preview]
~ properties.roleDefinitionId: "/subscriptions/xxxxxxxxxxxxxxxxxxx/providers/Microsoft.Authorization/roleDefinitions/xxxxxxxxxxxxxxxxxxxxxxxxx" => "/subscriptions/xxxxxxxxxxxxxxxxxxxx/resourceGroups/xxxxxxxxxxxxxxx-rg/providers/Microsoft.Authorization/roleDefinitions/xxxxxxxxxxxxxxxxxxx"
x properties.principalType: "Group"
Resource changes: 1 to modify
iF i again deploy also, the next time again will show the same output as 1 to modify
What is the issue here, Why ARM deployment is showing changes eventhough there are no changes.
Azure built-in role definitions are defined at the subscription level unless it is a custom role that you've created at the another scope.
In your bicep file, you can change the scope of the roleDefinition resource:
#description('the role deffinition is collected')
resource roleDefinition 'Microsoft.Authorization/roleDefinitions#2018-01-01-preview' existing = {
scope: subscription()
name: RoleDefinitionId
}
or you could also use subscriptionResourceId:
resource RoleAssignment 'Microsoft.Authorization/roleAssignments#2020-10-01-preview' = {
name: guid(resourceGroup().id, RoleDefinitionId, principalId)
properties: {
roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', RoleDefinitionId)
principalId: principalId
principalType: principalType
}
}

Azure Bicep multiple scopes in template

After using terraform for a very long time, I decided to start learning azure bicep. So far I am trying to have a grip on the logic. So far I have playing around on deployment of a storage account and keyvault. What I am doing here is the following.
create a storage account
use existing key vault to store storage account connection string as secret
create a key based on the storage account name
And this works as I am expected.
So I wanted to take one step forward. and here is where I am a bit confused.
What I wanted to do, is to use the same bicep template, to create a new secret but in a different resource group into a different key vault.
Now according to my understand of azure documentation, the template comes with a default scope which in my specific case target my default subscription and to run my bicep template from the terminal I use the command
az deployment group create -f ./template.bicep -g <resource-group-name>
and this is my template:
// Default values I'm using to test
param keyVaultName string = '<keyvault-name>'
param managedIdentityName string = 'test-managed-identity'
param tenantCodes array = [
'elhm'
'feor'
]
// I'm using prefix so I dont need to create additional arrays
var keyVaultKeyPrefix = 'Client-Key-'
var storagePrefix = 'sthrideveur'
// Get a reference to key vault
resource keyVault 'Microsoft.KeyVault/vaults#2021-06-01-preview' existing = {
name: keyVaultName
}
// Create a managed identity
resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities#2018-11-30' = {
name: managedIdentityName
location: resourceGroup().location
}
// Grant permissions to key vault
resource accessPolicy 'Microsoft.KeyVault/vaults/accessPolicies#2019-09-01' = {
name: '${keyVault.name}/add'
properties: {
accessPolicies: [
{
tenantId: subscription().tenantId
objectId: managedIdentity.properties.principalId
permissions: {
// minimum required permissions
keys: [
'get'
'unwrapKey'
'wrapKey'
]
}
}
]
}
}
// Create key vault keys
resource keyVaultKeys 'Microsoft.KeyVault/vaults/keys#2021-06-01-preview' = [for tenantCode in tenantCodes: {
name: '${keyVault.name}/${keyVaultKeyPrefix}${tenantCode}'
properties: {
keySize: 2048
kty: 'RSA'
// storage key should only needs these operations
keyOps: [
'unwrapKey'
'wrapKey'
]
}
}]
// Create storage accounts
resource storageAccount 'Microsoft.Storage/storageAccounts#2021-04-01' = [for tenantCode in tenantCodes: {
name: '${storagePrefix}${tenantCode}'
location: resourceGroup().location
kind: 'StorageV2'
sku: {
name: 'Standard_RAGRS'
}
// Assign the identity
identity: {
type: 'UserAssigned'
userAssignedIdentities: {
'${managedIdentity.id}': {}
}
}
properties: {
allowCrossTenantReplication: true
minimumTlsVersion: 'TLS1_2'
allowBlobPublicAccess: false
allowSharedKeyAccess: true
networkAcls: {
bypass: 'AzureServices'
virtualNetworkRules: []
ipRules: []
defaultAction: 'Allow'
}
supportsHttpsTrafficOnly: true
encryption: {
identity: {
// specify which identity to use
userAssignedIdentity: managedIdentity.id
}
keySource: 'Microsoft.Keyvault'
keyvaultproperties: {
keyname: '${keyVaultKeyPrefix}${tenantCode}'
keyvaulturi: keyVault.properties.vaultUri
}
services: {
file: {
keyType: 'Account'
enabled: true
}
blob: {
keyType: 'Account'
enabled: true
}
}
}
accessTier: 'Cool'
}
}]
// Store the connectionstrings in KV if specified
resource storageAccountConnectionStrings 'Microsoft.KeyVault/vaults/secrets#2019-09-01' = [ for (name, i) in tenantCodes :{
name: '${keyVault.name}/${storagePrefix}${name}'
properties: {
value: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount[i].name};AccountKey=${listKeys(storageAccount[i].id, storageAccount[i].apiVersion).keys[0].value};EndpointSuffix=${environment().suffixes.storage}'
}
}]
according to the documentation here https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/deploy-to-resource-group?tabs=azure-cli
When I need to target a specific resource group, I can use the scope in the resource, so I create this:
resource keyvaultApi 'Microsoft.KeyVault/vaults#2021-06-01-preview' existing = {
name: keyVaultApiName
scope: resourceGroup('secondresourcegroup')
}
So far no errors, but the problem happens when I had to create a managed identity resource.
resource keyvaultApi 'Microsoft.KeyVault/vaults#2021-06-01-preview' existing = {
name: keyVaultApiName
scope: resourceGroup('secondresourcegroup')
}
resource managedIdentityTwo 'Microsoft.ManagedIdentity/userAssignedIdentities#2018-11-30' = {
name: managedIdentityNameTwo
location: resourceGroup().location
}
resource accessPolicyApi 'Microsoft.Media/videoAnalyzers/accessPolicies#2021-11-01-preview' = {
name: '${keyvaultApi.name}/add'
properties: {
accessPolicies: [
{
tenantId: subscription().tenantId
objectId: managedIdentityTwo.properties.principalId
permissions: {
// minimum required permissions
keys: [
'get'
'unwrapKey'
'wrapKey'
]
}
}
]
}
}
In the key vault I could declare the scope, but to the underlying resources, such as access policy etc, I cannot declare the scope. So how can bicep understand that those resources needs to target a specific resource group and specific key vault?
Because when I run the terminal command, I am targeting a specific resource group, so I don't really understand how I can use one template to target different resource groups and resources accordingly.
I hope I made my point clear, and please if I didn't, just feel free to ask me more informations.
Thank you so much for your time and help
UPDATE:
When I try to run the code as it is, I get the following error:
{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"NotFound","message":"{\r\n \"error\": {\r\n \"code\": \"ParentResourceNotFound\",\r\n \"message\": \"Can not perform requested operation on nested resource. Parent resource 'secondkeyvault' not found.\"\r\n }\r\n}"}]}}
UPDATE:
So I followed the Daniel lead and in a second template I deployed the code I needed for the second template as follow:
template2.bicep
param deploymentIdOne string = newGuid()
param deploymentIdTwo string = newGuid()
output deploymentIdOne string = '${deploymentIdOne}-${deploymentIdTwo}'
output deploymentIdTwo string = deploymentIdTwo
// Default values I'm using to test
param keyVaultApiName string = 'secondkeyvaultapi'
param managedIdentityNameTwo string = 'second-second-identity'
var keyVaultKeyPrefixTw = 'Client-Key-'
param tenantCodes array = [
'tgrf'
]
resource keyvaultApi 'Microsoft.KeyVault/vaults#2021-06-01-preview' existing = {
name: keyVaultApiName
}
resource managedIdentityTwo 'Microsoft.ManagedIdentity/userAssignedIdentities#2018-11-30' = {
name: managedIdentityNameTwo
location: resourceGroup().location
}
resource accessPolicyApi 'Microsoft.KeyVault/vaults/accessPolicies#2019-09-01' = {
name: '${keyvaultApi.name}/add'
properties: {
accessPolicies: [
{
tenantId: subscription().tenantId
objectId: managedIdentityTwo.properties.principalId
permissions: {
// minimum required permissions
keys: [
'get'
'unwrapKey'
'wrapKey'
]
}
}
]
}
}
// Store the connectionstrings in KV if specified
resource clientApiKeys 'Microsoft.KeyVault/vaults/secrets#2019-09-01' = [ for name in tenantCodes :{
name: '${keyvaultApi.name}/${keyVaultKeyPrefixTw}${name}'
properties: {
value: '${deploymentIdOne}-${deploymentIdTwo}'
}
}]
and in my main template I added the module:
module clientKeyApi 'template2.bicep' = {
name: 'testfrgs'
scope: 'secondresourcegroup'
}
But there is something that is not clear 100% to me.
How does it work to override all the for loop and parameters name I have declared in my template2.bicep , and yet the module require a scope subscription, if I declare the scope, wouldn't this override the default value?
Sorry guys for the newbie questions, I am trying to break my mindset from terraform and understand better how bicep work.
Any explanation would be amazing and helpful
You can't specify scope on the resource, but you can specify it on a module. You'll need to turn the resource that adds the access policy to the keyvault into a separate module, then specify scope on the module. You can also make the scope for your deployment subscription, but then you'll need to break everything that targets a specific resource group into modules, as well.
This is due to how ARM deployments work. The default scope for an ARM deployment is at the resource group level. You can't point a resource at a different resource group because it's outside the scope of the deployment.
Modules, however, run as sub-deployments and therefore can have a different scope set.
This is a case where Terraform is more straightforward, since it calls the Azure APIs directly instead of using the ARM deployment model. Terraform doesn't care about deployment scopes because it doesn't use them.

Amplify JS API GraphQL Elasticsearch throws "ResolverExecutionLimitReached" error

I've implemented the Amplify JS Library with a Vue project and have had success with all of the features of the library except this issue. When I query a model with Elasticsearch, it returns the appropriate results, but also the error of "ResolverExecutionLimitReached".
This is the request:
let destinations = await API.graphql(graphqlOperation(queries.searchDestinations, {filter: { deviceId: { eq: params.id }}}))
This is the schema:
type Destination
#model
#searchable
#auth(rules: [{ allow: public }, { allow: private }])
#key(name: "byXpoint", fields: ["xpoint"])
#key(name: "byDevice", fields: ["deviceId"])
{
id: ID!
index: Int!
levels: [String]
name: String!
xpoint: String
sourceId: ID
Source: Source #connection
lock: Boolean
breakaway: Boolean
breakaways: String
probeId: ID!
probe: Probe #connection(fields: ["probeId"])
deviceId: ID!
device: Device #connection(fields: ["deviceId"])
orgId: ID!
org: Org #connection(fields: ["orgId"])
}
And this returns:
{
data: {
searchDestinations: {items: Array(100), nextToken: "ba1dc119-2266-4567-9b83-f7eee4961e63", total: 384}
},
errors: [
{
data: null
errorInfo: null
errorType: "ResolverExecutionLimitReached"
locations: []
message: "Resolver invocation limit reached."
path: []
}
]
}
My understanding is the AppSync API has a hard limit of returning more than 1000 entries, but this query is on a table with only ~600 entries and is only returning 384. I am executing the same command via AppSync directly via a NodeJS application and it works without issue.
Not sure where to investigate further to determine what is triggering this error. Any help or direction is greatly appreciated.
Connections in the schema were causing the single request to go beyond the 1000 request limit (exactly as stated by Mickers in the comments). Updated schema with less connections on fetch and issue was resolved.

Getting 'query in command must target a single shard'

I have a CosmosDB setup using the Mongo API. I have a collection with a hashed shard on one of the field of the document. When I run commands like db.collection.remove or db.collection.deleteMany I get the following error.
Command deleteMany failed: query in command must target a single shard key.: {"message":"Command deleteMany failed: query in command must target a single shard key."}
I'm not sure how can I mention a shard key as part of the query considering I want the query to run across all the shards.
You need to provide shard key when you want to run commands like db.collection.remove or db.collection.deleteMany.
For example :
My data source as below:
[
{
"id" : "2",
"name" : "b"
},
{
"id" : "1",
"name" : "a"
}
]
And my shared key is "/name". Use db.coll.deleteMany({"name":"a"}) to delete specific shard.
Hope it helps you.
It should be ShardKey which you have chosen when you created cosmosDb collection.
FilterDefinition<Product> filter = Builders<Product>.Filter.Eq("id", 2);
=== 2 is my shardKey
await this._dbContext.GetProducts.DeleteOneAsync(filter);
return RedirectToAction("Index");
Kindly refer an image below , how does it look like in CosmosDB
Shard Key(Partition Key) has to be provided during specification of schema model in the code. Once its provided, we can perform regular operation like save, update and delete as usual.
Example:
const mySchema = new Schema({
requestId: { type: String, required: true },
data: String,
documents: [{ docId: String, name: String, attachedBy: String }],
updatedBy: {
type: {
name: { type: String, required: true },
email: { type: String, required: true },
}, required: true
},
createdDate: { type: Date, required: true },
updatedDate: { type: Date },
}, { shardKey: { requestId: 1 } }
);
In the above code we specified requestId as Shard Key, now we can perform any mongo operations
Example:
let request:any = await myModel.findById(requestId);
request.data ="New Data";
await request.save();
Hope that helps.
This works with all Mongo operations

Error: Resolver associated with data sources

I'm having the serverless error:
Resolver associated with data sources when building from serverless.yml config file:
# serverless.yml
...
mappingTemplates:
- dataSource: Wallet
type: Query
field: walletFromId
request: "_dynamo-get-wallet.txt"
response: "_generic-result-response.txt"
- dataSource: Wallet
type: Query
field: walletsFromUser
request: "_dynamo-get-wallets-from-user.txt"
response: "_generic-result-response.txt"
- dataSource: Wallet
type: Mutation
field: registerWallet
request: "_dynamo-put-wallet.txt"
response: "_generic-result-response.txt"
dataSources:
- type: AMAZON_DYNAMODB
name: Wallet
description: 'Wallet DataSource'
config:
tableName: "${self:custom.stage}-Wallet"
serviceRoleArn: "arn:aws:iam::${self:custom.accountId}:role/${self:custom.appSync.serviceRole}"
...
I also have a schema.graphql:
type Query {
# query the wallet with given id and get the output with detail info
walletFromId(walletId: String!): Wallet!
# query wallets with given user id and get list of cards
walletsFromUser(userId: String!): [Wallet!]!
}
type Mutation {
# Add a wallet to an existing user
registerWallet(userId: String!, number: String!, cvx: String!, expirationDate: String!): Wallet!
}
type Wallet {
walletId: String!
userId: String!
number: String!
cvx: String!
expirationDate: String!
}
type Subscription {
addWallet: Wallet
#aws_subscribe(mutations: ["registerWallet"])
}
schema {
query: Query
mutation: Mutation
subscription: Subscription
}
I could not find a single clue as to what this error mean, and there isn't anything else I can get from the build logs.
This error usually means you are trying to delete a data source that is currently being used by a resolver. If you can identify the resolver pointing to the data source and delete it then you should no longer see the error message.

Resources