I am interested to see if anyone knows of any better alternative to using conditional count statements in Terraform. By "conditional count statement", I mean a statement where depending a condition like a variable input, count will evaluate to create either 0 or 1 of a resource.
A simple example:
resource "xxxxxxxx" "example" {
count = var.example != null ? 1 : 0
}
Here, the resource will only be created if var.example has a value (is not null), otherwise it will not get created.
Conditional counts usually work ok in practice, but sometimes in more complex uses than the one above it introduces a risk of getting an error during Terraform Plan where it cannot evaluate the result of the count pre-apply.
The "count" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the count depends on.
Is there any better way to achieve the same effect of creating resources on a conditional basis in Terraform?
Keeping in mind that your terraform config not shown I've used a general example of for_each instead of count.
This is an example of how 2 CNAME records will be created using & in this example terraform modules are used but the same can be done directly on terraform resources.
locals {
cname_records = {
"email" = ["email.domain.net."]
"imap" = ["imap.domain.net."]
}
}
module "aws_route53_record_CNAME" {
source = "app.terraform.io/terraform-cloud-org/route53-record/aws"
version = "1.0.0"
for_each = local.cname_records
records = each.value
name = each.key
zone_id = "YOUR-HOSTED-ZONE-ID"
type = "CNAME"
ttl = "300"
}
It varies by case, but there are some cases where you need to get around this. There is a clever trick, but it seems obtuse in the single item (binary exists or not) case. But in a case where there are actually multiple items, this should help.
This is a completely contrived example but I actually use the method quite a bit. In short, a list of things not known until apply, can be replaced by a list of objects with a key you don't care about. The purpose is that for_each doesn't mind if the value is unknown at plan-time, only if the key is.
Consider the following root module with these four modules.
main.tf
resource "aws_s3_bucket" "this" {
bucket = "h4s-test-bucket"
}
# module "with_count" { # cannot be determined until apply
# source = "./with-count"
# x = aws_s3_bucket.this.id
# }
module "with_for_each_over_item" { # handy workaround
source = "./with-for-each-over-item"
x = aws_s3_bucket.this.id
}
output "with_for_each_over_item" {
value = module.with_for_each_over_item
}
# module "with_for_each_list" { # cannot be determined until apply
# source = "./with-for-each-list"
# x = [aws_s3_bucket.this.id]
# }
module "with_for_each_list_better" { # handy workaround
source = "./with-for-each-list-better"
x = [{ y = aws_s3_bucket.this.id }]
}
output "with_for_each_list_better" {
value = module.with_for_each_list_better
}
module "with_for_each_list_best" { # handier workaround
source = "./with-for-each-list-best"
x = [aws_s3_bucket.this.id]
}
output "with_for_each_list_best" {
value = module.with_for_each_list_best
}
with-count/main.tf (problematic)
variable "x" {
type = string
default = null
}
resource "null_resource" "this" {
count = var.x != null ? 1 : 0
}
output "this" {
value = null_resource.this
}
with-for-each-over-item/main.tf (handy workaround)
variable "x" {
type = string
default = null
}
resource "null_resource" "this" {
for_each = { for i, v in [var.x] : i => v }
}
output "this" {
value = null_resource.this
}
with-for-each-list/main.tf (problematic)
variable "x" {
type = list(string)
default = []
}
resource "null_resource" "this" {
for_each = toset(var.x)
}
output "this" {
value = null_resource.this
}
with-for-each-list-better/main.tf (handy workaround)
variable "x" {
type = list(object({ y = string }))
default = []
}
resource "null_resource" "this" {
for_each = { for i, v in var.x : i => v }
}
output "this" {
value = null_resource.this
}
with-for-each-list-best/main.tf (handiest workaround)
variable "x" {
type = list(string)
default = []
}
resource "null_resource" "this" {
for_each = { for i, v in var.x : i => v }
}
output "this" {
value = null_resource.this
}
Summary
In cases where the variable has a value not known at plan-time, consider using an object where the key is known.
Related
I am trying to find a way how to pass/map the list variable to kubernetes_deployment container environment variables. I run an asp.net core application in the container and I trying to modify the app with setting env variables to override the app settings.json. It is not a problem to override single values from settings.json, the problem is if I need to define/override the whole array there.
I have variable list like this in terrafrom:
variable "allowed_cars" {
type = list(
object({
manufacturer = string
model = string
})
)
}
Then, I have resource definition of kubernetes_deployment with containers.
I believe it could work if I set the env variables for the container like this:
env {
name = "App__AllowedCars__0__Manufacturer"
value = "xxx"
}
env {
name = "App__AllowedCars__0__Model"
value = "xxx"
}
env {
name = "App__AllowedCars__1__Manufacturer"
value = "xxx"
}
env {
name = "App__AllowedCars__1__Model"
value = "xxx"
}
...
Is there a way how to pass these env variables to the container in a dynamic way based on allowed_cars terraform variable? I don't know how many items will be defined for each environments, etc...
Thank you very much.
Something of this definition (used example of azurerm_container_group in azure)
environment_variables = "${merge(var.env_vars,var.secure_env_vars,local.master_env)}"
and then variables can be passed in
variable "env_vars" {
type = "map"
description = "envvaars"
default = {
WEB_USER = "locust"
HATCH_RATE = 25
LOCUST_COUNT = 50
LOCUST_FILE = "/locust/locustfile.py"
ATTACKED_HOST = "https://api-perf.yrdy.com"
}
}
variable "secure_env_vars" {
type = "map"
description = "secure env vars"
default = {
WEB_PASSWORD = "dummy"
API_KEY = "test"
}
}
As #ydaetskcoR mentioned, the dynamic block was solution for me.
https://www.terraform.io/docs/language/expressions/dynamic-blocks.html
dynamic "env" {
for_each = var.allowed_cars
content {
name = "App__AllowedCars__0__${env.key}__Manufacturer"
value = env.value["manufacturer"]
}
}
The examples in typesafe config require you to know a path to specify literally, e.g.:
config1.getString("complex-app.something")
I need to be able to parse a config file with free-form labels in places. How can I handle that.
Say that I have HOCON like:
root : {
a : { x: 1, y: 2 },
b : { x: 2, z: 3 }
}
Without knowing what keys are in the file, how can I find out that there are keys 'a' and 'b' and that the value of the object with key 'a' has keys 'x' and 'y' and for 'b' there are keys 'x' and 'z'?
Here is one way to not do it:
Config config = ConfigFactory.load("configula");
Config root = config.getObject("root").toConfig();
for (Entry<String, ConfigValue> e: root.entrySet()) {
// The key is a.x, a.y etc. ??!?
// The value is not a map-like thing
}
With the keys of the entry set being a path, I have to parse the path to find what the keys were, which is pretty close to making it easier to simply parse a text file with StringTokenizer or the like.
What is the the config way of parsing nested objects? For example, with groovy and ConfigSlurper a similar structure:
root {
a {
x = 1
y = 2
}
b {
x = 2
z = 3
}
}
would be parsed like this:
cfg = new ConfigSlurper().parse(new File('configula.config'))
cfg.root.each { k, r ->
r.each { kk, v ->
println "${k}, ${kk} -> ${v}"
}
}
every combination of things I've tried ends up with the value being a ConfigValue. Is there a way to get a tree of keys that map to objects that themselves contain keys that map to objects?
To answer my own question: ConfigObject extends java.util.Map so, a way to navigate when a key can be any arbitrary value, is to use ConfigObject.keySet().
Is there something in R (either a package or base idiom) that is like an Option as found in Scala and other languages (see tag optional for details). Specifically, I'm looking for the following features some object that can:
signify the absence of a value but easily
hold attributes
return a default value in the face of having no contained value without requiring that the result of the default value be calculated unless it is actually needed
I'm sure there are a lot of other nice characteristics of Options that I haven't fully recognized as I'm relatively new to the idiom. Any answer that can provide more than the above listed features gets bonus points, especially if the additional features can be described well.
I tried writing a poor substitute using an R6 class (below). Anything that works better or is more idiomatically aligned with R would be greatly appreciated.
library(R6)
Option <- R6Class("Option",
public = list(
initialize = function(value=NULL) {
self$value <- value
}
,get = function() {
return(self$value)
}
,set = function(value) {
self$value <- value
return(value)
}
,getOrElse = function(...) {
if(self$isDefined()) {
return(self$value)
} else {
return(eval(...))
}
}
,isDefined = function() {
return(!all(is.null(self$value)) && !all(is.na(self$value)))
}
, value = NULL
)
,private = list()
,active = list()
) #end Option
Example:
bob <- Option$new()
bob$isDefined() == FALSE
bob$getOrElse("a") == "a"
bob$getOrElse({Sys.sleep(2);"b"})=="b"
bob$set(value = "a")
bob$isDefined() == TRUE
bob$getOrElse({Sys.sleep(2);"b"})=="a"
Powershell exposes some parameters, "dynamic parameters", based on context. The MSDN page explains the mechanism pretty well, but the skinny is that to find out about these one must call GetDynamicParameters(), which returns a class containing the additional parameters. I need to get these parameters via reflection, and (here's the crux of it), in a ReflectionOnly context (that is, the types are loaded with ReflectionOnlyLoadFrom). So, no Assembly.InvokeMember("GetDynamicParameters").
Can this be done?
No. Reflection works against static assembly metadata. Dynamic parameters in powershell are added at runtime by the command or function itself.
Perhaps this helps:
1: Defintion of the dynamic parameters
#===================================================================================
# DEFINITION OF FREE FIELDS USED BY THE CUSTOMER
#-----------------------------------------------------------------------------------
# SYNTAX: #{ <FF-Name>=#(<FF-Number>,<isMandatory_CREATE>,<isMandatory_UPDATE>); }
$usedFFs = #{
"defaultSMTP"=#(2,1,0); `
"allowedSMTP"=#(3,1,0); `
"secondName"=#(100,1,0); `
"orgID"=#(30001,1,0); `
"allowedSubjectTypeIDs"=#(30002,1,0); `
}
# FF-HelpMessage for input
$usedFFs_HelpMSG = #{ 2="the default smtp domain used by the organizaiton. Sampel:'algacom.ch'"; `
3="comma seperated list of allowed smtp domains. Sampel:'algacom.ch,basel.algacom.ch'"; `
100="an additional organization name. Sampel:'algaCom AG')"; `
30001="an unique ID (integer) identifying the organization entry"; `
30002="comma seperated list of allowed subject types. Sampel:'1,2,1003,10040'"; `
}
2: definition of function that builds the dynamic parameters
#-------------------------------------------------------------------------------------------------------
# Build-DynParams : Used to build the dynamic input parameters based on $usedFFs / $usedFFs_HelpMSG
#-------------------------------------------------------------------------------------------------------
function Build-DynParams($type) {
$paramDictionary = New-Object -Type System.Management.Automation.RuntimeDefinedParameterDictionary
foreach($ffName in $usedFFs.Keys) {
$ffID = $usedFFs.Item($ffName)[0]
$dynAttribCol = New-Object -Type System.Collections.ObjectModel.Collection[System.Attribute]
$dynAttrib = New-Object System.Management.Automation.ParameterAttribute
$dynAttrib.ParameterSetName = "__AllParameterSets"
$dynAttrib.HelpMessage = $usedFFs_HelpMSG.Item($ffID)
switch($type) {
"CREATE" { $dynAttrib.Mandatory = [bool]($usedFFs.Item($ffName)[1]) }
"UPDATE" { $dynAttrib.Mandatory = [bool]($usedFFs.Item($ffName)[2]) }
}
$dynAttribCol.Add($dynAttrib)
$dynParam = New-Object -Type System.Management.Automation.RuntimeDefinedParameter($ffName, [string], $dynAttribCol)
$paramDictionary.Add($ffName, $dynParam)
}
return $paramDictionary
}
3. Function that makes use of the dynamic params
#-------------------------------------------------------------------------------------------------------
# aAPS-OrganizationAdd : This will add a new organization entry
#-------------------------------------------------------------------------------------------------------
Function aAPS-OrganizationAdd {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$true,HelpMessage="The name of the new organization")]
[String]$Descr,
[Parameter(Mandatory=$false,HelpMessage="The name of the parent organization")]
[String]$ParentDescr=$null,
[Parameter(Mandatory=$false,HelpMessage="The status of the new organization [1=Active|2=Inactive]")]
[int]$Status = 1,
[Parameter(Mandatory=$false,HelpMessage="If you want to see the data of the deactivated object")]
[switch]$ShowResult
)
DynamicParam { Build-DynParams "CREATE" }
Begin {}
Process {
# do what oyu want here
}
End {}
}
I'm trying to write a trait (in Scala 2.8) that can be mixed in to a case class, allowing its fields to be inspected at runtime, for a particular debugging purpose. I want to get them back in the order that they were declared in the source file, and I'd like to omit any other fields inside the case class. For example:
trait CaseClassReflector extends Product {
def getFields: List[(String, Any)] = {
var fieldValueToName: Map[Any, String] = Map()
for (field <- getClass.getDeclaredFields) {
field.setAccessible(true)
fieldValueToName += (field.get(this) -> field.getName)
}
productIterator.toList map { value => fieldValueToName(value) -> value }
}
}
case class Colour(red: Int, green: Int, blue: Int) extends CaseClassReflector {
val other: Int = 42
}
scala> val c = Colour(234, 123, 23)
c: Colour = Colour(234,123,23)
scala> val fields = c.getFields
fields: List[(String, Any)] = List((red,234), (green,123), (blue,23))
The above implementation is clearly flawed because it guesses the relationship between a field's position in the Product and its name by equality of the value on those field, so that the following, say, will not work:
Colour(0, 0, 0).getFields
Is there any way this can be implemented?
Look in trunk and you'll find this. Listen to the comment, this is not supported: but since I also needed those names...
/** private[scala] so nobody gets the idea this is a supported interface.
*/
private[scala] def caseParamNames(path: String): Option[List[String]] = {
val (outer, inner) = (path indexOf '$') match {
case -1 => (path, "")
case x => (path take x, path drop (x + 1))
}
for {
clazz <- getSystemLoader.tryToLoadClass[AnyRef](outer)
ssig <- ScalaSigParser.parse(clazz)
}
yield {
val f: PartialFunction[Symbol, List[String]] =
if (inner.isEmpty) {
case x: MethodSymbol if x.isCaseAccessor && (x.name endsWith " ") => List(x.name dropRight 1)
}
else {
case x: ClassSymbol if x.name == inner =>
val xs = x.children filter (child => child.isCaseAccessor && (child.name endsWith " "))
xs.toList map (_.name dropRight 1)
}
(ssig.symbols partialMap f).flatten toList
}
}
Here's a short and working version, based on the example above
trait CaseClassReflector extends Product {
def getFields = getClass.getDeclaredFields.map(field => {
field setAccessible true
field.getName -> field.get(this)
})
}
In every example I've seen the fields are in reverse order: the last item in the getFields array is the first one listed in the case class. If you use case classes "nicely", then you should just be able to map productElement(n) onto getDeclaredFields()( getDeclaredFields.length-n-1).
But this is rather dangerous, as I don't know of anything in the spec that insists that it must be that way, and if you override a val in the case class, it won't even appear in getDeclaredFields (it'll appear in the fields of that superclass).
You might change your code to assume things are this way, but check that the getter method with that name and the productIterator return the same value and throw an exception if they don't (which means that you don't actually know what corresponds to what).
You can also use the ProductCompletion from the interpreter package to get to attribute names and values of case classes:
import tools.nsc.interpreter.ProductCompletion
// get attribute names
new ProductCompletion(Colour(1, 2, 3)).caseNames
// returns: List(red, green, blue)
// get attribute values
new ProductCompletion(Colour(1, 2, 3)).caseFields
Edit: hints by roland and virtualeyes
It is necessary to include the scalap library which is part of the scala-lang collection.
Thanks for your hints, roland and virtualeyes.