How to properly auto-scale groups of VMs in Cloudify? - cloudify

Question

I'm using cloudify community version 19.01.24.
Trying to figure out how to auto-scale a group of two VMs. Here's what I came up with so far (skipped irrelevant parts):
monitored_vm1_port:
type: cloudify.openstack.nodes.Port
properties:
openstack_config: *openstack_config
relationships:
- type: cloudify.relationships.contained_in
target: proxy_server_network
monitored_vm2_port:
type: cloudify.openstack.nodes.Port
properties:
openstack_config: *openstack_config
relationships:
- type: cloudify.relationships.contained_in
target: proxy_server_network
monitored_vm1_host:
type: cloudify.openstack.nodes.Server
properties:
image: { get_input: image }
flavor: { get_input: flavor }
resource_id: { concat: ['monitored_vm1-', { get_input: client_name }] }
agent_config:
user: { get_input: agent_user }
key: { get_property: [ keypair, private_key_path ] }
interfaces:
cloudify.interfaces.monitoring_agent:
install:
implementation: diamond.diamond_agent.tasks.install
inputs:
diamond_config:
interval: 10
start: diamond.diamond_agent.tasks.start
stop: diamond.diamond_agent.tasks.stop
uninstall: diamond.diamond_agent.tasks.uninstall
cloudify.interfaces.monitoring:
start:
implementation: diamond.diamond_agent.tasks.add_collectors
inputs:
collectors_config:
NetworkCollector: {}
relationships:
- type: cloudify.openstack.server_connected_to_port
target: monitored_vm1_port
- type: cloudify.openstack.server_connected_to_keypair
target: keypair
monitored_vm2_host:
type: cloudify.openstack.nodes.Server
properties:
image: { get_input: image }
flavor: { get_input: flavor }
resource_id: { concat: ['monitored_vm2-', { get_input: client_name }] }
agent_config:
user: { get_input: agent_user }
key: { get_property: [ keypair, private_key_path ] }
interfaces:
cloudify.interfaces.monitoring_agent:
install:
implementation: diamond.diamond_agent.tasks.install
inputs:
diamond_config:
interval: 10
start: diamond.diamond_agent.tasks.start
stop: diamond.diamond_agent.tasks.stop
uninstall: diamond.diamond_agent.tasks.uninstall
cloudify.interfaces.monitoring:
start:
implementation: diamond.diamond_agent.tasks.add_collectors
inputs:
collectors_config:
NetworkCollector: {}
relationships:
- type: cloudify.openstack.server_connected_to_port
target: monitored_vm2_port
- type: cloudify.openstack.server_connected_to_keypair
target: keypair
groups:
vm_group:
members: [monitored_vm1_host, monitored_vm2_host]
scale_up_group:
members: [monitored_vm1_host, monitored_vm2_host]
policies:
auto_scale_up:
type: scale_policy_type
properties:
policy_operates_on_group: true
scale_limit: 2 # max additional instances
scale_direction: '<'
scale_threshold: 31457280
service_selector: .*monitored_vm1_host.*network.eth0.rx.bit
cooldown_time: 60
triggers:
execute_scale_workflow:
type: cloudify.policies.triggers.execute_workflow
parameters:
workflow: scale
workflow_parameters:
delta: 1
scalable_entity_name: vm_group
scale_compute: true
policies:
vm_group_scale_policy:
type: cloudify.policies.scaling
properties:
default_instances: 1
targets: [vm_group]
So the blueprint gets deployed correctly, and scale workflow is being triggered according to specified condition (traffic on the VM's interface), but it fails during creation of the new VMs instances with following errors:
2019-11-18 14:54:46,591:ERROR: Task nova_plugin.server.create[f736f81c-7f8c-4f82-a280-8352c1d01bff] raised:
Traceback (most recent call last):
(...)
NonRecoverableError: Port 3b727b5e-a2ec-47cc-b711-37cb80a7b4e5 is still in use. [status_code=409]
Looks like Cloudify is trying to spawn new instances using existing ports, weird.
So I thought, maybe I should explicitly place VM's ports in the scaling group as well, to replicate them along with VMs. Tried it like this:
vm_group:
members: [monitored_vm1_host, monitored_vm1_port, monitored_vm2_host, monitored_vm2_port]
But in that case I get an error regarding some missing objects relationships, already at blueprint verification stage:
Invalid blueprint - Node 'monitored_vm1_host' and 'monitored_vm1_port' belong to some shared group but they are not contained in any shared node, nor is any ancestor node of theirs.
in: /opt/manager/resources/blueprint-with-scaling-d79fed3d-0b3b-4459-a851-fedd9ecf50c6/blueprint-with-scaling.yaml
I've gone through documentation and any examples I could find (there are not many), but they're unclear to me.
How can I get it scale correctly?

Answer 1

You got the first error as you said because Cloudify tried to scale the VM and connect it to the port that was already bound to the first VM.
The second error means that you cannot scale the port if it does not depend on a node that is also scaled, this is to avoid the scale of resources that can't be scaled.
The solution for this would be, to have a node of type cloudify.nodes.Root and connect to it by relationship the port, if the port depends on this node and this node will be part of the scale group you will be able to scale.
Your blueprint will have something like this:
my_relationship_node:
type: cloudify.nodes.Root
port:
type: cloudify.openstack.nodes.Port
properties:
openstack_config: *openstack_config
relationships:
- type: cloudify.relationships.connected_to
target: public_network
- type: cloudify.relationships.depends_on
target: public_subnet
- type: cloudify.openstack.port_connected_to_security_group
target: security_group
- type: cloudify.openstack.port_connected_to_floating_ip
target: ip
- type: cloudify.relationships.contained_in
target: my_relationship_node
I hope it helps.