According to official OpenStack SDK documentation I can pass "min_count" argument while creating a server to create more than one server. However, no matter whether I input 1, 2 or 5, OpenStack only creates one instance (without throwing any kind of exception, info etc.). I tried inputting the min_count as a string and as an int, to no avail.
So how do I create multiple instances in Python Openstack SDK? I know I can do that in a loop, but I need to create them all at once :)
Thanks in advance for all the help
instances_number=2
server = conn.compute.create_server(
name=name,
image_id=image.id,
flavor_id=flavor.id,
networks=[{"uuid": network.id}],
key_name=keypair.name,
user_data=init_script,
has_config_drive=True,
min_count=instances_number,
max_count=instances_number
)
Related
This is my first time setting up an OpenStack instance on Ubuntu and I'm having some difficulty with setting up additional compute nodes. I've set up a controller node following the devstack instructions here with the stable/xena release and I'm trying to add an additional compute node so I've gone through the set up here but I have a few questions.
The additional compute node does not show up as a hypervisor (although it shows up under the compute service list), does someone have a resource for how to add the compute node as a hypervisor?
I ran the discover_hosts tool within the devstack repo so that the compute node gets picked up by the db but what transport url and database connections should the additional compute node use? Do I copy the transport url and database connection url used by the controller node?
Does Openstack use the resources (storage, RAM, cpus) of the additional compute node to create new VMs as well?
If someone could provide advice on how to go about setting up this compute node that would be greatly appreciated.
Thanks in advance!
Note: In the comments below I mention some steps I tried so I'll just sum them up here with their results.
nova-manage cell_v2 discover_hosts --verbose gave this output:
Found 3 cell mappings.
Skipping cell0 since it does not contain hosts. Getting computes from cell 'cell1': <random_string1> .
Found 0 unmapped computes in cell <random_string1>.
Getting computes from cell: <random_string2>.
Checking host mapping for compute host 'vmname': <random_string3>.
Found 0 unmapped computes in cell <random_string2>
So the command runs but I think theres an issue with how things are set up in the db since the compute node doesn't seem to be linked to a cell.
nova-manage cell_v2 list_hosts output gives 2 hosts, the controller and the vm I am trying to add but the cell name for the compute node I'm trying to add is None.
nova-manage cell_v2 list_cells output gives 3 cells, one with no name value but it has the same cell uuid as <random_string2> in the above comment with a transport url that has no /nova_cell1 ending and the db connection string is the same as cell0.
So I think there is an issue with how the compute node is trying to be added to the db?
1, try run nova-manage cell_v2 discover_hosts in controller node to discover hypervisor.
2, you should not do anything if the step 1 works.
3, yes if the step 1 works.
We need to connect to on premise Teradata from Azure Databricks .
Is that possible at all ?
If yes please let me know how .
I was looking for this information as well and I recently was able to access our Teradata instance from Databricks. Here is how I was able to do it.
Step 1. Check your cloud connectivity.
%sh nc -vz 'jdbcHostname' 'jdbcPort'
- 'jdbcHostName' is your Teradata server.
- 'jdbcPort' is your Teradata server listening port. By default, Teradata listens to the TCP port 1025
Also check out Databrick’s best practice on connecting to another infrastructure.
Step 2. Install Teradata JDBC driver.
Teradata Downloads page provides JDBC drivers by version and archive type. You can also check the Teradata JDBC Driver Supported Platforms page to make sure you pick the right version of the driver.
Databricks offers multiple ways to install a JDBC library JAR for databases whose drivers are not available in Databricks. Please refer to the Databricks Libraries to learn more and pick the one that is right for you.
Once installed, you should see it listed in the Cluster details page under the Libraries tab.
Terajdbc4.jar dbfs:/workspace/libs/terajdbc4.jar
Step 3. Connect to Teradata from Databricks.
You can define some variables to let us programmatically create these connections. Since my instance required LDAP, I added LOGMECH=LDAP in the URL. Without LOGMECH=LDAP it returns “username or password invalid” error message.
(Replace the text in italic to the values in your environment)
driver = “com.teradata.jdbc.TeraDriver”
url = “jdbc:teradata://Teradata_database_server/Database=Teradata_database_name,LOGMECH=LDAP”
table = “Teradata_schema.Teradata_tablename_or_viewname”
user = “your_username”
password = “your_password”
Now that the connection variables are specified, you can create a DataFrame. You can also explicitly set this to a particular schema if you have one already. Please refer to Spark SQL Guide for more information.
Now, let’s create a DataFrame in Python.
My_remote_table = spark.read.format(“jdbc”)\
.option(“driver”, driver)\
.option(“url”, url)\
.option(“dbtable”, table)\
.option(“user”, user)\
.option(“password”, password)\
.load()
Now that the DataFrame is created, it can be queried. For instance, you can select some particular columns to select and display within Databricks.
display(My_remote_table.select(“EXAMPLE_COLUMN”))
Step 4. Create a temporary view or a permanent table.
My_remote_table.createOrReplaceTempView(“YOUR_TEMP_VIEW_NAME”)
or
My_remote_table.write.format(“parquet”).saveAsTable(“MY_PERMANENT_TABLE_NAME”)
Step 3 and 4 can also be combined if the intention is to simply create a table in Databricks from Teradata. Check out the Databricks documentation SQL Databases Using JDBC for other options.
Here is a link to the write-up I published on this topic.
Accessing Teradata from Databricks for Rapid Experimentation in Data Science and Analytics Projects
If you create a virtual network that can connect to on prem then you can deploy your databricks instance into that vnet. See https://docs.azuredatabricks.net/administration-guide/cloud-configurations/azure/vnet-inject.html.
I assume that there is a spark connector for terradata. I haven't used it myself but I'm sure one exists.
You can't. If you run Azure Databricks, all the data needs to be stored in Azure. But you can call the data using REST API from Teradata and then save data in Azure.
My original problem was that I want to increase my DynamoDB write throughput before I run the pipeline, and then decrease it when I'm done uploading (doing it max once a day, so I'm fine with the decreasing limitations).
They only way I found to do it is through a shell script that will issue the API commands to alter the throughput. How does it work with my AMI access_key and secret_key when it's a resource that pipeline creates for me? (I can't log in to set the ~/.aws/config file and don't really want to create an AMI just for this).
Should I write the script in bash? can I use ruby/python AWS SDK packages for example? (I prefer the latter..)
How do I pass my credentials to the script? do I have runtime variables (like #startedDate) that I can pass as arguments to the activity with my key and secret? Do I have any other way to authenticate with either the commandline tools or the SDK package?
If there is another way to solve my original problem - please let me know. I've only got to the ShellActivity solution because I couldn't find anything else in documentations/forums.
Thanks!
OK. found it - http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-concepts-roles.html
The resourceRole in the default object in your pipeline will be the one assigned to resources (Ec2Resource) that are created as a part of the pipeline activation.
The default one in configured to have all your permissions and AWS commandline and SDK packages are automatically looking for those credentials so no need to update ~/.aws/config of pass credentials manually.
I have a StoredProcedure that returns a simple table containing several records:
DECLARE #STEPS_TABLE AS TABLE (OrchestrationID uniqueidentifier, [Message] nvarchar(1000));
-- LOADING THE VALUES HERE
SELECT * FROM #STEPS_TABLE As Step FOR XML AUTO, XMLDATA, ELEMENTS
I used the SQL Transport Schema Generation Wizard to create my schema and could configure the port correctly. If I use this schema on my orchestration, it works perfectly. BizTalk starts one instance of the orchestration everytime the #STEPS_TABLE has more than 1 record.
Reading Microsoft technical documentation, they recommend to get several messages in one call and then use the XML pipeline to disassemble the multi-row BizTalk message into a single-row BizTalk message.
I haven't used the XML pipeline before, so I tried the provided steps but couldn't get it to work.
Could somebody provide me a link to a "how to" (didn't find anything until now, after several hours of searching) or give me some hints to succeed.
Thanks in advance.
... some hours later I could figure it out myself. So if anybody comes across the same issue as me, here you have some guidelines to make it work on your environment.
At the end I followed a different walkthrough from Microsoft and avoided the pipeline recommendation altogether. The documentation I found is called "Disassembling Result Sets Using the SQL Adapter" and does exactly what i was looking for. You can just follow the whole walkthrough from Microsoft but avoid the creation of the send port and make some little adjustment on the receive port.
After following the technical document you will end up with two schemas, I will call them message and envelope (contains several messages) for the sake of this excercise. In your orchestration you can create a receiving port that maps to the message and then when you configure it as a SQL Port and you link it to your stored procedure (or select statement), you only have to change the Document Root Element Name to the envelope root name; the XML Receive pipeline (provided by default in BizTalk 2006) will do the magic of disassembling the messages contained in the envelope and instantiating an orchestration for each message.
The Microsoft "Disassembling Result Sets Using the SQL Adapter" walkthrough can be found under:
http://msdn.microsoft.com/en-us/library/aa562098(v=bts.20).aspx
Mission accomplished :)
I am getting the ORA-24778: cannot open connections, what are the possible causes?
We have a number of applications deployed in WAS7 profile and they connect to a number of schemas in Oracle 11g.
One of the schema is connecting through other schema via public DB link.
I cannot identify a solution for this cause.
After restarting the WAS7 profile, it is ok for a while and again start hitting the error.
Pls help!!
I assume you missed to tell us a few details:
You are using XA
You are using XA in combination with database links
You are using shared database links
The ora-24778 is not happening all the time
Either you haven't configured shared server option or you are not connected to a shared server. However Oracle requires you to user shared server if you want to use XA and database links.
Or the parameter OPEN_LINKS_PER_INSTANCE is not set sufficiently. Keep in mind mind that there is also a open_links init.ora parameter. The open_links parameter does not apply to XA.
This error can occur when you invoke a dblink in a existing transaction.