I created an azure Snowflake trial account and an odbc dsn, it works.
Then I had to create a Snowflake trial in AWS to use the Snowflake training.
When creating a DSN it fails with this error; Incorrect username or password was specified.
For Azure, I use ..snowflakecomputing.com, this works.
For AWS I use .snowflakecomputing.com, I get the user error.
I tried other combinations but hten I always get a host unresolved error.
..snowflakecomputing.com
.sg..aws.snowflakecomputing.com
Thanks for hints
As per https://docs.snowflake.com/en/user-guide/organizations-connect.html your Snowflake URL / servername is in this format
https://<account_locator>.<region>.<cloud>.snowflakecomputing.com
The cloud is not needed if using AWS, but is if using Azure or GCP. The region is not needed if us-east1 but otherwise must be specified.
Now, it doesn't matter where you are trying to connect to Snowflake from - it only matters where your Snowflake instance is.
So, if you made your Snowflake instances in AWS it is the exact same URL / server name whether you connect to it via the web console, a command-line tool, ODBC from an AWS application, JDBC from an Azure application, or anything else.
Thus, if you made the Snowflake account in AWS you will use something like
https://FV23210.ap-southeast2.snowflakecomputing.com
no matter where you call it from; if you made the Snowflake account in Azure you will use something like
https://FV23210.ap-southeast2.azure.snowflakecomputing.com
no matter where you call it from.
Related
I am using Azure Document Db version 2.6 and I am unable to connect to Cosmos Db emulator. I am trying to connect to https://localhost:8081. I am getting below error after it executes this line:
// Open the connection
documentClient.OpenAsync().GetAwaiter().GetResult();
WebException: The remote name could not be resolved:
'localhost-eastus2euap'
I am not sure how it's connecting to this url "localhost-eastus2euap" when I am trying to connect to localhost:8081
It looks like you are adding regions to the ConnectionPolicy PreferredLocations, is that correct?
When working with the emulator, adding regions is not supported, there are no regions as it is running in an isolated environment.
If this is code that hits the emulator or a live account, some configuration might be added to know when to add the regions and when not to.
I have an existing database for which i was looking to create a new clustered environment. I tried the following steps:
Create a new database instance (OS & DB Server).
Take a backup / snapshot from existing database server for all the databases.
Import the snapshot to the new server.
Configure the cluster - referred to various sites but all giving same solution. Example reference site - https://vexxhost.com/resources/tutorials/how-to-configure-a-galera-cluster-with-mariadb-on-ubuntu-12-04/
Ran the command (sudo galera_new_cluster) on the primary server. (Primary server - no issue starting up). But when we tried starting the secondary server - it actually crashed for some reason.
Unfortunately at this point, dont have the logs stored / backed up with me where it failed. But it seemed like it tried to sync in with the primary server - had some failure with that.
As for additional part of the actions performed above. Both the server with same username / password - created a passwordless ssh connection between both the machines. Also, the method of syncing is set to rsync.
Am i missing something or doing it wrong? Is there a better way available on it?
We need to connect to on premise Teradata from Azure Databricks .
Is that possible at all ?
If yes please let me know how .
I was looking for this information as well and I recently was able to access our Teradata instance from Databricks. Here is how I was able to do it.
Step 1. Check your cloud connectivity.
%sh nc -vz 'jdbcHostname' 'jdbcPort'
- 'jdbcHostName' is your Teradata server.
- 'jdbcPort' is your Teradata server listening port. By default, Teradata listens to the TCP port 1025
Also check out Databrick’s best practice on connecting to another infrastructure.
Step 2. Install Teradata JDBC driver.
Teradata Downloads page provides JDBC drivers by version and archive type. You can also check the Teradata JDBC Driver Supported Platforms page to make sure you pick the right version of the driver.
Databricks offers multiple ways to install a JDBC library JAR for databases whose drivers are not available in Databricks. Please refer to the Databricks Libraries to learn more and pick the one that is right for you.
Once installed, you should see it listed in the Cluster details page under the Libraries tab.
Terajdbc4.jar dbfs:/workspace/libs/terajdbc4.jar
Step 3. Connect to Teradata from Databricks.
You can define some variables to let us programmatically create these connections. Since my instance required LDAP, I added LOGMECH=LDAP in the URL. Without LOGMECH=LDAP it returns “username or password invalid” error message.
(Replace the text in italic to the values in your environment)
driver = “com.teradata.jdbc.TeraDriver”
url = “jdbc:teradata://Teradata_database_server/Database=Teradata_database_name,LOGMECH=LDAP”
table = “Teradata_schema.Teradata_tablename_or_viewname”
user = “your_username”
password = “your_password”
Now that the connection variables are specified, you can create a DataFrame. You can also explicitly set this to a particular schema if you have one already. Please refer to Spark SQL Guide for more information.
Now, let’s create a DataFrame in Python.
My_remote_table = spark.read.format(“jdbc”)\
.option(“driver”, driver)\
.option(“url”, url)\
.option(“dbtable”, table)\
.option(“user”, user)\
.option(“password”, password)\
.load()
Now that the DataFrame is created, it can be queried. For instance, you can select some particular columns to select and display within Databricks.
display(My_remote_table.select(“EXAMPLE_COLUMN”))
Step 4. Create a temporary view or a permanent table.
My_remote_table.createOrReplaceTempView(“YOUR_TEMP_VIEW_NAME”)
or
My_remote_table.write.format(“parquet”).saveAsTable(“MY_PERMANENT_TABLE_NAME”)
Step 3 and 4 can also be combined if the intention is to simply create a table in Databricks from Teradata. Check out the Databricks documentation SQL Databases Using JDBC for other options.
Here is a link to the write-up I published on this topic.
Accessing Teradata from Databricks for Rapid Experimentation in Data Science and Analytics Projects
If you create a virtual network that can connect to on prem then you can deploy your databricks instance into that vnet. See https://docs.azuredatabricks.net/administration-guide/cloud-configurations/azure/vnet-inject.html.
I assume that there is a spark connector for terradata. I haven't used it myself but I'm sure one exists.
You can't. If you run Azure Databricks, all the data needs to be stored in Azure. But you can call the data using REST API from Teradata and then save data in Azure.
I want to access AWS Athena in Power BI with ODBC. I used the ODBC driver(1.0.3) that Amazon provides:
https://docs.aws.amazon.com/de_de/athena/latest/ug/connect-with-odbc.html
To access the AWS-Service I use the user=YYY and the password=XXX. To access the relevant data our administrator created a role “ExternalAthenaAccessRole#99999”.
99999 is the ID of the account where Athena runs.
To use the ODVC-driver in Power BI I created the following connection string:
Driver=Simba Athena ODBC Driver;AwsRegion=eu-central-1;S3OutputLocation=s3://query-results-bucket/testfolder;AuthenticationType=IAM Credentials;
But when I enter the User XXX with the password YYY It get the message “We couldn’t authenticate with the credentials provided. Please try again.”.
Normally I would think that I must include the role “ExternalAthenaAccessRole#99999” in the connection string, but I couldn’t find a parameter for it in the documentation.
https://s3.amazonaws.com/athena-downloads/drivers/ODBC/SimbaAthenaODBC_1.0.3/Simba+Athena+ODBC+Install+and+Configuration+Guide.pdf
Can anybody help me how I can change the connection string so that I can access the data with the ODBC driver in Power BI?
TL;DR;
When using Secret Keys, do not specify "User / password", but instead always click on "default credentials" in Power Bi, to force it to use the Local AWS Configuration (e.g. C:/...$USER_HOME/.aws/credentials)
Summarized Guide for newbies:
Prerequisites:
AWSCli installed locally, on your laptop. If you don’t have this, just download the MSI installer from here:
https://docs.aws.amazon.com/cli/latest/userguide/install-windows.html
Note: this quick guide is just to configure the connection using AWS Access Keys, and not federating the credentials through any other Security layer.
Configure locally your AWS credentials.
From the Windows command prompt (cmd), execute: aws configure
Enter your AWS Access Key ID, Secret Access Key and default region; for example "eu-west-1" for Ireland.
You can get these Keys from the AWS console, IAM service, Users, select your user, Security, Create/Download Access Keys.
You should never share these keys, and it’s highly recommended to rotate these, for example, every month.
Download Athena ODBC Driver:
https://docs.aws.amazon.com/athena/latest/ug/connect-with-odbc.html
Important: If you have Power Bi 64 bits, download the same (32 or 64) for the ODBC.
Install it on your laptop, where you have Power Bi.
Open Windows ODBCs, add a User DSN and select Simba-Athena as the Driver.
Use always "Default credentials" and not user/password, since it will use our local keys from Step 1.
Configure an S3 bucket, for the temporary results. You can use something like: s3://aws-athena-query-results-eu-west-1-power-bi
On the Power Bi app, click on Get Data and Type ODBC.
Choose Credentials "default", to use the local AWS keys (from step 1) and, optionally, enter a "select" query.
Click on Load the data.
Important concern: I’m afraid Power Bi will load all the results from the query into our local memory. So if, for example. we're bringing 3 months of data and that is equivalent to 3 GB, then we will consume this in our local laptop.
Another important concern:
- For security reasons, you'll need to implement a KMS Encryption keys. Otherwise, the data is being transmitted in clear text, instead of being encrypted.
Relevant reference (as listed above), where you can find the steps for this entire configuration process, but more in detail:
- https://s3.amazonaws.com/athena-downloads/drivers/ODBC/Simba+Athena+ODBC+Install+and+Configuration+Guide.pdf
Carlos.
I'm trying to create shiny app in shiny.io which requires access to a MySQL database which is located in my local machine. Is it possible to use RMySQL to connect to the local database by shiny.io? And how can I do that?
I'm a little "late" with my answer, but keep in mind that if you are deploying to shiny.io then "localhost" is pointing to shiny.io and there is no mysql for you there :-) So you should have to open your MySQL server and use your public IP address in "host" variable. Not that I am proposing this, from the security point of view it is not wise to open your MySQL server to the public. If you follow this way, at least, you should filter connections to only allow the IP address of shiny.io.
Another solution -more reasonable, in my opinion- might be to use an already public server that offers you a persistence layer, such as mongohq.com (now compose.io). They use MongoDB (not MySQL) but it could satisfy your needs (and you will get a 512MB RAM testing server for free). You may want to study this working example that uses shiny.io and mongo in order to start your project up.
The usage is essentially the same as in regular R script.
you can put this at the top of your server.R file
library(RMySQL)
con = dbConnect(MySQL(), user='myusername', password='mysillypassword', db='dataaa', host='localhost')
query = "SELECT 1"
result = dbGetQuery(con, query)
Now result will be available for the rest of the shiny app.
You can also build the query inside a reactive function, and dynamically query the database.
Within your mySQL database, whitelist shinyapps.io IP addresses.
See this helpful post from Rstudio support.