I can connect to an external db: database1 through flyway CLI and list all the schema versions etc.
This database1 connection does not require any custom-proxy in between.
But I have another DB: database2 which I generally connect with custom-proxy in sqlDeveloper.
How do we mention such advanced-proxy configuration in flyway.conf?
As of now, without proxy info in flyway.conf, I am not able to connect to database2 through flyway.
Related
I'm trying to connect my app engine flex service to the cloud sql database that is in a shared project.
I have added the app engine default and app engine flex user as SQL admin to the shared project.
Now I'm trying to connect the app engine container to the cloud sql instance with the following connection string:
"Server=cloudsql;Database=cliche;User Id=user;Password=password;"
However, the connection always fails with the given connection string.
[13:57:37 ERR] An error occurred using the connection to database 'cliche' on server 'cloudsql'.
[13:57:37 ERR] An exception occurred while iterating over the results of a query for context type 'IKL.Data.ApplicationDbContext'.
Microsoft.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server)
As requested I provide the app.yaml here:
runtime: aspnetcore
env: flex
service: platform
manual_scaling:
instances: 1
resources:
cpu: 2
memory_gb: 1.5
disk_size_gb: 10
beta_settings:
cloud_sql_instances: shared-resources-291914:europe-west3:edutec-mssql=tcp:1433
I found your issue! Your connexion string "Server=cloudsql;Database=cliche;User Id=user;Password=password;" isn't correct.
Cloud SQL proxy open a tunnel. one side of the tunnel is the Cloud SQL instance, the other is your local instance. Therefore you have to connect on the local host
"Server=127.0.0.1;Database=cliche;User Id=user;Password=password;"
If you are using VPC Peering between your App Engine Flex project and your Cloud SQL project, then I must inform you that this is not possible to do yet, when you use an internal IP on your Cloud SQL, it will automatically use a VPC peering (different from the one you are probably using), and traffic between 3 VPC peerings is not posible.
You can use a VM in your Cloud SQL project to serve as a NAT server to route traffic onto your Cloud SQL there in order to workaround this, or you can use a Cloud VPN between your App Engine and Cloud SQL project, if possible please make sure that the NAT VM is in the same region as your Cloud SQL server or that the VPN on your SQL project is on the same region as your SQL server.
Also, please keep in mind that you would need to have the appropriate GCP firewall rules on all the involved GCP projects.
I'm very new to Docker (in fact I've been only using it for one day) so maybe I'm misunderstanding some basic concept but I couldn't find a solution myself.
Here's the problem. I have an ASP.NET Core server application on a Windows machine. It uses MongoDB as a datastore. Everything works fine. I decided to pack all this stuff into Docker containers and put it to a Linux (Ubuntu Server 18.04) server. I've packed mongo to a container so now its PUBLISHED IP:PORT value is 192.168.99.100:32772
I've hardcoded this address to my ASP.NET server and also packed it to a container (IP 192.168.99.100:5000).
Now if I run my server and mongo containers together on my Windows machine, they work just fine. The server connects to a container with the database and can do whatever it needs.
But when I transfer both containers to Ubuntu and run them, the server cannot connect to the database because this IP address is not available there. I've beed googling for a few hours to find a solution and still I'm struggling with it.
What is the correct way to go about thes IP addresses? Is it possible to set an IP that will be the same for a container regardless of environment?
I recommend using docker-compose for the purpose you described above.
With docker-compose, you can access the database via a service name instead of an IP (which potentially is not available on another system). Here two links to get started
https://docs.docker.com/compose/gettingstarted/
https://docs.docker.com/compose/compose-file/
Updated answer (10.11.2019)
Here a concrete example for your asp.net app:
docker-compose.yaml
version: "3"
services:
frontend:
image: fqdn/aspnet:tag
ports:
- 8080:80
links:
- database
database:
image: mongo
environment:
MONGO_INITDB_DATABASE: "mydatabase"
MONGO_INITDB_ROOT_USERNAME: "root"
MONGO_INITDB_ROOT_PASSWORD: "example"
volumes:
- myMongoVolume:/data/db
volumes:
myMongoVolume: {}
From the frontend container, you can reach the mongo db container via the service name "database" (instead of an IP). Due to the link definition in the frontend service, the frontend service will start after the linked service (database).
Through volume definition, the mongo database will be stored in a volume that persists independently from the container lifecycle.
Additionally, I assume you want to reach the asp.net application via the host IP. I do not know the port that you expose in your application so I assume the default port 80. Via the ports section in the frontend, we define that container port 80 is exposed as port 8080 on the host IP. e.g. you can open your browser and type your host IP and port 8080 e.g. 127.0.0.1:8080 for localhost and reach your application.
With docker-compose installed, you can start your app, which consists of your frontend and database service via
docker-compose up
Available command options for docker-compose can be found here
https://docs.docker.com/compose/reference/overview/
Install instructions for docker-compose
https://docs.docker.com/compose/install/
Updated answer (10.11.2019, v2)
From the comment section
Keep in mind that you need to connect via the servicename (e.g. database) and the correct port. For MongoDB that port is 27017. That would tanslate to database:27017 in your frontend config
Q: will mongo also be available from the outside in this case?
A: No, since the service does not contain any port definition the database itself will not be directly reachable. From a security standpoint, this is preferable.
Q: could you expain this
volumes:
myMongoVolume: {}
A: in the service definition for your database service, we have specified a volume to store the database itself to make the data independent from the container lifecycle. However just by defining a volume in the service section the volume will not be created. Through the definition in the volume section, we create the volume myMongoVolume with the default settings (indicated through {}). If you would like to customize your volume you can do so in this volumes section of your docker-compose.yaml. More information regarding volumes can be found here
https://docs.docker.com/compose/compose-file/#volume-configuration-reference
e.g. if you would like to use a specific storage driver for your volume or use an external storage.
We configured Artifactory 5.5.1 by using yum install using default derby db able to access with derby db.
When we changed to external db postgres, I not able to access Artifatory, access server is timed out after 60000 miliseconds.
Do we need to do any additional configuration changes?
Trying to deploy a Symfony 3.2/Doctrine application to Swisscom PaaS.
Buildpack (PHP 7, httpd etc.) are installed, composer is running and installing dependencies, but when calling the composer after-commands, like cache:clear I get an:
[Doctrine\DBAL\Exception\ConnectionException]
An exception occured in driver: SQLSTATE[HY000] [2002] Connection refused
my manifest.yml:
applications:
- services:
- dbservice
buildpack: php_buildpack
host: myapp
name: MyApp
instances: 1
memory: 640M
env:
SYMFONY_ENV: prod
PHP_INI_SCAN_DIR: .bp-config/php/conf.d
my options.json:
"WEB_SERVER": "httpd",
"COMPOSER_INSTALL_OPTIONS": ["--no-dev --optimize-autoloader --no-progress --no-interaction"],
"COMPOSER_VENDOR_DIR": "vendor",
"SYMFONY_ENV": "prod",
"WEBDIR": "web",
"PHP_MODULES": "fpm",
"PHP_VERSION": "{PHP_70_LATEST}",
"PHP_EXTENSIONS": [
"bz2",
"zlib",
"curl",
"mcrypt",
"openssl",
"mbstring",
"pdo",
"pdo_mysql"
],
"ZEND_EXTENSIONS": [
"opcache"
]
And this is how I read database credentials from VCAP and set parameters in Symfony (which is perfectly working with a local setup of VCAPSERVICES env vars):
$vcapServices = json_decode($_ENV['VCAP_SERVICES']);
$container->setParameter('database_driver', 'pdo_mysql');
$db = $vcapServices->{'mariadb'}[0]->credentials;
$container->setParameter('database_host', $db->host);
$container->setParameter('database_port', $db->port);
$container->setParameter('database_name', $db->name);
$container->setParameter('database_user', $db->username);
$container->setParameter('database_password', $db->password);
// Just for debug:
echo 'User: ';
var_dump($container->getParameter('database_user'));
echo 'Db: ';
var_dump($db);
Service is running, both var_dump deliver the expected values. But still connection is refused.
What am I doing wrong?
****EDIT****
A similar problem seems to be here, but without a solution: Cloud foundry p-mysql
****EDIT****
I debugged down right to the statement where PDO constructor is called.
It is called with the following parameters:
$dsn = mysql:host=10.0.20.18;port=3306;dbname=CF_DB922DD3_CACB_4344_9948_746E585732B5;
$username = "myrealusername"; // as looked up in VCAP_SERVICES
$password = "myrealpassword"; // as looked up in VCAP_SERVICES
$options = array();
Anything looks exactly like it can be seen in the web console for the service binding.
Is an unix_socket required to successfully connect?
****EDIT****
As Symfony is using some composer post-install-commands (in this case e.g. to clear and warm up the cache) which require already a working database connection that this is not supported with DB services by cloudfoundry as long as the container is not fully built and deployed?
Am running out of ideas.
Sorry for the issue and thanks for feedback. Swisscom modified the security groups. See Application Security Groups for more info.
Cloud Foundry blocks all outbound network connections from application
containers by default. Administrators can override this
block-by-default behavior with Application Security Groups (ASGs).
ASGs are a collection of egress rules that specify one or more
individual protocols, ports, and destinations to allow network access
to.
Cloud Foundry has two default sets of ASGs: default-staging and
default-running. All application containers in Cloud Foundry use a
base policy from one of these.
The rule for Galera as a Service (MariaDB) was only done in running. We added the rule also to staging.
Just in case someone has a similar issue this was the solution:
In the end I could only solve this with PaaS provider (swisscomdev) support.
Obviously database connection was not provided/possible during the staging/deployment of our app, but Symfony's cache:clear/warmup required a full database connection during the composer post processing phase.
After a fix in the cloudfoundry based platform of swisscomdev everything worked as expected.
In my experience, Connection refused can mean one of the things:
The firewall (iptables?) is blocking the non-local access to the port
The mysql server is not even listening to the port
The mysql server is listening on ipv6 port 3306 while client tries to connect via ipv4.
First let's try to narrow it down by running basic telnet test:
telnet 10.0.20.18 3306
This should greet you with some half-garbled message with clear mention of MySQL. If it does not, than you need to go back to more general restrictions like firewalls and policies.
Since you are positive that server is running, I suggest checking if you are being blocked by firewall or SELinux. Don't know how much control do you have over the Swisscom's PaaS system, though.
If you have SSH access to the PaaS service, you could try running tcpdump do capture any traffic. See this article: https://serverfault.com/questions/507627/debugging-a-connection-refused-response-on-port-21
Hope this gives you some hint...
Could you please advise me, how to configure a DSN entry for SQL Azure on UNix AIX box with DataDirect 6.1 to connect from PowerCenter 9.1.0.
Aix Server: 10.10.10.10 : 2222
On this Port, ABCXYZ9PQR(Database Server name) Database is configured.
Database name: TestDatabase
Telent is working from application server. Telnet 10.10.10.10 2222 --> Connected
Able to connect to the SQL server from Windows.
Please advise me what are the tests do I need to perform, and configure the DSN entry in ODBC.ini.
Thanks,
Sarat
Below steps are necessary for configuring odbc:
Ensure that the env variables ODBCHOME & ODBCINI are set. These has to be set before Informatica Services are started. Else they wont get picked up.
configure DNS in obdc.ini file for the DB server.
Use ssgbodbc utility (download from mysupport site) to test if the odbc connection goes through
Copy the DNS name from the entry in odbc.ini variable and paste it in the Connection String section of the ODBC connection in Workflow Manager.
I'm not clear from your description though:
Can you clarify if the Informatica Services is installed on AIX box (10.10.10.10)? how did you configure the connection to azure on windows box?
-Sadagopan