I am looking for a good solution to on-the-fly connection of databases within Symfony utilizing Doctrine for entity management.
The scenario I have is that all inbound users to our service will be visiting *.website.com addresses, like client1.website.com.
We would like to use one Doctrine entity for the Client table to then look up their database credentials based on the URL of their account on the fly.
So far I have found the following topics here on stackoverflow that discuss dynamically changing the database credentials--but no clear workable solutions.
I'd like to propose collaborating to put together a working solution, and I'll put together a blog/tutorial post for other folks looking to modify database connection parameters within Symfony.
Here are some related posts:
Dynamic database connection symfony2
Symfony2, Dynamic DB Connection/Early override of Doctrine Service
Thanks!
If $em is existing entity manager and you want to reuse it's configuration, you can use this:
$conn = array(
'driver' => 'pdo_mysql',
'user' => 'root',
'password' => '',
'dbname' => 'foo'
);
$new = \Doctrine\ORM\EntityManager::create(
$conn,
$em->getConfiguration(),
$em->getEventManager()
);
I needed to do something similar - runtime discovery of an available database server. I did it by overriding the doctrine.dbal.connection_factory.class parameter and substituting my own derivation of the Doctrine bundle's ConnectionFactory class
My services.yml provides the parameter, pointing at my custom class
parameters:
doctrine.dbal.connection_factory.class: Path\To\Class\CustomConnectionFactory
Then fill in your discovery logic in Path\To\Class\CustomConnectionFactory.php
<?php
namespace Path\To\Class;
use Doctrine\Bundle\DoctrineBundle\ConnectionFactory;
use Doctrine\Common\EventManager;
use Doctrine\DBAL\Configuration;
class CustomConnectionFactory extends ConnectionFactory
{
public function createConnection(array $params, Configuration $config = null, EventManager $eventManager = null, array $mappingTypes = array())
{
// Discover and override $params array here.
// A real-world example might obtain them from zookeeper,
// consul or etcd for example. You'll probably want to cache
// anything you obtain from such a service too.
$params['driver'] = 'pdo_mysql';
$params['host'] = '10.1.2.3';
$params['port'] = 3306;
$params['dbname'] = 'foo';
$params['user'] = 'myuser';
$params['password'] = 'mypass';
//continue with regular connection creation using new params
return parent::createConnection($params, $config, $eventManager,$mappingTypes);
}
}
Note also that Symfony 3.2 features the ability to use environment variables in container configurations, and to use their values on-demand (rather than fixing them when the container is compiled). See the blog announcement for more details.
Related
I'm working on a Symfony 4 Web Project, and I have a database for every Client, so, in every request I have to connect to a database based on client id.
How to use doctrine to connect to a database manually ?
MyController:
/**
* #Route("/api/log", name="log", methods={"GET"})
*/
public function log(Request $request)
{
$this->denyAccessUnlessGranted(['ROLE_CLIENT','ROLE_ADMIN']);
$clientId = $request->query->get('client_id');
$dbName = 'project_'.$clientId;
//I have database credentials: $host,$port,$username,$password & $dbName:
$this->getDoctrine()->........
You can work with multiple connections and managers. Here the official symfony documentation.
So, if you could change manually config/packages/doctrine.yaml, you will solve your problem.
Another way, is to work with entity manager directly:
use Doctrine\ORM\Tools\Setup;
use Doctrine\ORM\EntityManager;
$paths = array('/path/to/entity/mapping/files');
$config = Setup::createAnnotationMetadataConfiguration($paths);
$dbParams = array('driver' => 'pdo_sqlite', 'memory' => true);
$entityManager = EntityManager::create($dbParams, $config);
Entity Manager API documenation
I'm trying to use this Symfony bundle:
https://github.com/KnpLabs/KnpPaginatorBundle
In the docs, they use it a controller. So they have easy access to service container or the request object.
But as far as I understand, the Doctrine query should be in a repository, not a controller, right? And I already do have a function returning records. It's just that the pagination service doesn't expect "results" upon instantiating. It wants the query. So I can't return the "results" to the controller, but rather in middle of this function use a paginator.
On the other hand, stuff like playing with services or requests indeed belong to controllers.
So how this should be done? At first I thought about injecting the "knp_paginator" service and the request object into the repository. But I don't think this is the right way.
I'd say that the Request object should not go further down the stack than from the Controller.
Nothing prevents you from injecting the paginator directly into your custom repository, so why not doing that?
your.repository.service.definition:
class: Your\Repository\Class
# for symfony 2.3
factory_service: doctrine
factory_method: getRepository
# for symfony 2.8 and higher
factory: ["#doctrine.orm.entity_manager", getRepository]
arguments:
- YourBundle:YourEntity
calls:
- [setPaginator, ["#knp_paginator"]]
In the repository, you then should have the paginator available for use with the QueryBuilder:
public function setPaginator($paginator)
{
$this->paginator = $paginator;
}
...
$this->paginator->paginate($qb->getQuery(), $page, $limit);
In order to get your $page and $limit variables into the repository, you don't need the Request object. Simply pass them as a parameter to the repository call:
// In your controller
// You can use forms here if you want, but for brevity:
$criteria = $request->get('criteria');
$page = $request->get('page');
$limit = $request->get('limit');
$paginatedResults = $myCustomRepository->fetchPaginatedData($criteria, $page, $limit);
Passing the request object further down the Controller means that you have a leak in your abstractions. It's no concern of your application to know about the Request object. Actually, the request might well come from other sources such as the CLI command. You don't want to be creating a Request object from there because of a wrong level of abstraction.
Assuming that you have a Custom Repository Class, you can have a method in that Repository, which returns a Query or a valid instance of Query Builder and then you call that method from the controller and pass it to the paginate() method.
For example where $qb is returned by the custom repository (not return result but just the querybuilder of it)
$paginator = $this->get('knp_paginator');
$pagination = $paginator->paginate(
$qb->getQuery(),
$request->query->getInt($pageParameterName, 1),
$perPage,
array('pageParameterName' => $pageParameterName)
);
am creating own framework based on Sf2 commponents and i try to create router service.
I i need that service for generateUrl() method
protected function generateUrl($route, $parameters = array(), $referenceType = UrlGeneratorInterface::ABSOLUTE_PATH)
{
return $this->get('router')->generate($route, $parameters, $referenceType);
}
I try this
$container = new ContainerBuilder();
$container->setDefinition('router_loader', new Definition('Symfony\Component\Config\Loader\LoaderInterface'));
$container->setDefinition('router', new Definition('Symfony\Component\Routing\Router', array()));
And when i execute in my methodAction
$this->generateUrl('home');
he return me:
Catchable fatal error: Argument 1 passed to
Symfony\Component\Routing\Router::__construct() must be an instance of
Symfony\Component\Config\Loader\LoaderInterface, none given in
D:\xampp\htdocs\my_fw\vendor\symfony\routing\Router.php on line 95
looking on router constructor i see. I need that interface
public function __construct(LoaderInterface $loader, $resource, array $options = array(), RequestContext $context = null, LoggerInterface $logger = null)
how to avoid that implementation in service?
**New update:** routing.php
use Symfony\Component\Routing\RouteCollection;
use Symfony\Component\Routing\Route;
// Routing
$routes = new RouteCollection();
// Home
$routes->add('home', new Route('/', array(
'_controller' => 'MyCompany\\Controller\\HomeController::indexAction',
)));
You are having this error because you are configuring the router service with no service definition the service definition should have your service arguments which is the source of your error because the container try to create the router with no arguments check this
For better use you can configure the router service in the service.yml/.xml file
Edit: The documentation page for the Routing component has detailed setup instructions
Try injecting the router_loader service as an argument into the router service. For this case you have to use the Symfony\Component\DependencyInjection\Reference class.
Routes have to be configured with a config file when using the all in one Router class. You have to use a FileLocator and a real Loader class, like the YamlFileLoader, you can't just use the interface (services generally can't be interfaces in Symfony).
The service container setup for the router service should look like this:
use Symfony\Component\DependencyInjection\Reference;
// Loads config files from the current directory, change this to
// your liking, or add more than one path
$container->setDefinition('router_config_locator', new Definition(
'Symfony\Component\Config\FileLocator', [[__DIR__]]
));
$container->setDefinition('router_loader', new Definition(
'Symfony\Component\DependencyInjection\Loader\YamlFileLoader', [
new Reference('router_config_locator'),
]
));
$container->setDefinition('router',
new Definition('Symfony\Component\Routing\Router', array(
'loader' => new Reference('router_loader'),
// Definition of routes in Yaml form
'resource' => 'routes.yml',
))
);
The routes.yml file contains your route definitions:
home:
path: /home
defaults: {_controller: "MyController:index"}
Also have a look at this documentation page about setting up the routing component, which also talks about setting up the routing without the all in one class and config file.
I have an application with an existing set of unit tests which are using SQLite as the DB. I have recently added search capabilities via ES which have replaced many of the endpoint actions that used to query the DB directly. I want to test all of the business logic involved with these endpoints without testing ES itself, which means no ES server available. I plan to test ES itself in a set of integration tests to be run less frequently.
My problem is trying to track down exactly what is going on with the execution flow.
My first inclination was to simply create a mock object of the ES Finder that FOSElasticaBundle creates for my index. Because I'm using pagination, it turned out to be more complex than I thought:
// code context: test method in unit test extending Symfony's WebTestCase
$client = $this->getClient();
$expectedHitCount = 10;
// Setup real objects which (as far as I can tell) don't act upon the ES client
// and instead only hold / manipulate the data.
$responseString = file_get_contents(static::SEARCH_RESULT_FILE_RESOURCE);
$query = SearchRepository::getProximitySearchQuery($lat, $lng, $radius, $offset, $limit);
$response = new Response($responseString, 200);
$resultSet = new RawPartialResults(new ResultSet($response, $query ));
// Create a mock pagination adapter which is what my service expects to be returned from
// the search repository.
$adapter = $this->getMockBuilder('FOS\ElasticaBundle\Paginator\RawPaginatorAdapter')
->disableOriginalConstructor()
->getMock();
$adapter->method('getTotalHits')->will($this->returnValue($expectedTotalCount));
$adapter->method('getResults')->will($this->returnValue($resultSet));
$adapter->method('getQuery')->will($this->returnValue($query));
$es = $this->getMockBuilder(get_class($client->getContainer()->get(static::ES_FINDER_SERVICE)))
->disableOriginalConstructor()
->getMock();
$es->method('createPaginatorAdapter')->will($this->returnValue($adapter));
// Replace the client container's service definition with our mock object
$client->getContainer()->set(static::ES_FINDER_SERVICE, $es);
This actually works all the way until I return the view from my controller. My service gets back the mock paginatior adapter with the pre-popuated result set from the JSON search response I have stored in a file (and subsequently passed into my ResultSet object). However, once I return the view, there seems to be a listener involved that tries to query ES again with the Query instead of using the ResultSet I already passed in.
I can't seem to find this listener. I also don't understand why it would try to query when a ResuletSet already exists.
I am using FOSRestBundle, as well, and making use of their ViewListener to auto-serialize whatever I return. I don't see any suspects in that flow, either. I think it may have something to do with the serialization of the result set, but so far haven't been able to track the offending code down.
Has anyone tried to do something similar to this before and have any suggestions on either how to debug my current setup or an alternative, better setup for mocking ES for this type of test?
After digging around I found an alternative solution that does not involve using mock objects. I am going to leave this open for the time being in case someone has a better approach, but the approach I decided to take in the mean time is to override the Client in my test environment.
FOSElasticaBundle has an example for overriding the client here: https://github.com/FriendsOfSymfony/FOSElasticaBundle/blob/master/Resources/doc/cookbook/suppress-server-errors.md
I was able to override the client in such a way that I could create a unique key from the request and then provide responses based on that key, essentially stubbing the server for all known requests. For requests that don't match I return a default empty response. This works well enough for me.
Client Code
<?php
namespace Acme\DemoBundle\Tests\Elastica;
use Elastica\Request;
use Elastica\Response;
use FOS\ElasticaBundle\Client as BaseClient;
class Client extends BaseClient
{
/**
* This array translates a key which is the md5 hash of the Request::toString() into
* a human friendly name so that we can load the proper response from a file in the
* file system.
*
* #var array
*/
protected $responseLookup = array(
'7fea3dda860a424aa974b44f508b6678' => 'proximity-search-response.json'
);
/**
* {#inheritdoc}
*/
public function request($path, $method = Request::GET, $data = array(), array $query = array())
{
$request = new Request($path, $method, $data, $query);
$requestKey = md5($request->toString());
$this->_log($request);
$this->_log("Test request lookup key: $requestKey");
if (!isset($this->responseLookup[$requestKey])
|| !$response = file_get_contents(__DIR__ . "/../DataFixtures/Resources/search/{$this->responseLookup[$requestKey]}")) {
return $this->getNullResponse();
}
return new Response($response);
}
public function getNullResponse()
{
$this->_log("Returning NULL response");
return new Response('{"took":0,"timed_out":false,"hits":{"total":0,"max_score":0,"hits":[]}}');
}
}
Configuration Change
// file: config_test.yml
parameters:
fos_elastica.client.class: Acme\DemoBundle\Tests\Elastica\Client
Sample Response File (proximity-search-response.json)
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": null,
"hits": [
{
"_index": "search",
"_type": "place",
"_id": "1",
"_score": null,
"_source": {
"location": "40.849100,-73.644800",
"id": 1,
"name": "My Place"
},
"sort": [
322.52855474383045
]
}
]
}
}
This solution works well and is fast, but the maintenance is a pain. If anything about the request changes, you need to retrieve the new request key from the log, update it in the array, and update the file with the new response data for the new request. I generally just curl the server directly and modify it from there.
I would love to see any other solutions that may be simpler, but I hope this helps someone else in the meantime!
you can try to disable the event listeners in your config_test.yml (or whatever is your test environment name).
fos_elastica:
indexes:
your_index_name:
types:
your_type_name:
persistence:
listener:
insert: false
update: false
delete: false
I know that using the Doctrinebundle in Symfony2 it is possible to instantiate multiple DB connections under Doctrine...
$connectionFactory = $this->container->get('doctrine.dbal.connection_factory');
$connection = $connectionFactory->createConnection(array(
'driver' => 'pdo_mysql',
'user' => 'foo_user',
'password' => 'foo_pass',
'host' => 'foo_host',
'dbname' => 'foo_db',
));
I'm curious if this is the case if you are using PURELY Doctrine though?, I've set up Doctrine via Composer like so...
{
"config": {
"vendor-dir": "lib/"
},
"require": {
"doctrine/orm": "2.3.4",
"doctrine/dbal": "2.3.4"
}
}
And have been looking for my ConnectionFactory class but am not seeing it anywhere? Am I required to use Symfony2 to do this?
Should I just download ConnectionFactory.php from the DoctrineBundle and include it in my DBAL folder?? idk?
A bundle is only in the context of symfony needed, it wraps the orm into symfony infrastructure (services, etc.). For pure use of the orm you should read the ORM: Installation and Configuration. As you see you must create an entity manager by yourself with EntityManager::create($dbParams, $config), so simply create different entity managers for your different databases.
For DBAL use you should read DBAL: Configuration and see, a connection can simply obtained trough DriverManager::getConnection($connectionParams, $config); But if you are sure the ConnectionFactory has no dependency to symfony stuff and you really need it, you can try copy it to your code and construct a new factory to obtain a DBAL connection.
$connectionFactory = new ConnectionFactory(array());
$connection = $connectionFactory->createConnection(array(
'driver' => 'pdo_mysql',
'user' => 'foo_user',
'password' => 'foo_pass',
'host' => 'foo_host',
'dbname' => 'foo_db',
));
But take care, this is a DBAL connection, i.e. it's a abstraction layer which sits on top of PDO and only for pure SQL queries. If you need a entity manager you have to initialize it as mentioned in the docs above, or maybe you find another entity manager factory class, which you can "copy".