I am using Doctrine with Symfony2.
My config.yml file looks something like this:-
Doctrine Configuration
doctrine:
dbal:
driver: %database_driver%
host: %database_host%
port: %database_port%
dbname: %database_name%
user: %database_user%
password: %database_password%
charset: UTF8
Unfortunately my tables are not collating to the UTF8_general_ci or UTF8_unicode_ci
I tried
collate: utf8_unicode_ci
But Doctrine2 didn't recognize the option.
How can I achieve the same?
The behavior of collate has changed in doctrine: http://www.doctrine-project.org/jira/browse/DDC-2139
The collation is now set to utf8_unicode_ci if you don't specify anything at the table level. The way to go right now is to add it to options in your table declaration:
/**
* #ORM\Table(options={"collate"="utf8_swedish_ci"})
* #ORM\Entity
*/
This will generate the correct collation for the table:
$ php app/console doctrine:schema:update --dump-sql --env=test | grep swedish
... DEFAULT CHARACTER SET utf8 COLLATE utf8_swedish_ci ENGINE = InnoDB;
I've filed an issue for the documentation to be updated to reflect this behaviour.
The charset: UTF8 option is just useful to ask Doctrine to execute SET NAMES UTF-8 on each page. I don't have any specific configuration for Doctrine, and my tables are by default in utf8_general_ci InnoDB.
Read this part of the documentation: https://www.doctrine-project.org/projects/doctrine-orm/en/latest/reference/faq.html#how-do-i-set-the-charset-and-collation-for-mysql-tables, it answers your question:
You can’t set these values inside the annotations, yml or xml mapping
files. To make a database work with the default charset and collation
you should configure MySQL to use it as default charset, or create the
database with charset and collation details. This way they get
inherited to all newly created database tables and columns.
UPDATE:
See Symfony3.1 book for reference (click here):
Also notice the use of utf8mb4 instead of plain utf8. ("Symfony recommends utf8mb4 against MySQL's utf8 character set, since it does not support 4-byte unicode characters, and strings containing them will be truncated. This is fixed by the newer utf8mb4 character set.")
Setting UTF8 defaults for MySQL is as simple as adding a few lines to
your configuration file (typically my.cnf):
[mysqld]
# Version 5.5.3 introduced "utf8mb4", which is recommended
collation-server = utf8mb4_general_ci # Replaces utf8_general_ci
character-set-server = utf8mb4 # Replaces utf8
You can also change the defaults for Doctrine so that the generated
SQL uses the correct character set.
# app/config/config.yml
doctrine:
dbal:
charset: utf8mb4
default_table_options:
charset: utf8mb4
collate: utf8mb4_unicode_ci
I suggest you try add this setting to your Doctrine configuration:
options:
1002: "SET NAMES 'UTF8' COLLATE 'utf8_unicode_ci'"
So it looks like this:
doctrine:
dbal:
driver: %database_driver%
host: %database_host%
port: %database_port%
dbname: %database_name%
user: %database_user%
password: %database_password%
charset: UTF8
options:
1002: "SET NAMES 'UTF8' COLLATE 'utf8_unicode_ci'"
As a reference Doctrine Configuration:
http://symfony.com/doc/2.0/reference/configuration/doctrine.html#reference-dbal-configuration
And in case you are using MySql:
http://dev.mysql.com/doc/refman/5.0/en/charset-connection.html
BTW. I had problems with displaying polish characters and adding only set names without collation (as below) helped.
options:
1002: "SET NAMES 'UTF8'
Use code below to set collation, engine and charset (annotation example):
/**
* #ORM\Table(
* name="temporary",
* options={"collate":"utf8_general_ci", "charset":"utf8", "engine":"MyISAM"}
* )
* #ORM\Entity
*/
Source: http://doctrine-orm.readthedocs.org/projects/doctrine-orm/en/latest/reference/annotations-reference.html#annref-column
You can refer to the documentation here http://dev.mysql.com/doc/refman/5.0/en/charset-applications.html if you are using ORMs like Doctrine.
Particularly: adding/editing the [mysqld] block in your my.cnf (usually found in /etc/my.cnf or xampp/mysql/my.cnf)
[mysqld]
character-set-server=utf8
collation-server=utf8_general_ci
I've got this problems with fetching data which includes polish characters from database. It looks like:
effects of displaying data from db
my config.yml is like:
doctrine:
dbal:
driver: pdo_mysql
host: "%database_host%"
port: "%database_port%"
dbname: "%database_name%"
user: "%database_user%"
password: "%database_password%"
charset: UTF8
options:
1002: "SET NAMES 'UTF8'"
and I've been searching for an answer for couple of hours - Im so tired of this. "1002: SET NAMES 'utf8'" Doesnt work for me. But when I tryed to access my db from simple php script (out of symfony) the effect was the same but when I added line:
mysql_query("SET NAMES 'utf8'");
it worked properly. So it seems to be little strange. All the tables in my db have 'utf8_unicode_ci' set.
I met the same problems. By search the code, I find the code in
vendor/doctrine/dbal/lib/Doctrine/DBAL/Platforms/MySqlPlatform.php
if ( ! isset($options['collate'])) {
$options['collate'] = 'utf8_unicode_ci';
}
So I changed it to 'utf8_general_ci', and it worked. All of tables' collation changed to utf8_general_ci after rebuild, but I still have one question: when I changed the annotations, I got the following error message:
[Creation Error] The annotation #ORM\Table declared on class Gvsun\UserBundle\Entity\User does not have a property named "collation". Available properties: name, schema, indexes, uniqueConstraints, options
I search the documentation of Doctrine, but I didn't find the 'option' property, can someone tell me how to use options property, I think it might be able to change collate in annotations settings?
I create manually my data base with with UTF8 collate (ex. with phpmyadmin). If I do this, all tables create with command doctrine:schema:create will have collate utf8.
I have successfully used options: collate in YML configuration of Symfony 2.7.
MyBundle\Entity\Company:
type: entity
repositoryClass: MyBundle\Repository\CompanyRepository
table: company
options:
collate: utf8_general_ci
if you are looking for a way to have migrations be created correctly you should setup the connection
you can set the default_table_options connection option to achieve this:
in symfony this is done through:
doctrine:
dbal:
default_table_options:
charset: utf8
collate: utf8_general_ci
for those looking to do it in plain doctrine this translates to doctrine connection option defaultDatabaseOptions, and is passed to the entity manager together with your database credentials etc:
[
...
'driver' => ...
'user' => ...
...
'defaultDatabaseOptions' => [
'charset' => 'utf8',
'collate' => 'utf8_general_ci'
]
]
I had have the same problem, and after reading this documentation from doctrine project page I decided to abandon solution with yml file.
Since DoctrineBundle 2.0, utf8mb4 is the default value for MySQL. So, you don't need to configure anything anymore.
See https://github.com/doctrine/DoctrineBundle/blob/master/UPGRADE-2.0.md
#ORM\Table(name="user", options={"collate"="utf8_unicode_ci", "charset"="utf8 ", "engine"="InnoDB"})
Related
On a brand new project (Symfony 4.1), I tried to work with a SQLite database.
So I change the .env as it said :
###> doctrine/doctrine-bundle ###
# Format described at http://docs.doctrine-project.org/projects/doctrine-dbal/en/latest/reference/configuration.html#connecting-using-a-url
# For an SQLite database, use: "sqlite:///%kernel.project_dir%/var/data.db"
# Configure your db driver and server_version in config/packages/doctrine.yaml
#DATABASE_URL=mysql://db_user:db_password#127.0.0.1:3306/db_name
DATABASE_URL=sqlite:///%kernel.project_dir%/var/data.db
###< doctrine/doctrine-bundle ###
File doctrine.yaml look like
parameters:
# Adds a fallback DATABASE_URL if the env var is not set.
# This allows you to run cache:warmup even if your
# environment variables are not available yet.
# You should not need to change this value.
env(DATABASE_URL): ''
doctrine:
dbal:
# configure these for your database server
driver: 'pdo_sqlite'
server_version: '5.7'
charset: utf8mb4
default_table_options:
charset: utf8mb4
collate: utf8mb4_unicode_ci
url: '%env(resolve:DATABASE_URL)%'
But when I try to create the database :
php bin/console doctrine:database:create
This error occurs :
In CreateDatabaseDoctrineCommand.php line 78:
Connection does not contain a 'path' or 'dbname' parameter and cannot be dropped.
The fact is that url: '%env(resolve:DATABASE_URL)%' can't recursivly resolve the %kernel.project_dir% part of my .env config.
It seems to be a known issue that has been fixed for Symfony 3.4 in 2017 (So I can bet that has been fixed for more recent branches like 4.1 since 2017)
Interresting Source :
https://github.com/symfony/flex/issues/129
https://github.com/symfony/symfony/pull/23901
Please note that I found 2 workarounds in file doctrine.yaml :
parameters:
# Adds a fallback DATABASE_URL if the env var is not set.
# This allows you to run cache:warmup even if your
# environment variables are not available yet.
# You should not need to change this value.
env(DATABASE_URL): 'sqlite:///%kernel.project_dir%/var/data.db'
But cleary (as mentioned) I should not modify that parameter...
Or
doctrine:
dbal:
# configure these for your database server
driver: 'pdo_sqlite'
server_version: '5.7'
charset: utf8mb4
default_table_options:
charset: utf8mb4
collate: utf8mb4_unicode_ci
url: 'sqlite:///%kernel.project_dir%/var/data.db'
Of course, it is more clean to fill in the sqlite connexion string in .env file than doctrine.yaml file because it depends on environment settings...
Do you have any idea how to make it works in .env file ?
How can one configure Symfony's DoctrineMigrationsBundle to use different database authentication credentials to its DoctrineBundle—or at very least, a different DoctrineBundle connection to that used elsewhere in the app?
We would like the app to connect to the database with only limited permissions, e.g. no ability to issue DDL commands such as CREATE, ALTER or DROP. However, migrations will need to execute such DDL commands and so should connect as a user with elevated permissions. Is this possible?
I know it's a very old post, but as it is the one which shows on a Google search on the subject, I add my solution, working with Symfony 4.
First, you just have to define a new database connection in config/doctrine.yml (a new entity manager is NOT needed):
doctrine:
dbal:
default_connection: default
connections:
default:
# This will be the connection used by the default entity manager
url: '%env(resolve:DATABASE_URL)%'
driver: 'pdo_pgsql'
server_version: '11.1'
charset: UTF8
migrations:
# This will be the connection used for playing the migrations
url: '%env(resolve:DATABASE_MIGRATIONS_URL)%'
driver: 'pdo_pgsql'
server_version: '11.1'
charset: UTF8
orm:
# As usual...
You also have to define the DATABASE_MIGRATIONS_URL with the admin credentials in the .env file or in environnement variables:
###> doctrine/doctrine-bundle ###
# Format described at http://docs.doctrine-project.org/projects/doctrine-dbal/en/latest/reference/configuration.html#connecting-using-a-url
DATABASE_URL=postgresql://app_user:app_user_pass#localhost:5432/db
# Database url used for migrations (elevated rights)
DATABASE_MIGRATIONS_URL=postgresql://admin_user:admin_user_pass#localhost:5432/db
###< doctrine/doctrine-bundle ###
Then, just execute your migrations with the --db option, passing the name of your new connection:
php bin/console doctrine:migrations:migrate --db=migrations
Yes. Just define a new entity manager with the correct connection details and then use that entity manager when running migration commands
$ php app/console doctrine:migrations:version --em=new_entity_manager
Is there a way to configure test parameters with Symfony2?
I tried importing parameters_test.yml from the config_test.yml file but the variables still come from the parameters.yml file which is imported by config.yml which is imported by config_dev.yml which is imported by config_test.yml.
Basically I want to setup different variables in a test parameter file so I can access the values and also use them in the config files. If I hardcode the variables directly into config_test.yml I don't have access to them.
The variable I'm interested in particular is database_name or doctrine->dbal->dbname.
Thanks
Of course ensure your two first lines your config_test.yml are:
imports:
- { resource: config_dev.yml }
- { resource: parameters_test.yml }
In config_test.yml simple write this configuration:
# Doctrine Configuration
doctrine:
dbal:
driver: %database_driver_test%
host: %database_host_test%
port: %database_port_test%
dbname: %database_name_test%
user: %database_user_test%
password: %database_password_test%
With Doctrine in Symfony2 there is a simple way to start a project with reverse-engineered Entities created from existing database schema. It is quite well documented here. There is not mentioned how to reverse-engineer data from a non-default database when using multiple databases (which is documented here).
I found the solution here, it works like this:
php app/console doctrine:mapping:convert --em="troller" --from-database yml ./src/NAMESPACE/NAMEBundle/Resources/config/doctrine/metadata/orm
However, I'm just getting the exception as if the second entity manager didn't exist. Even though I have config.yml according to docs.
[InvalidArgumentException]
Doctrine ORM Manager named "troller" does not exist.
Any ideas?
Did you specify an entity manager with "troller" name?
You can do this with a snippet of code like this (into app/config/config.yml file)
orm:
default_entity_manager: default
entity_managers:
default:
connection: default
mappings:
AcmeDemoBundle: ~
AcmeStoreBundle: ~
troller:
connection: troller
mappings:
YourTrollerBundle: ~
In this example, you've defined two entity managers called default and troller. The default entity manager manages entities in the AcmeDemoBundle and AcmeStoreBundle, while the troller entity manager manages entities in the YourTrollerBundle. You've also defined two connections, one for each entity manager.
Obiously define a new connection and entity manager, isn't enaugh: you have to specify "connection parameters" also (like db name, user, password, driver, and so on)
troller:
driver: "%database_driver2%"
host: "%database_host2%"
port: "%database_port2%"
dbname: "%database_name2%"
user: "%database_user2%"
password: "%database_password2%"
charset: UTF8
I'm looking for a way of using single EntityManager with multiple databases for a MySql and sqlite connection. Mysql is used in production while we'd like to use Sqlite for tests.
It works well with mysql. I don't specify dbname in my default connection in config.yml and then reference table names in entity annotations like #ORM\Table(name="Orders.tblAccount"). We do this because we have cross database relationships.
But when changing the driver and using this for Sqlite i'm not able to drop / create database for this connection as there's no database name, but I don't need to specify EntityManager name to be able to use entities from different databases.
Example: config.yml for mysql connection
doctrine:
dbal:
default_connection: Default
connections:
Default:
driver: %database_driver%
host: %database_host%
user: %database_user%
password: %database_password%
charset: %database_charset%
orm:
default_entity_manager: Default
entity_managers:
Default:
connection: Default
mappings:
AcmeOrdersBundle: ~
AcmeStoreBundle: ~
Example: annotations for entities
/**
* Acme\OrdersBundle\Entity\Account
*
* #ORM\Table(name="Orders.tblAccount")
* #ORM\Entity(repositoryClass="Acme\OrdersBundle\Repository\AccountRepository")
* #ORM\HasLifecycleCallbacks
*/
class Account
/**
* Acme\OrdersBundle\Entity\AccountType
*
* #ORM\Table(name="Orders.ublAccountType")
* #ORM\Entity(repositoryClass="Acme\CoreBundle\Repository\DoctrineRepository")
*/
class AccountType extends BaseEntity
/**
* Acme\StoreBundle\Entity\Employee
*
* #ORM\Table(name="Store.tblEmployee")
* #ORM\Entity(repositoryClass="Acme\StoreBundle\Repository\EmployeeRepository")
*/
class Employee
Example: current config.yml for sqlite connection
SQLite connection #config.yml
doctrine:
dbal:
default_connection: Default
connections:
Default:
driver: pdo_sqlite
path: :memory:
memory: true
orm:
auto_generate_proxy_classes: %kernel.debug%
auto_mapping: true
Entity annotations are the same, but because "." is not legal character in sqlite table names, symfony replaces it with "__". Unfortunately, it's not taken into account when querying the database, so I'm getting the following errors:
SQLSTATE[HY000]: General error: 1 no such table: Store.tblEmployee
(Obviously, table created by symfony is called Store__tblEmployee)
Question:
Is there any way to use multiple sqlite databases on the same EntityManager? I need to be able to join across the databases, as I'm doing when connected to mysql database.
Any ideas or help will be greatly appreciated.