How to generate a sqlite3-compatible dump from Propel's schema.xml - sqlite

To speed up the unit tests I want to use SQLite instead of MySQL, so I'm trying to get a SQLite-compatible dump of the databases from Propel's schema.xml. The application is built in Symfony2, and this is how the db configuration in config.yml looks like:
propel:
dbal:
driver: sqlite
dsn: sqlite:/tmp/test_db1.sq3
default_connection: db1
connections:
db1:
user: %database_user%
password: %database_password%
dsn: sqlite:/tmp/test_db1.sq3
db2:
user: %database_user%
password: %database_password%
dsn: sqlite:/tmp/test_db2.sq
However, the command app/console propel:build-sql is always generating MySQL-specific dumps that fail to load into SQLite.
I've also removed any references to MySQL from properties.ini and still don't get it right.
Any ideas?

Each connection name has to be related to a database name, so if you declare a connection c1, you need to have a database tag with the attribute name="c1" (in your schema.xml).
Assuming you have the following schema.xml:
<database name="default">
…
</database>
Write the following section in your config.yml:
propel:
dbal:
default_connection: default
connections:
default:
driver: mysql
username: root
dsn: mysql:host=localhost;dbname=my_db
And the following definition in config_test.yml:
propel:
dbal:
connections:
default:
driver: sqlite
dsn: sqlite:/tmp/test_db1.sq3
Then, if you want to generate SQL statements for your tests, just run:
php app/console --env=test propel:build-sql
To generate SQL statements for your dev env:
php app/console propel:build-sql
or
php app/console --env=dev propel:build-sql

Related

DB create Doctrine2

I'm new to the Symfony and Doctrine world.
So i'm following the doc to install Symfony. All went well.
Next Step : create the DB. Again I'm following the documentation (Databases and the Doctrine ORM) and run the command php bin/console doctrine:database:create
This command give me no output and is running for more than 30 minuts which seems to be a lot just to create a DB.
I've got a local mysql server running on 127.0.0.1:3000
here is my .env file
###> doctrine/doctrine-bundle ###
# Format described at https://www.doctrine-project.org/projects/doctrine-dbal/en/latest/reference/configuration.html#connecting-using-a-url
# For an SQLite database, use: "sqlite:///%kernel.project_dir%/var/data.db"
# For a PostgreSQL database, use: "postgresql://db_user:db_password#127.0.0.1:5432/db_name?serverVersion=11&charset=utf8"
# IMPORTANT: You MUST configure your server version, either here or in config/packages/doctrine.yaml
DATABASE_URL=mysql://root:root#127.0.0.1:3000/my_project?serverVersion=8.0.19
###< doctrine/doctrine-bundle ###
Does anyone have a clue why this command does not give me any output ?
Problem solved even if I don't understand why.
I edit the doctrine.yaml that was like
doctrine:
dbal:
url: '%env(resolve:DATABASE_URL)%'
to
doctrine:
dbal:
default_connection: default
connections:
default:
dbname: my_project
user: root
password: root
host: localhost
driver: pdo_mysql
server_version: '8.0.19' # your database server version here
#url: '%env(resolve:DATABASE_URL)%'

How can I setup a second db connection with doctrine2 and symfony?

I use doctrine ORM in a Symfony 2.8 project.
My project contains several Bundles. For one Bundle, which generates Reports I want to use the db server with the slave replication as not to stress the master db server.
How to set this up?
What I tried so far:
In the config.yml
Doctrine Configuration
doctrine:
dbal:
default_connection: default
connections:
default:
driver: pdo_mysql
host: '%database_host%'
port: '%database_port%'
dbname: '%database_name%'
user: '%database_user%'
password: '%database_password%'
charset: UTF8
slave:
driver: pdo_mysql
host: '%database_host_slave%'
port: '%database_port_slave%'
dbname: '%database_name_slave%'
user: '%database_user_slave%'
password: '%database_password_slave%'
charset: UTF8
Here I created my second db connection with the values stored in my parameters.yml.
I seem to get the orm configuration I tried to setup in the same file not correctly.
Let my first explain what I need:
I have a "ReportingBundle" which runs a console command. The entity manager is only needed to provide the proper authorization to the needed db server. The queries itself are oure SQL and I don't use the entities.
my service.yml for this bundle:
services:
myproject.reporting.service.csv_report_attachment:
class: Myproject\ReportingBundle\Service\DefaultCsvReportAttachmentService
arguments: ['#doctrine.orm.slave_entity_manager', '#logger', '#myproject.reporting.service.php_template_engine', 'reportingHtmlMailTemplate.php']
Now my non functioning orm setup in the config.yml:
ORM configuration
orm:
auto_generate_proxy_classes: '%kernel.debug%'
# naming_strategy: doctrine.orm.naming_strategy.underscore
default_entity_manager: default
entity_managers:
slave:
connection: slave
mappings:
MyprojectReportingBundle: ~
default:
connection: default
auto_mapping : true
metadata_cache_driver: redis
query_cache_driver: redis
result_cache_driver: redis
This results in
[Doctrine\ORM\ORMException]
Unknown Entity namespace alias 'MyprojectReportingBundle'.
I tried to follow the documentation here:
http://symfony.com/doc/current/reference/configuration/doctrine.html#custom-mapping-entities-in-a-bundle
Question:
What is the correct syntax, so that my query is run on the slave server instead of the default server?
I believe you have everything configured correctly. However, you need to make sure the Entity classes are defined in the Bundle namespace. So for example you'd want to have all the entities defined within Myproject\ReportingBundle\Entity in your example.

symfony2.6 doctrine2 schema_filter parameter not exists

I am using symfony 2.6 (composer.json equal to its github repo) and I am trying to use the schema filter of DBAL.
in config.yml
# Doctrine Configuration
doctrine:
dbal:
schema_filter: ^sf2_
but error returned on shell:
[Symfony\Component\Config\Definition\Exception\InvalidConfigurationException]
Unrecognized option "schema_filter" under "doctrine.dbal"
What am I missing?
EDIT:
config.yml (doctrine part only)
doctrine:
dbal:
default_connection: default
connections:
default:
driver: "%database.driver%"
host: "%database.master.host%"
port: "%database.master.port%"
dbname: "%database.master.dbname%"
user: "%database.master.user%"
password: "%database.master.password%"
charset: UTF8
options:
1002: "SET NAMES 'UTF8'"
# if using pdo_sqlite as your database driver, add the path in parameters.yml
# e.g. database_path: "%kernel.root_dir%/data/data.db3"
# path: "%database_path%"
slaves:
slave1:
host: "%database.slave1.host%"
port: "%database.slave1.port%"
dbname: "%database.slave1.dbname%"
user: "%database.slave1.user%"
password: "%database.slave1.password%"
mapping_types:
enum: string
set: string
bit: boolean
types:
# some types
orm:
auto_generate_proxy_classes: "%kernel.debug%"
default_entity_manager: default
entity_managers:
default:
connection: default
(POST) CONSIDERATION:
Anyway the schema_filter does not fit my requirements it is too vague to define with a reg exp (I mean for my actual schema is required a too complex reg exp and it is not pratical at all). I posted in doctrine2 groups a request for "enhancing" this option.
https://groups.google.com/forum/#!topic/doctrine-user/Tr4kkpIxwRk
What is your exact symfony version? I tried this config option with Symfony 2.6.5 and everything works fine.
Do you happen to have multiple connections? There is a note in documentation about this, at the end of the page:
Note that if you have multiple connections configured then the schema_filter configuration will need to be placed per-connection.
Manual tables
The schema_filter parameter is part of the Doctrine Migrations bundle. Do you have that installed?

Symfony2 + Doctrine: Create Entities from specific database that is not the default

The Symfony2 Cookbook provides information on how to create entities from existing schemas, but I haven't been able to find a way to import from another database called 'legacy' (see config.yml below) that isn't the default.
Running the following command creates YAML files only from the default database
$ php app/console doctrine:mapping:convert yml ./src/Soapbox/DashboardBundle/Resources/config/doctrine/metadata/orm --from-database --force
If I don't provide any arguments, I receive suggestions, but not sure which ones are applicable.
doctrine:mapping:convert [--filter="..."] [--force] [--from-database] [--extend[="..."]] [--num-spaces[="..."]] [--namespace[="..."]] [--em[="..."]] to-type dest-path
app/config/config.yml
# Doctrine Configuration
doctrine:
dbal:
default_connection: default
connections:
default:
driver: %database_driver%
host: %database_host%
port: %database_port%
dbname: %database_name%
user: %database_user%
password: %database_password%
charset: UTF8
legacy:
driver: pdo_mysql
host: my.host
port: null
dbname: magazines
user: soapbox
password: XXXXX
charset: UTF8
If I'm thinking clearly which I might not be, you have two EntityManagers, default and legacy.
As such you can use the flag --em "legacy" to get it to import from that specific database.

How to use sqlite database on symfony2 project?

In a Symfony2 project, you can configure the databases connections at the app/config/parameters.ini file. Documentation states that you can use, among others, sqlite3 PDO driver.
But configuring sqlite doesn't works well:
[parameters]
database_driver = pdo_sqlite
database_host = localhost
database_port =
database_name = test_project.db
database_user = root
database_password =
Using app/console doctrine:database:create, successfully creates a test_project.db file at the project root directory.
But after creating some entities, then running app/console doctrine:schema:update --force should create the tables on the database file, but it doesn't, file appears empty, with O bytes size.
Note that using any other PDO driver works well, but not with SQLite...
I've also tried to use the full path for the db file in the database_name parameter, but to no avail, database still doesn't gets updated.
For reference, here's the doctrine dbal section of the config.yml file:
doctrine:
dbal:
driver: %database_driver%
host: %database_host%
port: %database_port%
dbname: %database_name%
user: %database_user%
password: %database_password%
charset: UTF8
Is there a way around this? configurations missing? something not stated on the official doc of symfony2 project?
According to Doctrine the elements used for sqlite DBAL configuration are:
user (string): Username to use when connecting to the database.
password (string): Password to use when connecting to the database.
path (string): The filesystem path to the database file. Mutually exclusive with memory. path takes precedence.
memory (boolean): True if the SQLite database should be in-memory (non-persistent). Mutually exclusive with path. path takes precedence.
This is also listed in the full reference for Doctrine configuration in Symfony2, although not elaborated on.
So you need to switch up your config params to match whats appropriate for sqlite.
Here is what I needed to get SQLite to work, just after doing symfony new myapp :
in app/config.yml :
# Doctrine Configuration
doctrine:
dbal:
driver: pdo_sqlite
path: "%database_path%"
In app/config/parameters.yml:
parameters:
database_path: "%kernel.root_dir%/db/myapp_%kernel.environment%.db3"
...
Next I could do a composer install, create a new entity and it just worked.
I've found that if I add a path line pointing at the database_name to my config.yml, sqlite seems to pick that up, and MySQL doesn't seem to complain.
doctrine:
dbal:
driver: %database_driver%
host: %database_host%
port: %database_port%
dbname: %database_name%
path: %database_name%
user: %database_user%
password: %database_password%
charset: UTF8
This means you can still keep all database information in the parameters file, you don't need separate configs depending on which database you are using.
Mainly the file path or the file path permisssion will have issue.
In config.yml, set path to full path like
/home/{name}/NB/PHP/Symfony/test/src/Database/data.db3
Dont give %database_path% or what ever. Try this it will work.
If it works you can give as
%kernel.root_dir%/../src/Database/%database_path%
Also check sqlite is ok by
phpinfo(INFO_MODULES);
In view/output you can see pdo_sqlite and its version.
In my case setting a username and password in config/packages/doctrine.yaml did not create a username/password protected sqlite database.
doctrine:
dbal:
charset: UTF8
url: '%DATABASE_URL%'
user: 'foo'
password: 'bar'
It seems like the parameters username and password are ignored?

Resources