I'm trying to use Doctrine Result Caching with Redis and Predis and SncRedisBundle and Symfony.
I would like to know if my cache+doctrine+redis+predis+SncRedisBundle configuration is set properly
snc_redis:
clients:
default:
type: predis
alias: default
dsn: redis://localhost
doctrine:
metadata_cache:
client: default
entity_manager: default
document_manager: default
result_cache:
client: default
entity_manager: [default, read]
query_cache:
client: default
entity_manager: default
because below query display what expected, but how do I know if it come from cache or not?
$query = $repository->createQueryBuilder('p')
->where('p.id < :id')
->setParameter('id', '100')
->orderBy('p.id', 'ASC')
->getQuery()
->useQueryCache(true)
->useResultCache(true);
$products = $query->getResult();
As you can see in this test https://github.com/doctrine/doctrine2/blob/master/tests/Doctrine/Tests/ORM/Functional/ResultCacheTest.php#L40 there is no function to do this.
You can however achieve this by either defining your own cacheId or using ->getQuery()->getResultCacheId()deprecated ->getQuery()->getQueryCacheProfile()->getCacheKey() and ->getQuery()->getResultCacheDriver()->fetch($cacheKey)
source: https://github.com/doctrine/doctrine2/blob/master/lib/Doctrine/ORM/AbstractQuery.php#L945
Related
I want to cache results of some HTTP requests to my Symfony 3.4 application using Symfony HttpCache.
Following their docs I now have those lines in app_dev.php
$kernel = new AppKernel('dev', true);
$kernel = new AppCache($kernel);
This is my controller code
$overview = $this->channelRepository->getOverview();
$response = new JsonResponse($overview);
$response->setSharedMaxAge(3600);
return $response;
And this is my config.yml
framework:
cache:
app: cache.adapter.redis
default_redis_provider: snc_redis.default
#...
snc_redis:
clients:
default:
type: phpredis
alias: default
dsn: redis://localhost
logging: false
However, I don't see new entries appear in my redis server. Instead, those entries appear in filesystem cache (var/cache/dev/http/cache). How do I fix my application to use Redis HTTP cache?
I dug a bit deeper and found out that AppCache that I use extends Symfony\Bundle\FrameworkBundle\HttpCache\HttpCache which only accepts filesystem store. Does that mean that I should somehow use Symfony\Component\HttpKernel\HttpCache\HttpCache, which accepts any Store in its constructor?
How to use DoctrineCacheBundle? I want to cache query (so that it is not translated to SQL each time, right?). Also I want to cache the result.
I have set it up like this:
doctrine:
orm:
metadata_cache_driver:
cache_provider: metadata_cache
query_cache_driver:
cache_provider: query_cache
result_cache_driver:
cache_provider: result_cache
and
doctrine_cache:
providers:
metadata_cache:
apc: ~
query_cache:
apc: ~
result_cache:
apc: ~
And in my controller:
$cache = $this->get('doctrine_cache.providers.query_cache');
if ($cache->contains('someid')) {
$surveysEntities = $cache->fetch('someid');
} else {
$surveysEntities = $this->getDoctrine()->getRepository('MyBundle:Survey')->getSurveys();
$cache->save('someid', $surveysEntities);
}
This will cache only query? Or both query and result? Is it enough? OR I should also do sth on query in my repository?
OK, got it. This cache result. Query is cached with doctrine via config. settings.
My application pulls data from an external database and then stores in the application database after minor processing. How would I set-up the mappings for the external database since it's not tied to entities? This is what I currently have:
dbal:
default_connection: default
types:
json: Sonata\Doctrine\Types\JsonType
connections:
default:
driver: pdo_mysql
host: "%database_host%"
port: "%database_port%"
dbname: "%database_name%"
user: "%database_user%"
password: "%database_password%"
charset: UTF8
rnr:
driver: pdo_mysql
host: "%database_host2%"
port: "%database_port2%"
dbname: "%database_name2%"
user: "%database_user2%"
password: "%database_password2%"
charset: UTF8
orm:
default_entity_manager: default
auto_generate_proxy_classes: "%kernel.debug%"
entity_managers:
default:
connection: default
naming_strategy: doctrine.orm.naming_strategy.underscore
auto_mapping: true
rnr:
connection: rnr
mappings:
AppBundle: ~
With this implementation, I get the error below:
[Doctrine\ORM\ORMException]
Unknown Entity namespace alias 'AppBundle'.
Here's how I implement the function:
protected function execute(InputInterface $input, OutputInterface $output)
{
$em = $this->getContainer()->get('doctrine')->getManager();
$q = $em->createQueryBuilder();
$q->select('t')->from('AppBundle:TransactionSync', 't')->orderBy('t.id', 'DESC')->setMaxResults(1);
$sync = $q->getQuery()->getResult();
$em1 = $this->getContainer()->get('doctrine')->getManager('rnr');
$conn = $em1->getConnection();
$query = "SELECT id, merchant, client, phone, traderTransIdent AS member_id, transaction_id, transaction_type_id, value AS amount, points, DATE_FORMAT(STR_TO_DATE( transaction_date, '%d-%m-%Y' ), '%Y-%m-%d') AS transaction_date FROM merchant_transactions WHERE id > ". $sync->getId();
$stmt = $conn->prepare($query);
$stmt->execute();
$results = $stmt->fetchAll();
if(count($results) > 1)
{
$ts = new TransactionSync();
$ts->setStartTime(new \DateTime());
$id = 0;
foreach($results as $result)
{
$transaction_type = $em->getRepository('AppBundle:TransactionType')->find($result['transaction_type_id']);
$member = $em->getRepository('AppBundle:Member')->find($result['member_id']);
$transaction = new Transaction();
$transaction->setAmount($result['amount']);
$transaction->setPoints($result['points']);
$transaction->setClient($result['client']);
$transaction->setPhone($result['phone']);
$transaction->setTransactionId($result['transaction_id']);
$transaction->setTransactionDate(new \DateTime($result['transaction_date']));
$transaction->setTransactionType($transaction_type);
$transaction->setMember($member);
$em->persist($transaction);
$id = $result['id'];
}
$ts->setLastId($id);
$ts->setRecords(count($results));
$ts->setEndTime(new \DateTime());
$em->persist($ts);
$em->flush();
}
$output->writeln($text);
}
Unfortunately, you cannot use auto_mapping with multiple connections and you cannot map the same Bundle and/or the same alias to different connections. Doctrine is probably looking for AppBundle entities in connection default, completely ignoring rnr; to check all entities known by Doctrine run:
app/console doctrine:mapping:info
and see if your classes are listed correctly.
To workaround your problem, you need to move the TransactionSync entity out of AppBundle. For example, you can create a 'ExtEntity' directory and change your configuration to something like this:
orm:
default_entity_manager: default
auto_generate_proxy_classes: "%kernel.debug%"
entity_managers:
default:
connection: default
naming_strategy: doctrine.orm.naming_strategy.underscore
mappings:
AppBundle: ~
rnr:
connection: rnr
mappings:
ExtEntityMapping:
arbitrary_key:
type: xml # or annotation/yml
dir: %kernel.dir%/../src/AppBundle/ExtEntity
prefix: AppBundle\ExtEntity\
alias: ExtEntity
To avoid some headache, you can also use the method Registry:: getEntityManagerForClass to retrieve the proper ObjectManager for the object.
For example you can do the following in your code:
$registry = $this->getContainer()->get('doctrine');
$transSyncManager = $registry->getEntityManagerForClass('ExtEntity:TransactionSync');
$transTypeManager = $registry->getEntityManagerForClass('AppBundle:TransactionType');
Going off-track, note that you are using prepared statement in a wrong way, which is prone to SQL-injection (maybe not if IDs are always generated by DBMS); use parameters like this:
$query = "SELECT id, merchant, client, phone, traderTransIdent AS member_id, transaction_id, transaction_type_id, value AS amount, points, DATE_FORMAT(STR_TO_DATE( transaction_date, '%d-%m-%Y' ), '%Y-%m-%d') AS transaction_date FROM merchant_transactions WHERE id > :minId";
$stmt = $conn->prepare($query);
$stmt->execute(array('minId' => $sync->getId());
And even better -- if you can modify your database schema -- use DateTime for transaction_date column and avoid the string conversion madness. :)
i have read the documentation of symfony2 in relation to the performance and I have realized the following steps.
Install APC 'php-apc' on my webserver and restart my webserver
Modify my doctrine configuration
doctrine:
dbal:
driver: "%database_driver%"
host: "%database_host%"
port: "%database_port%"
dbname: "%database_name%"
user: "%database_user%"
password: "%database_password%"
charset: UTF8
orm:
auto_generate_proxy_classes: "%kernel.debug%"
auto_mapping: true
metadata_cache_driver: apc
result_cache_driver: apc
query_cache_driver: apc
Now if i call a action to retrieve all users from database i see in the information bar at the bottom that doctrine execute every time 114 queries. Why the queries not cached?
My action look like this:
$users = $this->getDoctrine()->getRepository('AppUserBundle:User')->findAll();
return $this->render('AppUserBundle:User:index.html.twig', array('users' => $users));
Doctrine doesn't cache query results by default. You have to explicitly point that you want to cache query using useResultCache method. For example, if you'd like to cache getting all users, write your own method in User repository class:
use Doctrine\ORM\EntityRepository;
class UserRepository extends EntityRepository
{
public function fetchAll()
{
$query = $this->createQueryBuilder('u')->getQuery();
return $query->useResultCache(true)->getResult();
}
}
The method may take additional arguments:
public function useResultCache($bool, $lifetime = null, $resultCacheId = null)
$bool - set to true if you want to cache query result
$lifetime - TTL of cached result in seconds
$resultCacheId - you can pass your own id, in case of null Doctrine will handle that
I'm using symfony2 snc-redis bundle for caching.
On my server, redis has been installed and working correctly.
My problem is; when i try to clear or flush db with redis, all sites on my server that using redis, crashes. Giving internal server error because of prod env.
I'v tried to change redis configuration ports in my config.yml for every single site on my server but i think didn't work.
My sample snc-redis configuration:
snc_redis:
clients:
default:
type: predis
alias: default
dsn: redis://localhost
logging: %kernel.debug%
cache:
type: predis
alias: cache
dsn: redis://localhost/1
logging: true
cluster:
type: predis
alias: cluster
dsn:
- redis://127.0.0.1/5
- redis://127.0.0.2/6
- redis://pw#/var/run/redis/redis-1.sock/7
- redis://127.0.0.1:6379/8
options:
profile: 2.4
connection_timeout: 10
connection_persistent: true
read_write_timeout: 30
iterable_multibulk: false
throw_errors: true
cluster: Snc\RedisBundle\Client\Predis\Connection\PredisCluster
monolog:
type: predis
alias: monolog
dsn: redis://localhost/1
logging: false
options:
connection_persistent: true
session:
client: default
prefix: foo
use_as_default: true
doctrine:
metadata_cache:
client: cache
entity_manager: default
document_manager: default
result_cache:
client: cache
entity_manager: [default, read]
document_manager: [default, slave1, slave2]
namespace: "dcrc:"
query_cache:
client: cache
entity_manager: default
monolog:
client: monolog
key: monolog
swiftmailer:
client: default
key: swiftmailer
monolog:
handlers:
main:
type: service
id: monolog.handler.redis
level: debug
What i'm doing wrong? How can i get it work correctly and will not cause crashing.
My Redis Bundle for Symfon2:
Snc\RedisBundle\SncRedisBundle()
https://github.com/snc/SncRedisBundle
You can define prefix for each site like this:
snc_redis:
clients:
default:
dsn: "redis://localhost:6379"
options:
prefix : "site_name"
type: phpredis
alias: default
logging: %kernel.debug%
Note: You must to be considered to put this prefix to all clients ;)
Did you try to change client alias for every site ?