I am using a redis cache in my symfony application.
When inspecting the internally used keys in redis-cli I found out that all entries share a common prefix (like AWVAugkC6-:, or gBphwvED1G:). The prefix changes if I toggle APP_ENV between dev and prod.
Can I disable this behavior?
Currently, I cannot use the prod entries in dev mode, and vice versa.
A call to
$this->cache->getItem('2f560421-1b6d-4251-8392-e9c0f5e824a7-1514764800-1546300799-noref-nostr');
translates in devmode to a redis key
gBphwvED1G:2f560421-1b6d-4251-8392-e9c0f5e824a7-1514764800-1546300799-noref-nostr
however in prod, it's
AWVAugkC6-:2f560421-1b6d-4251-8392-e9c0f5e824a7-1514764800-1546300799-noref-nostr
So the prefix is exchanged!
I have traced this back to AbstractTrait::getId($key) https://github.com/symfony/cache/blob/e5e9a6d35558b43cca49cde9f3a7ad22980812cb/Traits/AbstractTrait.php#L269, but I think this line should just hash the give key, not the environment...?
framework:
cache:
default_redis_provider: 'redis://%env(REDIS_HOST)%'
pools:
data_evaluator_cache_items_cache:
adapter: cache.adapter.redis
default_lifetime: 0
public: true
data_evaluator_cache_tags_cache:
adapter: cache.adapter.redis
default_lifetime: 0
public: true
services:
tagged_data_cache:
class: Symfony\Component\Cache\Adapter\TagAwareAdapter
arguments:
$itemsPool: "#data_evaluator_cache_items_cache"
$tagsPool: "#data_evaluator_cache_tags_cache"
public: true
I am unsure if this relates to the TagAwareAdapter. In reality an additional tag entry is stored, which
The prefix is computed to ensure dev and prod don't share the same cached items, to be safe by default. This is done when using semantic configuration, in a compiler pass, here:
https://github.com/symfony/cache/blob/master/DependencyInjection/CachePoolPass.php#L55
If you want to opt out from this, you might need to define your cache pools directly as a service - without using semantic configuration. You'll then have full control over the namespace and everything else.
You can also keep using semantic and redefine the kernel.container_class parameter. I think this can be done by overriding the getContainerClass on your Kernel.
But you should double check first why you need same namespace for prod and dev.
Related
To deploy, I 'git pull' then 'bin/console cache:clear' to see the updates to my website.
It works, but I don't want to lose MY cache pool's data when this happens. (I cache results to 3rd party api calls)
I expected the symfony command to clear it's own cache stores only and not the cache items I've submitted since they are still valuable to me after I clear the symfony cache to deploy new code.
Q1: How can I clear all the caches except my app's cache pool 'api_call_cache_pool' below?
Q2: Is there a different cache clear call I should use instead of 'bin/console cache:clear'?
Q3: Would api_call_cache_pool not be cleared if it's adapter was Redis instead of filesystem?
framework:
cache:
directory: '%kernel.cache_dir%/pools'
app: cache.adapter.filesystem
system: cache.adapter.system
pools:
api_call_cache_pool:
adapter: cache.adapter.filesystem
default_lifetime: 6000
tags: true
As cache:clear will clear all your caches, I would reccomend to create a script by leveraging the php bin/console cache:pool:list and exclude what you want to exclude (api_call_cache_pool in your example).
I'm not sure why a "blacklist" policy can't be applied here.
(Or, maybe, use cache.system_clearer and cache.app_clearer can be enough; you should try by yourself)
I have a library used by two Symfony apps, this library defines a set of services that I want to be public (I want to be able to retrieve them directly through the container. When I try to access one service I have this:
The "Library\Service\DerivedServices\OneSpecificImplementation" service or alias has been removed or inlined when the container was compiled. You should either make it public, or stop using the container directly and use dependency injection instead.
The problem is that said service is declared public.
Basically there is:
a Library\Service\BaseService class that has two setters for the common dependencies (doctrine and a logger in this snippet);
several derived class (in the Library\Service\DerivedServices namespace), each defines a new service (with its own constructor to handle DI directly).
So here are my services definitions:
# Base: inject common dependencies
Library\Service\BaseService:
abstract: true
calls:
- [setDoctrine, ['#doctrine.orm.entity_manager']]
- [setLogger, ['#Library\Service\Logger']]
# These services are public to be retrieved directly by the container interface
Library\Service\DerivedServices\:
resource: '../vendor/company/library/src/Library/Service/DerivedServices'
public: true
autowire: true
autoconfigure: false
parent: Library\Service\BaseService
Then, the Symfony application retrieves one derived service like:
$this->get('Library\Service\DerivedServices\OneSpecificImplementation');
These didn't make any difference:
I have changed the order of the service definitions
Both apps run Symfony 4.3.3
I think this is something trivial configuration-wise, but I can't pinpoint it (and after 2 hours trying to debug why the framework compiles my service as private, I thought that someone probably had this and could probably help me).
It turns out the order of declaration of the services matter a lot. As I thought so, the problem was configuration-wise.
I had:
Library\Service\BaseService:
...
Library\Service\DerivedServices\:
...
Library\Service\:
resource: '../vendor/company/library/src/Library/Service'
The last instruction redeclared all services as private (by default).
I changed this to:
Library\Service\:
resource: '../vendor/company/library/src/Library/Service'
Library\Service\BaseService:
...
Library\Service\DerivedServices\:
...
This declared all the services private first, then redeclared them with the new declaration: use of parent + public.
When I run the command below, I can see that Symfony considers every class in the app as a potential injectable service:
$ bin/console debug:container
Symfony Container Services
==========================
------------------------------- -----------------------------
Service ID Class name
------------------------------- -----------------------------
App\Domain\Model\Product App\Domain\Model\Product
App\Domain\Model\Site App\Domain\Model\Site
App\Domain\Model\User App\Domain\Model\User
...
This includes entities and value objects, that obviously should not, and will not, ever be injected, and as such should not be registered in the container.
Is this expected? Is there a way to either:
explicitly blacklist classes from being registered in the container,
or explicitly whitelist services that should be registered in the container?
it is expected.
have a look at config/services.yaml where the magic happens (at least this happens in my services.yaml):
# removed lines for readability before, after and in-between the following
services:
App\:
resource: '../src/*'
exclude: '../src/{Entity,Migrations,Tests}'
this essentially should turn everything except for entities, migrations and tests in the src directory into services. you can adapt this to your needs obviously.
So essentially ... if you remove that App\ entry, you would be bound to explicitly add whitelist all classes that are supposed to be services.
Blacklisting on the other hand is shown in the example, although the example is quite general. Maybe there are more sophisticated ways to do this ...
This is quite a long question, but there's quite a lot to it.
It feels like it should be a reasonably common use case, so I'm hoping the Stack Overflow community can provide me with a 'best practice in Symfony2' answer.
The solution I describe below works, but there are several consequences I'd like to avoid:
In my local dev environment, if I have used the wrong db connection the test will work in dev but fail on production
The routes of the ADMIN API are accessible on the PUBLIC API url, just denied.
If I have a mirror of live in my dev environment (3 separate checkouts with the corresponding parameters.yml file) then the feature tests for the other bundles fail
Is there a 'best practice in Symfony2' way to set up my project?
We're running a LAMP stack. We use git/(Atlassian) stash for version control.
We're using doctrine for the ORM and FOS-REST with OAuth plus symfony firewalls to authenticate and authorise the users.
We're committed to use Symfony2, so I am trying to find a 'best practice' solution:
I have a project with 3 applications:
A public-facing API (which gives read-only access to the data)
A protected API (which provides admin functionality)
A set of batch processes (to e.g. import data and monitor data quality)
Each application uses a set of shared models.
I have created 4 bundles, one each for the application and a 4th for the shared models.
Each application must use a different database user to access the database.
There's only one database.
There's several tables, one is called 'prices'
The admin API only must be accessible from one hostname (e.g. admin-api.server1)
The public API only must be accessible from a different hostname (e.g. public-api.server2)
Each application is hosted on a different server
In parameters.yml in my dev environment I have this
// parameters.yml
api_public_db_user: user1
api_public_db_pass: pass1
api_admin_db_user: user2
api_admin_db_pass: pass2
batch_db_user: user3
batch_db_pass: pass3
In config.yml I have this:
// config.yml
doctrine:
dbal:
connections:
api_public:
user: "%api_public_db_user%"
password: "%api_public_db_pass%"
api_admin:
user: "%api_admin_db_user%"
password: "%api_admin_db_pass%"
batch:
user: "%batch_db_user%"
password: "%batch_db_pass%"
In my code I can do this (I believe this can be done from the service container too, but I haven't got that far yet)
$entityManager = $this->getContainer()->get('doctrine')->getManager('api_public');
$entityRepository = $this->getContainer()->get('doctrine')->getRepository('CommonBundle:Price', api_admin');
When I deploy my code to each of the live servers, I put junk values in the parameters.yml for the other applications
// parameters.yml on the public api server
api_public_db_user: user1
api_public_db_pass: pass1
api_admin_db_user: **JUNK**
api_admin_db_pass: **JUNK**
batch_db_user: **JUNK**
batch_db_pass: **JUNK**
I have locked down my application so that the database isn't accessible (and thus the other API features don't work)
I have also set up Symfony firewall security so that the different routes require different permissions
There's also security in the apache vhost to deny access to say the admin api path from the public api directory.
So, I have secured my application and met the requirement of the security audit, but the dev process isn't ideal and something feels wrong.
As background:
We have previously looked at splitting it up into different applications within the same project (like this Symfony2 multiple applications and api centric application. Actually followed this method http://jolicode.com/blog/multiple-applications-with-symfony2) , but ran into difficulties, and in any case, Fabien says not to (https://groups.google.com/forum/#!topic/symfony-devs/yneojUuFiqw). That this existed in Symfony1 and was removed in Symfony2 is enough of an argument for me.
We have previously gone down the route of splitting up each bundle and importing it using composer, but this caused too many development overheads (for example, having to modify many repositories to implement a feature; it not being possible to see all of the changes for a feature in a single pull request).
We are receiving an ever growing number of requests to create APIs, and we're similarly worried about putting each application in its own repository.
So, putting each of the three applications in a separate Symfony project / git repository is something we want to avoid too.
The configuration uses :
doctrine:
dbal:
driver: "%database_driver%"
....
orm:
auto_generate_proxy_classes: "%kernel.debug%"
auto_mapping: true
What is the exact meaning of auto_mapping? It is used in tons of examples with true and false, and no precise description.
When does occurs the proxy generation if it's not auto ? By doctrine command-line tools ?
auto_mapping is where doctrine will automatically load the mapping from your bundle Resources/config/doctrine directory.
Setting it to false will mean that you will need to load the mappings yourself. It can be handy if you have mappings for entities rather than mapped superclasses in a vendor bundle that you want to override.
You can do this either by way of stating the mappings in the doctrine config ...
doctrine:
orm:
entity_managers:
default:
mappings:
AcmeUnknownBundle:
mapping: true
type: yml
dir: "Resources/config/doctrine"
alias: ~
prefix: Acme\UnknownBundle\Entity
is_bundle: true
adding them in some sort of mappings pass ...
class AcmeUnknownBundle extends Bundle
{
public function build(ContainerBuilder $container)
{
parent::build($container);
// ...
$modelDir = realpath(__DIR__.'/Resources/config/doctrine/model');
$mappings = array(
$modelDir => 'Acme\UnknownBundle\Model',
);
$ormCompilerClass = 'Doctrine\Bundle\DoctrineBundle\DependencyInjection\Compiler\DoctrineOrmMappingsPass';
if (class_exists($ormCompilerClass)) {
$container->addCompilerPass(
DoctrineOrmMappingsPass::createYamlMappingDriver(
$mappings,
array('acme_unknown.model_manager_name'),
true
));
}
}
}
I know this is super late, but the answer to the second part of the question regarding auto_generate_proxy_classes can be found in Doctrine's ORM documentation, here: Auto-Generating Proxy Classes (ORM v2.11 docs).
I recommend also reading the "Proxy Objects" section a little further down on the same page. That section says, in part:
A proxy object is an object that is put in place or used instead of the real object. A proxy object can add behavior to the object being proxied without that object being aware of it. In ORM, proxy objects are used to realize several features but mainly for transparent lazy-loading.
Proxy objects with their lazy-loading facilities help to keep the subset of objects that are already in memory connected to the rest of the objects. This is an essential property as without it there would always be fragile partial objects at the outer edges of your object graph.
Doctrine ORM implements a variant of the proxy pattern where it generates classes that extend your entity classes and adds lazy-loading capabilities to them. Doctrine can then give you an instance of such a proxy class whenever you request an object of the class being proxied.
WARNING: The ORM docs are very clear that this should be allowed in development only. Read more at the link above, but the bottom line is this:
Furthermore you should have the Auto-generating Proxy Classes option to true in development and to false in production. If this option is set to TRUE it can seriously hurt your script performance if several proxy classes are re-generated during script execution. Filesystem calls of that magnitude can even slower than all the database queries Doctrine issues. Additionally writing a proxy sets an exclusive file lock which can cause serious performance bottlenecks in systems with regular concurrent requests.
tl;dr -- and I know that I'm glossing over a lot of nuances -- this is somewhat like a PHP version of Javascript's Promise model. Don't have an actual object yet? No worries, we'll give you a fake (proxy) one and fill in the details later if you need them. The auto_generate_proxy_classes setting defines whether or not this is done on the fly at runtime.
For those who want more detailed info on the Proxy Pattern in PHP, try here or here.