i know this has been asked over and over again, i read the topics, but it's always focused on specific cases and i generally try to understand why its not best practise to use a service inside an entity.
Given a very simple service :
Class Age
{
private $date1;
private $date2;
private $format;
const ym = "%y years and %m month"
const ...
// some DateTime()->diff() methods, checking, formating the entry formats, returning different period formats for eg.
}
and a simple entity :
Class People
{
private $firstname;
private $lastname;
private $birthday;
}
From a controller, i want to do :
$som1 = new People('Paul', 'Smith', '1970-01-01');
$som1->getAge();
Off course i can rewrite the getAge() function inside my entity, its not long, but im very lazy and as i've already written all the possible datetime->diff() i need in the above service, i dont understand why i shouldnt use'em...
NB : my question isnt about how to inject the container in my entity, i can understand why this doesnt make sense, but more what wld be the best practise to avoid to rewrite the same function in different entities.
Inheritance seems to be a bad "good idea" as i could use the getAge() inside a class BlogArticle and i doubt that this BlogArticle Class should be inheriting from the same class as a People class...
Hope i was clear, but not sure...
One major confusion for many coders is to think that doctrine entities "are" the model. That is a mistake.
See edit of this post at the end, incorporating ideas related to CQRS+ES -
Injecting services into your doctrine entities is a symptom of "trying to do more things than storing data" into your entities. When you see that "anti-pattern" most probably you are violating the "Single Responsibility" principle in SOLID programming.
http://en.wikipedia.org/wiki/Anti-pattern
http://en.wikipedia.org/wiki/SOLID_%28object-oriented_design%29
http://en.wikipedia.org/wiki/Single_responsibility_principle )
Symfony is not an MVC framework, it is a VC framework only. Lacks the M part. Doctrine entities (I'll call them entities from now on, see clarification at the end) are a "data persistence layer", not a "model layer". Symfony has lots of things for views, web controllers, command controllers... but has no help for domain modelling ( http://en.wikipedia.org/wiki/Domain_model ) - even the persistence layer is Doctrine, not Symfony.
Overcoming the problem in SF2
When you "need" services in a data-layer, trigger an antipattern alert. Storage should be only a "put here - get from there" system. Nothing else.
To overcome this problem, you should inject the services into a "logic layer" (Model) and separate it from "pure storage" (data-persistence layer). Following the single responsibility principle, put the logics in one side, put the getters and setters to MySQL in another.
The solution is to create the missing Model layer, not present in Symfony2, and make it to give "the logic" of the domain objects, completely separated and decoupled from the layer of data-persistence which knows "how to store" the model into a MySQL database with doctrine, or to a redis, or simply to a text file.
All those storage systems should be interchangeable and your Model should still expose the very same public methods with absolutely no change to the consumer.
Here's how you do it:
Step 1: Separate the model from the data-persistence
To do so, in your bundle, you can create another directory named Model at the bundle-root level (besides tests, DependencyInjection and so), as in this example of a game.
The name Model is not mandatory, Symfony does not say anything about it. You can choose whatever you want.
If your project is simple (say one bundle), you can create that directory inside the same bundle.
If your project is many bundles wide, you could consider
either putting the model splitted in the different bundles, or
or -as in the example image- use a ModelBundle that contains all the "objects" the project needs (no interfaces, no controllers, no commands, just the logic of the game, and its tests). In the example, you see a ModelBundle, providing logical concepts like Board, Piece or Tile among many others, structures in directories for clarity.
Particularly for your question
In your example, you could have:
Entity/People.php
Model/People.php
Anything related to "store" should go inside Entity/People.php - Ex: suppose you want to store the birthdate both in a date-time field, as well as in three redundant fields: year, month, day, because of any tricky things related to search or indexing, that are not domain-related (ie not related withe lo 'logics' of a person).
Anything related to the "logics" should go inside Model/People.php - Ex: how to calculate if a person is over the majority of age just now, given a certain birthdate and the country he lives (which will determine the minumum age). As you can see, this has nothing to do on the persistence.
Step 2: Use factories
Then, you must remember that the consumers of the model, should never ever create model objects using "new". They should use a factory instead, that will setup the model objects properly (will bind to the proper data-storage layer). The only exception is in unit-testing (we'll see it later). But apart from unitary tests, grab this with fire in your brain, and tattoo it with a laser in your retina: Never do a 'new' in a controller or a command. Use the factories instead ;)
To do so, you create a service that acts as the "getter" of your model. You create the getter as a factory accessible thru a service. See the image:
You can see a BoardManager.php there. It is the factory. It acts as the main getter for anything related to boards. In this case, the BoardManager has methods like the following:
public function createBoardFromScratch( $width, $height )
public function loadBoardFromJson( $document )
public function loadBoardFromTemplate( $boardTemplate )
public function cloneBoard( $referenceBoard )
Then, as you see in the image, in the services.yml you define that manager, and you inject the persistence layer into it. In this case, you inject the ObjectStorageManager into the BoardManager. The ObjectStorageManager is, for this example, able to store and load objects from a database or from a file; while the BoardManager is storage agnostic.
You can see also the ObjectStorageManager in the image, which in turn is injected the #doctrine to be able to access the mysql.
Your managers are the only place where a new is allowed. Never in a controller or command.
Particularly for your question
In your example, you would have a PeopleManager in the model, able to get the people objects as you need.
Also in the Model, you should use the proper singular-plural names, as this is decoupled from your data-persistence layer. Seems you are currently using People to represent a single Person - this can be because you are currently (wrongly) matching the model to the database table name.
So, involved model classes will be:
PeopleManager -> the factory
People -> A collection of persons.
Person -> A single person.
For example (pseudocode! using C++ notation to indicate the return type):
PeopleManager
{
// Examples of getting single objects:
Person getPersonById( $personId ); -> Load it from somewhere (mysql, redis, mongo, file...)
Person ClonePerson( $referencePerson ); -> Maybe you need or not, depending on the nature the your problem that your program solves.
Person CreatePersonFromScratch( $name, $lastName, $birthDate ); -> returns a properly initialized person.
// Examples of getting collections of objects:
People getPeopleByTown( $townId ); -> returns a collection of people that lives in the given town.
}
People implements ArrayObject
{
// You could overload assignment, so you can throw an exception if any non-person object is added, so you can always rely on that People contains only Person objects.
}
Person
{
private $firstname;
private $lastname;
private $birthday;
}
So, continuing with your example, when you do...
// **Never ever** do a new from a controller!!!
$som1 = new People('Paul', 'Smith', '1970-01-01');
$som1->getAge();
...you now can mutate to:
// Use factory services instead:
$peopleManager = $this->get( 'myproject.people.manager' );
$som1 = $peopleManager->createPersonFromScratch( 'Paul', 'Smith', '1970-01-01' );
$som1->getAge();
The PeopleManager will do the newfor you.
At this point, your variable $som1 of type Person, as it was created by the factory, can be pre-populated with the necessary mechanics to store and save to the persistence layer.
The myproject.people.manager will be defined in your services.yml and will have access to the doctrine either directly, either via a 'myproject.persistence.manager` layer or whatever.
Note: This injection of the persistence layer via the manager, has several side effects, that would side track from "how to make the model have access to services". See steps 4 and 5 for that.
Step 3: Inject the services you need via the factory.
Now you can inject any services you need into the people.manager
You, if your model object needs to access that service, you have now 2 choices:
When the factory creates a model object, (ie when PeopleManager creates a Person) to inject it via either the constructor, either a setter.
Proxy the function in the PeopleManager and inject the PeopleManager thru the constructor or a setter.
In this example, we provide the PeopleManager with the service to be consumed by the model. When the people manager is requested a new model object, it injects the service needed to it in the new sentence, so the model object can access the external service directly.
// Example of injecting the low-level service.
class PeopleManager
{
private $externalService = null;
class PeopleManager( ServiceType $externalService )
{
$this->externalService = $externalService;
}
public function CreatePersonFromScratch()
{
$externalService = $this->externalService;
$p = new Person( $externalService );
}
}
class Person
{
private $externalService = null;
class Person( ServiceType $externalService )
{
$this->externalService = $externalService;
}
public function ConsumeTheService()
{
$this->externalService->nativeCall(); // Use the external API.
}
}
// Using it.
$peopleManager = $this->get( 'myproject.people.manager' );
$person = $peopleManager->createPersonFromScratch();
$person->consumeTheService()
In this example, we provide the PeopleManager with the service to be consumed by the model. Nevertheless, when the people manager is requested a new model object, it injects itself to the object created, so the model object can access the external service via the manager, which then hides the API, so if ever the external service changes the API, the manager can do the proper conversions for all the consumers in the model.
// Second example. Using the manager as a proxy.
class PeopleManager
{
private $externalService = null;
class PeopleManager( ServiceType $externalService )
{
$this->externalService = $externalService;
}
public function createPersonFromScratch()
{
$externalService = $this->externalService;
$p = new Person( $externalService);
}
public function wrapperCall()
{
return $this->externalService->nativeCall();
}
}
class Person
{
private $peopleManager = null;
class Person( PeopleManager $peopleManager )
{
$this->peopleManager = $peopleManager ;
}
public function ConsumeTheService()
{
$this->peopleManager->wrapperCall(); // Use the manager to call the external API.
}
}
// Using it.
$peopleManager = $this->get( 'myproject.people.manager' );
$person = $peopleManager->createPersonFromScratch();
$person->ConsumeTheService()
Step 4: Throw events for everything
At this point, you can use any service in any model. Seems all is done.
Nevertheless, when you implement it, you will find problems at decoupling the model with the entity, if you want a truly SOLID pattern. This also applies to decoupling this model from other parts of the model.
The problem clearly arises at places like "when to do a flush()" or "when to decide if something must be saved or left to be saved later" (specially in long-living PHP processes), as well as the problematic changes in case the doctrine changes its API and things like this.
But is also true when you want to test a Person without testing its House, but the House must "monitor" if the Person changes its name to change the name in the mailbox. This is specially try for long-living processes.
The solution to this is to use the observer pattern ( http://en.wikipedia.org/wiki/Observer_pattern ) so your model objects throw events nearly for anything and an observer decides to cache data to RAM, to fill data or to store data to the disk.
This strongly enhances the solid/closed principle. You should never change your model if the thing you change is not domain-related. For example adding a new way of storing to a new type of database, should require zero edition on your model classes.
You can see an example of this in the following image. In it, I highlight a bundle named "TurnBasedBundle" that is like the core functionality for every game that is turn-based, despite if it has a board or not. You can see that the bundle only has Model and Tests.
Every game has a ruleset, players, and during the game, the players express the desires of what they want to do.
In the Game object, the instantiators will add the ruleset (poker? chess? tic-tac-toe?). Caution: what if the ruleset I want to load does not exist?
When initializing, someone (maybe the /start controller) will add players. Caution: what if the game is 2-players and I add three?
And during the game the controller that receives the players movements will add desires (for example, if playing chess, "the player wants to move queen to this tile" -which may be a valid, or not-.
In the picture you can see those 3 actions under control thanks to the events.
You can observe that the bundle has only Model and Tests.
In the model, we define our 2 objects: Game, and the GameManager, to get instances of Game objects.
We also define Interfaces, like for example the GameObserver, so anyone willing to receive the Game events should be a GameObserver folk.
Then you can see that for any action that modifies the state of the model (for example adding a player), I have 2 events: PRE and POST. See how it works:
Someone calls the $game->addPlayer( $player ) method.
As soon as we enter the addPlayer() function, the PRE event is raised.
The observers then can catch this event to decide if a player can be added or not.
All PRE events should come with a cancel passed by reference. So if someone decides this is a game for 2 players and you try to add a 3rd one, the $cancel will be set to true.
Then you are again inside the addPlayer function. You can check if someone wanted to cancel the operation.
Do the operation if allowed (ie: mutate the $this-> state).
After the state has been changed, raise a POST event to indicate the observers that the operation has been completed.
In the picture you see three, but of course it has a lot lot more. As a rule of thumb, you will have nearly 2 events per setter, 2 events per method that can modify the state of the model and 1 event for each "unavoidable" action. So if you have 10 methods on a class that operate on it, you can expect to have about 15 or 20 events.
You can easily see this in the typical simple text box of any graphyc library of any operating system: Typical events will be: gotFocus, lostFocus, keyPress, keyDown, keyUp, mouseDown, mouseMove, etc...
Particularly, in your example
The Person will have something like preChangeAge, postChangeAge, preChangeName, postChangeName, preChangeLastName, postChangeLastName, in case you have setters for each of them.
For long-living actions like "person, do walk for 10 seconds" you maybe have 3: preStartWalking, postStartWalking, postStopWalking (in case a stop of 10 seconds cannot be programatically prevented).
If you want to simplify, you can have two single preChanged( $what, & $cancel ) and postChanged( $what ) events for everything.
If you never prevent your changes to happen, you can even just have one single event changed() for all and any change to your model. Then your entity will just "copy" the model properties in the entity properties at every change. This is OK for simple classes and projects or for structures you are not going to publish for third-party consumers, and saves some coding. If the model class becomes a core class to your project, spending a bit of time adding all the events list will save you time in the future.
Step 5: Catch the events from the data layer.
It is at this point that your data-layer bundle enters in action!!!
Make your data layer an observer of your model. When the model Changes its internal state then make your Entity to "copy" that state into the entity state.
In this case, the MVC acts as expected: The Controller, operates on the Model. The consequences of this are still hidden from the controller (as the controller should not have access to Doctrine). The model "broadcasts" the operation made, so anyone interested knows, which in turn triggers that the data-layer knows about the model change.
Particularly, in your project
The Model/Person object will have been created by the PeopleManager. When creating it, the PeopleManager, which is a service, and therefore can have other services injected, can have the ObjectStorageManager subsystem handy. So the PeopleManager can get the Entity/People that you reference in your question and add the Entity/People as an observer to Model/Person.
In the Entity/People mainly you substitute all the setters by event catchers.
You read your code like this: When the Model/Person changes its LastName, the Entity/People will be notified and will copy the data into its internal structure.
Most probably, you are tempted to inject the entity inside the model, so instead of throwing an event, you call the setters of the Entity.
But with that approach, you 'break' the Open-Closed principle. So if at any given point you want to migrate to MongoDb, you need to "change" your "entities" by "documents" in your model. With the observer-pattern, this change occurs outside the model, who never knows the nature of the observer beyond that is its a PersonObserver.
Step 6: Unit test everything
Finally, you want to unit test your software. As this pattern I have explained overcomes the anti-pattern that you discovered, you can (and you should) unit-test the logics of your model independently of how that is stored.
Following this pattern, helps you to go towards the SOLID principles, so each "unit of code" is independent on the others. This will allow you to create unit-tests that will test the "logics" of your Model without writing to the database, as it will inject a fake data-storage layer as a test-double.
Let me use the game example again. I show you in the image the Game test. Assume all games can last several days and the starting datetime is stored in the database. We in the example currently test only if getStartDate() returns a dateTime object.
There are some arrows in it, that represent the flow.
In this example, from the two injecting strategies I told you, I choose the first one: To inject into the Game model object the services it needs (in this case a BoardManager, PieceManager and ObjectStorageManager) and not to inject the GameManager itself.
First, you invoke phpunit that will call look for the Tests directory, recursively in all the directories, finding classes named XxxTest. Then will desire to invoke all the methods named textSomething().
But before calling it, for each test method it calls the setup().
In the setup we will create some test-doubles to avoid "real access" to the database when testing, while correctly testing the logics in our model. In this case a double of my own data layer manager, ObjectStorageManager.
It is assigned to a temporary variable for clarity...
...that is stored in the GameTest instance...
...for later use in the test itself.
The $sut (system under test) variable is then created with a new command, not via a manager. Do you remember that I said that tests were an exception? If you use the manager (you still can) here it is not a unit-test, it's an integration test because tests two classes: the manager and the game. In the new command we fake all the dependencies that the model has (like a board manager, and like a piece manager). I am hardcoding GameId = 1 here. This relates to data-persistance, see below.
We then may call the system under test (a simple Game model object) to test its internals.
I am hardcoding "Game id = 1" in the new. In this case we are only testing that the returned type is a DateTime object. But in case we want to test also that the date that it gets is the proper one, we can "tune" the ObjectStorageManager (data-persistance layer) mock to return whatever we want in the internal call, so we could test that for example when I request the date to the data-layer for game=1 the date is 1st-jun-2014 and for game=2 the date is 2nd-jun-2014. Then in the testGetStartDate I would create 2 new instances, with Ids 1 and 2 and check the content of the result.
Particularly, in your project
You will have a Test/Model/PersonTest unit test that will be able to play with the logics of the person, and in case of needing a person from the database, you will fake it thru the mock.
In case you want to test the storing of the person to the database, it is enough that you unit-test that the event is thrown, no matter who listens to it. You can create a fake listener, attach to the event, and when the postChangeAge happens mark a flag and do nothing (no real database storage). Then you assert that the flag is set.
In short:
Do not confuse logics and data-persistance. Create a Model that has nothing to do with entities, and put all the logics in it.
Do never use new to get your models from any consumer. Use factory services instead. Special attention to avoid news in controllers and commands. Exception: The unit-test is the only consumer that can use a new.
Inject the services you need in the Model via the factory, which in turn receives it from the services.yml configuration file.
Throw events for everything. When I say everything, means everything. Just imagine you observe the model. What would you like to know? Add an event for it.
Catch the events from controllers, views, commands and from other parts of the model, but, specially, catch them in the data-storage layer, so you can "copy" the object to the disk without being intrusive to the model.
Unit test your logics without depending on any real database. Attach the real database storage system in production and attach a dummy implementation for your tests.
Seems a lot of work. But it is not. It is a matter of getting used to it. Just think about the "objects" you need, create them and make the data-layer be "monitors" of your objects. Then your objects are free to run, decoupled. If you create the model from a factory, inject any needed service in the model to the model, and leave the data alone.
Edit apr/2016 - Separating Domain from Persistance
All occurences of the word entity in this answer are referring to the "doctrine entities" which is what causes confusion to the majority of coders, between the model layer and the persistance layer which should be always different.
Doctrine is infrastructure, so doctrine is outside the model by definition.
Doctrine has entities. So, by definition, then doctrine entities are also outside the model.
Instead, the increasing popularity of the DDD building blocks makes a need to clarify even more my answer, as DDD uses the word Entity within the model too.
Domain entities (not Doctrine entities) are similar to what I refer in this answer to Domain objects.
In fact, there are many types of Domain objects:
Domain entities (different from the Doctrine entites).
Domain value objects (could be thought similar to basic types, with logic).
Domain events (also distinct from those Symfony events and also different from the Doctrine events).
Domain commands (different from those Symfony command line controller-like helpers).
Domain services (different from the Symfony framework services).
etc.
Therefore, take all my explanation as this: when I say "Entities are not model objects" just read "Doctrine entities are not Domain entities".
Edit jun/2019 - CQRS+ES analogy
Ancients already used persistant history methods to recod things (for example placing marks on a stone to register transactions).
Since a decade long the CQRS+ES approach (Command Query Responsability Segregation + Event Sourcing) in programming has been growing in popularity, bringing that idea of "the history is immutable" to the programs we code and today many coders think of separating the command side vs the query side. If you don't know what I'm talking about, no worries, just skip the next paragraphs.
The growing popularity of CQRS+ES in the last 3 or 4 years makes me think to consider a comment here and how it relates to what I answered here 5 years ago:
This answer was thought as 1 single model, not a write-model and a read-model. But I'm happy to see many overlapping ideas.
Think of the PRE events, I mention here, as the "commands and the write-model". Think of the POST events as the "Event Sourcing part going towards the read-model".
In CQRS you can easily find that "commands can be accepted or not" in function of the internal state. Usually one implements them throwing exceptions but there are other alternatives there, like answering if the command was accepted or not.
For example, in a "Train" I can "set it to X speed". But if the state is that the train is in a rail that cannot go further 80Km/h, then setting it to 200 should be rejected.
This is ANALOGOUS to the cancel boolean passed by reference where an entity could just "reject" something PRIOR to its state change.
Instead the POST events do not carry the "cancel" event and are thrown AFTER the state change happened. This is why you could not cancel them: They talk about the "state change that actually occurred" and therefore it cannot be cancelled: It aleady happened.
So...
In my answer of 2014, the "pre" events match with the "Command acceptance" of the CQRS+ES systems (the command can be accepted or rejected), and the "post" events match the "Domain events" of the CQRS+ES systems (it just informs that the change actually already happened, do whatever you want with that information).
You already mentioned a very good point. Instances of class Person are not the only thing that can have an age. BlogArticles can also age along with many other types. If you're using PHP 5.4+ you can utilize traits to add little pieces of functionality instead of having service objects from the container (or maybe you can combine them).
Here is a quick mockup of what you could do to make it very flexible. This is the basic idea:
Have one age calculating trait (Aging)
Have a specific trait which can return the appropriate field ($birthdate, $createdDate, ...)
Use the trait inside your class
Generic
trait Aging {
public function getAge() {
return $this->calculate($this->start());
}
public function calculate($startDate) { ... }
}
For person
trait AgingPerson {
use Aging;
public function start() {
return $this->birthDate;
}
}
class Person {
use AgingPerson;
private $birthDate = '1999-01-01';
}
For blog article
// Use for articles, pages, news items, ...
trait AgingContent {
use Aging;
public function start() {
return $this->createdDate;
}
}
class BlogArticle {
use AgingContent;
private $createDate = '2014-01-01';
}
Now you can ask any instance of the above classes for their age.
echo (new Person())->getAge();
echo (new BlogArticle())->getAge();
Finally
If you need type hinting traits won't do you any favors. In that case you will need to provide an interface and let every class that uses the trait implement it (the actual implementation is the trait but the interface enables type hinting).
interface Ageable {
public function getAge();
}
class Person implements Ageable { ... }
class BlogArticle implements Ageable { ... }
function doSomethingWithAgeable(Ageable $object) { ... }
This may seem like a lot of hassle when in reality it's much easier to maintain and extend this way.
A big part is that there is no easy way to inject dependencies when using the database.
$person = $personRepository->find(1); // How to get the age service injected?
One solution might be to pass the age service as an argument.
$ageCalculator = $container('age_service');
$person = $personRepository->find(1);
$age = $person->calcAge($ageCalculator);
But really, you would probably be better off just adding the age stuff to your Person class. Easier to test and all that.
It sounds like you might have some output formatting going on? That sort of thing should probably be done in twig. getAge should really just return a number.
Likewise, your date of birth really should be a date object and not a string.
You are right, it's generally discouraged. However, there are several approaches how you can extend the functionality of an entity beyond the purpose of a data container. Of course, all of them can be considered (more or less) bad practice … but somehow you gotta do the job, right?
You can indeed create an AbstractEntity super class, from which all other entities inherit. This AbstractEntity would contain helper methods that other entities may need.
You can work with custom Doctrine repositories, if you need an entity context to work with an entity manager and return “more special” results than what the common getters would give you. As you have access to the entity manager in a repository, you can perform all kinds of special queries.
You can write a service that is in charge of the entity/entities in question. Downside: you cannot control that other parts of your code (or other developers) know of this service. Advantage: There's no limit to what you can do, and it's all nicely encapsuled.
You can work with Lifecycle Events/Callbacks.
If you really need to inject services into entities, you could consider setting a static property on the entity and only set it once in a controller or a dedicated service. Then you don't need to take care on each initialization of an object. Could be combined with the AbstractEntity approach.
As mentioned before, all of these are have their advantages and disadvantages. Pick your poison.
I come from low level languages - C++ is the highest level I program in.
Recently I came across Reflection, and I just cannot fathom how it could be used without code smells.
The idea of inspecting a class/method/function during runtime, in my opinion, points to a flaw in design - I think most problems Reflection (tries to) solve could be used with either Polymorphism or proper use of inheritance.
Am I wrong? Do I misunderstand the concept and utility of Reflection?
I am looking for a good explanation of when to utilize Reflection where other solutions will fail or be too cumbersome to implement as well as when NOT to use it.
Please enlighten this low-level lubber.
Reflection is most commonly used to circumvent the static type system, however it also has some interesting use cases:
Let's write an ORM!
If you're familiar with NHibernate or most other ORMs, you write classes which map to tables in your database, something like this:
// used to hook into the ORMs innards
public class ActiveRecordBase
{
public void Save();
}
public class User : ActiveRecordBase
{
public int ID { get; set; }
public string UserName { get; set; }
// ...
}
How do you think the Save() method is written? Well, in most ORMs, the Save method doesn't know what fields are in derived classes, but it can access them using reflection.
Its wholly possible to have the same functionality in a type-safe manner, simply by requiring a user to override a method to copy fields into a datarow object, but that would result in lots of boilerplate code and bloat.
Stubs!
Rhino Mocks is a mocking framework. You pass an interface type into a method, and behind the scenes the framework will dynamically construct and instantiate a mock object implementing the interface.
Sure, a programmer could write the boilerplate code for the mock object by hand, but why would she want to if the framework will do it for her?
Metadata!
We can decorate methods with attributes (metadata), which can serve a variety of purposes:
[FilePermission(Context.AllAccess)] // writes things to a file
[Logging(LogMethod.None)] // logger doesn't log this method
[MethodAccessSecurity(Role="Admin")] // user must be in "Admin" group to invoke method
[Validation(ValidationType.NotNull, "reportName")] // throws exception if reportName is null
public void RunDailyReports(string reportName) { ... }
You need to reflect over the method to inspect the attributes. Most AOP frameworks for .NET use attributes for policy injection.
Sure, you can write the same sort of code inline, but this style is more declarative.
Let's make a dependency framework!
Many IoC containers require some degree of reflection to run properly. For example:
public class FileValidator
{
public FileValidator(ILogger logger) { ... }
}
// client code
var validator = IoC.Resolve<FileValidator>();
Our IoC container will instantiate a file validator and pass an appropriate implementation of ILogger into the constructor. Which implementation? That depends on how its implemented.
Let's say that I gave the name of the assembly and class in a configuration file. The language needs to read name of the class as a string and use reflection to instantiate it.
Unless we know the implementation at compile time, there is no type-safe way to instantiate a class based on its name.
Late Binding / Duck Typing
There are all kinds of reasons why you'd want to read the properties of an object at runtime. I'd pick logging as the simplest use case -- let say you were writing a logger which accepts any object and spits out all of its properties to a file.
public static void Log(string msg, object state) { ... }
You could override the Log method for all possible static types, or you could just use reflection to read the properties instead.
Some languages like OCaml and Scala support statically-checked duck-typing (called structural typing), but sometimes you just don't have compile-time knowledge of an objects interface.
Or as Java programmers know, sometimes the type system will get your way and require you to write all kinds of boilerplate code. There's a well-known article which describes how many design patterns are simplified with dynamic typing.
Occasionally circumventing the type system allows you to refactor your code down much further than is possible with static types, resulting in a little bit cleaner code (preferably hidden behind a programmer friendly API :) ). Many modern static languages are adopting the golden rule "static typing where possible, dynamic typing where necessary", allowing users to switch between static and dynamic code.
Projects such as hibernate (O/R mapping) and StructureMap (dependency injection) would be impossible without Reflection. How would one solve these with polymorphism alone?
What makes these problems so difficult to solve any other way is that the libraries don't directly know anything about your class hierarchy - they can't. And yet they need to know the structure of your classes in order to - for example - map an arbitrary row of data from a database to a property in your class using only the name of the field and the name of your property.
Reflection is particularly useful for mapping problems. The idea of convention over code is becoming more and more popular and you need some type of Reflection to do it.
In .NET 3.5+ you have an alternative, which is to use expression trees. These are strongly-typed, and many problems that were classically solved using Reflection have been re-implemented using lambdas and expression trees (see Fluent NHibernate, Ninject). But keep in mind that not every language supports these kinds of constructs; when they're not available, you're basically stuck with Reflection.
In a way (and I hope I'm not ruffling too many feathers with this), Reflection is very often used as a workaround/hack in Object-Oriented languages for features that come for free in Functional languages. As functional languages become more popular, and/or more OO languages start implementing more functional features (like C#), we will most likely start to see Reflection used less and less. But I suspect it will always still be around, for more conventional applications like plugins (as one of the other responders helpfully pointed out).
Actually, you are already using a reflective system everyday: your computer.
Sure, instead of classes, methods and objects, it has programs and files. Programs create and modify files just like methods create and modify objects. But then programs are files themselves, and some programs even inspect or create other programs!
So, why is it so OK for a Linux install to be reflexive that nobody even thinks about it, and scary for OO programs?
I've seen good usages with custom attributes. Such as a database framework.
[DatabaseColumn("UserID")]
[PrimaryKey]
public Int32 UserID { get; set; }
Reflection can then be used to get further information about these fields. I'm pretty sure LINQ To SQL does something similar...
Other examples include test frameworks...
[Test]
public void TestSomething()
{
Assert.AreEqual(5, 10);
}
Without reflection you often have to repeat yourself a lot.
Consider these scenarios:
Run a set of methods e.g. the testXXX() methods in a test case
Generate a list of properties in a gui builder
Make your classes scriptable
Implement a serialization scheme
You can't typically do these things in C/C++ without repeating the whole list of affected methods and properties somewhere else in the code.
In fact C/C++ programmers often use an Interface description language to expose interfaces at runtime (providing a form of reflection).
Judicious use of reflection and annotations combined with well defined coding conventions can avoids rampant code repetition and increase maintainability.
I think that reflection is one of these mechanisms that are powerful but can be easily abused. You're given the tools to become a "power user" for very specific purposes, but it is not meant to replace proper object oriented design (just as object oriented design is not a solution for everything) or to be used lightly.
Because of the way Java is structured, you are already paying the price of representing your class hierarchy in memory at runtime (compare to C++ where you don't pay any costs unless you use things like virtual methods). There is therefore no cost rationale for blocking it fully.
Reflection is useful for things like serialization - things like Hibernate or digester can use it to determine how to best store objects automatically. Similarly, the JavaBeans model is based on names of methods (a questionable decision, I admit), but you need to be able to inspect what properties are available to build things like visual editors. In more recent versions of Java, reflections is what makes annotations useful - you can write tools and do metaprogramming using these entities that exist in the source code but can be accessible at runtime.
It is possible to go through an entire career as a Java programmer and never have to use reflection because the problems that you deal with don't require it. On the other hand, for certain problems, it is quite necessary.
As mentioned above, reflection is mostly used to implement code that needs to deal with arbitrary objects. ORM mappers, for instance, need to instantiate objects from user-defined classes and fill them with values from database rows. The simplest way to achieve this is through reflection.
Actually, you are partially right, reflection is often a code smell. Most of the time you work with your classes and do not need reflection- if you know your types, you are probably sacrificing type safety, performance, readability and everything that's good in this world, needlessly. However, if you are writing libraries, frameworks or generic utilities, you will probably run into situations best handled with reflection.
This is in Java, which is what I'm familiar with. Other languages offer stuff that can be used to achieve the same goals, but in Java, reflection has clear applications for which it's the best (and sometimes, only) solution.
Unit testing software and frameworks like NUnit use reflection to get a list of tests to execute and executes them. They find all the test suites in a module/assembly/binary (in C# these are represented by classes) and all the tests in those suites (in C# these are methods in a class). NUnit also allows you to mark a test with an expected exception in case you're testing for exception contracts.
Without reflection, you'd need to specify somehow what test suites are available and what tests are available in each suite. Also, things like exceptions would need to be tested manually. C++ unit testing frameworks I've seen have used macros to do this, but some things are still manual and this design is restrictive.
Paul Graham has a great essay that may say it best:
Programs that write programs? When
would you ever want to do that? Not
very often, if you think in Cobol. All
the time, if you think in Lisp. It
would be convenient here if I could
give an example of a powerful macro,
and say there! how about that? But if
I did, it would just look like
gibberish to someone who didn't know
Lisp; there isn't room here to explain
everything you'd need to know to
understand what it meant. In Ansi
Common Lisp I tried to move things
along as fast as I could, and even so
I didn't get to macros until page 160.
concluding with . . .
During the years we worked on Viaweb I
read a lot of job descriptions. A new
competitor seemed to emerge out of the
woodwork every month or so. The first
thing I would do, after checking to
see if they had a live online demo,
was look at their job listings. After
a couple years of this I could tell
which companies to worry about and
which not to. The more of an IT flavor
the job descriptions had, the less
dangerous the company was. The safest
kind were the ones that wanted Oracle
experience. You never had to worry
about those. You were also safe if
they said they wanted C++ or Java
developers. If they wanted Perl or
Python programmers, that would be a
bit frightening-- that's starting to
sound like a company where the
technical side, at least, is run by
real hackers. If I had ever seen a job
posting looking for Lisp hackers, I
would have been really worried.
It is all about rapid development.
var myObject = // Something with quite a few properties.
var props = new Dictionary<string, object>();
foreach (var prop in myObject.GetType().GetProperties())
{
props.Add(prop.Name, prop.GetValue(myObject, null);
}
Plugins are a great example.
Tools are another example - inspector tools, build tools, etc.
I will give an example of a c# solution i was given when i started learning.
It contained classes marked with the [Exercise] attribute, each class contained methods which were not implemented (throwing NotImplementedException). The solution also had unit tests which all failed.
The goal was to implement all the methods and pass all the unit tests.
The solution also had a user interface which it would read all class marked with Excercise, and use reflection to generate a user interface.
We were later asked to implement our own methods, and later still to understand how the user interface 'magically' was changed to include all the new methods we implemented.
Extremely useful, but often not well understood.
The idea behind this was to be able to query any GUI objects properties, to provide them in a GUI to get customized and preconfigured. Now it's uses have been extended and proved to be feasible.
EDIT: spelling
It's very useful for dependency injection. You can explore loaded assemblies types implementing a given interface with a given attribute. Combined with proper configuration files, it proves to be a very powerful and clean way of adding new inherited classes without modifying the client code.
Also, if you are doing an editor that doesn't really care about the underlying model but rather on how the objects are structured directly, ala System.Forms.PropertyGrid)
Without reflection no plugin architecture will work!
Very simple example in Python. Suppose you have a class that have 3 methods:
class SomeClass(object):
def methodA(self):
# some code
def methodB(self):
# some code
def methodC(self):
# some code
Now, in some other class you want to decorate those methods with some additional behaviour (i.e. you want that class to mimic SomeClass, but with an additional functionality).
This is as simple as:
class SomeOtherClass(object):
def __getattr__(self, attr_name):
# do something nice and then call method that caller requested
getattr(self.someclass_instance, attr_name)()
With reflection, you can write a small amount of domain independent code that doesn't need to change often versus writing a lot more domain dependent code that needs to change more frequently (such as when properties are added/removed). With established conventions in your project, you can perform common functions based on the presence of certain properties, attributes, etc. Data transformation of objects between different domains is one example where reflection really comes in handy.
Or a more simple example within a domain, where you want to transform data from the database to data objects without needing to modify the transformation code when properties change, so long as conventions are maintained (in this case matching property names and a specific attribute):
///--------------------------------------------------------------------------------
/// <summary>Transform data from the input data reader into the output object. Each
/// element to be transformed must have the DataElement attribute associated with
/// it.</summary>
///
/// <param name="inputReader">The database reader with the input data.</param>
/// <param name="outputObject">The output object to be populated with the input data.</param>
/// <param name="filterElements">Data elements to filter out of the transformation.</param>
///--------------------------------------------------------------------------------
public static void TransformDataFromDbReader(DbDataReader inputReader, IDataObject outputObject, NameObjectCollection filterElements)
{
try
{
// add all public properties with the DataElement attribute to the output object
foreach (PropertyInfo loopInfo in outputObject.GetType().GetProperties())
{
foreach (object loopAttribute in loopInfo.GetCustomAttributes(true))
{
if (loopAttribute is DataElementAttribute)
{
// get name of property to transform
string transformName = DataHelper.GetString(((DataElementAttribute)loopAttribute).ElementName).Trim().ToLower();
if (transformName == String.Empty)
{
transformName = loopInfo.Name.Trim().ToLower();
}
// do transform if not in filter field list
if (filterElements == null || DataHelper.GetString(filterElements[transformName]) == String.Empty)
{
for (int i = 0; i < inputReader.FieldCount; i++)
{
if (inputReader.GetName(i).Trim().ToLower() == transformName)
{
// set value, based on system type
loopInfo.SetValue(outputObject, DataHelper.GetValueFromSystemType(inputReader[i], loopInfo.PropertyType.UnderlyingSystemType.FullName, false), null);
}
}
}
}
}
}
// add all fields with the DataElement attribute to the output object
foreach (FieldInfo loopInfo in outputObject.GetType().GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.GetField | BindingFlags.Instance))
{
foreach (object loopAttribute in loopInfo.GetCustomAttributes(true))
{
if (loopAttribute is DataElementAttribute)
{
// get name of field to transform
string transformName = DataHelper.GetString(((DataElementAttribute)loopAttribute).ElementName).Trim().ToLower();
if (transformName == String.Empty)
{
transformName = loopInfo.Name.Trim().ToLower();
}
// do transform if not in filter field list
if (filterElements == null || DataHelper.GetString(filterElements[transformName]) == String.Empty)
{
for (int i = 0; i < inputReader.FieldCount; i++)
{
if (inputReader.GetName(i).Trim().ToLower() == transformName)
{
// set value, based on system type
loopInfo.SetValue(outputObject, DataHelper.GetValueFromSystemType(inputReader[i], loopInfo.FieldType.UnderlyingSystemType.FullName, false));
}
}
}
}
}
}
}
catch (Exception ex)
{
bool reThrow = ExceptionHandler.HandleException(ex);
if (reThrow) throw;
}
}
One usage not yet mentioned: while reflection is generally thought of as "slow", it's possible to use Reflection to improve the efficiency of code which uses interfaces like IEquatable<T> when they exist, and uses other means of checking equality when they do not. In the absence of reflection, code that wanted to test whether two objects were equal would have to either use Object.Equals(Object) or else check at run-time whether an object implemented IEquatable<T> and, if so, cast the object to that interface. In either case, if the type of thing being compared was a value type, at least one boxing operation would be required. Using Reflection makes it possible to have a class EqualityComparer<T> automatically construct a type-specific implementation of IEqualityComparer<T> for any particular type T, with that implementation using IEquatable<T> if it is defined, or using Object.Equals(Object) if it is not. The first time one uses EqualityComparer<T>.Default for any particular type T, the system will have to go through more work than would be required to test, once, whether a particular type implements IEquatable<T>. On the other hand, once that work is done, no more run-time type checking will be required since the system will have produced a custom-built implementation of EqualityComparer<T> for the type in question.