I have a collection of entities with both parent keys and string ids. Sometimes I need to change the string id (update the entity with a new id). From this question (Modify a Google App Engine entity id?), it looks like I need to create a new entity and delete the old one.
Of course, I want to preserve all of the properties in the old entity when creating the new one, but there doesn't seem to be a clone method for NDB entities.
Is this the best way to change an entity's id, while preserving the parent?
# clone the old_entity and parent as new_entity
new_entity = MyModel(**old_entity.to_dict(), id=new_id, parent=old_entity.parent())
And then, I should be able to do this to replace the old entity with the new one:
new_entity.put() # save the new entity
old_entity.key.delete() # delete the old entity
def clone_entity(e, **extra_args):
klass = e.__class__
props = dict((v._code_name, v.__get__(e, klass)) for v in klass._properties.itervalues() if type(v) is not ndb.ComputedProperty)
props.update(extra_args)
return klass(**props)
example
b = clone_entity(a, id='new_id_here')
#sanch's answer works fine in most cases, but for some reason it will not copy attributes of type ndb.PickleProperty.
This modification will work for all attributes, including PickleProperty, and will also accept an optionnal new_class parameter to make a clone of another class.:
def clone_entity(e, **extra_args):
"""
Clone an ndb entity and return the clone.
Special extra_args may be used to:
- request cloned entity to be of a different class (and yet have attributes from original entity)
- define a specific parent, id or namespace for the cloned entity.
:param e: The ndb entity to be cloned.
:param extra_args: May include special args 'parent', 'id', 'namespace', that will be used when initializing new entity.
other extra_args, may set values for specific attributes.
:return: The cloned entity
"""
if 'new_class' in extra_args:
klass = extra_args.pop('new_class')
else:
klass = e.__class__
props = dict((v._code_name, v.__get__(e, klass)) for v in klass._properties.itervalues() if type(v) is not ndb.ComputedProperty)
init_args = dict()
for arg in ['parent', 'id', 'namespace']:
if arg in extra_args:
init_args[arg] = extra_args.pop(arg)
clone = klass(**init_args)
props.update(**extra_args)
clone.populate(**props)
return clone
Related
I want to create a new task from Command in a custom Bundle. But having troubles with setting the task status. I took the fixture from Backend developer's guide and adapted it to Command
$task = new Task();
$task->setSubject('Important task');
$task->setDescription('This is an important task');
$defaultPriority = $this->doctrine->getRepository(TaskPriority::class)->find('normal');
if ($defaultPriority) {
$task->setTaskPriority($defaultPriority);
}
$task->setOwner($taskDataArray['user']);
$task->setOrganization($this->getOrganization());
$this->getEntityManager(Task::class)->persist($task);
$this->getEntityManager(Task::class)->flush();
The database record is created, but by default status_id field is empty.
Without status it isn't shown in data grid.
The status has type AbstractEnumValue. The method which sets the status is defined in Model as * #method Task setStatus(AbstractEnumValue $status)
How to set up the status correctly in CRUD operation?
Thanks ahead.
Status is a enum field.
Internally enums are entities with autogenerated class names.
To work with the enum entity, first, you have to generate its name, then you can access it using doctrine, as a regular entity.
// generate enum entity class name by the enum code
$statusClass = ExtendHelper::buildEnumValueClassName('task_status');
// find existing status entity
$statusOpen = $this->doctrine->find($statusClass, 'open');
// assign status to the task
$task->setStatus($statusOpen);
See the reference at the OroEntityExtendBundle documentation.
I am working an Symfony 2.8 based web app project which currently uses Doctrine 2. The project is basically a simple ToDo list application which can be synced with a mobile app (iOS/Android).
While reading the Update notes of Doctrine 3 I discovered, that EntityManager::merge will no longer be supported.
An alternative to EntityManager#merge() is not provided by ORM 3.0,
since the merging semantics should be part of the business domain
rather than the persistence domain of an application. If your
application relies heavily on CRUD-alike interactions and/or PATCH
restful operations, you should look at alternatives such as
JMSSerializer.
I am not sure what is the best/correct way to replace EntityManager::merge?
Where do I use merge:
During the sync of the mobile apps with the web app the data is transferred as serialized JSON which is than de-serialized by JMSSerializer to an entity object. When the web app receives a ToDoEntry object this way, it can be a new ToDo-Entry (not known in the web app yet) or an updated existing entry. Either way, the received object is not managed by the EntityManager. Thus $em->persist($receivedObject) will always try to insert a new object. This will fail (due to the unique constraint of the id) if the ToDo-Entry already exists in the web app and needs to be updated.
Instead $em->merge($receivedObject) is used which automatically checks wether an insert or update is required.
Hot wo solve this?
Of course I could check for every received objects if an entity with the same ID already exists. In this case could load the existing object and update its properties manually. However this would be very cumbersome. The real project of course uses many different entities and each entity type/class would need its own handling to check which properties needs to be updated. Isn't there a better solution?
You can try to use registerManaged() method of Doctrine\ORM\UnitOfWork.
// $this->em <--- Doctrine Entity Manager
// $entity <--- detached Entity (and we know that this entity already exists in DB for example)
$id = [$entity->getId()]; //array
$data = $entity->toArray(); //array
$this->em->getUnitOfWork()->registerManaged($entity, $id, $data);
Of course, You can check the state of Your Entity using getEntityState() of Doctrine\ORM\UnitOfWork before/after perfoming needed actions.
$this->eM->getUnitOfWork()->getEntityState($entity, $assert = 3)
$assert <-- This parameter can be set to improve performance of entity state detection by potentially avoiding a database lookup if the distinction between NEW and DETACHED is either known or does not matter for the caller of the method.
While I have posted this question quite a while ago, it is still quite active. Until now my solution was to stick with Doctrine 2.9 and keep using the merge function. Now I am working on new project which should be Doctrine 3 ready and should thus not use the merge anymore.
My solution is of course specific for my special use case. However, maybe it is also useful for other:
My Solution:
As described in the question I use the merge method to sync deserialized, external entities into the web database where a version of this entity might already exist (UPDATE required) or not (INSERT required).
#Merge Annotation
In my case entities have different properties where some might be relevant for syncing and must be merged while others are only used for (web) internal housekeeping and must not be merged. To tell these properties appart, I have created a custom #Merge annotation:
use Doctrine\Common\Annotations\Annotation;
/**
* #Annotation
* #Target("PROPERTY")
*/
final class SyncMerge { }
This annotation is then be used to mark the entities properties which should be merged:
class ToDoEntry {
/*
* #Merge
*/
protected $date;
/*
* #Merge
*/
protected $title;
// only used internally, no need to merge
protected $someInternalValue;
...
}
Sync + Merge
During the sync process the annotation is used to merge the marked properties into existing entities:
public function mergeDeserialisedEntites(array $deserializedEntities, string $entityClass): void {
foreach ($deserializedEntities as $deserializedEntity) {
$classMergingInfos = $this->getMergingInfos($class);
$existingEntity = $this->entityManager->find($class, $deserializedEntity->getId());
if (null !== $existingEntity) {
// UPDATE existing entity
// ==> Apply all properties marked by the Merge annotation
foreach ($classMergingInfos as $propertyName => $reflectionProperty) {
$deserializedValue = $reflectionProperty->getValue($deserializedEntity);
$reflectionProperty->setValue($existingEntity, $deserializedEntity);
}
// Continue with existing entity to trigger update instead of insert on persist
$deserializedEntity = $existingEntity;
}
// If $existingEntity was used an UPDATE will be triggerd
// or an INSERT instead
$this->entityManager->persist($deserializedEntity);
}
$this->entityManager->flush();
}
private $mergingInfos = [];
private function getMergingInfos($class) {
if (!isset($this->mergingInfos[$class])) {
$reflectionClass = new \ReflectionClass($class);
$classProperties = $reflectionClass->getProperties();
$propertyInfos = [];
// Check which properties are marked by #Merge annotation and save information
foreach ($classProperties as $reflectionProperty) {
$annotation = $this->annotationReader->getPropertyAnnotation($reflectionProperty, Merge::class);
if ($annotation instanceof Merge) {
$reflectionProperty->setAccessible(true);
$propertyInfos[$reflectionProperty->getName()] = $reflectionProperty;
}
}
$this->mergingInfos[$class] = $propertyInfos;
}
return $this->mergingInfos[$class];
}
That's it. If new properties are added to an entity I have only to decide whether it should be merged or not and add the annotation if needed. No need to update the sync code.
Actually the code to handle this can be just a few lines. In background Doctrine will issue a query to search for your entity if not already in memory, so you can do the same by doing the query yourself with result cache enabled, and then just use PropertyAccessor to map the data.
https://symfony.com/doc/current/components/property_access.html
See this gist for a POC https://gist.github.com/stevro/99060106bbe54d64d3fbcf9a61e6a273
This question completely follows on from a related question I had asked (and was answered) here: Error when trying to retrieve a single entity
As I understand, to retrieve a single entity from the datastore using a property other than helper methods already provided (e.g. 'id') requires turning a simple data property into an EndpointsAliasProperty? If yes, how would I go about doing that? Or is it that we can only use 'id' (helper methods provided by EndpointsModel) and we cannot use any of the properties that we define (in this case 'title')?
The distinction between the custom EndpointsAliasPropertys and one of the data properties you defined is how they are used. They are all used to create a protorpc message, and that message is then converted into an EndpointsModel with your custom data in it. THIS is where the magic happens.
Breaking it down into steps:
1. You specify your data
from google.appengine.ext import ndb
from endpoints_proto_datastore.ndb import EndpointsModel
class MyModel(EndpointsModel):
my_attr = ndb.StringProperty()
2. You pick your fields for your method
class MyApi(...):
#MyModel.method(request_fields=('id', 'my_attr'), ...)
def my_method(self, my_model_entity):
...
3. A protorpc message class is defined from your fields
>>> request_message_class = MyModel.ProtoModel(fields=('id', 'my_attr'))
>>> request_message_class
<class '.MyModelProto_id_my_attr'>
>>> for field in request_message_class.all_fields():
... print field.name, ':', field.variant
...
id : INT64
my_attr : STRING
This happens every time a request is handled by a method decorated with #MyModel.method.
4. A request comes in your application and a message is created
Using the protorpc message class, a message instance is parsed from the JSON which gets passed along to your Endpoints SPI (which is created by endpoints.api_server).
When the request comes in to your protorpc.remote.Service it is decoded:
>>> from protorpc import remote
>>> protocols = remote.Protocols.get_default()
>>> json_protocol = protocols.lookup_by_content_type('application/json')
>>> request_message = json_protocol.decode_message(
... request_message_class,
... '{"id": 123, "my_attr": "some-string"}'
... )
>>> request_message
<MyModelProto_id_my_attr
id: 123
my_attr: u'some-string'>
5. The protorpc message is cast into a datastore model
entity = MyModel.FromMessage(request_message)
THIS is the step you really care about. The FromMessage class method (also provided as part of EndpointsModel) loops through all the fields
for field in sorted(request_message_class.all_fields(),
key=lambda field: field.number):
and for each field with a value set, turns the value into something to be added to the entity and separates based on whether the property is an EndpointsAliasProperty or not:
if isinstance(value_property, EndpointsAliasProperty):
alias_args.append((local_name, to_add))
else:
entity_kwargs[local_name] = to_add
After completing this loop, we have an ordered list alias_args of all key, value pairs and a dictionary entity_kwargs of the data attributes parsed from the message.
Using these, first a simple entity is created
entity = MyModel(**entity_kwargs)
and then each of the alias property values are set in order:
for name, value in alias_args:
setattr(entity, name, value)
The extended behavior happens in setattr(entity, name, value). Since EndpointsAliasProperty is a subclass of property, it is a descriptor and it has a setter which can perform some custom behavior beyond simply setting a value.
For example, the id property is defined with:
#EndpointsAliasProperty(setter=IdSet, property_type=messages.IntegerField)
def id(self):
and the setter performs operations beyond simply setting data:
def IdSet(self, value):
self.UpdateFromKey(ndb.Key(self.__class__, value))
This particular method attempts to retrieve the entity stored in the datastore using the id and patch in any values from the datastore that were not included in the entity parsed from the request.
If you wanted to do this for a field like my_attr, you would need to construct a custom query which could retrieve the item with that unique my_attr value (or fail if not exactly one such entity exists).
This is problematic and you'd be better off using a unique field like the key or ID used to store the entity in the datastore.
The keys with ancestors sample gives a great example of creating your own custom properties.
If you REALLY insist on using my_attr to retrieve an entity, you could do so using a different property name (since my_attr is already used for the data property) such as fromMyAttr:
class MyModel(EndpointsModel):
def MyAttrSet(self, value):
...
#EndpointsAliasProperty(setter=MyAttrSet)
def fromMyAttr(self):
...
Here, the MyAttrSet instance method would form the query:
def MyAttrSet(self, value):
query = MyModel.query(MyModel.my_attr == value)
results = query.fetch(2)
reject results that aren't unique for my_attr:
if len(results) == 0:
raise endpoints.NotFoundException('Not found.')
if len(results) == 2:
raise endpoints.BadRequestException('Colliding results.')
and copy over the values for the already stored entity if we do find a unique one:
matching_entity = results[0]
self._CopyFromEntity(matching_entity)
self._from_datastore = True
Using Breeze, what is the simplest way to populate a GUID key when an entity is created?
I'll assume that your entity is configured such that the client is responsible for setting the Guid key for new entities. That's the default for the Guid key of an Entity Framework Code First entity; it is as if the key property were adorned with [DatabaseGenerated(DatabaseGeneratedOption.None)]
The obvious approach is to set the key after creating the entity and before adding it to the manager, e.g.:
function createFoo() {
var foo = fooType.createEntity();
foo.id(breeze.core.getUuid()); // Knockout implementation
manager.addEntity(foo);
}
This may be all you ever need.
On the other hand, you may find that you're creating new Foos in many places and for some strange reason you can't use the createFoo function. You certainly don't want to repeat that code.
You can extend the Foo entity type with id-setting behavior after which you'd be able to write:
function createFoo() {
var foo = fooType.createEntity(); // foo.id is set for you
manager.addEntity(foo);
}
There are two approaches to consider - custom constructor and type initializer; both are described in "Extending Entities"
Constructor
You can initialize the key inside a custom constructor. Breeze calls the constructor both when you create the entity and when it materializes a queried entity. Breeze will replace the initial key value when materializing.
Here's an example that assumes the Knockout model library.
function Foo() {
foo.id(breeze.core.getUuid()); // using KO
}
// one way to get the MetadataStore
var store = manager.metadataStore;
// register the ctor with the Foo type
store.registerEntityTypeCtor("Foo", Foo);
Pretty simple. The only downside is that Breeze will generate a Guid every time it makes an entity, whether creating a new one or materializing one from a query. It's wasted effort during materialization but so what? Well, I suppose that might become a performance issue although I wouldn't assume so until I had measured it.
Initializer
Suppose you measured and the repeated Guid generation is a serious problem (really?). You could set the key in a type initializer instead and only call the Guid generator when creating a new entity.
Breeze calls a type initializer after the entity has been created or materialized from query just before returning that entity to the application. Clearly you don't want to overwrite a materialized key from the database so you'll test the key value to make sure it's not real (i.e. to make sure you're fixing a created entity) before assigning it. Here's an example.
function fooInitializer(foo) {
var emptyGuid = "00000000-0000-0000-0000-000000000000";
if (foo.id() !=== emptyGuid) {
foo.id(breeze.core.getUuid());
}
}
var store = manager.metadataStore;
// register the initializer; no ctor in this example
store.registerEntityTypeCtor("Foo", function(){}, fooInitializer);
Assuming you have a Guid surrogate Key on all your entities like we have in our case, you could code a createInstance factory that does the following in a very generic approach:
function createInstance(breezeEntityManager, typeName) {
var keyProperty = breezeEntityManager.metadataStore.getEntityType(typeName, false).dataProperties.filter(function (p) {
return p.isPartOfKey;
})[0];
var config = {};
config[keyProperty.name] = breeze.core.getUuid();
return breezeEntityManager.createEntity(typeName, config);
}
This way, you won't have to create an initializer for all your entities.
I've got a data provider that contains a collection of entities. I only want to be able to create a new entity through the data provider.
I.e, to create a new record I need to use:
Entity entity = Provider.AddNew();
enity.set_Properties... etc
My issue is that if I set my entities to Internal, System.Activator cannot create an Instance of them. Each of my Data Providers uses a Base class with the generic type of the entity passed through.
So at the moment my AddNew() method contains the following:
public T AddNew()
{
T added = Activator.CreateInstance<T>();
this.Collection.Add(added);
return added;
}
It's obviously not the end of the world if I can instantiate a new entity manually outside of the Data Provider namespaces, but it seems pointless considering there's no way to ever save them, so why give the option to do so?
EDIT: Forgot to mention that all my providers, entities, etc are in the same namespace.
Don't use the Activator, which relies on a public constructor. Instead use reflection to find the parameterless constructor and then call it. Something along these lines:
Type t = typeof(MyType);
var parameterlessCtor = (from c in t.GetConstructors(
BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance)
where c.GetParameters().Length == 0
select c).FirstOrDefault;
if(parameterlessCtor != null) instance = parameterlessCtor.Invoke(null);