Updating and creating entities in same loop - symfony

I have a loop that iterates through some imported Product data, and uses Doctrine2 to persist it to a database.
For each product I check to see if that productID exists already. If so, update it. If not, create it and persist it.
I do the same with associated entities, which is where I run into problems, for example each Product is related to a Manufacturer.
On each loop I will check to see is the ManufacturerID exists, and if not create/persist it.
If I create ManufacturerID=3 in one iteration, and then later on I have another product with ManufacturerID3, Doctrine doesn't know about it yet because it hasn't been flushed.
I can fix this by doing a flush() after every loop, as opposed to when the loop is completed, but I am wondering if there is a better way, maybe some way for Doctrine to search for objects with ManufacturerID=3 both in the repository and in newly persisted objects?
Flush()ing after every loop works but it doesn't seem like the right way to do it.
$manufacturer = $this->em
->getRepository('AMyBundle:Manufacturer')
->findOneByPosId($item->manufacturerID);
if (!$manufacturer)
{
$manufacturer = new Manufacturer();
$manufacturer->setPosId($item->manufacturerID);
$this->em->persist($manufacturer);
}

You know what they say: "Early optimization is the root of all evil" D. Knuth
Check this simple optimization and if you need better times then pull up your sleeves and go down, otherwise just move on.
I added a small benchmark for testing, remember to check both variants with the empty database.
$time1 = microtime(true);
// function start
if (!$manufacturer)
{
$manufacturer = new Manufacturer();
$manufacturer->setPosId($item->manufacturerID);
$this->em->persist($manufacturer);
$this->em->flush(); // only flush when there's a new manufacturer
}
// end of function
$time2 = microtime(true);
$time = $time2 - $time1;
printr("Time elapsed: $time");

Manufacturer m = new Manufacturer();
m.ManufacturerID = 123;
Database.Load(m);
m.Name = "abc";
Database.Store(m);
'much easier than symfony. Sim. Phony. J/k.

Related

Programically complete course and set grades in Moodle

I am new to Moodle and I am working on an old application, here are the specs:
$version = 2016120502.05;
$release = '3.2.2+ (Build: 20170407)';
$branch = '32';
My ultimate goal is to be able to create a generator class to create dummy data for some PHPUnit unit tests. (https://docs.moodle.org/dev/Writing_PHPUnit_tests) I found that there is a prebuilt one for creating courses and users. But I need to be able to also mark them complete in the course with their grades.
I was looking at this https://docs.moodle.org/dev/Gradebook_API which might be at least part of what I need. However, there are lots of tables in the system, and I'm not confident that this is what I need.
Here is my code up until this last point:
// Create user;
$this->user = $this->getDataGenerator()->create_user();
// Create courses.
$courseCount = 0;
$courses = [];
while ($courseCount < 5) {
$courses[] = $this->getDataGenerator()->create_course();
$courseCount++;
}
/** #var \myGlobal_generator $generator */
$generator = $this->getDataGenerator()->get_plugin_generator('myGlobal_generator');
// Create curriculum.
$this->curriculum = $generator->createCurriculum($courses);
// Now we need to set a user to have completed each one
// of the courses and set their grades for each as well.
Notice that grading and completion tracking are not necessarily related things. For example, you can mark activities and courses as completed without any grading involved, like so:
$cmassign = get_coursemodule_from_id('assign', $cmid);
$completion = new completion_info($course);
$completion->update_state($cmassign, COMPLETION_COMPLETE, $user->id);
$ccompletion = new completion_completion(['course' => $course->id, 'userid' => $user->id]);
$ccompletion->mark_complete();
If you really need to test/generate data with grading and grade-based completion you may need to code it along this lines:
create the course with completion tracking enabled.
create the activity (add it to the course) with grade-based completion tracking (completion tracking automatic, completion use grade to true).
create the new grading item (related to the activity and a user) with the global helper grade_update.
Calculate the internal completion state of the activity (for example with the public method $completion->internal_get_state )
Test the state of the activity (completed pass or completed fail).

Simpler way to get advanced meta data in ilUIHookPluginGUI?

I am currently coding a plugin for ILIAS. The plugin itself is not at all complex but it contains several issues whereas I think we could make it simpler as it is.
The situation is following: We have a global advanced meta data field added in the user defined meta data section with a bijective identifier. The field is activated at a repository objected named course. We have manipulated the GUI with the plugin based on ilUIHookPluginGUI.
The code for this is ... well ... see it for yourself.
First of all we save the ID of the new meta data field in the settings at the ConfigGUI for the plugin:
$field_settings = new ilSetting("foo");
$field_id_value = $field_settings->set("field_id",$_POST["field_id"]);
In our class which extends ilUIHookPluginGUI we are loading the setting as following and we have the ID of the field:
$field_settings = new ilSetting("foo");
$field_id_value = $field_settings->get("field_id");
Now the fun part. With this ID and the ref_id of the object (well, we also load the object to get the ObjId) we can load the value of the meta data field setted at the course:
$object = \ilObjectFactory::getInstanceByRefId($_GET[ 'ref_id' ]);
$obj_id = $object->getId();
$result = $DIC->database()->query("SELECT value FROM adv_md_values_text WHERE obj_id = '".$obj_id."' AND field_id = '".$field_id_value."'");
$value = $DIC->database()->fetchAssoc($result);
$is_active = $value['value'];
The question is ... is there an easier way to achieve my result?
Best,
Laura
Nice question. First of all, note that I consider the advanced metadata service in ILIAS to be lacking a good readme making clear, which hooks the interface is offering for tasks such as yours. Some time ago, I had to deal with this service as well and run into similar issues. Hopefully, your question helps to document this a little better an I myself am looking forward to other suggestions, knowing that mine is not really good as well. If you have any resources, helping pushing the introduction of good readme for services and also pushing services towards using the repository pattern with a clear interface would be highly appreciated.
Concering your question of what can be improved: I see three main issues in the lines of code:
Storing an ID in the config of your plugin. Your plugin will unconfigurable for non-technical people. However, also for you this will be error prone, think about exporting-importing stuff from a test-installation to production.
Access the value by query instead of the service.
Using new and static functions inside your code making it untestable.
Step 1
Lets start with the first one. Note, that I did not manage to solve this one without introducing a new one (a new query). Bad I know. I hope that there is a better solution, I did not find one after quick research. You store the id, since the field title is not securely unique, right? This is correct, however, you could think about storing the tripplet of field_title, record_title and (maybe) scope. Note that you maybe do not need the scope since you want to use this globally. A function return you and array containing field_id and record_id could look like so:
function getFieldAndRecordIdByFieldTitles($field_title, $record_title, $scope_title){
$query = "select field.field_id,field.record_id from adv_mdf_definition as field
INNER JOIN adv_md_record as record ON record.record_id = field.record_id
INNER JOIN adv_md_record_scope as scope ON scope.record_id = field.record_id
INNER JOIN object_reference as ref ON scope.ref_id = ref.ref_id
INNER JOIN object_data as scope_data ON ref.obj_id = scope_data.obj_id
WHERE field.title='$field_title' AND record.title='$record_title' AND scope_data.title = '$scope_title'";
$set = $this->dic()->database()->query($query);
if($row = $this->dic()->database()->fetchAssoc($set))
{
return array_values($row);
}
}
Then get your values like so:
list($field_id,$record_id) = getFieldAndRecordIdByFieldTitles("my_field", "my_record", "my_scope");
Note that I am aware that I am introducing a new query here. Sorry, was the best I could come up with. I am sure there you find a better solution, if your research a bit, let us know if successful. However, we will remove one in the next step.
Step 2
Use the undocumented service, the get your value out of the advance meta data. Since you now have the record id and the field id, you can to that like so:
$record_values = new ilAdvancedMDValues($record_id, $obj_id);
$record_values->read();
$ADTGroup = $ilAdvancedMDValues->getADTGroup();
$ADT = $ilADTGroup->getElement($field_id);
$value = $ADT->getText();
/**if you have text, others are possible, such as:
switch (true) {
case ($ADT instanceof ilADTText):
break;
case ($ADT instanceof ilADTDate):
$value = $ADT->getDate();
break;
case ($ADT instanceof ilADTExternalLink):
$... = $ADT->getUrl();
$... = $ADT->getTitle();
break;
case ($ADT instanceof ilADTInternalLink):
$... = $ADT->setTargetRefId($value);
}
**/
Note that ADT's are also undocumented. There might be a better way, to get a value out of this.
Step 3
Wrap your statics and new into some injectable dependency. I usually use the bloated constructor pattern to do this. Looks like so:
public function __construct(InjectedSettings $mySettings = null)
{
if (!$mySettings) //Case in the default scenario
{
$this->mySettings = new InjectedSettings();
} else //used e.g. for unit tests, where you can stuff the constructor with a mock
{
$this->mySettings = $mySettings;
}
$this->mySettings->doSometing();
}
Note that this is not real dep. injection, still you still use new, but I think a very workable fix to use dep. injection at least for the test context in ilias.
Does this help? I hope there will be other (better answers as well).

Shortcuting Doctrine Code

Is there is any way to Shortcut this Code
$questions = $this->getdoctrine()->getrepository('AppBundle:TicketQuestions')->find($id = 1);
$question1 = $questions->getQuestions();
$option1 = $questions->getOption1();
$option2 = $questions->getOption2();
$option3 = $questions->getOption3();
Because i want to use more than 20 times.
i use this code to get some questions and options from database for a Ticket system and i hope there is a way to not write a very long code for that.
It's hard to say without seeing your Entities. For example, do you have 20 hardcoded "options" in your "Questions" entity?
If you were to use something like an array of options, you could just use a foreach loop and get all those options. But, that also has some foundation in what your data model looks like.
If you want to use an underlying datastructure (such as an array or Doctrine ArrayCollection) you'll first have to structure your database and data scheme with the correct architecture.
This is a much deeper question than can you just write a shortcut, I don't think your underlying data and architecture are implemented correctly to use data structures (array, ArrayCollections).
Thanks any way guys, i solved it like this -> i wrote new function
public function callQuestions($id){
$questions = $this->getDoctrine()->getRepository('AppBundle:TicketQuestions')->find($id);
$question = $questions->getQuestions();
$option1 = $questions->getOption1();
$option2 = $questions->getOption2();
$option3 = $questions->getOption3();
return array($question,$option1,$option2,$option3);
}
and now i can call it easily like this:
$firstQuestion = TicketsController::callQuestions(1);
i will try also to make it shorter but this one works good now.

qt multiple QSqlTableModels edited together in one transaction

I have a window in a Qt application using PostgreSQL 9.3 database. The window is a form used do display, edit and insert new data. t looks like that:
I have data from 3 sql tables in that view. the tables are related with foreign keys:
contractors (main table) - mapped to "personal data" section
contacts (has foreign key to contractors.ID)
addresses (has foreign key to contractors.ID)
So - in my window's class I have 3 main models (+ 2 proxy models to transpose tables in "personal data" an "address data" sections). I use QSqlTableModel for theese sesctions, and a QSqlRelationalTableModel for contactData section. when opening that window "normally" (to view some contractor), i simply pass contractor's ID to the constructor and store it in proper variable. Also, I call the QSqlTableModel::​setFilter(const QString & filter) method for each of the models, and set the proper filtering. When opening that window in "add new" mode i simply pass a "-1" or "0" value to the ID variable, so no data gets loaded to the model.
All 3 models have QSqlTableModel::OnManualSubmit editStrategy. When saving the data (triggered by clicking a proper button), I start a transaction. And then I submit models one-by-one. personalData model gets submitted first, as I need to obtain it's PK after insert (to set in the FK fields in other models).
When submitting of the model fails, I show a messageBox with the QSqlError content, rollback the transaction and return from the method.
When I have an error on the first model being processed - no problem, as nothing was inserted. But when the first model is saved, but the second or third fails - there is a little problem. So I rollback the transacion as before, and return from the function. But after correcting the data and submitting it again - the first model is not trying to submit - as it doesn't know that there was a rollback, and the data needs to be inserted again. What would be a good way to notice such a model, that it needs to be submited once again?
At the moment I ended up with something like that:
void kontrahenciSubWin::on_btnContractorAdd_clicked() {
//QStringList errorList; // when error occurs in one model - whole transacion gets broken, so no need for a list
QString error;
QSqlDatabase db = QSqlDatabase::database();
//backup the data - in case something fails and we have to rollback the transaction
QSqlRecord personalDataModelrec = personalDataModel->record(0); // always one row. will get erased by SubmitAll, as no filter is set, because I don't have its ID.
QList<QSqlRecord> contactDataModelRecList;
for (int i = 0 ; i< contactDataModel->rowCount(); i++) {
contactDataModelRecList.append( contactDataModel->record(i) );
}
QList<QSqlRecord> addressDataModelRecList;
for (int i = 0 ; i< addressDataModel->rowCount(); i++) {
addressDataModelRecList.append( addressDataModel->record(i) );
}
db.transaction();
if ( personalDataModel->isDirty() && error.isEmpty() ) {
if (!personalDataModel->submitAll()) //submitAll calls select() on the model, which destroys the data as the filter is invalid ("where ID = -1")
//errorList.append( personalDataModel->lastError().databaseText() );
error = personalDataModel->lastError().databaseText();
else {
kontrahentid = personalDataModel->query().lastInsertId().toInt(); //only here can I fetch ID
setFilter(ALL); //and pass it to the models
}
}
if ( contactDataModel->isDirty() && error.isEmpty() )
if (!contactDataModel->submitAll()) //slot on_contactDataModel_beforeInsert() sets FK field
//errorList.append( contactDataModel->lastError().databaseText() );
error = contactDataModel->lastError().databaseText();
if ( addressDataModel->isDirty() && error.isEmpty() )
if (!addressDataModel->submitAll()) //slot on_addressDataModel_beforeInsert() sets FK field
//errorList.append( addressDataModel->lastError().databaseText() );
error = addressDataModel->lastError().databaseText();
//if (!errorList.isEmpty()) {
// QMessageBox::critical(this, tr("Data was not saved!"), tr("The following errors occured:") + " \n" + errorList.join("\n"));
if (!error.isEmpty()) {
QMessageBox::critical(this, tr("Data was not saved!"), tr("The following errors occured:") + " \n" + error);
db.rollback();
personalDataModel->clear();
contactDataModel->clear();
addressDataModel->clear();
initModel(ALL); //re-init models: set table and so on.
//re-add data to the models - backup comes handy
personalDataModel->insertRecord(-1, personalDataModelrec);
for (QList<QSqlRecord>::iterator it = contactDataModelRecList.begin(); it != contactDataModelRecList.end(); it++) {
contactDataModel->insertRecord(-1, *it);
}
for (QList<QSqlRecord>::iterator it = addressDataModelRecList.begin(); it != addressDataModelRecList.end(); it++) {
addressDataModel->insertRecord(-1, *it);
}
return;
}
db.commit();
isInEditMode = false;
handleGUIOnEditModeChange();
}
Does anyone have a better idea? I doubt if it's possible to ommit backing-up the records before trying to insert them. But maybe there is a better way to "re-add" them to the model? I tried to use "setRecord", and "remoweRows" & "insertRecord" combo, but no luck. Resetting the whole model seems easiest (I only need to re-init it, as it loses table, filter, sorting and everything else when cleared)
I suggest you to use a function written in the language PLPGSQL. It has one transaction between BEGIN and END. If it goes wrong at a certain point of the code then will it rollback all data flawlessly.
What you are doing now is not a good design, because you handle the control over a certain functionality (rollback) to an external system with regard to the rollback (it is happening in the database). The external system is not designed to do that, while the database on the contrairy is created and designed for dealing with rollbacks and transactions. It is very good at it. Rebuilding and reinventing this functionality, which is quite complex, outside the database is asking for a lot of trouble. You will never get the same flawless rollback handling as you will have using functions within the database.
Let each system do what it can do best.
I have met your problem before and had the same line of thought to work this problem out using Hibernate in my case. Until I stepped back from my efforts and re-evaluated the situation.
There are three teams working on the rollback mechanism of a database:
1. the men and women who are writing the source code of the database itself,
2. the men and women who are writing the Hibernate code, and
3. me.
The first team is dedicated to the creation of a good rollback mechanism. If they fail, they have a bad product. They succeeded. The second team is dedicated to the creation of a good rollback mechanism. Their product is not failing when it is not working in very complex situations.
The last team, me, is not dedicated to this problem. Who am I to write a better solution then the people of team 2 or team 1 based on the work of team 2 who were not able to get it to the level of team 1?
That is when I decided to use database functions instead.

How does Entity Framework decide whether to reference an existing object or create a new one?

Just for my curiosity (and future knowledge), how does Entity Framework 5 decide when to create a new object vs. referencing an existing one? I might have just been doing something wrong, but it seems that every now and then if I do something along the lines of:
using (TestDB db = new TestDB())
{
var currParent = db.Parents.Where(p => p.Prop == passedProp).FirstOrDefault();
if(currParent == null) {
Parent newParent = new Parent();
newParent.Prop = passedProp;
currParent = newParent;
}
//maybe do something to currParent here
var currThing = db.Things.Where(t => t.Prop == passedPropTwo).FirstOrDefault();
currThing.Parent = currParent;
db.SaveChanges();
}
EF will create a new Parent in the database, basically a copy of the currParent, and then set the Parent_ID value of currThing to that copy. Then, if I do it again (as in, if there's already two of those parents), it won't make a new Parent and instead link to the first one. I don't really understand this behavior, but after playing around with it for a while something like:
using (TestDB db = new TestDB())
{
var currParent = db.Parents.Where(p => p.Prop == passedProp).FirstOrDefault();
if(currParent == null) {
Parent newParent = new Parent();
newParent.Prop = passedProp;
currParent = newParent;
}
//maybe do something to currParent here
var currThing = db.Things.Where(t => t.Prop == passedPropTwo).FirstOrDefault();
currThing.Parent = db.Parents.Where(p => p.ID == currParent.ID).First();
db.SaveChanges();
}
seemed to fix the problem. Is there any reason this might happen that I should be aware of, or was there just something weird about the way I was doing it at the time? Sorry I can't be more specific about what the exact code was, I encountered this a while ago and fixed it with the above code so I didn't see any reason to ask about it. More generally, how does EF decide whether to reference an existing item instead of creating a new one? Just based on whether the ID is set or not? Thanks!
If your specific instance of your DBContext provided that specific instance of that entity to you, then it will know what record(s) in the database it represents and any changes you make to it will be proper to that(those) record(s) in the database. If you instantiate a new entity yourself, then you need to tell the DBContext what exactly that record is if it's anything but a new record that should be inserted into your database.
In the special scenario where you have multiple DBContext instances and one instance provides you this entity but you want to use another instance to work with and save the entity, then you have to use ((IObjectContextAdapter)firstDbContext).ObjectContext.Detach() to orphan this entity and then use ((IObjectContextAdapter)secondDbContext).ObjectContext.Parents.Attach() to attach it (or ApplyChanges() if you're also editing it - this will call Attach for you).
In some other special scenarios (your object has been serialized and/or you have self-tracking entities), some additional steps may be required, depending on what exactly you are trying to do.
To summarize, if your specific instance of your DBContext is "aware" of your specific instance of an entity, then it will work with it as if it is directly tied to that specific row in the database.

Resources