Drupal - Importing a taxonomy with migrate module from a table and creating/updating existing terms - drupal

I need to import a list of terms into my taxonomy from a source I loaded in the database.
The problem is I allready have this taxonomy on my site (loaded wihtout migrate) with terms that are used by reference in other content, so I got to keep existing term and update them or create the new ones.
To link my taxonomy source and the existing taxonomy I have an unique code for each term, so I added a code field to my vocabulary and filled it for each existing term.
I am currently able to create and update terms with my current Migration class, but if the name of my term on the site and the name of the term in my source is different, the import will create a new term instead of updating its name even if the code is the same.
Here my Migration Class :
class TotoMigration extends Migration {
private $list_term = array();
public function __construct($arguments) {
parent::__construct();
$this->softDependencies = array('TotoParent');
// get data from the custom table containing the new terms to create or update
$query = db_select('toto', 'f')
->fields('f', array(
'CODE', // code
'LIBLONG', // name
'PARENT', // parent
)
);
$this->source = new MigrateSourceSQL($query);
$this->destination = new MigrateDestinationTerm('toto_tax');
$this->map = new MigrateSQLMap($this->machineName,
array(
'CODE' => array('type' => 'varchar',
'length' => 5,
'not null' => TRUE,
'description' => 'Code',
)
),
MigrateDestinationTerm::getKeySchema()
);
$this->addFieldMapping('name', 'LIBLONG');
$this->addFieldMapping('field_code', 'CODE');
$this->addFieldMapping('parent', 'PARENT')
->arguments(array('source_type' => 'tid'))
->sourceMigration('TotoParent');
// create a list of existing toto terms with code => tid
$list_term = db_query("select fc.field_code_value, ttd.tid
from taxonomy_term_data ttd
left join taxonomy_term_hierarchy tth on tth.tid=ttd.tid
left join field_data_field_code fc on fc.entity_id = ttd.tid
where ttd.vid=10
and tth.parent!=0;")->fetchAllKeyed();
}
public function prepareRow($row) {
// Always include this fragment at the beginning of every prepareRow()
// implementation, so parent classes can ignore rows.
if (parent::prepareRow($row) === FALSE) {
return FALSE;
}
// if the destination is not mapped in migrate we tell him where to go
if (!isset($row->migrate_map_destid1) && isset($list_term[$row->CODE])) {
$row->migrate_map_destid1 = $list_term[$row->CODE];
}
}
}
I then load the import with drush (and --update option).
I must be missing something, if anyone got a clue it will be welcome.

After many tries, the problem reside in the fact the module Migrate does not support Creating content and Updating content in the same migration class (I even read it will sometime claim to update content and just do nothing).
So the solution is pretty simple, create 2 classes :
One for Creating content
One for Updating content
Your Creating class will be the same.
Your Updating class will need to have a systemeOfRecord set to DESTINATION :
$this->systemOfRecord = Migration::DESTINATION;
So it knows to only update and not recreate the content, it will keep current fields not mapped and update fields mapped that are not part of the MigrateSQLMap :
$this->map = new MigrateSQLMap($this->machineName,array(...));
The tricky part will be to find corresponding nid/tid of your content so you can map it to your imported data and then to separate data used to update or create content.

Related

CMIS add custom type to cm:folder

I've one repository where I create some Site for store all documents and folders that my organization create.
I've one web app that include this php library https://github.com/dkd/php-cmis-client (port of Apache Chemistry Java implementation).
So I can creates documents, folder, set some properties, etc... but I would like to do "more".
I would like to extend the model and create my own model. In this case, It's very simple, I create "myPersonal" model, and create one custom type 'folderAmp' (his parents is cm:folder). And I extend this custom type and create a new property "myP:idNew" where I want to store some id code.
So, When I create a folder with this library (or maybe in java) I create it with...
JAVA
Folder parent = ....
Map<String, Object> properties = new HashMap<String, Object>();
properties.put(PropertyIds.NAME, "a new folder");
properties.put(PropertyIds.OBJECT_TYPE_ID, "cmis:folder");
// create the folder
Folder newFolder = parent.createFolder(properties);
PHP
$properties = array(
\Dkd\PhpCmis\PropertyIds::OBJECT_TYPE_ID => 'cmis:folder',
\Dkd\PhpCmis\PropertyIds::NAME => 'Demo Folder'
);
try {
$folder = $session->createFolder(
$properties,
$session->createObjectId($session->getRepositoryInfo()->getRootFolderId())
);
So if I want to add my new custom type... What I've to do?
I try to add with this:
properties.put(PropertyIds.OBJECT_TYPE_ID, "cmis:folder, myP:folderAmp" but it doesn't works.
You are looking for createType(). Not all repositories support it.
Get the CMIS Workbench from https://chemistry.apache.org/java/download.html , login into your repository, and open the "Types window". Select the folder type (cmis:folder). If the "Create Type" button on the top is enabled you can create a new subtype of cmis:folder. If not, either the repository doesn't support it or you don't have the permissions to do this.
But...if create type button is not enable... this repository is not "CMIS standar" ¿no?
Finally, the problem I solved it...very simple... I change
$properties = array(
\Dkd\PhpCmis\PropertyIds::OBJECT_TYPE_ID => 'cmis:folder',
\Dkd\PhpCmis\PropertyIds::NAME => 'Demo Folder'
);
try {
$folder = $session->createFolder(
$properties,
$session->createObjectId($session->getRepositoryInfo()->getRootFolderId())
);
By this...
$properties = array(
\Dkd\PhpCmis\PropertyIds::OBJECT_TYPE_ID => 'F:oto:Historia',
\Dkd\PhpCmis\PropertyIds::NAME => 'Demo Folder'
);
try {
$folder = $session->createFolder(
$properties,
$session->createObjectId($session->getRepositoryInfo()->getRootFolderId())
);
Where oto it's a model and Historia it's a custom type that I create with Alfresco Console Admin. I included all parent (cm:folder) properties in oto:Historia and create same custom properties.
When I create the folder, it's create with oto:Historia type.

Is it possible to have versioned many_many relations?

I already used versioning on DataObjects when they contain a lot of content, now I'm wondering if it's possible to apply versioning to a many_many relation?
Assuming I have the following:
class Page extends SiteTree
{
private static $many_many = array(
'Images' => 'Image'
);
}
Then the ORM will create a Page_Images table for me to store the relations. In order to have a versioned relation, more tables would be required (eg. Page_Images_Live).
Is there any way to tell the ORM to create versioned relations? When looking at the above example with a Page * – * Images relation, I don't want the Image class to be versioned, but rather the relation. Eg. something like this:
Version Stage:
---
PageA
Images ( ImageA, ImageB, ImageC )
Version Live:
---
PageA
Images ( ImageA, ImageC, ImageD, ImageE )
Is that even possible out of the box?
I've spent a lot of time looking into this and without fundamentally modifying ManyManyList (as it doesn't expose the necessary hooks through the extension system), there isn't many choices.
I am a dessert-first kind of person, how CAN we do it?
My only suggestion to accomplish this feat is essentially a many-to-many bridge object (ie. a separate entity joining Page and Image) via $has_many though it still requires quite a bit of modification.
This is partially discussed on the forum where a solution about subverting the actual relationship by storing the versioned items against the actual object rather than in a joining table. That would work but I think we can still do better than that.
I am personally leaning towards tying the version of the relationship to the Page itself and my partial solution below covers this. Read below the fold for more info trying this as an update to ManyManyList.
Something like this is a start:
class PageImageVersion extends DataObject
{
private static $db = array(
'Version' => 'Int'
);
private static $has_one = array(
'Page' => 'Page',
'Image' => 'Image'
);
}
This contains our 2-way relationship plus we have our version number stored. You will want to specify the getCMSFields function to add the right fields required allowing you to relate it to an existing image or upload a new one. I am avoiding covering this as it should be relatively straight forward compared to the actual version handling part.
Now, we have a has_many on Page like so:
private static $has_many = array(
'Images' => 'PageImageVersion'
);
In my tests, I also added an extension for Image adding the matching $has_many onto it as well like so:
class ImageExtension extends DataExtension
{
private static $has_many = array(
'Pages' => 'PageImageVersion'
);
}
Honestly, not sure if this is necessary beyond adding the Pages
function on the Image side of the relationship. As far as I can see, it won't really matter for this particular usecase.
Unfortunately, because of this way of versioning, we can't use the standard way of calling the Images, we will need to be a bit creative. Something like this:
public function getVersionedImages($Version = null)
{
if ($Version == null)
{
$Version = $this->Version;
}
else if ($Version < 0)
{
$Version = max($this->Version - $Version, 1);
}
return $this->Images()->filter(array('Version' => $Version));
}
When you call getVersionedImages(), it will return all images that have the Version set on it aligning with the version of the current page. Also supports getting previous versions via getVersionedImages(-1) for the last version or even gets images for a specific version of the page by passing any position number.
OK, so far so good. We now need to make sure that every page write we are getting a duplicate list of images for this new version of the page.
With an onAfterWrite function on Page, we can do this:
public function onAfterWrite()
{
$lastVersionImages = $this->getVersionedImages(-1);
foreach ($lastVersionImages as $image)
{
$duplicate = $image->duplicate(false);
$duplicate->Version = $this->Version;
$duplicate->write();
}
}
For those playing at home, this is where things get a bit iffy relating to how restoring previous versions of Page would affect this.
Because we would be editing this in GridField, we will need to do a few things. First is make sure our code can handle the Add New function.
My idea is an onAfterWrite on the PageImageVersion object:
public function onAfterWrite()
{
//Make sure the version is actually saved
if ($this->Version == 0)
{
$this->Version = $this->Page()->Version;
$this->write();
}
}
To get your versioned items displaying in GridField, you would have it set up similar to this:
$gridFieldConfig = GridFieldConfig_RecordEditor::create();
$gridField = new GridField("Images", "Images", $this->getVersionedImages(), $gridFieldConfig);
$fields->addFieldToTab("Root.Images", $gridField);
You might want to link to images directly from the GridField via GridFieldConfig_RelationEditor however this is when things get sour.
Time for the veggies...
One of the big difficulties is GridField, for both linking and unlinking these entities. Using the standard GridFieldDeleteAction will directly update the relationship without the right version.
You will need to extend GridFieldDeleteAction and override the handleAction to write your Page object (to trigger another version), duplicate every version of our versioned image object for the last version while making it skip the one you don't want in the new version.
I'll admit, this last bit is just guesswork by me. From my understanding and debugging, it should work but simply there is a lot of fiddling to get it right.
Your extension of GridFieldDeleteAction then needs to be added to your specific GridField.
This would essentially be your last step away from making this solution work. Once you have the adding, removing, duplicating, version updating part down, it really is a matter of just using getVersionedImages() to get the right images.
Conclusion
Avoid. I get why you want to do this but I really don't see a clean way of being able to handle this without a decent sized update to how many_many relationships are handled in Silverstripe.
But I really want it as a ManyManyList!
The changes I see required for ManyManyList are having a 3-way key (Foreign Key, Local Key, Version Key) and the various methods for adding/removing/fetching etc updated.
If there were hooks in the add and remove functions, you might be able to sneak in the functionality as an extension (via Silverstripe's extension system) and add the needed data to the extra fields that many_many relationships allow.
While I could get this happening by extending ManyManyList directly and then forcing ManyManyList to be replaced with my custom class via Object::useCustomClass, it would be even more of a messy solution.
It is simply too long/complex for me to give a full answer for a pure ManyManyList solution at this stage (though I may get back to this later and give it a shot).
Disclaimer: I am not a Silverstripe Core dev, there may be a neater solution to this entire thing but I simply can't see how.
You can define second relation with "_Live" suffix and update it when the page is published. Note: This solution stores only two versions (live and stage).
Bellow is my implementation which automatically detects whether many-many relation is versioned or not. It then handles publishing and data retrieval. All what is needed is to define one extra many-many relation with "_Live" suffix.
$page->Images() returns items according to the current stage (stage/live).
class Page extends SiteTree
{
private static $many_many = array(
'Images' => 'Image',
'Images_Live' => 'Image'
);
public function publish($fromStage, $toStage, $createNewVersion = false)
{
if ($toStage == 'Live')
{
$this->publishManyToManyComponents();
}
parent::publish($fromStage, $toStage, $createNewVersion);
}
protected function publishManyToManyComponents()
{
foreach (static::getVersionedManyManyComponentNames() as $component_name)
{
$this->publishManyToManyComponent($component_name);
}
}
protected function publishManyToManyComponent($component_name)
{
$stage = $this->getManyManyComponents($component_name);
$live = $this->getManyManyComponents("{$component_name}_Live");
$live_table = $live->getJoinTable();
$live_fk = $live->getForeignKey();
$live_lk = $live->getLocalKey();
$stage_table = $stage->getJoinTable();
$stage_fk = $live->getForeignKey();
$stage_lk = $live->getLocalKey();
// update or add items from stage to live
foreach ($stage as $item)
{
$live->add($item, $stage->getExtraData(null, $item->ID));
}
// delete remaining items from live table
DB::query("DELETE l FROM $live_table AS l LEFT JOIN $stage_table AS s ON l.$live_fk = s.$stage_fk AND l.$live_lk = s.$stage_lk WHERE s.ID IS NULL");
// update new items IDs in live table (IDs are incremental so the new records can only have higher IDs than items in ID => should not cause duplicate IDs)
DB::query("UPDATE $live_table AS l INNER JOIN $stage_table AS s ON l.$live_fk = s.$stage_fk AND l.$live_lk = s.$stage_lk SET l.ID = s.ID WHERE l.ID != s.ID;");
}
public function manyManyComponent($component_name)
{
if (Versioned::current_stage() == 'Live' && static::isVersionedManyManyComponent($component_name))
{
return parent::manyManyComponent("{$component_name}_Live");
}
else
{
return parent::manyManyComponent($component_name);
}
}
protected static function isVersionedManyManyComponent($component_name)
{
$many_many_components = (array) Config::inst()->get(static::class, 'many_many', Config::INHERITED);
return isset($many_many_components[$component_name]) && isset($many_many_components["{$component_name}_Live"]);
}
protected static function getVersionedManyManyComponentNames()
{
$many_many_components = (array) Config::inst()->get(static::class, 'many_many', Config::INHERITED);
foreach ($many_many_components as $component_name => $dummy)
{
$is_live = 0;
$stage_component_name = preg_replace('/_Live$/', '', $component_name, -1, $is_live);
if ($is_live > 0 && isset($many_many_components[$stage_component_name]))
{
yield $stage_component_name;
}
}
}
}

Modifying a field collection programmatically missing hostEntity fields

I am trying to modify a field collection in a node that already exists so I can change an image on the first element in an array of 3. The problem is, the hostEntity info is not set when I do a entity_load or entity_load_single so when I do a:
$field_collection_item->save(true); // with or without the true
// OR
$fc_wrapper->save(true); // with or without the true
I get the following error:
Exception: Unable to save a field collection item without a valid reference to a host entity. in FieldCollectionItemEntity->save()
When i print_r the field collection entity the hostEntity:protected fields are indeed empty. My field collection is setup as follows:
field_home_experts
Expert Image <--- Want to change this data only and keep the rest below
field_expert_image
Image
Expert Name
field_expert_name
Text
Expert Title
field_expert_title
Text
Here is the code I am trying to use to modify the existing nodes field collection:
$node = getNode(1352); // Get the node I want to modify
// There can be up to 3 experts, and I want to modify the image of the first expert
$updateItem = $node->field_home_experts[LANGUAGE_NONE][0];
if ($updateItem) { // Updating
// Grab the field collection that currently exists in the 0 spot
$fc_item = reset(entity_load('field_collection_item', array($updateItem)));
// Wrap the field collection entity in the field API wrapper
$fc_wrapper = entity_metadata_wrapper('field_collection_item', $fc_item);
// Set the new image in place of the current
$fc_wrapper->field_expert_image->set((array)file_load(4316));
// Save the field collection
$fc_wrapper->save(true);
// Save the node with the new field collection (not sure this is needed)
node_save($node);
}
Any help would be greatly appreciated, I am still quite new to Drupal as a whole (end-user or developer)
Alright so I think I have figured this out, I wrote up a function that will set a field collection values:
// $node: (obj) node object returned from node_load()
// $collection: (string) can be found in drupal admin interface:
// structure > field collections > field name
// $fields: (array) see usage below
// $index: (int) the index to the element you wish to edit
function updateFieldCollection($node, $collection, $fields = Array(), $index = 0) {
if ($node && $collection && !empty($fields)) {
// Get the field collection ID
$eid = $node->{$collection}[LANGUAGE_NONE][$index]['value'];
// Load the field collection with the ID from above
$entity = entity_load_single('field_collection_item', array($eid));
// Wrap the loaded field collection which makes setting/getting much easier
$node_wrapper = entity_metadata_wrapper('field_collection_item', $entity);
// Loop through our fields and set the values
foreach ($fields as $field => $data) {
$node_wrapper->{$field}->set($data);
}
// Once we have added all the values we wish to change then we need to
// save. This will modify the node and does not require node_save() so
// at this point be sure it is all correct as this will save directly
// to a published node
$node_wrapper->save(true);
}
}
USAGE:
// id of the node you wish to modify
$node = node_load(123);
// Call our function with the node to modify, the field collection machine name
// and an array setup as collection_field_name => value_you_want_to_set
// collection_field_name can be found in the admin interface:
// structure > field collections > manage fields
updateFieldCollection(
$node,
'field_home_experts',
array (
'field_expert_image' => (array)file_load(582), // Loads up an existing image
'field_expert_name' => 'Some Guy',
'field_expert_title' => 'Some Title',
)
);
Hope this helps someone else as I spent a whole day trying to get this to work (hopefully I won't be a noob forever in Drupal7). There may be an issue getting formatted text to set() properly but I am not sure what that is at this time, so just keep that in mind (if you have a field that has a format of filtered_html for example, not sure that will set correctly without doing something else).
Good luck!
Jake
I was still getting the error, mentioned in the question, after using the above function.
This is what worked for me:
function updateFieldCollection($node, $collection, $fields = Array(), $index = 0) {
$eid = $node->{$collection}[LANGUAGE_NONE][$index]['value'];
$fc_item = entity_load('field_collection_item', array($eid));
foreach ($fields as $field => $data) {
$fc_item[$eid]->{$field}[LANGUAGE_NONE][0]['value'] = $data;
}
$fc_item[$eid]->save(TRUE);
}
I hope this helps someone as it took me quite some time to get this working.

Silverstripe Gridfield Relation

I am attempting to update the secure files silverstripe module to SS3.
In it, the author uses the following ComplexTableField:
class SecureFileTokenPermissionDecorator extends DataExtension {
static $has_many = array(
'AccessTokens' => 'SecureFileAccessToken'
);
....
$tokenList = new ComplexTableField(
$this->owner,
'ContainedFileTokens',
'SecureFileAccessToken',
null,
null,
"File.ParentID = '{$this->owner->ID}'",
$sourceSort = null,
"JOIN File ON FileID = File.ID"
));
$tokenList->setParentIdName('FolderID');
$tokenList->setRelationAutoSetting(false);
....
}
I was wondering how I would go about representing the same data/relation with gridfField.
Thanks!
Looking at the secure files module source code, I would suggest the following setup:
$tokenList = $gridField = new GridField(
'AccessTokens',
'Tokens',
$this->owner->AccessTokens(),
GridFieldConfig_RelationEditor::create()
);
This works directly on the relation getter, which is a lazy loaded list (not queried until necessary), and automatically paginated. I'm not quite sure how the setParentIdName("FolderID") fits in here, probably unnecessary. Caution: Haven't tried this on the actual codebase.
If you need some help understanding the API on a higher level, have a look in the GridField docs and the "datamodel" topic.
Thanks for helping to make modules ready for SS3! :)

Creating new event in a module

I'm trying to execute some code right before content is deleted. The Rules module has events for
After updating existing content
Before saving content
After saving new content
After deleting content
However, none of these execute my code at the right time.
I discovered a simple module called Predelete, which provides hooks for executing code before deletion. This seemed like an excellent candidate to call the Rules event from.
So, I created a very simple module based on the "predelete_field" example module contained within Predelete. The folder is called "predelete_field", is in the "modules" folder, and contains the following files:
1: predelete_field.info
core = "7.x"
dependencies[] = "rules"
dependencies[] = "list"
dependencies[] = "predelete"
description = "Example for the predelete module with a content type and a node"
name = "Predelete Field"
package = Other
project = "predelete_field"
version = "7.x-1.0"
; Information added by drupal.org packaging script on 2011-07-11
version = "7.x-1.1"
core = "7.x"
project = "predelete"
datestamp = "1310360219"
2: predelete_field.module
<?php
/**
* Implements hook_predelete_node().
*/
function predelete_field_predelete_node($node) {
drupal_set_message( "PREDELETE HOOK CALLED", 'warning' );
rules_invoke_event('predelete_field', $node);
$deletable = TRUE;
$reason = t('Deletable by default.');
return array('result' => $deletable, 'reason' => $reason);
}
3: predelete_field.rules.inc
<?php
/**
* Implements hook_rules_event_info() on behalf of the predelete_field module.
*/
function rules_predelete_field_event_info() {
$items = array(
'predelete_field_predelete' => array(
'label' => t('Before deleting content'),
'group' => t('Node'),
),
);
return $items;
}
Unfortunately, this does not appear to work: The event does not show up in the event list, even after clearing cache and disabling and re-enabling the module. Likewise, the drupal_set_message function does not appear to fire.
Is anyone able to spot any errors I may have made, or provide a solution?
The Predelete module just seems to hook into the confirmation form for multiple node deletions, it won't ever be fired using Rules as Rules doesn't invoke the form but uses the node API instead.
If you look at the node_delete_multiple() function though you'll see that several hooks are called before any content is actually deleted from the database (namely hook_node_delete and hook_entity_delete). One of these is the hook you'll want to implement in your custom module, like so:
function mymodule_node_delete($node) {
// Perform some action based on values in $the node object.
// Nothing has been deleted from the database at this point.
}
You'll need to clear Drupal's caches again when you've added that hook but according to the documentation it should work.

Resources