I am trying to clean up content annotations (IAnnotations) of uninstalled Plone add-on. I have succefully walked through all site content and removed annotations in live objects.
However, CMFEditions / versioning keeps old content copies around and I don't know how to walk through them and remove all bad annotations they have on them.
Unless I can clean up these object references, after removing add-on from buildout one starts getting ZODB BrokenObject errors when editing content having annotations with missing objects in history.
That is nasty. Another reason addons should ONLY use primitive types for all their data storage.
I have some pseudo code that might get you close:
repo_tool = getToolByName(obj, "portal_repository")
if not repo_tool.isVersionable(obj):
return
history = repo_tool.getHistoryMetadata(obj)
if not history:
return
length = history.getLength(countPurged=False)
for i in xrange(length - 1, -1, -1):
try:
version = repo_tool.retrieve(obj, i)
annotations = IAnnotations(version.object)
del annotations[KEY_TO_DELETE]
except POSKeyError:
pass
Related
I have an enumerand of around 150 entries, which I need to get into IBM Rhapsody.
Doing this by hand is clearly lengthy and error prone. I have google extensively but found only things that tell me how to edit the generated code -- not go the other way.
The question is: How is this done? And if there is no way -- please someone post that as an answer.
David,
I would jump into the Java API (plugin subsystem) and do it that way. If you haven't learned how to use the API, there is a bit of a learning curve. There are two ways to go about it: Implement a Java (or your favorite JVM language--I use Scala) app that realizes the Rhapsody Plugin framework and then you choose to package it up and deploy it so that it gets loaded when you load your model, or, if it is a one off job, do everything up to the point of packaging it up and then run it from within your IDE and you are done. If you are comfortable with Scala, I can post some code.
So what I did in the end was I edited the relevant .sbs file, used a small python program to generate the items I required, and then update the length of the array accordingly.
all_the_literals = ["enum_name = 0x4e", enum_name2 = 0xF2", ... ,]
for field1, waste, field1_value in map(lambda x: x.split(" "),
all_the_literals):
literal_string = f""" {{ IEnumerationLiteral
- _id = GUID {uuid.uuid4()};
- _name = \"{field1}\";
- codeUpdateCGTime = 5.16.2022::19:24:18;
- _modifiedTimeWeak = 5.16.2022::19:24:18;
- _value = \"{field1_value}\";
}}"""
print(literal_string)
Note the above "code" snippet purely prints the items, which you then copy-paste into the relevant field in the sbs file. YMMV -- this was the correct format for an enum in Rhapsody (and note how I fudged the update time, but it worked successfully, so you'll need to do the same if you use this answer).
Also note it's probably better to use bauhaus9's answer, but I definitely didn't have time for it.
after upgrading 2sxc to 09.08.00 we get an error:
error CS1502: The best overloaded method match for 'ToSic.SexyContent.Razor.SexyContentWebPage.AsDynamic(ToSic.Eav.Interfaces.IEntity)' has some invalid arguments at System.Web.Compilation.BuildManager.PostProcessFoundBuildResult
So the part it talks about is:
var Content = AsDynamic(dict.Value);
What should I do about this?????
Just for context - because of new features related to using data from any kind of source (json, sql, web-api, csv, etc.) we had to change how we work with lists of items.
Previously, all items were handled in dictionaries, so the list was a Dictionary<entityId, entity>, which meant that you had to use .Value. This ID=Key would have caused a lot of problems in the future, because lists containing persons and lists containing pages could both have the same ID, basically breaking any dictionary-setup.
So in 9.8 we were forced to introduce a breaking change
See also: https://github.com/2sic/2sxc/releases/tag/v09.08.00
We didn't do this lightly, but it's an essential step towards opening up the EAV & 2sxc for upcoming features. Sorry about this!
I have a Plone 4 site which stopped to rename new Archetypes objects; after creation (as something like /temp/portaltype.2015-04-23.1234567890) and saving the first changes, including giving it a title, it should be renamed to something nicer (/temp/an-object-with-a-meaningful-name), but this doesn't happen anymore.
Perhaps the problem arose when I applied some changes to update Plone from 4.3.3 to 4.3.4 (to make one step at a time); but I have inherited a long versions.cfg which is solely sorted by package names and doesn't include any hints why certain versions were chosen ...
I'm able to go back two months and have a version which does the renaming, but without more knowledge about what to look for, it will be a very time-consuming process of re-applying every single change, rebuilding, starting and testing; but there have not been any changes to my schema definitions. I have a temp browser which is involved in delivering the primary edit form. but this doesn't seem to be the case for the saving action.
Sadly I don't fully understand yet the mechanisms of the base_edit action which should - as far as I understand - call Archetypes.BaseObject.processForm and implicitly ._renameAfterCreation, so I'd be grateful for some pointers how to debug this. Thank you!
Update:
I have a few triggers in my product's configure.zcml, e.g.:
<subscriber
for=".content.portaltype.PortalType
Products.Archetypes.interfaces.IObjectInitializedEvent"
handler=".events.onInitPortalType"/>
… with, in events.py:
def onInitPortalType(self, event):
"""
Called after first edit of new objects?
"""
print '/// onInitPortalType(%(self)r, %(event)r)' % locals()
setInitialOwner(self, event)
setStateToPrivate(self, event)
However, the event doesn't seem to be triggered, since I couldn't find the output in an instance fg session.
Update 2:
I noticed that zope.event had been pinned to a quite old version (3.5.2), so I'm trying to update to 4.3.4 more seriously now (following this how-to). This got me zope.event v4.0.3, but I have a version conflict now:
There is a version conflict.
We already have: zc.recipe.egg 1.3.2.
While:
Installing.
Getting section test.
Initializing section test.
Installing recipe zc.recipe.testrunner.
There seems to be a requirement for zc.recipe.egg < 2dev somewhere, but I can't find it.
Nothing significant changed between Plone 4.3.3 and 4.3.4 on Archetypes. Products.Archetypes changed from 1.9.7 to 1.9.8 and Products.ATContentTypes remains on the same version.
Pointers could be:
There's a _at_rename_after_creation flag, which is True by default. This can be changed on the content type class.
Is your type still activated in portal_factorytool? (Afaik this should have no impact on renaming after creation - but who knows :-))
Any Products.Archetypes.interfaces.IObjectInitializedEvent subscriber?
Issue I had once was, that the tmp id portaltype.2015-04-23.1234567890 had the wrong format and AT did no recognise it as tmp id and therefore it did not rename it after creation. The method AT uses to check if the id is autogenerated --> https://github.com/plone/Products.CMFPlone/blob/4.3.4/Products/CMFPlone/utils.py#L111 AFAIK the problem was, that the meta_type and portal_type was not the same anymore.
We had a whole collection of Plone 3 sites with a custom Image type
subclassed from ATImage. This allowed us to add an extra image scaling to the
standard list ("'logo':(454, 58)", used by our theme package).
While this still works in Plone 4, it isn't really the correct approach now that
plone.app.imaging is part of the standard toolkit. That can define custom scales on
the fly.
It looks like I can enable plone.app.imaging on any type subclassed
from ATImage by simply setting "sizes = None" for the collection of custom
scales on the type. I am however then left with a redundant subclass of ATImage.
Looking long term, it would be useful to replace all of our existing "FalconImage"
content items (hundreds in total) with standard "Image" content items.
A brief experiment on a test site reveals that if I just walk through the document
tree updating the portal_type attribute from "FalconImage" to "Image" then
the content behaves as an "Image": each object suddenly acquires a
Transform tab and all of the scales defined by ##imaging-controlpanel.
I am sure that there would be fallout from such a brute force approach.
Is there a recommended approach for transforming one type into another?
(I'm happy to add source for our custom ATImage type if anyone thinks that
it is relevant. It is really just a very minimal tweak of ATImage, with a
different collection of sizes on the ImageField)
Yes, there is a recommended approach:
http://pypi.python.org/pypi/Products.contentmigration
The only thing that you have to do is to write a custom migration from FalconImage to Image.
Bye,
Giacomo
You need to use Products.contentmigration but the docs there are no place to start. Use the docs at plone.org for a step-by-step on migrating content.
Thanks to Giacomo and Ross for the pointers.
Just in case it is useful to others, my migration code ended up looking like the following:
from Products.contentmigration.walker import CustomQueryWalker
from Products.contentmigration.archetypes import InplaceATItemMigrator
class FalconImageMigrator(InplaceATItemMigrator):
walker = CustomQueryWalker
src_meta_type = "FalconImage"
src_portal_type = "FalconImage"
dst_meta_type = "ATBlob"
dst_portal_type = "Image"
# Following stolen from plone.app.blob.migrations, ATImageToBlobImageMigrator
# migrate all fields except 'image', which needs special handling...
fields_map = {
'image': None,
}
def migrate_data(self):
self.new.getField('image').getMutator(self.new)(self.old)
# ATFileToBlobMigrator reindexes certain fields. Otherwise we
# need to clear and rebuild the entire catalog.
def last_migrate_reindex(self):
self.new.reindexObject(idxs=['object_provides', 'portal_type',
'Type', 'UID'])
migrator = FalconImageMigrator
walker = migrator.walker(portal, FalconImageMigrator)
walker.go()
print walker.getOutput()
Complications:
Image is a little odd as a destination type, as data gets migrated into the blob store.
We need to update the catalog so that "resolveuid/UID" links generated by TinyMCE
continue to work. last_migrate_reindex() method on Migrator class should be faster than clearing and rebuilding the entire catalog from scratch.
We're running a Zope server with an eventually large-ish number of Plone (4) sites. Every now and then, an extension product update comes along and requires a re-install to pick up changes in the profile settings, e.g. new content types.
Manually, this would mean clicking through to every Plone site's portal_quickinstaller, tick the products, press update. This is not very feasible if we're talking about dozens of sites, so I'm trying to automate this. Essentially so far, I have the following living as a Script(Python) in the Zope root:
a = context.restrictedTraverse('/')
p = a['Plone']
print p.getSiteManager()
qi = p.restrictedTraverse('portal_quickinstaller')
print qi
qi.reinstallProducts('LinguaPlone')
(Simplified; in reality I have a longer list instead of the single Plone instance, and I might want to reinstall a longer list of products.)
This fails with the following:
Module Products.CMFQuickInstallerTool.QuickInstallerTool, line 613, in uninstallProducts
Module Products.CMFQuickInstallerTool.InstalledProduct, line 272, in uninstall
Module Products.CMFQuickInstallerTool.InstalledProduct, line 351, in _cascadeRemove
AttributeError: 'BaseGlobalComponents' object has no attribute 'objectItems'
From my debugging attempts so far, the BaseGlobalComponents is the Zope SiteManager returned by the zope.component.getSiteManager. How do I convince quickinstaller to pick up the right one, i.e. the one from the Plone Site it's living in?
Alternatively, how would I go about automating re-installing products in a way that will remain vaguely feasible for larger installations? (ETA: I'm aware this is not the kind of thing you do automatically with a cronjob, but updates of inhouse-developed products can't be avoided, I'm afraid.)
Here's how to change the active local site manager. You won't be able to do this in Restricted Python, so you'll need to turn your Python script into an External Method or browser view.
from zope.app.component.hooks import setHooks, setSite
setHooks()
setSite(site)
The setHooks call only needs to be done once. In Zope 2.12 these calls should be imported instead from zope.site.hooks and in Zope 2.13 from zope.component.hooks.
Keep in mind that calling reinstallProducts is not appropriate for all add-on products, and not recommended unless you've carefully checked what reinstalling does and are sure it won't cause problems. Some products provide upgrade steps that run actions more selectively.
Disclaimer: are you sure you want to do this? Automatically reinstalling and upgrading products to the latest version, blindly and without any testing on a staging instance, is asking for trouble.
Anyway, you can do such a thing using XML-RPC and a little tweaking. This is how you install a product on a live running instance using XML-RPC:
>>> import xmlrpclib
>>> proxy = xmlrpclib.ServerProxy(
"http://admin:passwd#localhost:8080/Plone/portal_quickinstaller"
)
>>> proxy.getProductVersion('Marshall')
'2.0'
>>> proxy.isProductInstalled('Marshall')
'False'
>>> proxy.installProduct('Marshall')
'Registry installed sucessfully.\n'
>>> proxy.isProductInstalled('Marshall')
'True'
To reinstall you need subclass Products.CMFQuickInstallerTool.QuickInstallerTool.py and provide you custom QuickInstallerTool with a method that has the keyword argument "reinstall" set as 'True' by default; something like:
442c442
< swallowExceptions=None, reinstall=False,
---
> swallowExceptions=None, reinstall=True,
452,457c452,457
< if self.isProductInstalled(p):
< prod = self._getOb(p)
< msg = ('This product is already installed, '
< 'please uninstall before reinstalling it.')
< prod.log(msg)
< return msg
---
> #if self.isProductInstalled(p):
> # prod = self._getOb(p)
> # msg = ('This product is already installed, '
> # 'please uninstall before reinstalling it.')
> # prod.log(msg)
> # return msg
Even better: provide your own method for gathering information about versions and reinstalling a product, compatible with the XML-RPC protocol (as you cannot pass keyword arguments).
There might be cleaner ways of doing this via XML-RPC, but portal_quickinstaller is not meant to be used in this way and there may be caveats. Use with caution.
I have got this python script in the Zope root of an instance with 7 Plone Sites. Looks pretty much the same as what you have. It might be that it only works on this Plone 2.5 site (yes, old), but I think it should work on 3.x and 4.x as well. Maybe an an innocent looking difference (that I am overlooking) causes the error in your script; maybe the restrictedTraverses that you do trip it up. (Script edited for clarity.)
SITES = ['site-1', 'site-2']
for site in SITES:
print "Reinstalling LinguaPlone in %s." % site
portal = context[site]
qi = portal.portal_quickinstaller
qi.reinstallProducts(['LinguaPlone'])
First don't do a reinstall it can break your website in many cases.
Next you have to consider that add-ons may provide an upgrade step (usually they will). Use the quickinstaller api to achieve this in PythonScript. It is good but it can also be achieved with a script on the file system. Check the examples here: http://svn.plone.org/svn/plone/plone.org/Products.PloneOrg/trunk/scripts/
Another solution can be to use the Selenium IDE to record the quickinstaller stuff in one site and make a copy paste the results of that tests to run it on another website (very weird isn't it ?)