Plone 3.3.5: We have a middle sized Plone site and we'd like to update its workflows. Since it's a long-running process we noticed something strange going on. Our Archetypes accessors, not security related, where called when hitting "Update security settings" in portal_workflow.
Looks like the culprit is update_metadata=1 default setting in ZCatalog:
-> self.plone_log("treatmentToImagingHours: %s"%str(treatmentToImagingHours))
(Pdb) bt
/Users/moo/sits/parts/zope2/lib/python/ZServer/PubCore/ZServerPublisher.py(25)__init__()
-> response=b)
/Users/moo/sits/parts/zope2/lib/python/ZPublisher/Publish.py(401)publish_module()
-> environ, debug, request, response)
/Users/moo/sits/parts/zope2/lib/python/ZPublisher/Publish.py(202)publish_module_standard()
-> response = publish(request, module_name, after_list, debug=debug)
/Users/moo/sits/parts/zope2/lib/python/ZPublisher/Publish.py(119)publish()
-> request, bind=1)
/Users/moo/sits/parts/zope2/lib/python/ZPublisher/mapply.py(88)mapply()
-> if debug is not None: return debug(object,args,context)
/Users/moo/sits/parts/zope2/lib/python/ZPublisher/Publish.py(42)call_object()
-> result=apply(object,args) # Type s<cr> to step into published object.
<string>(4)_facade()
/Users/moo/sits/parts/zope2/lib/python/AccessControl/requestmethod.py(64)_curried()
-> return callable(*args, **kw)
/Users/moo/sits/eggs/Products.CMFCore-2.1.2-py2.4.egg/Products/CMFCore/WorkflowTool.py(457)updateRoleMappings()
-> count = self._recursiveUpdateRoleMappings(portal, wfs)
/Users/moo/sits/eggs/Products.CMFCore-2.1.2-py2.4.egg/Products/CMFCore/WorkflowTool.py(609)_recursiveUpdateRoleMappings()
-> count = count + self._recursiveUpdateRoleMappings(v, wfs)
/Users/moo/sits/eggs/Products.CMFCore-2.1.2-py2.4.egg/Products/CMFCore/WorkflowTool.py(609)_recursiveUpdateRoleMappings()
-> count = count + self._recursiveUpdateRoleMappings(v, wfs)
/Users/moo/sits/eggs/Products.CMFCore-2.1.2-py2.4.egg/Products/CMFCore/WorkflowTool.py(609)_recursiveUpdateRoleMappings()
-> count = count + self._recursiveUpdateRoleMappings(v, wfs)
/Users/moo/sits/eggs/Products.CMFCore-2.1.2-py2.4.egg/Products/CMFCore/WorkflowTool.py(609)_recursiveUpdateRoleMappings()
-> count = count + self._recursiveUpdateRoleMappings(v, wfs)
/Users/moo/sits/eggs/Products.CMFCore-2.1.2-py2.4.egg/Products/CMFCore/WorkflowTool.py(609)_recursiveUpdateRoleMappings()
-> count = count + self._recursiveUpdateRoleMappings(v, wfs)
/Users/moo/sits/eggs/Products.CMFCore-2.1.2-py2.4.egg/Products/CMFCore/WorkflowTool.py(600)_recursiveUpdateRoleMappings()
-> ob.reindexObject(idxs=['allowedRolesAndUsers'])
/Users/moo/sits/eggs/Products.Archetypes-1.5.11-py2.4.egg/Products/Archetypes/CatalogMultiplex.py(115)reindexObject()
-> c.catalog_object(self, url, idxs=lst)
/Users/moo/sits/eggs/Plone-3.3rc2-py2.4.egg/Products/CMFPlone/CatalogTool.py(417)catalog_object()
-> update_metadata, pghandler=pghandler)
/Users/moo/sits/parts/zope2/lib/python/Products/ZCatalog/ZCatalog.py(536)catalog_object()
-> update_metadata=update_metadata)
/Users/moo/sits/parts/zope2/lib/python/Products/ZCatalog/Catalog.py(348)catalogObject()
-> self.updateMetadata(object, uid)
/Users/moo/sits/parts/zope2/lib/python/Products/ZCatalog/Catalog.py(277)updateMetadata()
-> newDataRecord = self.recordify(object)
/Users/moo/sits/parts/zope2/lib/python/Products/ZCatalog/Catalog.py(417)recordify()
-> if(attr is not MV and safe_callable(attr)): attr=attr()
/Users/moo/sits/products/SitsPatient/content/SitsPatient.py(2452)outSichECASS()
portal_workflow calls ob.reindexObject(idxs=['allowedRolesAndUsers']). However, this triggers refresh to all metadata.
1) Is this normal behavior?
2) Is this desired behavior?
3) Can I turn update_metadata off to speed up the process without breaking anything? Does portal security rely on metadata in any point?
Yes, this is normal behaviour. The catalog stores a subset of information that an object provides as a cache, so you can render pages with just catalog results without having to wake up the original objects. This includes the current workflow state for an object.
When reindexing, the catalog must update the metadata too, as it has no means of determining if that data has changed or not.
In this particular process, you cannot turn update_metadata off without patching; you'd have to either:
patch Products.ZCatalog.Catalog.catalogObject to switch off update_metadata there,
patch Products.Archetypes.CatalogMultiplex.CatalogMultiplex.reindexObject to call catalogObject with the update_metadata flag set to False,
patch the workflow tool to call reindexObjectSecurity instead of reindexObject.
You'd have to audit your catalog schema (metadata) columns to see if nothing will indeed change when you update workflow security.
Related
In order to test my flask application, I setup a local SQLite DB with create_engine. To delete, I have my route call TablesEtl().delete_record_by_id().
class TablesEtl:
def __init__(self) -> None:
self.engine = create_engine(get_database_engine_uri(), connect_args={"check_same_thread": False})
def get_session(self) -> Session:
session = Session(self.engine, autoflush=False)
return session
def delete_record_by_id(self, record_id: int, table: Base) -> None:
session = self.get_session()
session.query(table).filter(table.id == record_id).delete()
session.commit()
session.close()
def get_record_by_id(self, record_id: int, table: Base) -> dict:
session = self.get_session()
query: Base = Query(table, session=session).filter(table.id == record_id).first()
session.close()
if query is None:
return {}
info = {}
counting = 0
for field in query.__table__.columns.keys():
info[field.replace("_", "-")] = getattr(query, field)
counting += 1
return info
The test_delete is done by creating a record, deleting it through this route and then requesting to the "information" route, information about the record I've just created. It should return a 404, since the record doesn't exist.
The problem is that it returns 200 and the correct information about the record I've just created.
This didn't occured when I ran the tests with pytest alone, but now I'm running with pytest-xdist and I think the second request is coming before the record was actually deleted. I also noticed that a db-name.db-journal was created side by side with my original db-name.db.
def test_not_finding_model_info_after_delete(self) -> None:
model_id = create_model_on_client(self.client, self.authorization_header, "Model Name")
self.client.delete(API_DELETE_MODEL, headers=self.authorization_header, json={"model-id": model_id})
response = get_model(self.client, self.authorization_header, model_id)
self.assertEqual(404, response.status_code)
When I run it with some time.sleep it works, but I don't want to depend upon a sleep for it to work and when I run pytest a second time with --lf option it works, but I believe it's because there are less calls being made to the db. When I get all the records that were asked to be deleted and check against the db, they are in fact deleted. I believe it's something to do with this .db-journal.
Can someone shade a light to it? Another DB would be better for this problem?
Just for the knowledge I just want to see how WRITE triggers execute for the query below. Is it possible to see them?.
FOR EACH Customer EXCLUSIVE-LOCK WHERE NAME = "Go Fishing Ltd":
ASSIGN Customer.Balance = 600.
END.
Add:
-clientlog path/to/log.log -logginglevel 4 -logentrytypes 4GLTrace
to your startup command.
This will create a log of all of the calls that your code makes.
For more information: https://knowledgebase.progress.com/articles/Knowledge/P9893
You can also use the LOG-MANAGER system handle within your code to dynamically control the logging at runtime:
https://docs.progress.com/bundle/openedge-abl-troubleshoot-applications/page/LOG-MANAGER-system-handle-attributes-and-methods.html
but for simple purposes like this it is easier to just add the startup parameters.
Watch the log-manager show the write trigger being executed on the sports2020 database:
def var clog as char no-undo.
def var lclog as longchar no-undo.
assign
clog = guid + '.log'.
log-manager:logfile-name = clog
log-manager:log-entry-types = '4gltrace:5,4glmessages'
.
for each Customer exclusive-lock where name = 'Go Fishing Ltd':
Customer.Balance = Customer.Balance + 1. // write trigger only fires when record changes
end.
log-manager:close-log().
copy-lob from file clog to lclog.
message string( lclog ).
https://abldojo.services.progress.com/?shareId=62978a833fb02369b25479f0
Relevant snippet from the output:
4GLTRACE Return from Main Block "Customer Customer" [sports2020trgs/wrcust.p]
I see I can use collective.folderishtypes to add new types (folderish) to be used instead of default news item and event. But I want to convert existing news items and events to folderish content types and keep it as simple as possible. Is it possible to override (monkey-patching?) the default types in a simple way (as result to have existing objects with folderish behavior)?
Or what is the good way of solving this issue? I just need existing objects to be solved, too and to have not confusing duplicate content types like: Add new News Item, News Item Folderish... etc. Also, if possible to keep existing listings (like latest events) working.
I have no experience with collective.folderish, the description sounds promising though, too bad it seems not to work for you.
If I needed to solve this and it's not a requirement to keep the histories (workflow- & content-history), I'd go create a new folderish type with the same fields, create for each event and news an instance of the new type and copy the field-values over.
That would change the modification-date, yet could be overcome by copying the mod-date to the publication-date-field (if not used already) and do the 'Latest news/events'-listings with collections sorted by pub-date.
But if you wanted to keep histories and leave mod-date untouched, you could create a folder for each news/event-item, put the item into the folder, set the item as default-view of the folder and rename the folder to the same id as the item. That will make the folder and item appear as one item in the UI and links to the item will not break because the folder is at the destination.
I tested this with a browser-view-script. Alas, adding a folder and moving the item within one script-run does not work for reasons I couldn't track down in short time. So one needs to call the browser-view three times:
from Acquisition import aq_parent, aq_inner
from Products.Five.browser import BrowserView
class View(BrowserView):
report = ''
def __call__(self):
portal = self.context
catalog = portal.portal_catalog
news_items = catalog.searchResults(portal_type='News Item')
event_items = catalog.searchResults(portal_type='Event')
items = news_items + event_items
for i, item in enumerate(items):
self.processItem(item, i, len(items))
return self.report
def processItem(self, item, i, itemsAmount):
item = item.getObject()
item_id = item.id
parent = aq_parent(aq_inner(item))
folder = None
folder_id = item_id + '-container'
if item_id == parent.id:
if i == itemsAmount-1: self.report += '\
Nothing to do, all ' + str(itemsAmount) + ' items have the same id as their parent.'
else:
if parent.id == folder_id:
parent = getParent(parent)
folder = parent[folder_id]
folder.setDefaultPage(item_id)
parent.manage_renameObject(folder.id, item_id)
if i == itemsAmount-1: self.report += '\
Step 3/3: Renamed ' + str(itemsAmount) + ' folder-ids.'
else:
try:
folder = addFolder(parent, folder_id)
if i == itemsAmount-1: self.report += '\
Step 1/3: Added ' + str(itemsAmount) + ' folders.'
folder.setTitle(item_id) # set same title as item has
folder.reindexObject()
except:
folder = parent[folder_id]
try:
cutAndPaste(item, folder)
if i == itemsAmount-1: self.report += '\
Step 2/3: Moved ' + str(itemsAmount) + ' items into folders.'
except:
pass
def addFolder(parent, folder_id):
parent.invokeFactory('Folder', folder_id)
folder = parent[folder_id]
folder.setTitle(folder_id)
folder.reindexObject()
return folder
def cutAndPaste(item, folder):
""" Move item into folder. """
parent = aq_parent(aq_inner(item))
clipboard = parent.manage_cutObjects([item.id])
folder.manage_pasteObjects(clipboard)
folder.reindexObject()
def getParent(item):
return aq_parent(aq_inner(item))
Disclaimers:
You need to do this procedure also every time a new event/news-item is created, with an event-listener.
It would be better to create new event-listeners for each step of the process and start the next one when the preceding step has ended.
The temporary-id for the folder (composed of item-id and the arbitrary suffix "-container") is assumed to not exist already within the parent of an item. Although it is very unlikely to happen, you might want to grab that exception in the script, too.
I have not tested this, but based on the collective.folderishtypes documentation ("How to migrate non-folderishtypes to folderish ones") you should be able to call the ##migrate-btrees view on your Plone site root to migrate non-folderish content types to folderish.
Warning: do a backup of the database before attempting the migration, and test in development environment first before applying this on production data.
I am writing a Frama-C Plugin.
I want to develop a plugin, that sets the value of a local variable. By this idea I try to do the value-analysis afterwards, and then I can analyze the reachablility, path analysis and other things by my second plugin.
Is it possible to set the value of a local variable within a plugin (at the start of a function where I know the name)?
EDIT
I now found out how to make new local variables, how to get the Varinfo of variables and how to create new varinfos. The only missing thing is setting the variable's value.
I started with a code like this:
match kf_cil_fun with
| Cil_types.Definition(a,b) ->
let val_visitor = new value_set_visitor kf in
Visitor.visitFramacFileSameGlobals (val_visitor :> Visitor.frama_c_visitor) (Ast.get());
let old_varinfo = Cil.makeLocalVar a "x" Cil.intType in
let new_varinfo = Cil.makeVarinfo false false "x" Cil.intType in
val_visitor#doStuff old_varinfo new_varinfo;
()
| _ -> ()
where the visitor is a simple visitor with a method doStuff, and the builtin-methods vfile, vglob_aux and vstmt_aux that simply call Cil.DoChildren
method doStuff old_varinfo new_varinfo =
Cil.set_varinfo self#behavior old_varinfo new_varinfo;
Does anyone have an idea of how to set the value of x to 1 (or a fixed other value)? Am I doing the things right?
I have the following code which is meant to programmatically assign relation values to a custom content type.
publications = # some data
catalog = getToolByName(context, 'portal_catalog')
for pub in publications:
if pub['custom_id']:
results = catalog(custom_id=pub['custom_id'])
if len(results) == 1:
obj = results[0].getObject()
measures = []
for m in pub['measure']:
if m in context.objectIds():
m_id = intids.getId(context[m])
relation = RelationValue(m_id)
measures.append(relation)
obj.measures = measures
obj.reindexObject()
notify(ObjectModifiedEvent(obj))
Snippet of schema for custom content type
measures = RelationList(
title=_(u'Measure(s)'),
required=False,
value_type=RelationChoice(title=_(u'Measure'),
source=ObjPathSourceBinder(object_provides='foo.bar.interfaces.measure.IMeasure')),
)
When I run my script everything looks good. The problem is when my template for the custom content tries to call "pub/from_object/absolute_url" the value is blank - only after a restart. Interestingly, I can get other attributes of pub/from_object after a restart, just not it's URL.
from_object retrieves the referencing object from the relation catalog, but doesn't put the object back in its proper Acquisition chain. See http://docs.plone.org/external/plone.app.dexterity/docs/advanced/references.html#back-references for a way to do it that should work.