Plone 4: collective.formcriteria uninstall now can't import site - plone

I had collective.formcriteria installed on a site, then it was uninstalled & removed from Buildout.
When I export my site it works fine, but trying to import the site gives me this error:
Traceback (innermost last):
Module ZPublisher.Publish, line 127, in publish
Module ZPublisher.mapply, line 77, in mapply
Module ZPublisher.Publish, line 47, in call_object
Module OFS.ObjectManager, line 628, in manage_importObject
Module OFS.ObjectManager, line 646, in _importObjectFromFile
Module ZODB.ExportImport, line 92, in importFile
Module transaction._transaction, line 253, in savepoint
Module transaction._transaction, line 250, in savepoint
Module transaction._transaction, line 647, in __init__
Module ZODB.Connection, line 1128, in savepoint
Module ZODB.Connection, line 570, in _commit
Module ZODB.ExportImport, line 175, in _importDuringCommit
ImportError: No module named formcriteria.criteria.relativepath
Will installing this addon again and attempting to import/export my site work or is there another fix?
Thanks.

During import the ZODB has to remap the persistent ids in the dumped pickles, and to do so it has to 'load' all the entries in your export file.
During that remapping process it thus will try and import every referenced class. Your export still holds references to collective.formcriteria classes, and that's why you get the error during import.
There are two things you can do:
Install collective.formcriteria into your buildout just for this import to work. There is no need to run install code or anything, the Zope process just needs to be able to import the class during the import process. You can remove it again afterwards.
Fill in the missing imports with empty stubs. This is a bit more involved, as you'll need to create a dummy module that you'll insert into sys.modules to be loaded for this process.
The dummy module could look like this; I've named it 'placeholder.py':
from persistent import Persistent
class PlaceHolderClass(Peristent):
def __repr__(self):
return "<Persistent placeholder class for now-removed instances>"
And you stub out missing imports with:
import sys
import placeholder
sys.modules['collective.formcriteria.criteria.relativepath'] = placeholder
placeholder.FormRelativePathCriterion = placeholder.PlaceHolderClass
It is quite likely you'll need to stub out more such classes, based on the collective.formcriteria package layout.
Using this technique does not remove the broken references; it'll only let you import your .zexp file into a new instance.

Related

Airflow: ValueError: Unable to configure handler 'processor' - wasb logger

I am trying to configure remote logging with Azure blob.
Airflow version: 1.10.2
Python: 3.6.5
Ubuntu: 18.04
Following are the step I did:
In $AIRFLOW_HOME/config/log_config.py, I have put REMOTE_BASE_LOG_FOLDER = 'wasb-airflow-logs' (This is a folder inside the container (container name: airflow-logs))
Empty init.py is in $AIRFLOW_HOME/config/
$AIRFLOW_HOME/config/ is added in $PYTHONPATH
Renamed DEFAULT_LOGGING_CONFIG to LOGGING CONFIG everywhere in $AIRFLOW_HOME/config/log_config.py
User defined in Airflow blob connection has read/write access to REMOTE_BASE_LOG_FOLDER
$AIRFLOW_HOME/airflow.cfg it has remote_logging = True
logging_config_class = log_config.LOGGING_CONFIG
remote_log_conn_id =
Following is the error:
Unable to load the config, contains a configuration error.
Traceback (most recent call last):
File "/home/gsingh/anaconda3/lib/python3.6/logging/config.py", line 382, in resolve
found = getattr(found, frag)
AttributeError: module 'airflow.utils.log' has no attribute 'wasb_task_handler'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/gsingh/anaconda3/lib/python3.6/logging/config.py", line 384, in resolve
self.importer(used)
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/utils/log/wasb_task_handler.py", line 23, in <module>
from airflow.contrib.hooks.wasb_hook import WasbHook
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/contrib/hooks/wasb_hook.py", line 22, in <module>
from airflow.hooks.base_hook import BaseHook
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 28, in <module>
from airflow.models import Connection
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/models.py", line 86, in <module>
from airflow.utils.dag_processing import list_py_file_paths
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/utils/dag_processing.py", line 49, in <module>
from airflow.settings import logging_class_path
ImportError: cannot import name 'logging_class_path'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/gsingh/anaconda3/lib/python3.6/logging/config.py", line 558, in configure
handler = self.configure_handler(handlers[name])
File "/home/gsingh/anaconda3/lib/python3.6/logging/config.py", line 708, in configure_handler
klass = self.resolve(cname)
File "/home/gsingh/anaconda3/lib/python3.6/logging/config.py", line 391, in resolve
raise v
File "/home/gsingh/anaconda3/lib/python3.6/logging/config.py", line 384, in resolve
self.importer(used)
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/utils/log/wasb_task_handler.py", line 23, in <module>
from airflow.contrib.hooks.wasb_hook import WasbHook
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/contrib/hooks/wasb_hook.py", line 22, in <module>
from airflow.hooks.base_hook import BaseHook
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 28, in <module>
from airflow.models import Connection
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/models.py", line 86, in <module>
from airflow.utils.dag_processing import list_py_file_paths
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/utils/dag_processing.py", line 49, in <module>
from airflow.settings import logging_class_path
ValueError: Cannot resolve 'airflow.utils.log.wasb_task_handler.WasbTaskHandler': cannot import name 'logging_class_path'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/gsingh/venv/bin/airflow", line 21, in <module>
from airflow import configuration
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/__init__.py", line 36, in <module>
from airflow import settings, configuration as conf
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/settings.py", line 262, in <module>
logging_class_path = configure_logging()
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/logging_config.py", line 73, in configure_logging
raise e
File "/home/gsingh/venv/lib/python3.6/site-packages/airflow/logging_config.py", line 68, in configure_logging
dictConfig(logging_config)
File "/home/gsingh/anaconda3/lib/python3.6/logging/config.py", line 795, in dictConfig
dictConfigClass(config).configure()
File "/home/gsingh/anaconda3/lib/python3.6/logging/config.py", line 566, in configure
'%r: %s' % (name, e))
ValueError: Unable to configure handler 'processor': Cannot resolve 'airflow.utils.log.wasb_task_handler.WasbTaskHandler': cannot import name 'logging_class_path'
I am not sure which configuration I am missing. Has anyone faced the same issue?
You need to install the azure package.
pip install 'apache-airflow[azure_blob_storage,azure_data_lake,azure_cosmos,azure_container_instances]
As per updating.md
This now should be installed with
pip install apache-airflow[azure]
But this didn't work for me.
sudo chown 50000:0 dags logs plugins in my case.
I tried to run official docker-compose.yml with all these containers (which are dependent on these 3 volume forwards) or simply wrap airflow standalone into a single container for a debug purpose. Turned out volumes were created with root ownerships instead of airflows.
I had the same error however if I scrolled up higher I could see that there was another exception thrown before the ValueError. Which was a PermissionError.
PermissionError: [Errno 13] Permission denied: '/usr/local/airflow/logs/scheduler'
The reason I got that error is because I didn't create the initial 3 folders (dags, logs, plugins) before running airflow docker container. So docker seems to have created then automatically but the permissions were wrong.
Steps to fix:
Stop current container
docker-compose down --volumes --remove-orphans
Delete folders dags, logs, plugins
Just in case Destroy the images and volumes already created (in Docker Desktop)
Create folders again from command line
mkdir logs dags plugins
run airflow docker again
docker-compose up airflow-init
docker-compose up

Make quintagroup.transmogrifier work (version conflict, configuration_id) (Plone)

I'd like to use transmogrifier to copy a little excerpt of the one Plone 4 site to another. I followed the instructions in the quintagroup documentation, e.g. here.
I added both collective.transmogrifier and quintagroup.transmogrifier to my buildout:
[instance]
eggs +=
Products.Marshall
collective.transmogrifier
quintagroup.transmogrifier
zcml +=
collective.transmogrifier
quintagroup.transmogrifier
However, the installed version 1.4 of collective.transmogrifier was not sufficient, since it lacks the traverse function in the utils module:
File ".../eggs/plone.app.transmogrifier-1.3-py2.7.egg/plone/app/transmogrifier/atschemaupdater.py", line 8, in <module>
from collective.transmogrifier.utils import traverse
zope.configuration.xmlconfig.ZopeXMLConfigurationError: File ".../parts/instance/etc/site.zcml", line 15.2-15.55
ZopeXMLConfigurationError: File ".../parts/instance/etc/package-includes/027-quintagroup.transmogrifier-configure.zcml", line 1.0-1.70
ZopeXMLConfigurationError: File ".../eggs/quintagroup.transmogrifier-0.5-py2.7.egg/quintagroup/transmogrifier/configure.zcml", line 11.4-11.50
ZopeXMLConfigurationError: File ".../eggs/plone.app.transmogrifier-1.3-py2.7.egg/plone/app/transmogrifier/configure.zcml", line 9.2-12.8
ImportError: cannot import name traverse
I specified collective.transmogrifier = 1.5 in my versions.cfg, and then changed eggs/quintagroup.transmogrifier-0.5-py2.7.egg/EGG-INFO/requires.txt which insisted in collective.transmogrifier<1.5 (why?) to
collective.transmogrifier<=1.5
After rebuilding and restarting I was able to add the quintagroup.transmogrifier in the quickinstaller view.
However, when I tried to export the site in the Generic Setup Tool, I failed miserably:
Traceback (innermost last):
Module ZPublisher.Publish, line 138, in publish
Module ZPublisher.mapply, line 77, in mapply
Module ZPublisher.Publish, line 48, in call_object
Module Products.GenericSetup.tool, line 583, in manage_exportSelectedSteps
Module Products.GenericSetup.tool, line 1038, in _doRunExportSteps
Module quintagroup.transmogrifier.exportimport, line 72, in exportSiteStructure
Module collective.transmogrifier.utils, line 118, in constructPipeline
Module quintagroup.transmogrifier.sitewalker, line 32, in __init__
Module collective.transmogrifier.utils, line 225, in __init__
AttributeError: Transmogrifier instance has no attribute 'configuration_id'
It didn't make a difference whether I saved the default export profile before or not.
Is there some configuration step missing, or is there some reason for the collective.transmogrifier <1.5 constraint? I'd like to see this work before taking on the task of selecting the whitelisted contents ...
I've come across this before, but I was able to avoid using quintagroup.transmogrifier in that case.
The pin was removed in this commit in master (though it was a little more complicated than that if you check the history of setup.py).
So basically you are going to have to check out that product in your buildout:
[sources]
...
quintagroup.transmogrifier = git https://github.com/collective/quintagroup.transmogrifier.git
and rerunning buildout (probably bin/buildout -c develop.cfg) should do it, though that should be considered a short term hack rather than a long term solution if you are doing it on production.
The long term solution is to build your own '0.5-tobias' egg using jarn.mkrelease or zest.releaser (more modern that mkrelease, but trickier to figure out) or raise an issue on https://github.com/collective/quintagroup.transmogrifier.git to ask someone to make an official release for you!

AttributeError: 'module' object has no attribute 'NavigationViewlet'

here is my traceback from a Plone 4.2.1 installation (on SuSE Linux):
Traceback (innermost last):
Module ZPublisher.Publish, line 126, in publish
Module ZPublisher.mapply, line 77, in mapply
Module ZPublisher.Publish, line 46, in call_object
Module OFS.ObjectManager, line 620, in manage_importObject
Module OFS.ObjectManager, line 638, in _importObjectFromFile
Module ZODB.ExportImport, line 92, in importFile
Module transaction._transaction, line 260, in savepoint
Module transaction._transaction, line 257, in savepoint
Module transaction._transaction, line 690, in __init__
Module ZODB.Connection, line 1123, in savepoint
Module ZODB.Connection, line 587, in _commit
Module ZODB.ExportImport, line 176, in _importDuringCommit
AttributeError: 'module' object has no attribute 'NavigationViewlet'
The strange thing, that I do not understand is, why only one Plone Site shows this error during import into an instance newer than Plone 4.1.5?
The Plone site has only the following additional plugins/products installed (as others too, but they all work fine):
LinguaPlone
PloneFormGen
Products.ZSyncer
collective.quickupload
So, where is my mistake/error?
Thanx for any help, because the site is too big now to "rebuild" it in a new Plone instance from scratch :-(
The plone.app.layout.viewlets.links.NavigationViewlet was removed in plone.app.layout version 2.2 (see this entry into the HTML5 PLIP and this commit).
ZEXP imports into different version stacks are generally fraught with problems, exactly because of changing persistent object locations. You'll have to import to an earlier version of the software stack, then follow the regular Plone upgrade path to bring yourself up to a more recent version of Plone.
The upgrade scripts take care of removing references to removed viewlets like these, for example.

I changed an archetype attr (field) name and now I'm getting permission errors. What can I do?

I named an attribute (field) in a content-type 'string' instead of 'company'.
I've installed my product and after seeing this error, I changed it in my content type class, reinstalled the product, ran update security settings and update catalog. But I keep getting:
Module Products.CMFPlone.utils, line 392, in _createObjectByType
Module Products.CMFCore.TypesTool, line 290, in _finishConstruction
Module Products.CMFCore.CMFCatalogAware, line 148, in notifyWorkflowCreated
Module Products.CMFCore.WorkflowTool, line 291, in notifyCreated
Module Products.DCWorkflow.DCWorkflow, line 346, in notifyCreated
Module Products.DCWorkflow.DCWorkflow, line 430, in _changeStateOf
Module Products.DCWorkflow.DCWorkflow, line 529, in _executeTransition
Module Products.DCWorkflow.DCWorkflow, line 389, in updateRoleMappingsFor
Module Products.DCWorkflow.utils, line 64, in modifyRolesForPermission
Module AccessControl.Permission, line 92, in setRoles
AttributeError: string
How do I delete this string attribute that is persisted somewhere in ny ZODB since I've already removed it from my content type? I've reinstalled the product, restarted my instance, but I keep getting the same problem. Ideas? I can delete my Data.fs since it's a development machine, but if it happened in a production site, this wouldn't be an option.
PS: When I ran bin/instance fg:
2011-07-28 19:01:59 WARNING Init Class mynamespace.mypackage.content.mycontent.MyContent has a security declaration for nonexistent method 'string'
I found the problem. I changed in my content class, but didn't change in my content type interface.
paster creates a # -*- schema definition goes here -*- on interfaces.py as well. I did a grep string on my product directory and found it.

Problem with a local utility used as a dependency in a product

Here's the problem. I have mynamespace.mypackage that has as a dependency mynamespace.mydependencypackage, that is a local utility. It's registered using component registry.
In config.py from mynamespace.mypackage, I have
DEPENDENCIES = ['mynamespace.mydependencypackage']
And in my mynamespace.mypackage's setuphandlers.py this dependency is installed if not already installed.
Problem is: if I reinstall mynamespace.mypackage through ZMI, everything seems to perfectly install (since no errors are shown) but I keep getting a ComponentLookupError when using methods in mynamespace.mypackage that gets the utility:
Module zope.component._api, line 207, in getUtility
ComponentLookupError: (<InterfaceClass MY_UTILITY_INTERFACE, '')
I can 'fix' this problem, by reinstalling the mynamespace.mydependencypackage in my setuphandlers.py or through the ZMI as well when reinstalling mynamespace.mypackage, but this doesn't seem the best solution for me.
What am I missing about Generic Setup here? I didn't make this utility persist any value on ZODB whatsoever. I can just forget about all these problems and create a BrowserView with utilities methods, but I want to understand first why am I having these problems.
EDIT: Now, I have a bigger problem. The TypeError: ('object.__new__(MyClass) is not safe, use Persistence.Persistent.__new__()', <function _reconstructor at 0xb7783e9c>, (<class 'mynamespace.mydependencypackage.package.MyClass'>, <type 'object'>, None)) is being shown. Full traceback:
Traceback (innermost last):
Module ZPublisher.Publish, line 110, in publish
Module ZPublisher.BaseRequest, line 429, in traverse
Module ZPublisher.BeforeTraverse, line 99, in __call__
Module Products.CMFCore.PortalObject, line 94, in __before_publishing_traverse__
Module zope.event, line 23, in notify
Module zope.component.event, line 26, in dispatch
Module zope.component._api, line 130, in subscribers
Module zope.component.registry, line 290, in subscribers
Module zope.interface.adapter, line 535, in subscribers
Module zope.component.event, line 33, in objectEventNotify
Module zope.component._api, line 130, in subscribers
Module zope.component.registry, line 290, in subscribers
Module zope.interface.adapter, line 535, in subscribers
Module zope.app.component.site, line 375, in threadSiteSubscriber
Module zope.app.component.hooks, line 61, in setSite
Module Products.CMFCore.PortalObject, line 75, in getSiteManager
Module ZODB.Connection, line 761, in setstate
Module ZODB.Connection, line 819, in _setstate
Module ZODB.serialize, line 604, in setGhostState
Module ZODB.serialize, line 597, in getState
Module copy_reg, line 48, in _reconstructor
TypeError: ('object.__new__(MyClass) is not safe, use Persistence.Persistent.__new__()', <function _reconstructor at 0xb7783e9c>, (<class 'mynamespace.mydependencypackage.package.MyClass'>, <type 'object'>, None))
It sounds like you have custom Python code in your setuphandlers.py file to install the dependency. Is there a reason you don't note the dependency in the metadata.xml file? Or can you show us that code?
When activating an add-on in Plone, it does a before/after comparison of various entities to support deactivation. Among these are local persistent utilities, as defined by the componentregistry.xml file. Note: anything defined in GenericSetup xml files results in persistent changes - if you want non-persistent utilities, use ZCML files to register them.
So when you have custom code to add a local utility in your setuphandlers.py code, Plone thinks this utility belongs to your main add-on. If you reinstall that add-on the utility gets removed alongside anything else and then everything is installed again.
I'm guessing your "is it already installed" check fails after the reinstall and the utility isn't added again.
You can avoid all of this, by simple giving both the dependency and main add-on their own GenericSetup profiles and than noting a dependency in the main's metadata.xml, like:
<?xml version="1.0"?>
<metadata>
<version>1</version>
<dependencies>
<dependency>profile-my.dependency:default</dependency>
</dependencies>
</metadata>
Once you do that the dependencies profile will be activated independently and Plone keeps track of it on its own. If you then decide to reinstall the main add-on, the dependency won't be touched at all and remains intact.

Resources