I've got a product installed in one Plone Site, this product change the visibility of a field of the Event content type.
It use IBrowserLayerAwareExtender to restrain the change to only the Plone Site where the product is installed.
This work on the development server on which buildout is made with the develop.cfg option, but in production, the layer is not respected, and all others Plone Site have this change.
Here is the code:
schemaextender.py:
class EventModifier(object):
"""
Masque certains champs inutiles pour le projet
"""
implements(ISchemaModifier, IBrowserLayerAwareExtender)
adapts(IATEvent)
layer = IBswMonasticLayer
def __init__(self, context):
self.context = context
# noinspection PyMethodMayBeStatic
def fiddle(self, schema):
"""
:param schema:
:return:
"""
schema['attendees'].widget.visible = {'edit': 'invisible', 'view': 'invisible'}
schema['location'].widget.label = _(u'Adresse')
return schema
configure.zcml:
<adapter for="Products.ATContentTypes.interface.IATEvent"
provides="archetypes.schemaextender.interfaces.ISchemaModifier"
factory=".schemaextender.EventModifier"
name="bsw.monastic.schemaextender.EventModifier"/>
Is it a bug or am I missing something ?
IMHO it's weird it works on you dev machine and not in production. I assume the browserlayer is indeed there on the production site.
You can check this by running the following code in a debug session on the server:
>>> from zope.component.hooks import setSite
>>> plone = app.path.to.plone.site
>>> setSite(plone) # Setup component registry
>>> from plone.browserlayer.utils import registered_layers
>>> registered_layers()
[...] # Bunch of layer interface active on the plone site.
I assume it's there, so you should remove it.
If so remove it using from plone.browserlayer.utils import unregister_layer
>>> from plone.browserlayer.utils import unregister_layer
>>> unregister_layer(layername)
>>> import transaction
>>> transaction.commit()
Related
I'm trying to implement custom XCOM backend.
Those are the steps I did:
Created "include" directory at the main Airflow dir (AIRFLOW_HOME).
Created these "custom_xcom_backend.py" file inside:
from typing import Any
from airflow.models.xcom import BaseXCom
import pandas as pd
class CustomXComBackend(BaseXCom):
#staticmethod
def serialize_value(value: Any):
if isinstance(value, pd.DataFrame):
value = value.to_json(orient='records')
return BaseXCom.serialize_value(value)
#staticmethod
def deserialize_value(result) -> Any:
result = BaseXCom.deserialize_value(result)
result = df = pd.read_json(result)
return result
Set at config file:
xcom_backend = include.custom_xcom_backend.CustomXComBackend
When I restarted webserver I got:
airflow.exceptions.AirflowConfigException: The object could not be loaded. Please check "xcom_backend" key in "core" section. Current value: "include.cust...
My guess is that it not recognizing the "include" folder
But how can I fix it?
*Note: There is no docker. It is installed on a Ubuntu machine.
Thanks!
So I solved it:
Put custom_xcom_backend.py into the plugins directory
set at config file:
xcom_backend = custom_xcom_backend.CustomXComBackend
Restart all airflow related services
*Note: Do not store DataFrames that way (bad practice).
Sources I used:
https://www.youtube.com/watch?v=iI0ymwOij88
We are using Apache 1.9.0. I have written a snowflake hook plugin. I have placed the hook in the $AIRFLOW_HOME/plugins directory.
$AIRFLOW_HOME
+--plugins
+--snowflake_hook2.py
snowflake_hook2.py
# This is the base class for a plugin
from airflow.plugins_manager import AirflowPlugin
# This is necessary to expose the plugin in the Web interface
from flask import Blueprint
from flask_admin import BaseView, expose
from flask_admin.base import MenuLink
# This is the base hook for connecting to a database
from airflow.hooks.dbapi_hook import DbApiHook
# This is the snowflake provided Connector
import snowflake.connector
# This is the default python logging package
import logging
class SnowflakeHook2(DbApiHook):
"""
Airflow Hook to communicate with Snowflake
This is implemented as a Plugin
"""
def __init__(self, connname_in='snowflake_default', db_in='default', wh_in='default', schema_in='default'):
logging.info('# Connecting to {0}'.format(connname_in))
self.conn_name_attr = 'snowflake_conn_id'
self.connname = connname_in
self.superconn = super().get_connection(self.connname) #gets the values from Airflow
{SNIP - Connection stuff that works}
self.cur = self.conn.cursor()
def query(self,q,params=None):
"""From jmoney's db_wrapper allows return of a full list of rows(tuples)"""
if params == None: #no Params, so no insertion
self.cur.execute(q)
else: #make the parameter substitution
self.cur.execute(q,params)
self.results = self.cur.fetchall()
self.rowcount = self.cur.rowcount
self.columnnames = [colspec[0] for colspec in self.cur.description]
return self.results
{SNIP - Other class functions}
class SnowflakePluginClass(AirflowPlugin):
name = "SnowflakePluginModule"
hooks = [SnowflakeHook2]
operators = []
So I went ahead and put some print statements in Airflows plugin_manager to try and get a better handle on what is happening. After restarting the webserver and running airflow list_dags, these lines were showing the "new module name" (and no errors
SnowflakePluginModule [<class '__home__ubuntu__airflow__plugins_snowflake_hook2.SnowflakeHook2'>]
hook_module - airflow.hooks.snowflakepluginmodule
INTEGRATING airflow.hooks.snowflakepluginmodule
snowflakepluginmodule <module 'airflow.hooks.snowflakepluginmodule'>
As this is consistent with what the documentation says, I should be fine using this in my DAG:
from airflow import DAG
from airflow.hooks.snowflakepluginmodule import SnowflakeHook2
from airflow.operators.python_operator import PythonOperator
But the web throws this error
Broken DAG: [/home/ubuntu/airflow/dags/test_sf2.py] No module named 'airflow.hooks.snowflakepluginmodule'
So the question is, What am I doing wrong? Or have I uncovered a bug?
You need to import as below:
from airflow import DAG
from airflow.hooks import SnowflakeHook2
from airflow.operators.python_operator import PythonOperator
OR
from airflow import DAG
from airflow.hooks.SnowflakePluginModule import SnowflakeHook2
from airflow.operators.python_operator import PythonOperator
I don't think that airflow automatically goes through the folders in your plugins directory and runs everything underneath it. The way that I've set it up successfully is to have an __init__.py under the plugins directory which contains each plugin class. Have a look at the Astronomer plugins in Github, it provides some really good examples for how to set up your plugins.
In particular have a look at how they've set up the mysql plugin
https://github.com/airflow-plugins/mysql_plugin
Also someone has incorporated a snowflake hook in one of the later versions of airflow too which you might want to leverage:
https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/hooks/snowflake_hook.py
I installed Django and was able to double check that the module was in fact in Python, but when attempting to implement basic commands such as runserver or utilize manage.py; I get DJANGO_SETTEINGS_MODULE error. I already used "set DJANGO_SETTINGS_MODULE = mysite.settings" as advised and inserted mysite.settings into the PATH for Python as some documentation online directed me to.
Now instead of undefined it says no such module exists. I can't find anything else in the documentation and I used my test site name instead of "mysite" without any change. Does anyone know what am I missing? All I can find in the module library for Django in my Python is this code.
from future import unicode_literals
from django.utils.version import get_version
VERSION = (1, 11, 5, 'final', 0)
__version__ = get_version(VERSION)
def setup(set_prefix=True):
"""
Configure the settings (this happens as a side effect of accessing the first setting), configure logging and populate the app registry.
Set the thread-local urlresolvers script prefix if `set_prefix` is True.
"""
from django.apps import apps
from django.conf import settings
from django.urls import set_script_prefix
from django.utils.encoding import force_text
from django.utils.log import configure_logging
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
if set_prefix:
set_script_prefix(
'/' if settings.FORCE_SCRIPT_NAME is None else force_text(settings.FORCE_SCRIPT_NAME)
)
apps.populate(settings.INSTALLED_APPS)
Are you sure you wrote properly the environment variable? I'm asking cause I see you get the error DJANGO_SETTEINGS_MODULE (settings has a misspelling)...
I am following the tests here: https://github.com/plone/plone.app.referenceablebehavior/blob/master/plone/app/referenceablebehavior/referenceable.txt
I added plone.app.referenceablebehavior to Plone 4.3, created a type TTW and made it referenceable:
Then I created an instance of the type in the site root called "My Referenceable Type Instance", and tried the following in debug mode:
>>> from plone.app.referenceablebehavior.referenceable import IReferenceable
>>> IReferenceable.providedBy(app.Plone['my-referenceable-type-instance'])
False
I would expect the result to be True. Is this a bug, or am I missing something?
[0] My buildout:
[buildout]
extends = https://raw.github.com/pythonpackages/buildout-plone/master/4.3.x-dev
[plone]
eggs +=
plone.app.referenceablebehavior
In a debug session, you need to set the local site manager before attempting this. Try:
>>> from zope.component.hooks import setSite
>>> setSite(app.Plone)
...prior to attempting to check if IReferenceable is provided by the object. The reason that this is necessary is that Dexterity uses something called an Object Specification Descriptor that looks up interfaces dynamically from the Factory Type Information of the type, which is site-specific (you cannot retrieve site-specific configuration without first having the local site configured for lookups).
I would like to check internet connexion from my plone site. I tried a ping in a python script
## Script (Python) "pwreset_action.cpy"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##title=Reset a user's password
##parameters=randomstring, userid=None, password=None, password2=None
from Products.CMFCore.utils import getToolByName
from Products.PasswordResetTool.PasswordResetTool import InvalidRequestError, ExpiredRequestError
import ping, socket
status = "success"
pw_tool = getToolByName(context, 'portal_password_reset')
try:
pw_tool.resetPassword(userid, randomstring, password)
except ExpiredRequestError:
status = "expired"
except InvalidRequestError:
status = "invalid"
except RuntimeError:
status = "invalid"
context.plone_log("TRYING TO PING")
try :
ping.verbose_ping('www.google.com' , run=3)
context.plone_log("PING DONE")
except socket.error, e:
context.plone_log("PING FAILED")
return state.set(status=status)
I got these errors :
2012-07-20T11:37:08 INFO SignalHandler Caught signal SIGTERM
------
2012-07-20T11:37:08 INFO Z2 Shutting down fast
------
2012-07-20T11:37:08 INFO ZServer closing HTTP to new connections
------
2012-07-20T11:37:42 INFO ZServer HTTP server started at Fri Jul 20 11:37:42 2012
Hostname: 0.0.0.0
Port: 8080
------
2012-07-20T11:37:42 WARNING SecurityInfo Conflicting security declarations for "setText"
------
2012-07-20T11:37:42 WARNING SecurityInfo Class "ATTopic" had conflicting security declarations
------
2012-07-20T11:37:46 INFO plone.app.theming Patched Zope Management Interface to disable theming.
------
2012-07-20T11:37:48 INFO PloneFormGen Patching plone.app.portlets ColumnPortletManagerRenderer to not catch Retry exceptions
------
2012-07-20T11:37:48 INFO Zope Ready to handle requests
------
Python Scripts in Zope are sandboxed (via RestrictedPython, which means that any module imports have to be declared safe first. Adding modules to the declared-safe list is generally a Bad Idea unless you know what you are doing.
To declare a module as importable into Python Scripts, you'll need to create a python package, then add the following code to it so it is executed when Zope starts:
from Products.PythonScripts.Utility import allow_module
allow_module('ping')
This'll allow any import from that module (use with caution)!
It's better to allow only specific methods and classes from a module; use a ModuleSecurity declaration for that:
from AccessControl import ModuleSecurityInfo
ModuleSecurityInfo('ping').declarePublic('verbose_ping')
ModuleSecurityInfo('socket').declarePublic('error')
This is documented in the Security chapter of the Zope Developers Guide, specifically the section on module security assertions.
Note that it nearly always is a better idea to do all this work in a tightly constrained method in unrestricted code (e.g. a regular python package), then allow that method to be used from a python script instead.
It won't work.
You CANNOT import arbitrary Python modules in RestrictedPython scripts, as in the answer you were told yesterday:
https://stackoverflow.com/a/11568316/315168
If you need to use arbitraty Python modules you need to write your own Plone add-on for that and use a BrowserView for the purpose. RestrictedPython through-the-web-browser development is not enough:
http://collective-docs.readthedocs.org/en/latest/getstarted/index.html