I have a Zope / Plone 4.3 environment and we use z3c.saconfig to configure the database (Oracle) settings in a internal product.
I need to change the usual SQLAlchemy's pool type from QueuePool to NullPool. But, is there a way to do it using z3c.saconfig?
z3c.saconfig delegates creating the SQLAlchemy engine to a IEngineFactory utility; see the interface source.
You can create your own subclass of it to register as a local utility. Your subclass could reuse the existing utility implementation, overriding the configuration() method:
from persistent import Persistent
from z3c.saconfig.utility import EngineFactory
from sqlalchemy.pool import NullPool
class NullPoolEngineFactory(Persistent, EngineFactory)
def configuration(self):
kwargs = self._kw.copy()
kwargs['poolclass'] = NullPool
return self._args, kwargs
The above augments the arguments for the sqlalchemy.create_engine() function by adding a poolclass argument.
You'd register this utility as a component in your GenericSetup profile:
<?xml version="1.0"?>
<componentregistry>
<utilities>
<utility
interface="z3c.saconfig.interfaces.IEngineFactory"
factory="yourproject.yourmodule.NullPoolEngineFactory"/>
</utilities>
</componentregistry>
After running your generic setup profile, this registers a persistent version of the utility, and it'll be found instead of the default global utility.
Related
Making a basic https-get request from a pipeline transform results in "connectionError".
How should one consume an API using the "requests" library to extend some data within a pipeline?
from transforms.api import Input, Output, transform_pandas
import requests
#transform_pandas(
Output("..."),
df=Input("..."),
)
def compute(df):
# Random example
response = requests.get('https://api.github.com')
print(response.content)
return df
results in
Is this a configuration issue?
Following #nicornk's comment and the docs on external transforms, using external APIs from within Palantir is restricted by default and requires several administrative steps. The steps to call external APIs from within pipeline transforms are:
Check the settings of your code repository
Add the library "transforms-external-systems" to your code repositories libraries.
Adding the library, adds a new icon in the control panel on the left.
Click on the new icon and import an egress policy, if any available. Otherwise try to create a "network egress policy", which usually involves an approval process.
After having managed to "import"/install an egress policy into your code repository, import methods from the package "transforms.external.systems" and decorate your transformation, following the docs.
FastAPI documentation recommends using lru_cache decorated functions to retrieve the config file. That makes sense to avoid I/O getting the env file.
config.py
from pydantic import BaseSettings
class Settings(BaseSettings):
app_name: str = "Awesome API"
admin_email: str
items_per_user: int = 50
class Config:
env_file = ".env"
and then in other modules, the documentation implemented a function that gets the settings
#module_omega.py
from . import config
#lru_cache()
def get_settings():
return config.Settings()
settings = get_settings()
print(settings.ENV_VAR_ONE)
I am wondering if this method is better practice or advantageous to just initializing a settings object in the config module and then importing it like below.
#config.py
from pydantic import BaseSettings
class Settings(BaseSettings):
app_name: str = "Awesome API"
admin_email: str
items_per_user: int = 50
class Config:
env_file = ".env"
settings = Settings()
#module_omega.py
from .config import settings
print(settings.ENV_VAR_ONE)
I realize it's been a while since you asked, and though I agree with the commenters that these can be functionally equivalent, I can point out another important difference that I think motivates the use of #lru_cache.
What the #lru_cache approach can help with is limiting the amount of code that is executed when the module is imported.
settings = Settings()
By doing this, like you suggested, you are exporting an instance of your settings. Which means that you're transitively executing any code that needs to be run to create your settings immediately when your module is imported.
While module exports are cached similar to how #lru_cache would do, you don't have as much control over deferring the loading of your settings, since in python we typically place our imports at the top of a file.
The #lru_cache technique is especially useful if you have more expensive settings, like looking at the filesystem, or going to the network. That way you can defer loading your settings until you really actually need them.
from . import get_settings
def do_something_with_deferred_settings():
print(get_settings().my_setting)
if __name__ == "__main__":
do_something_with_deferred_settings()
Other things to look into:
#cache in python 3.9 instead of #lru_cache
Module __getattr__ doesn't add anything here IMO, but it can be useful when working with dynamism and the import system.
I've enabled RBAC as environment variable in docker-compose file.
- AIRFLOW__WEBSERVER__RBAC=True
I want to capture the user who kicked off a dag inside my dag files.
I tried using from flask_login import current_user. But, I get the value of current_user as None.
May I know how to capture user details using RBAC?
According to Airflow documentation as part of RBAC security model that is handled by Flask AppBuilder (FAB):
Airflow uses flask_login and exposes a set of hooks in the
airflow.default_login module. You can alter the content and make it
part of the PYTHONPATH and configure it as a backend in
airflow.cfg.
Flask-login module provides user management operations, thus you can fetch current user within a dedicated property flask_login.current_user, adding
some extra fields, as described in #3438 pull request:
if current_user and hasattr(current_user, 'user'):
user = current_user.user.username
elif current_user and hasattr(current_user, 'username'):
I suppose that you can use current_user.user.username to fetch a user login.
I have a nameko service that deals with lots of entities, and having the entrypoints in a single service.py module would render the module highly unreadable and harder to maintain.
So I've decided to split up the module in to multiple Services which are then used to extend the main service. I am kind of worried about dependency injection and thought a dependency like the db, might have multiple instances due to this approach. This is what I have so far:
the customer service module with all customer-related endpoints
# app/customer/service.py
class HTTPCustomerService:
"""HTTP endpoints for customer module"""
name = "http_customer_service"
db = None
context = None
dispatch = None
#http("GET,POST", "/customers")
def customer_listing(self, request):
session = self.db.get_session()
return CustomerListController.as_view(session, request)
#http("GET,PUT,DELETE", "/customers/<uuid:pk>")
def customer_detail(self, request, pk):
session = self.db.get_session()
return CustomerDetailController.as_view(session, request, pk)
and main service module that inherits from customer service, and possibly other abstract services
# app/service.py
class HTTPSalesService(HTTPCustomerService):
"""Nameko http service."""
name = "http_sales_service"
db = Database(Base)
context = ContextData()
dispatch = EventDispatcher()
and finally I run it with:
nameko run app.service
So this works well, but is the approach right? Especially with regards to dependency injection?
Yep, this approach works well.
Nameko doesn't introspect the service class until run-time, so it sees whatever standard Python class inheritance produces.
One thing to note is that your base class is not "abstract" -- if you point nameko run at app/customer/service.py it will attempt to run it. Related, if you put your "concrete" subclass in the same module, nameko run will try to run both of them. You can mitigate this by specifying the service class, i.e. nameko run app.services:HTTPSalesService
I have a scheduled job (i'm using apscheduler.scheduler lib) that needs access to the plone site object, but I don't have the context in this case. I subscribed IProcessingStart event, but unfortunately getSite() function returns None.
Also, is there a programmatic way to obtain a specific Plone Site from Zope Server root?
Additional info:
I have a job like this:
from zope.site import hooks
sched = Scheduler()
#sched.cron_schedule(day_of_week="*", hour="9", minute="0")
def myjob():
site = hooks.getSite()
print site
print site.absolute_url()
catalogtool = getToolByName(site, "portal_catalog")
print catalogtool
The site variable is always None inside a APScheduler job. And we need informations about the site to run correctly the job.
We have avoided to execute using a public URL because an user could execute the job directly.
Build a context first with setSite(), and perhaps a request object:
from zope.app.component.hooks import setSite
from Testing.makerequest import makerequest
app = makerequest(app)
site = app[site_id]
setSite(site)
This does require that you open a ZODB connection yourself and traverse to the site object yourself.
However, it is not clear how you are accessing the Plone site from your scheduler. Instead of running a full new Zope process, consider calling a URL from your scheduling job. If you integrated APScheduler into your Zope process, you'd have to create a new ZODB connection in the job, traverse to the Plone site from the root, and use the above method to set up the site hooks (needed for a lot of local components anyway).