request.meta vs class attributes scrapy - web-scraping

In order to handle data between callbacks in scrapy, it is possible to define class attributes:
def __init__(self, *args, **kwargs):
self.my_var = 1
and use:
self.my_var
instead of:
request.meta['my_var'] = 1

request.meta is the perfect and reliable way to pass meta information tied to a specific request. For example, if you parse basic product information from the product list page and want to pass along this information to the product details page callback - request.meta is the perfect solution. meta also has a number of special keys that allow to control the behavior of a specific request.

Related

How can I define http.route(‘URL’) which will generic for all pages or dynamic url with dynamic page rendering in odoo 11 web controller?

I want to use below method which is call while rendering any page. If I pass ‘/’ in http.route it will call only for homepage not for others like ‘/shop’, ‘/blog’ etc. Also wanted to pass dynamic template rendering in return on the basis of http.route(‘URL’).
#http.route(['/'], type='http', auth="public", website=True)
def cusotm_controller_func(self, **kwargs):
values= { # values which is passing in template }
return request.render('website.homepage', values)
Can anyone help me out?
Thanks.
Maybe you need to call your function in another moment, like after the models registry(pool) is loaded. You could do this by implementing the method _register_hook in your model. Odoo will always call that method in your model to allow you to initialize whatever you want, but you just have the self and cr arguments.
Maybe helps
Thanks to from odoo forum.
Solution :
Controllers are registering itself in the openerp.http.controllers_per_module dict (see code)... so you can get controller instance using module name and controller name under it:
from odoo.http import controllers_per_module
controller_instance = None
for name,instance in controllers_per_module.get('my_module'):
if name.endswith('my_module.controller.name'):
controller_instance = instance
break
if controller_instance != None:
controller_instance.a_function(*args)

How to use annotations with z3c.form's DictionaryField

There is documentation on using Python dict with z3c.form (loading & storing form data).
However, the z3c.form datamanager used for dicts is not registered for other types or interfaces (see reference), whereas annotations typically use something like PersistentDict.
How can I use the DictionaryField datamanager in this scenario? Ie. so that in my form's getContent method I merely return the PersistentDictannotation.
Well, unfortunately there seems no simple solution for this requirement.
I once faced the same problem using the datagrid field in a z3c form.
The following instruction solves the problem for the datagrid field, which is a list (PersistentList of dicts (PersistentMappings).
I guess you may adapt this solution for your case.
First you need to add the following code to the getContent method:
from plone.directives import form
class MyForm(form.SchemaEditForm):
schema = IMyFormSchema
ignoreContext = False
def getContent(self):
annotations = IAnnotations(self.context)
if ANNOTATION_KEY not in annotations:
annotations[ANNOTATION_KEY] = PersistentMapping()
return YourStorageConfig(annotations[ANNOTATION_KEY])
Important note: I wrap the annotation storage to satisfy the get/set behavior of the z3c form. Check the following YourStorageConfig implementation and you will see why :-).
class YourStorageConfig(object):
implements(IMyFormSchema)
def __init__(self, storage):
self.storage = storage
def __getattr__(self, name):
if name == 'storage':
return object.__getattr__(self, name)
value = self.storage.get(name)
return value
def __setattr__(self, name, value):
if name == 'storage':
return object.__setattr__(self, name, value)
if name == 'yourfieldname':
self.storage[name] = PersistentList(map(PersistentMapping, value))
return
raise AttributeError(name)
yourfieldname should be the field name you are using in the form schema.
To implement the a datagrid field, there is some more work to do, but this may be enough for your case.
Please post comments, or tracebacks, so I can provide further help. I'll gonna add more details/explanation if necessary ;-)
It turns out the answer is as easy as the following ZCML adapter registration:
<adapter
for="persistent.dict.PersistentDict zope.schema.interfaces.IField"
provides="z3c.form.interfaces.IDataManager"
factory="z3c.form.datamanager.DictionaryField"
/>
With that, the following customization of a form is sufficient to use (PersistentDict) annotations for loading & storing form data:
def getContent(self):
"return the object the form will manipulate (load from & store to)"
annotations = IAnnotations(self.context)
return annotations[SOME_ANNOTATIONS_KEY_HERE]
This is assuming that a PersistentDict has been previously stored at annotations[SOME_ANNOTATIONS_KEY_HERE] - otherwise the above code will result in KeyError. It would probably be a good idea to change above getContent so that if the annotation does not yet exist, it is created and initialized with some default values.
Finally, note that for some reason, z3c.form warns against enabling DictionaryField for every mapping type, so it may be prudent to for example subclass PersistentDict for form storage, rather than use it directly. I submitted an issue to z3c.form asking for clarification of that warning.

Setting field defaults on programmatically created Dexterity items

I have a Dexterity content type based on plone.directives.form.Schema which has a number of form hints for assigning defaults:
#form.default_value(field=ITrial['start_on'])
def default_start_on(data):
return datetime.now()
Some of the defaults are more complex, passing back objects that are themselves instances of Dexterity types. These objects are essential for the main type's setup, which is triggered by various events.
I'm now in the process of testing. Ideally, I'd like to be able to use something like:
item = createContentInContainer(folder, 'ctcc.model.trial', 'item')
That is, I'd like the defaults to be picked up by the item without having to be manually passed into the constructor.
If I was using zope.schema I could use FieldProperty to set up proxies to the schema fields. Is there something equivalent for Dexterity, or perhaps a function for pushing an object through form creation?
Solution: I ended up going with David's option #1, intercepting ObjectCreatedEvent.
#grok.subscribe(ITrial, IObjectCreatedEvent)
def create_trial(trial, event):
if getattr(trial, 'start_on', None) is None:
trial.start_on = default_start_on(None)
It stills feels like I'm replicating part of form behaviour, but at least it's using the same functions that are providing the form defaults.
As you've discovered, the #form.default_value decorator is respected by z3c.form forms, but not when items are created in other ways. You have several options:
Write a handler for the ObjectCreatedEvent for your content type which sets up the necessary default values. This is the simplest approach to implement, but may not work if there are other handlers of this event that need the values in place.
Create your own subclass of Dexterity's generic Item class, and use it instead of Item as the basis for your content type. Then you can customize the __init__ method to set whatever you want. This would require migration of existing content items though, if you already have some.
This is a more complicated option. Replace the factory utility used to construct the content type. createContentInContainer ends up looking for an IFactory utility with a name equal to the factory attribute of the content type's FTI. The default implementation is in plone.dexterity.factory but you could replace it with a different one that does more.

Authorize request in ASP.NET Web API based on specific user

I followed this tutorial http://www.tugberkugurlu.com/archive/api-key-authorization-through-query-string-in-asp-net-web-api-authorizationfilterattribute
to create custom Authorization filter.
I have CarController with my custom Authorize Attribute:
[ApiKeyAuth("apiKey", typeof(ApiKeyAuthorizer))]
I send two parameters in the url .. host/Car/4?username=xxx&pass=xxx
It works basically fine, however I want to allow only car owners to see information about their cars.
E.g. user ABC can see only host/Car/5 and user DEF can see host/Car/6 and host/Car/10
how can I solve this scenario?
How can I access the id of the car used in query (host/Car/ID) in my ApiKeyAuthorizer.
Greetings
If you look at his code, https://github.com/tugberkugurlu/ASPNETWebAPISamples/tree/master/TugberkUg.Web.Http/src/samples and https://github.com/tugberkugurlu/ASPNETWebAPISamples/tree/master/TugberkUg.Web.Http/src/TugberkUg.Web.Http, I think you'll find that he's pulling the data directly from the query string. It should simply be a matter of extending that method to pull in the id parameter. You might also want to look at the RequestContentKeyValueModel on the HttpActionContext parameter passed into the OnAuthorization method. The documentation is sketchy and I haven't played with it yet, but that seems like a likely candidate to me. However, the route data is available indirectly through the HttpRequestMessage via an extension method, specifically:
message.GetRouteData();

Anonymous users creating content types: which approaches instead of invokeFactory can be used?

I've come across this little function that let anonymous users call invoke factory.
security.declarePrivate('anonymousInvokeFactory')
def anonymousInvokeFactory(self, container, type_name, id,
REQUEST=None, *args, **kw):
"""
Anonymous cannot add objects with InvokeFactory, so this is a
special
method to do it with. Must be called from other function to limit
possibillities of abuse.
"""
# remember original user
mtool = getToolByName(self, 'portal_membership')
originalUser = mtool.getAuthenticatedMember()
# wrap the request in new security to be able to add content
user = self.getWrappedOwner()
newSecurityManager(REQUEST, user)
container.invokeFactory(type_name, id, REQUEST=REQUEST, *args, **kw)
# set original user again
newSecurityManager(REQUEST, originalUser)
return id
I seems perfect for a situation where I'm using some proxyManager metadata. But I haven't seen this little snippet anywhere besides this nabble entry - is it safe? Which disadvantages can you see in this approach? EDIT: I've found now in official community plone docs effort some references.
My scenario: the anonymous user is creating an Archetype object on ZODB, in a specific context only, that only accepts this type of object. He can not see any objects, he is just calling a form that is going to create these objects. These objects are going to be created, and their attributes (fields) need to be populated as well. The _createObjectType approach creates the object but it doesn't add the fields even using **kwargs. EDIT2: It's possible to edit using default acessors like obj.setTitle. I'm now using this approach, and it works flawlessly.
I would be weary of using anything that sets up a new security manager. A better way to do this would be to bypass security when creating the object.
You can do something like:
pt = getToolByName(context, 'portal_types')
type_info = pt.getTypeInfo('portal_type')
ob = type_info._constructInstance(context, id)
# CMFCore compatibility
if hasattr(type_info, '_finishConstruction'):
return type_info._finishConstruction(ob)
else:
return ob
source: uwosh.pfg.d2c

Resources