Modelling association in Elm leads to dependencies cycle - associations

My database consists of a table of items and a table of associated ads.
I would like my elm app to display either an item with its associated ads, or an ad with information on the parent item.
To match those interface needs, I would have liked to write the following two modules in elm, matching what my API already sends :
module Item.Model exposing (..)
import Ad.Model exposing (Ad)
type alias Item =
{ ads : Maybe List Ad
}
and
module Ad.Model exposing (..)
import Item.Model exposing (Item)
type alias Ad =
{ item : Maybe Item
}
This definition however results in the following dependencies cycle error :
ERROR in ./elm-admin/Main.elm
Module build failed: Error: Compiler process exited with error Compilation failed
Your dependencies form a cycle:
┌─────┐
│ Item.Model
│ ↓
│ Ad.Model
└─────┘
You may need to move some values to a new module to get rid of the cycle.
at ChildProcess.<anonymous> (/opt/app/assets/node_modules/node-elm-compiler/index.js:141:27)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:191:7)
at maybeClose (internal/child_process.js:886:16)
at Socket.<anonymous> (internal/child_process.js:342:11)
at emitOne (events.js:96:13)
at Socket.emit (events.js:188:7)
at Pipe._handle.close [as _onclose] (net.js:497:12)
# ./js/admin.js 3:12-44
It looks like defining both type alias in the same module doesn't make the compiler stop, but this is not a satisfying solution to me because I don't want every Model of my app to end up in the same file.
Is there a way solve this without defining the two type aliases Ad and Item in the same module ?

I can't think of an immediate solution to the question you pose, but I would not store data that way - I would use pointers instead
module Item.Model exposing (..)
type alias Items = Dict String Item
type alias Item =
{ ads : List String
}
-- you don't need a Maybe and a List
-- as the absence of data can be conveyed by the empty list
and
module Ad.Model exposing (..)
import Item.Model exposing (Item)
type alias Ads = Dict String Ad
type alias Ad =
{ itemKey : Maybe String
}
Then you could something like
model.ads
|> L.filterMap (\key -> Dict.get key model.items)
|> ...
model.itemKey
|> Maybe.andThen (\itemKey -> Dict.get itemKey model.ads)
|> ...
Such an approach works, and also offers opportunities for memoisation subsequently

Related

Why did not found return type in Custom UDAF of presto

Custom Function what I made did not work in presto on EMR.
I want to create a simple UDAF that just return 42.
First, my custom function what I wrote a simple functions, but did not work in presto.
A error is following in presto-cli:
presto> select answer_to_life('the universe');
Query 20180324_120433_00000_7n6s6 failed: answer_to_life(varchar):bigint not found
com.facebook.presto.spi.PrestoException: answer_to_life(varchar):bigint not found
at com.facebook.presto.metadata.FunctionRegistry.doGetSpecializedFunctionKey(FunctionRegistry.java:972)
at com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:146)
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3716)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2424)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2298)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2211)
at com.google.common.cache.LocalCache.get(LocalCache.java:4154)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4158)
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5147)
at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5153)
at com.facebook.presto.metadata.FunctionRegistry.getSpecializedFunctionKey(FunctionRegistry.java:898)
at com.facebook.presto.metadata.FunctionRegistry.getAggregateFunctionImplementation(FunctionRegistry.java:875)
at com.facebook.presto.sql.planner.LocalExecutionPlanner$Visitor.buildAccumulatorFactory(LocalExecutionPlanner.java:1973)
at com.facebook.presto.sql.planner.LocalExecutionPlanner$Visitor.planGlobalAggregation(LocalExecutionPlanner.java:1984)
at com.facebook.presto.sql.planner.LocalExecutionPlanner$Visitor.visitAggregation(LocalExecutionPlanner.java:955)
at com.facebook.presto.sql.planner.LocalExecutionPlanner$Visitor.visitAggregation(LocalExecutionPlanner.java:596)
at com.facebook.presto.sql.planner.plan.AggregationNode.accept(AggregationNode.java:167)
at com.facebook.presto.sql.planner.LocalExecutionPlanner$Visitor.visitExchange(LocalExecutionPlanner.java:1919)
at com.facebook.presto.sql.planner.LocalExecutionPlanner$Visitor.visitExchange(LocalExecutionPlanner.java:596)
at com.facebook.presto.sql.planner.plan.ExchangeNode.accept(ExchangeNode.java:196)
at com.facebook.presto.sql.planner.LocalExecutionPlanner$Visitor.visitAggregation(LocalExecutionPlanner.java:952)
at com.facebook.presto.sql.planner.LocalExecutionPlanner$Visitor.visitAggregation(LocalExecutionPlanner.java:596)
at com.facebook.presto.sql.planner.plan.AggregationNode.accept(AggregationNode.java:167)
at com.facebook.presto.sql.planner.LocalExecutionPlanner$Visitor.visitOutput(LocalExecutionPlanner.java:638)
at com.facebook.presto.sql.planner.LocalExecutionPlanner$Visitor.visitOutput(LocalExecutionPlanner.java:596)
at com.facebook.presto.sql.planner.plan.OutputNode.accept(OutputNode.java:82)
at com.facebook.presto.sql.planner.LocalExecutionPlanner.plan(LocalExecutionPlanner.java:393)
at com.facebook.presto.sql.planner.LocalExecutionPlanner.plan(LocalExecutionPlanner.java:324)
at com.facebook.presto.execution.SqlTaskExecution.<init>(SqlTaskExecution.java:161)
at com.facebook.presto.execution.SqlTaskExecution.createSqlTaskExecution(SqlTaskExecution.java:121)
at com.facebook.presto.execution.SqlTaskExecutionFactory.create(SqlTaskExecutionFactory.java:71)
at com.facebook.presto.execution.SqlTask.updateTask(SqlTask.java:340)
at com.facebook.presto.execution.SqlTaskManager.updateTask(SqlTaskManager.java:321)
at com.facebook.presto.server.TaskResource.createOrUpdateTask(TaskResource.java:128)
at sun.reflect.GeneratedMethodAccessor311.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
at org.glassfish.jersey.internal.Errors.process(Errors.java:268)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
at io.airlift.http.server.TraceTokenFilter.doFilter(TraceTokenFilter.java:63)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
at io.airlift.http.server.TimingFilter.doFilter(TimingFilter.java:52)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:454)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:169)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:61)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:564)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:317)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:128)
at org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Invocable.java:222)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:294)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:199)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:673)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:591)
at java.lang.Thread.run(Thread.java:748)
Code of AggregationFunction is following.
I referenced to this presto code
package sample;
import com.facebook.presto.spi.block.BlockBuilder;
import com.facebook.presto.spi.function.*;
import com.facebook.presto.spi.type.BigintType;
import com.facebook.presto.spi.type.StandardTypes;
import io.airlift.slice.Slice;
#AggregationFunction("answer_to_life")
public final class AnswerToLife {
private AnswerToLife() {
}
#InputFunction
public static void input(#AggregationState NullState state, #SqlType(StandardTypes.VARCHAR) Slice value) {
}
#CombineFunction
public static void combine(#AggregationState NullState state, #AggregationState NullState other) {
}
#OutputFunction(StandardTypes.BIGINT)
public static void output(#AggregationState NullState state, BlockBuilder out) {
BigintType.BIGINT.writeLong(out, 42);
}
}
Detail code is here (https://github.com/asari-mtr/presto-udaf/tree/stackoverflow).
The structure what I deployed is follwoing:
$ ls -1 /usr/lib/presto/plugin/my-udaf/
commons-codec-1.4.jar
guava-21.0.jar
hive-udf-1.0-SNAPSHOT.jar
presto-array-0.197.jar
stats-0.155.jar
presto-udaf-1.0-SNAPSHOT.jar
I use emr-5.12.0(Presto 0.188)
Thank you for your time.
Edited1
list in jar file.
% jar -tf target/presto-udaf-1.0-SNAPSHOT.jar
Picked up _JAVA_OPTIONS: -Dfile.encoding=UTF-8
META-INF/
META-INF/MANIFEST.MF
META-INF/services/
sample/
META-INF/services/com.facebook.presto.spi.Plugin
sample/AnswerToLife.class
sample/AnswerToLifePlugin.class
sample/NullState.class
META-INF/maven/
META-INF/maven/sample/
META-INF/maven/sample/presto-udaf/
META-INF/maven/sample/presto-udaf/pom.xml
META-INF/maven/sample/presto-udaf/pom.properties
sever.log
$ grep answer -i /mnt/var/log/presto/server.log
2018-03-26T06:37:14.213Z INFO main com.facebook.presto.server.PluginManager Installing sample.AnswerToLifePlugin
2018-03-26T06:37:14.214Z INFO main com.facebook.presto.server.PluginManager Registering functions from sample.AnswerToLife
And execute show funcstions
presto> show functions;
Function | Return Type | Argument Types
---------------------------------+-----------------------------------------------------+------------------------------------------------------------------------------
ST_Area | double | Geometry
ST_AsText | varchar | Geometry
...
answer_to_life | bigint | varchar
...
Edited2
Delete src/main/resources/META-INF/services/com.facebook.presto.spi.Plugin
Add <packaging>presto-plugin</packaging> to pom.xml
https://github.com/asari-mtr/presto-udaf/commit/f2a2ddbf0339e08f418b378a7ead511020e98a3b
I deployed zip made by Maven under /usr/lib/presto/plugin/.
But, The contents of the error does not change.
Edited3
I got the source from github (branch 0.188) on my Mac and built presto.
When we placed the above UDAF on its presto, it worked perfectly.
Perhaps there is a mistake in the installation procedure for presto on EMR.
Solved!!
The Custom plugin did not work because it was not a plugin creation, but a way of deploying on EMR was wrong.
Since I did not deploy to all nodes in the procedure I performed, I confirmed the operation by registering the following shell as a bootstrap action.
#/bin/bash
aws s3 cp s3://hogehoge/presto-udaf-1.0-SNAPSHOT.zip /tmp/
sudo mkdir -p /usr/lib/presto/plugin/
sudo unzip -d /usr/lib/presto/plugin/ /tmp/presto-udaf-1.0-SNAPSHOT.zip
10.1. SPI Overview — Presto 0.198 Documentation
Plugins must be installed on all nodes in the Presto cluster
(coordinator and workers).
Create Bootstrap Actions to Install Additional Software - Amazon EMR
You can use a bootstrap action to copy objects from Amazon S3 to each
node in a cluster before your applications are installed. The AWS CLI
is installed on each node of a cluster, so your bootstrap action can
call AWS CLI commands.
From what you wrote there are two things which are missing.
make sure that you used maven packaging presto-plugin in your pom.xml
also implement com.facebook.presto.spi.Plugin which in getFunctions method would return your function.
Take a look in presto-geospatial Presto module and see com.facebook.presto.plugin.geospatial.GeoPlugin and its pom.xml as reference.

Gulp group media error "must start with number, buffer, array or string"

I use
var cmq = require('gulp-combine-media-queries')
gulp.task('cmq', function () {
gulp.src('css/style.css')
.pipe(cmq({
log: true
}))
.pipe(gulp.dest('dist'));
});
And this error is output after gulp command:
File css/style.css found.
buffer.js:169
throw new TypeError('must start with number, buffer, array or
string');
^
TypeError: must start with number, buffer, array or string
at fromObject (buffer.js:169:9)
at new Buffer (buffer.js:62:10)
at Transform.transform [as _transform] (/home/user/node_modules/gulp-
combine-media-queries/index.js:152:21)
at Transform._read (/home/user/node_modules/gulp-combine-media-
queries/node_modules/readable-stream/lib/_stream_transform.js:184:10)
at Transform._write (/home/user/node_modules/gulp-combine-media-
queries/node_modules/readable-stream/lib/_stream_transform.js:172:12)
at doWrite (/home/user/node_modules/gulp-combine-media-
queries/node_modules/readable-stream/lib/_stream_writable.js:237:10)
at writeOrBuffer (/home/user/node_modules/gulp-combine-media-
queries/node_modules/readable-stream/lib/_stream_writable.js:227:5)
at Transform.Writable.write (/home/user/node_modules/gulp-combine-
media-queries/node_modules/readable-
stream/lib/_stream_writable.js:194:11)
at write (/home/user/node_modules/vinyl-fs/node_modules/readable-
stream/lib/_stream_readable.js:623:24)
at flow (/home/user/node_modules/vinyl-fs/node_modules/readable-
stream/lib/_stream_readable.js:632:7)
at DestroyableTransform.pipeOnReadable (/home/user/node_modules/vinyl-
fs/node_modules/readable-stream/lib/_stream_readable.js:664:5)
at emitNone (events.js:67:13)
at DestroyableTransform.emit (events.js:166:7)
at emitReadable_ (/home/user/node_modules/vinyl-
fs/node_modules/readable-stream/lib/_stream_readable.js:448:10)
at emitReadable (/home/user/node_modules/vinyl-
fs/node_modules/readable-stream/lib/_stream_readable.js:444:5)
at readableAddChunk (/home/user/node_modules/vinyl-
fs/node_modules/readable-stream/lib/_stream_readable.js:187:9)
Node - 4.2.6, Gulp 3.9.1, npm - 3.5.2
I also used another npms for grouping media and have the same errors. I read many articles with the same problems, but none has helped
looking at media-query code (https://github.com/konitter/gulp-combine-media-queries/blob/master/index.js)
I could see that your file "css/style.css" is not accessible by gulp hence the error.
Please recheck the path and try again

Robot Framework Variable Class File with triple Nested Dictionary is not dot notation accessible

Using the Robot Framework Documentation on Variable Files as a guide I implemented a Variable File Class with the get_variables. The basic example works as described.
When I implement a triple nested Dictionary (${A.B.C}) I can access the first two using ${A} and ${A.B} notation. However, when I want to access the third node ${A.B.C} the result is that I get an error
Resolving variable '${A.B.C}' failed: AttributeError: 'OrderedDict' object
has no attribute 'C'
In the below three examples I have an RF generated nested dictionary that I can access all nodes through the dot notation. The second example is a plain Python dictionary that is given back by the Variable Class. In the last example the Variable class returned is of the type OrderedDict.
Although the ${A['B']['C']['key']} works, it is more difficult to read in the code. When I load a similar yaml structure it supports the dot notation fully but this is not an option as the yaml file is static, and I require flexibility of Python for some of the key values.
So, I'm looking for some support on how to return a data structure that allows Robot Framework to interpret the full nested structure with the dot notation.
Variable Class File
from collections import OrderedDict
class OrderDict(object):
def get_variables(self):
C = OrderedDict([(u'key', u'value')])
B = OrderedDict([(u'C', C)])
A = OrderedDict([(u'B', B)])
D = {
u'E':
{
u'F':
{
u'key': u'value'
}
}
}
return OrderedDict([(u'DICT__A', A), (u'DICT__D', D)])
Robot Framework Script
*** Test Cases ***
Dictionary RF
${Z} Create Dictionary key=value
${Y} Create Dictionary Z=${Z}
${X} Create Dictionary Y=${Y}
Log To Console ${EMPTY}
Log To Console ${X}
Log To Console ${X['Y']['Z']['key']}
Log To Console ${X.Y}
Log To Console ${X.Y.Z}
Log To Console ${X.Y.Z.key}
Plain Dictionary Variable Class
Log To Console ${EMPTY}
Log To Console ${D}
Log To Console ${D['E']['F']['key']}
Log To Console ${D.E}
Log To Console ${D.E.F}
Log To Console ${D.E.F.key}
Ordered Dictionary Variable Class
Log To Console ${EMPTY}
Log To Console ${A}
Log To Console ${A['B']['C']['key']}
Log To Console ${A.B}
Log To Console ${A.B.C}
Log To Console ${A.B.C.key}
Robot Framework Console Log
Suite Executor: Robot Framework 3.0.2 (Python 2.7.9 on win32)
Dictionary RF
{u'Y': {u'Z': {u'key': u'value'}}}
value
{u'Z': {u'key': u'value'}}
{u'key': u'value'}
value
| PASS |
------------------------------------------------------------------------------
Plain Dictionary Variable Class
{u'E': {u'F': {u'key': u'value'}}}
value
{u'F': {u'key': u'value'}}
| FAIL |
Resolving variable '${D.E.F}' failed: AttributeError: 'dict' object has no attribute 'F'
------------------------------------------------------------------------------
Ordered Dictionary Variable Class
{u'B': OrderedDict([(u'C', OrderedDict([(u'key', u'value')]))])}
value
OrderedDict([(u'C', OrderedDict([(u'key', u'value')]))])
| FAIL |
Resolving variable '${A.B.C}' failed: AttributeError: 'OrderedDict' object has no
attribute 'C'
In the Robot Framework Slack channel Pekka Klarck pointed out that Robot Framework internally uses the robot.utils.DotDic class. Having get_variables() return a DotDic structure resolved my issue and I can now use the dot notation. Below is the code for the Variable Class DotDic (stored as DotDic.py).
from robot.utils import DotDict
class DotDic(object):
def get_variables(self):
G = {
u'H':
{
u'I':
{
u'key': u'value'
}
}
}
return {u'G': self.dict_to_dotdict(G)}
def dict_to_dotdict(self, dct):
dd = DotDict({})
for key, val in dct.items():
if isinstance(val, dict):
dd[key] = self.dict_to_dotdict(val)
else:
dd[key] = val
return dd

Create a portal_user_catalog and have it used (Plone)

I'm creating a fork of my Plone site (which has not been forked for a long time). This site has a special catalog object for user profiles (a special Archetypes-based object type) which is called portal_user_catalog:
$ bin/instance debug
>>> portal = app.Plone
>>> print [d for d in portal.objectMap() if d['meta_type'] == 'Plone Catalog Tool']
[{'meta_type': 'Plone Catalog Tool', 'id': 'portal_catalog'},
{'meta_type': 'Plone Catalog Tool', 'id': 'portal_user_catalog'}]
This looks reasonable because the user profiles don't have most of the indexes of the "normal" objects, but have a small set of own indexes.
Since I found no way how to create this object from scratch, I exported it from the old site (as portal_user_catalog.zexp) and imported it in the new site. This seemed to work, but I can't add objects to the imported catalog, not even by explicitly calling the catalog_object method. Instead, the user profiles are added to the standard portal_catalog.
Now I found a module in my product which seems to serve the purpose (Products/myproduct/exportimport/catalog.py):
"""Catalog tool setup handlers.
$Id: catalog.py 77004 2007-06-24 08:57:54Z yuppie $
"""
from Products.GenericSetup.utils import exportObjects
from Products.GenericSetup.utils import importObjects
from Products.CMFCore.utils import getToolByName
from zope.component import queryMultiAdapter
from Products.GenericSetup.interfaces import IBody
def importCatalogTool(context):
"""Import catalog tool.
"""
site = context.getSite()
obj = getToolByName(site, 'portal_user_catalog')
parent_path=''
if obj and not obj():
importer = queryMultiAdapter((obj, context), IBody)
path = '%s%s' % (parent_path, obj.getId().replace(' ', '_'))
__traceback_info__ = path
print [importer]
if importer:
print importer.name
if importer.name:
path = '%s%s' % (parent_path, 'usercatalog')
print path
filename = '%s%s' % (path, importer.suffix)
print filename
body = context.readDataFile(filename)
if body is not None:
importer.filename = filename # for error reporting
importer.body = body
if getattr(obj, 'objectValues', False):
for sub in obj.objectValues():
importObjects(sub, path+'/', context)
def exportCatalogTool(context):
"""Export catalog tool.
"""
site = context.getSite()
obj = getToolByName(site, 'portal_user_catalog', None)
if tool is None:
logger = context.getLogger('catalog')
logger.info('Nothing to export.')
return
parent_path=''
exporter = queryMultiAdapter((obj, context), IBody)
path = '%s%s' % (parent_path, obj.getId().replace(' ', '_'))
if exporter:
if exporter.name:
path = '%s%s' % (parent_path, 'usercatalog')
filename = '%s%s' % (path, exporter.suffix)
body = exporter.body
if body is not None:
context.writeDataFile(filename, body, exporter.mime_type)
if getattr(obj, 'objectValues', False):
for sub in obj.objectValues():
exportObjects(sub, path+'/', context)
I tried to use it, but I have no idea how it is supposed to be done;
I can't call it TTW (should I try to publish the methods?!).
I tried it in a debug session:
$ bin/instance debug
>>> portal = app.Plone
>>> from Products.myproduct.exportimport.catalog import exportCatalogTool
>>> exportCatalogTool(portal)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File ".../Products/myproduct/exportimport/catalog.py", line 58, in exportCatalogTool
site = context.getSite()
AttributeError: getSite
So, if this is the way to go, it looks like I need a "real" context.
Update: To get this context, I tried an External Method:
# -*- coding: utf-8 -*-
from Products.myproduct.exportimport.catalog import exportCatalogTool
from pdb import set_trace
def p(dt, dd):
print '%-16s%s' % (dt+':', dd)
def main(self):
"""
Export the portal_user_catalog
"""
g = globals()
print '#' * 79
for a in ('__package__', '__module__'):
if a in g:
p(a, g[a])
p('self', self)
set_trace()
exportCatalogTool(self)
However, wenn I called it, I got the same <PloneSite at /Plone> object as the argument to the main function, which didn't have the getSite attribute. Perhaps my site doesn't call such External Methods correctly?
Or would I need to mention this module somehow in my configure.zcml, but how? I searched my directory tree (especially below Products/myproduct/profiles) for exportimport, the module name, and several other strings, but I couldn't find anything; perhaps there has been an integration once but was broken ...
So how do I make this portal_user_catalog work?
Thank you!
Update: Another debug session suggests the source of the problem to be some transaction matter:
>>> portal = app.Plone
>>> puc = portal.portal_user_catalog
>>> puc._catalog()
[]
>>> profiles_folder = portal.some_folder_with_profiles
>>> for o in profiles_folder.objectValues():
... puc.catalog_object(o)
...
>>> puc._catalog()
[<Products.ZCatalog.Catalog.mybrains object at 0x69ff8d8>, ...]
This population of the portal_user_catalog doesn't persist; after termination of the debug session and starting fg, the brains are gone.
It looks like the problem was indeed related with transactions.
I had
import transaction
...
class Browser(BrowserView):
...
def processNewUser(self):
....
transaction.commit()
before, but apparently this was not good enough (and/or perhaps not done correctly).
Now I start the transaction explicitly with transaction.begin(), save intermediate results with transaction.savepoint(), abort the transaction explicitly with transaction.abort() in case of errors (try / except), and have exactly one transaction.commit() at the end, in the case of success. Everything seems to work.
Of course, Plone still doesn't take this non-standard catalog into account; when I "clear and rebuild" it, it is empty afterwards. But for my application it works well enough.

SBT Subprojects do not recognize plugin commands

I'm having an issue with getting SBT Subprojects to recognize commands provided by plugins. I have the following plugin source:
object DemoPlugin extends AutoPlugin {
override lazy val projectSettings = Seq(commands += demoCommand)
lazy val demoCommand =
Command.command("demo") { (state: State) =>
println("Demo Plugin!")
state
}
}
Which is used by a project configured as follows:
lazy val root = project in file(".")
lazy val sub = (project in file("sub")).
enablePlugins(DemoPlugin).
settings(
//...
)
The plugin is, of course, listed in project/plugins.sbt. However, when I open up sbt in the project, I see the following:
> sub/commands
[info] List(sbt.SimpleCommand#413d2cd1)
> sub/demo
[error] Expected ':' (if selecting a configuration)
[error] Not a valid key: demo (similar: doc)
[error] sub/demo
Even stranger, using consoleProject, I can see that the command in the project is the one defined by DemoPlugin!
scala> (commands in sub).eval.map { c => c.getClass.getMethod("name").invoke(c) }
res0: Seq[Object] = List(demo)
I'm looking to be able to type sub/demo, and have it perform the demo command. Any help would be much appreciated!
Commands aren't per-project. They only work for the top-level project.
It's also recommended to try and use tasks, or if needed input tasks where you might want to use a command.
If you really need a command, there's a way to have a sort of "holder" task, see the answer to Can you access a SBT SettingKey inside a Command?

Resources