How to use use ndb library schemeless - google-cloud-datastore

using datastore client library.
Can be used regardless of the state of the datastore property.
datastore_client.put({
"foo":"foo",
"bar":"bar",
"xxx":"xxx", # any property can put
...
})
But, using ndb.
Only predefined properties can be used.
Class Test(ndb.Model):
foo = StringProperty()
...
Test(
foo="foo",
bar="bar" # error
).put()
If I want to put()/get() schemeless.
I cannot use ndb?

Look for ndb.Expando .This is what you want

Related

why airflow googlecloudstorageobjectsensor is not serching for the partial object names (eg:file*.csv)

gcs_sensor in airflow is not working for the partial object name.
for example in object I have given myfile* but it is not working.
can you suggest a solution that takes partial names to search in the google cloud storage
file_watcher = GoogleCloudStorageObjectSensor(
task_id='filesensor',
bucket='poc-1',
object='myfile*',
google_cloud_conn_id='google_cloud_default',
dag=example_dag
)
You may want to look at GoogleCloudStoragePrefixSensor.
Reference:
https://airflow.apache.org/_api/airflow/contrib/sensors/gcs_sensor/index.html?highlight=googlecloudstorageprefixsensor#airflow.contrib.sensors.gcs_sensor.GoogleCloudStoragePrefixSensor
file_watcher = GoogleCloudStoragePrefixSensor(
task_id='filesensor',
bucket='poc-1',
prefix='myfile',
google_cloud_conn_id='google_cloud_default',
dag=example_dag
)

Sending vector data in the bus

I have a vector data (an array variable for example float32 mydata[5];). for transmitting a single primitve/basic data in a bus its pretty simple.
inside_data=Simulink.BusElement;
inside_data.Name='somename';
inside_data.SampleTime = -1;
inside_data.datatype='single';
this element can be put inside a using
Bus=Simulink.Bus;
Bus.Elements=inside_data;
But this works when the input is a primitive. But what if my data is a vector. like float32 a[5]; then how can i send this data element in a bus.
UPDATE
So I tried to use a constant block named a with datatype single in which the input part i changed it as [1 2 3] which is a vector input.
another element is b with uint8 datatype.
i used the s-function builder just to check the working of this model. i already set everything (bus_mode on , datatype to be bus type etc). in the output part i used something like:
y0[0]=u0->a[0];
y0[1]=u0->a[1];
y0[2]=u0->a[2];
y1[0]=u0->b;
But it throws error as
c:\program files (x86)\matlab_v7111_r10bsp1\extern\include\matrix.h(313) : error C2061: syntax error : identifier 'mxLogical'
c:\program files (x86)\matlab_v7111_r10bsp1\extern\include\matrix.h(313) : error C2059: syntax error : ';'
my final aim is to use it for s_function
so if i declare a variable in s_func as
real32_T *a_output[5]=(real32_T *)ssGetOutputPortRealSignal(S,0);
and then i have a strcuture(because am transmitting data with a bus so the bus header file has this structure) and how do i declare and assign the input to the output.
a_output[0]=my_struct->a_input[0];
a_output[1]=my_struct->a_input[1];
a_output[2]=my_struct->a_input[2];
a_output[3]=my_struct->a_input[3];
a_output[4]=my_struct->a_input[4];
but the problem is with the declaration. it gives me error cannot convert from real32_T to real32_T * .
The main idea is to create Bus of type you need.
I did it this way:
num = zeros(15,15,15);
move = zeros(15,15,15);
a = struct('number',num,'movement', move);
busInfo = Simulink.Bus.createObject(a);
You can see it's possible to create any data structure, array, vector anything you want and then create Bus Signal of the same type.
I use this in callbacks/preLoadFcn (Model Explorer) to define this type in workspace, it creates slBus1 variable (its Bus Signal of my type), so I need to define output (or input if necessary) of any block like slBus1 only. And then use Bus Selector to work array data.
Can it helps to you?
ADD NEW INFORMATION
It depends of what you want.
For example: I create s-function for feedback system. It use my structure like this:
function a = fcn(BusSignal)
%#codegen
num = zeros(15,15,15);
move = zeros(15,15,15);
%...some math actions with num and move...
a = struct('number',num,'movement', move);
%...and some action with a structure... for example:
if (c>b)
a.number(1,2,3) = 15;
a.movement(1,2,3) = 42;
else
a = BusSignal;
end
Look at this - I use Bus signal at entrance and at exit and use Bus Selector to work it's data with.
REMEMBER to define input and output data as Bus Signals!

Grokless way for dynamic vocabularies?

I have a python-script, which returns returns a context-based, dynamically generated simple list:
def myVocabMethod(self):
mylist = ['a','list','apart']
# do sth dynamic
return mylist
I would like to pass the result to the selection-field with the vocabulary-attribute looking s.th. like this:
atapi.StringField('theField'
vocabulary=.myPythonScript.myVocabMethod(),
(...)
),
How to glue the script-results and and the vocab-value together?
The documentation I found, always requires Grok. Also it's just a simple list, no i18n or other more complex features needed.
Grokless way to register a named vocabulary:
http://developer.plone.org/forms/vocabularies.html#registering-a-named-vocabulary-provider-in-zcml
Basically you point it to a function which returns SimpleVocabulary instance.
The post, where I found what I was looking for is this one:
http://www.universalwebservices.net/web-programming-resources/zope-plone/dynamic-vocabularies-in-plone-archetypes/
And is referenced in the official docs here:
http://developer.plone.org/content/archetypes/fields.html#dynamic-vocabularies
For anyone who might be interested, this is the code:
from Acquisition import aq_parent
from Products.Archetypes import atapi
from Products.Archetypes.public import DisplayList
YourArchetypeSchema = schemata.ATContentTypeSchema.copy() + atapi.Schema((
atapi.StringField(
'somefieldname',
vocabulary = yourVocabulary,
),
))
class YourArchetype(base.ATCTContent):
def yourVocabulary(self):
dl = DisplayList()
# We can do something context-based here,
# f.e. get all Events in parent-folder:
eventlist = aq_parent(self).contentValues(filter={'portal_type' : 'Event'})
for event in eventlist:
dl.add(event['id'], event['title'])
return dl
atapi.registerType(YourArchetype, PROJECTNAME)

What's wrong with my filter query to figure out if a key is a member of a list(db.key) property?

I'm having trouble retrieving a filtered list from google app engine datastore (using python for server side). My data entity is defined as the following
class Course_Table(db.Model):
course_name = db.StringProperty(required=True, indexed=True)
....
head_tags_1=db.ListProperty(db.Key)
So the head_tags_1 property is a list of keys (which are the keys to a different entity called Headings_1).
I'm in the Handler below to spin through my Course_Table entity to filter the courses that have a particular Headings_1 key as a member of the head_tags_1 property. However, it doesn't seem like it is retrieving anything when I know there is data there to fulfill the request since it never displays the logs below when I go back to iterate through the results of my query (below). Any ideas of what I'm doing wrong?
def get(self,level_num,h_key):
path = []
if level_num == "1":
q = Course_Table.all().filter("head_tags_1 =", h_key)
for each in q:
logging.info('going through courses with this heading name')
logging.info("course name filtered is %s ", each.course_name)
MANY MANY THANK YOUS
I assume h_key is key of headings_1, since head_tags_1 is a list, I believe what you need is IN operator. https://developers.google.com/appengine/docs/python/datastore/queries
Note: your indentation inside the for loop does not seem correct.
My bad apparently '=' for list is already check membership. Using = to check membership is working for me, can you make sure h_key is really a datastore key class?
Here is my example, the first get produces result, where the 2nd one is not
import webapp2 from google.appengine.ext import db
class Greeting(db.Model):
author = db.StringProperty()
x = db.ListProperty(db.Key)
class C(db.Model): name = db.StringProperty()
class MainPage(webapp2.RequestHandler):
def get(self):
ckey = db.Key.from_path('C', 'abc')
dkey = db.Key.from_path('C', 'def')
ekey = db.Key.from_path('C', 'ghi')
Greeting(author='xxx', x=[ckey, dkey]).put()
x = Greeting.all().filter('x =',ckey).get()
self.response.write(x and x.author or 'None')
x = Greeting.all().filter('x =',ekey).get()
self.response.write(x and x.author or 'None')
app = webapp2.WSGIApplication([('/', MainPage)],
debug=True)

rJava: using java/lang/Vector with a certain template class

I'm currently programming an R-script which uses a java .jar that makes use of the java/lang/Vector class, which in this case uses a class in a method that is not native. In java source code:
public static Vector<ClassName> methodname(String param)
I found nothing in the documentation of rJava on how to handle a template class like vector and what to write when using jcall or any other method.
I'm currently trying to do something like this:
v <- .jnew("java/util/Vector")
b <- .jcall(v, returnSig = "Ljava/util/Vector", method = "methodname",param)
but R obviously throws an exception:
method methodname with signature (Ljava/lang/String;)Ljava/util/Vector not found
How do I work the template class into this command? Or for that matter, how do I create a vector of a certain class in the first place? Is this possible?
rJava does not know java generics, there is no syntax that will create a Vector of a given type. You can only create Vectors of Objects.
Why are you sticking with the old .jcall api when you can use the J system, which lets you use java objects much more nicely:
> v <- new( J("java.util.Vector") )
> v$add( 1:10 )
[1] TRUE
> v$size()
[1] 1
# code completion
> v$
v$add( v$getClass() v$removeElement(
v$addAll( v$hashCode() v$removeElementAt(
v$addElement( v$indexOf( v$retainAll(
v$capacity() v$insertElementAt( v$set(
v$clear() v$isEmpty() v$setElementAt(
v$clone() v$iterator() v$setSize(
v$contains( v$lastElement() v$size()
v$containsAll( v$lastIndexOf( v$subList(
v$copyInto( v$listIterator( v$toArray(
v$elementAt( v$listIterator() v$toArray()
v$elements() v$notify() v$toString()
v$ensureCapacity( v$notifyAll() v$trimToSize()
v$equals( v$remove( v$wait(
v$firstElement() v$removeAll( v$wait()
v$get( v$removeAllElements()

Resources