I am using gcloud-python library for querying data from the cloud datastore. Consider my snippet to be like this
from google.appengine.ext import ndb
from datetime import datetime
class Order(ndb.Model):
order_name = ndb.StringProperty(required=True)
date_created = ndb.DateTimeProperty(default= datetime.now())
#code for querying the cloud datastore
from gcloud.datastore.query import Query
date_start = datetime.combine(date(year=2015, month=08, day=01), time())
date_end = datetime.combine(date(year=2015, month=08, day=03), time())
query = Query(kind='Order')
query.add_filter('order_name', '=', 'grand-line-order')
query.add_filter('date_created', '<', date_end)
query.add_filter('date_created', '>', date_start)
iterator = query.fetch(limit=10)
records, more, cursor = iterator.next_page()
print records
For the above snippet i am getting
File "/Users/sathyanarrayanan/Desktop/app/services/cdr_helper.py", line 528, in fetch_cdr
records, more, cursor = iterator.next_page()
File "/Users/sathyanarrayanan/Desktop/app/gcloud/datastore/query.py", line 388, in next_page
transaction_id=transaction and transaction.id,
File "/Users/sathyanarrayanan/Desktop/app/gcloud/datastore/connection.py", line 257, in run_query
datastore_pb.RunQueryResponse)
File "/Users/sathyanarrayanan/Desktop/app/gcloud/datastore/connection.py", line 108, in _rpc
data=request_pb.SerializeToString())
File "/Users/sathyanarrayanan/Desktop/app/gcloud/datastore/connection.py", line 85, in _request
raise make_exception(headers, content, use_json=False)
PreconditionFailed: 412 no matching index found.
My Index.yaml file is like this.
indexes:
- kind: Order
ancestor: yes
properties:
- name: date_created
- kind: Order
ancestor: yes
properties:
- name: date_created
direction: desc
- kind: Order
ancestor: yes
properties:
- name: order_name
direction: asc
- name: date_created
direction: desc
- kind: Order
ancestor: yes
properties:
- name: order_name
direction: asc
- name: date_created
direction: asc
Am I doing something wrong? Please help me out.
All of your indexes using ancestor:yes so ancestor key should be added in your query. without ancestor your index configuration require another index with 'ancestor:no'
- kind: Order
ancestor: no
properties:
- name: order_name
direction: asc
- name: date_created
direction: desc
Note: specific indexes for each query
The index configuration docs indicate that the index configuration should be in an XML file called datastore-indexes.xml.
Related
Here is my YAML:
---
version: 1.0
employee:
name:
employee1
headers:
I have the dict_data as below:-
dict_data = {'Key1': 'value1', 'Key2': 'value2}
Now we have to append this dict to headers attribute in yaml.
And the yaml should look like below:-
---
version: 1.0
employee:
name:
employee1
headers:
Key1: value1
Key2: value2
I tried doing this with PyYaml using update keyword and dumped only the dict in yaml file and removed all other contents.
You can read, update and rewrite as,
with open('filename.yaml','r') as yamlfile:
current_yaml = yaml.safe_load(yamlfile)
current_yaml['headers'].update(dict_data)
if current_yaml:
with open('filename.yaml','w') as yamlfile:
yaml.safe_dump(current_yaml, yamlfile)
I have a query with hardcoded dates used in the parameters section.Instead I want to pass them as environment variables.Any suggestions on how to parameterize the QueryString parameter?
service: service-name
frameworkVersion: '2'
provider:
name: aws
runtime: go1.x
lambdaHashingVersion: 20201221
stage: ${opt:stage, self:custom.defaultStage}
region: us-east-1
tags: ${self:custom.tagsObject}
logRetentionInDays: 1
timeout: 10
deploymentBucket: lambda-repository
memorySize: 128
tracing:
lambda: true
plugins:
- serverless-step-functions
configValidationMode: error
stepFunctions:
stateMachines:
callAthena:
name: datasorting-dev
type: STANDARD
role: ${self:custom.datasorting.${self:provider.stage}.iam}
definition:
Comment: "Data Refersh"
StartAt: Refresh Data
States:
Refresh Data:
Type: Task
Resource: arn:aws:states:::athena:startQueryExecution.sync
Parameters:
QueryString: >-
ALTER TABLE table.raw_data ADD IF NOT EXISTS
PARTITION (YEAR=2021, MONTH=02, DAY=15, hour=00)
WorkGroup: primary
ResultConfiguration:
OutputLocation: s3://output/location
End: true
you can replace any value in your serverless.yml enclosed in ${} brackets,
Serverless Framework Guide to Variables:
https://www.serverless.com/framework/docs/providers/aws/guide/variables/
for example, you can create a custom: section looking for environment variables, and if they are not present, you can have default values:
service: service-name
frameworkVersion: '2'
custom:
year: ${env:YEAR, 'default-year'}
month: ${env:MONTH, 'default-month'}
day: ${env:DAY, 'default-day'}
hour: ${env:HOUR, 'default-hour'}
stepFunctions:
stateMachines:
callAthena:
...
Parameters:
QueryString: >-
ALTER TABLE table.raw_data ADD IF NOT EXISTS
PARTITION (YEAR=${self:custom.year}, MONTH=${self:custom.month}, DAY=${self:custom.day}, hour=${self:custom.hour})
...
I am trying to write a HOT template for Openstack volume, and need to have the volume_type as a parameter. I also need to support a case when the parameter is not given, and default to the Cinder default volume type.
First attempt was to pass null to the volume_type , hoping it would give the default volume type. However no matter what I pass (null, ~, default, "" ) , seems there is no way to get the default volume type.
type: OS::Cinder::Volume
properties:
name: test
size: 1
volume_type: { if: ["voltype_given" , {get_param:[typename]} , null] }
Is there any way to get the default volume type , when you have the "volume_type" property defined?
Alternatively, is there any way to have the "volume_type" property itself behind a conditional? I tried several ways, but no luck. Something like:
type: OS::Cinder::Volume
properties:
if: ["voltype_given" , [ volume_type: {get_param:[typename]} ] , ""]
name: test
size: 1
ERROR: TypeError: : resources.kk-test-vol: : 'If' object is not iterable
Could you do something like this?
---
parameters:
typename:
type: string
conditions:
use_default_type: {equals: [{get_param: typename}, '']}
resources:
MyVolumeWithDefault:
condition: use_default_type
type: OS::Cinder::Volume
properties:
name: test
size: 1
MyVolumeWithExplicit:
condition: {not: use_default_type}
type: OS::Cinder::Volume
properties:
name: test
size: 1
volume_type: {get_param: typename}
# e.g. if you need to refer to the volume from another resource
MyVolumeAttachment:
type: OS::Cinder::VolumeAttachment
properties:
instance_uid: some-instance-uuid
volume_id:
if:
- use_default_type
- get_resource: MyVolumeWithDefault
- get_resource: MyVolumeWithExplicit
I' following a symfony2 tutorial and I've problems with one step.
Tutorial link: http://intelligentbee.com/blog/2013/08/12/symfony2-jobeet-day-6-more-with-the-model/
I'm in 'Refactoring' step. I've one this 3 steps:
1- I've correctly modified /src/Ibw/JobeetBundle/Resources/config/doctrine/Job.orm.yml file specifiying the repository
2- I've run the command: php app/console doctrine:generate:entities IbwJobeetBundle
3- And I've added the specified tutorial function por JobRepository.php
$qb = $this->createQueryBuilder('j')
->where('j.expires_at > :date')
->setParameter('date', date('Y-m-d H:i:s', time()))
->orderBy('j.expires_at', 'DESC');
BUT when I refresh my code I get this error:
An exception occurred while executing 'SELECT j0_.id AS id0, j0_.type
AS type1, j0_.company AS company2, j0_.logo AS logo3, j0_.url AS url4,
j0_.position AS position5, j0_.location AS location6, j0_.description
AS description7, j0_.how_to_apply AS how_to_apply8, j0_.token AS
token9, j0_.is_public AS is_public10, j0_.is_activated AS
is_activated11, j0_.email AS email12, j0_.expires_at AS expires_at13,
j0_.created_at AS created_at14, j0_.updated_at AS updated_at15,
j0_.category_id AS category_id16 FROM Job j0_ WHERE j0_.expires_at > ?
ORDER BY j0_.expires_at DESC' with params ["2016-03-17 15:47:19"]:
SQLSTATE[42S02]: Base table or view not found: 1146 Table 'jobeet.Job'
doesn't exist
In the symfony profiler I can see the full query:
SELECT j0_.id AS id0, j0_.type AS type1, j0_.company AS company2,
j0_.logo AS logo3, j0_.url AS url4, j0_.position AS position5,
j0_.location AS location6, j0_.description AS description7,
j0_.how_to_apply AS how_to_apply8, j0_.token AS token9, j0_.is_public
AS is_public10, j0_.is_activated AS is_activated11, j0_.email AS
email12, j0_.expires_at AS expires_at13, j0_.created_at AS
created_at14, j0_.updated_at AS updated_at15, j0_.category_id AS
category_id16 FROM Job j0_ WHERE j0_.expires_at > '2016-03-17
15:47:19' ORDER BY j0_.expires_at DESC
the table name it's in uppercase! 'Job', not 'job'
Any can help me, please?
You can try by modifying the entity definition in Job.orm.yml by adding the table attribute:
IbwJobeetBundleEntityJob:
type: entity
table: job
repositoryClass: IbwJobeetBundleRepositoryJobRepository
Consider my SLS file,
state1:
cmd.run:
- order: 1
- name: |
USER_NAME='username'
USERPWD='password'
DB_NAME='test'
USER_TOBE_CREATED='new_user'
PASSWORD='newpass'
mysql_user.present:
- order: 2
- host: localhost
- username: USER_TOBE_CREATED
- password: PASSWORD
- connection_user: USER_NAME
- connection_pass: USERPWD
- connection_charset: utf8
- saltenv:
- LC_ALL: "en_US.utf8"
mysql_grants.present:
- order: 3
- grant: all privileges
- database: DB_NAME.*
- user: USER_TOBE_CREATED
In the states mysql_user.present and mysql_grants.present I am using the variables USER_TOBE_CREATED,USER_NAME,USERPWD etc whose values are assigned in state cmd.run. How will I make these two following states to use the actual values of those variables?. Here it's taking variable name itself as the value.
You may want to declare the variables in the state file itself, i.e.:
{% set user_name = "name" %}
and:
state1:
cmd.run:
- order: 1
- name: |
USER_NAME='{{ user_name }}'
You can re-use the variable as many times as you want inside the state file.
Let me know if this helped.