I am looking for a general solution to a problem with couchdb views.
For example, have a view result like this:
{"total_rows":4,"offset":0,"rows":[
{"id":"1","key":["imported","1"],"value":null},
{"id":"2","key":["imported","2"],"value":null},
{"id":"3","key":["imported","3"],"value":null},
{"id":"4","key":["mapped","4"],"value":null},
{"id":"5,"key":["mapped","5"],"value":null}
]
1) If I want to select only "imported" documents I would use this:
view?startkey=["imported"]&endkey=["imported",{}]
2) If I want to select all imported documents with an higher id then 2:
view?startkey=["imported",2]&endkey=["imported",{}]
3) If I want to select all imported documents with an id between 2 and 4:
view?startkey=["imported",2]&endkey=["imported",4]
My Questtion is: How can I select all Rows with an id between 2 and 4?
You can try to extend the solution above, but prepend keys with a kind of "emit index" flag like this:
map: function (doc) {
emit ([0, doc.number, doc.category]); // direct order
emit ([1, doc.category, doc.number]); // reverse order
}
so you will be able to request them with
view?startkey=[0, 2]&endkey=[0, 4, {}]
or
view?startkey=[1, 'imported', 2]&endkey=[1, 'imported', 4]
But 2 different views will be better anyway.
I ran into the same problem a little while ago so I'll explain my solution. Inside of any map function you can have multiple emit() calls. A map function in your case might look like:
function(doc) {
emit([doc.number, doc.category], null);
emit([doc.category, doc.number], null);
}
You can also use ?include_docs=true to get the documents back from any of your queries. Then your query to get back rows 2 to 4 would be
view?startkey=[2]&endkey=[4,{}]
You can view the rules for sorting at CouchDB View Collation
Related
I am using #paginator directive on my query and my client wants to get all records of posts from the query. This is my code:
posts: [Post!]! #paginate
I tested this querys:
posts(first:0) {id} #works but don't get all records
posts(first:-1) {id} #error
One way was to get all records was to use the value of total inside the paginatorInfo and make a new query with that value on the first:.
posts(first:0) {
paginatorInfo {
total
}
}
For optimization making 2 querys to get all records is very bad.
The best approach I got was to make a new query (now having two querys for posts with different directives and names) like:
allPosts: [Post!]! #all
Other approaches but not so clean:
Set pagination.max_count to null in the config/lighthouse.php and do (first: 100000000) (the int is 32 bits so has a limit).
Change the #paginantor to a #field(resolver:) and do pagination with pure php.
I'm trying to use SQL IN clause kind of feature in dynamoDB. I tried using withFilterExpression but I'm not sure how to do it. I looked at similar questions as they were too old. Is there a better method to do this? This is the segment of code I have got. I have used a static List as example but it is actually dynamic.
def getQuestionItems(conceptCode : String) = {
val qIds = List("1","2","3")
val querySpec = new QuerySpec()
.withKeyConditionExpression("concept_id = :c_id")
.withFilterExpression("question_id in :qIds") // obviously wrong
.withValueMap(new ValueMap()
.withString(":c_id", conceptCode));
questionsTable.query(querySpec);
}
I need to pass qID list to fetch results similar to IN clause in SQL Query.
Please refer to this answer. Basically you need to form key list/value list dynamically
.withFilterExpression("question_id in (:qId1, :qId2, ... , :qIdN)")
.withValueMap(new ValueMap()
.withString(":qId1", ..) // just do this for each element in the list in a loop programmatically
....
.withString(":qIdN", ..)
);
Mind there is a restriction on maxItems in 'IN'
Having some trouble understanding how to create a Perl hash from a DB select statement.
$sth=$dbh->prepare(qq{select authorid,titleid,title,pubyear from books});
$sth->execute() or die DBI->errstr;
while(#records=$sth->fetchrow_array()) {
%Books = (%Books,AuthorID=> $records[0]);
%Books = (%Books,TitleID=> $records[1]);
%Books = (%Books,Title=> $records[2]);
%Books = (%Books,PubYear=> $records[3]);
print qq{$records[0]\n}
print qq{\t$records[1]\n};
print qq{\t$records[2]\n};
print qq{\t$records[3]\n};
}
$sth->finish();
while(($key,$value) = each(%Books)) {
print qq{$key --> $value\n};
}
The print statements work in the first while loop, but I only get the last result in the second key,value loop.
What am I doing wrong here. I'm sure it's something simple. Many thanks.
OP needs better specify the question and do some reading on DBI module.
DBI module has a call for fetchall_hashref perhaps OP could put it to some use.
In the shown code an assignment of a record to a hash with the same keys overwrites the previous one, row after row, and the last one remains. Instead, they should be accumulated in a suitable data structure.
Since there are a fair number of rows (351 we are told) one option is a top-level array, with hashrefs for each book
my #all_books;
while (my #records = $sth->fetchrow_array()) {
my %book;
#book{qw(AuthorID TitleID Title PubYear)} = #records;
push #all_books, \%book;
}
Now we have an array of books, each indexed by the four parameters.
This uses a hash slice to assign multiple key-value pairs to a hash.
Another option is a top-level hash with keys for the four book-related parameters, each having for a value an arrayref with entries from all records
my %books;
while (my #records = $sth->fetchrow_array()) {
push #{$books{AuthorID}}, $records[0];
push #{$books{TitleID}}, $records[1];
...
}
Now one can go through authors/titles/etc, and readily recover the other parameters for each.
Adding some checks is always a good idea when reading from a database.
I am using Entity Framework Reverse POCO generator v2.37.5.
I need to map an external database. It is not possible for me modify the schema. But the tables don't have primary key and all columns are set to null.
However, the combination of certain columns will always be unique.
For example, the following 3 columns can be combine to form the primary key:
EnrollNumb
CubNumb
SeqVal
Is there any setting in the template that helps me to set combination of columns as primary key?
Any suggestion / direction will be greatly appreciated.
First, find your Entities.ttinclude file.
Protip / entirely optional side-quest and distraction for better T4 editing:
Extract all the C# code (inside the T4 <#+ #> blocks) and move it to a new file named Entities.ttinclude.cs
Add <## Include File = "Entities.ttinclude.cs" #> to the Entities.ttinclude file.
Change the project build-action for Entities.ttinclude.cs to None.
Now you can get (basic) syntax-coloring for the C# code.
Now back to your regularly scheduled stackoverflowing:
Look for this somewhere around lines 250-320:
Settings.UpdateColumn = (Column column, Table table) =>
Inside the function you can tell EF that there totally is a PK defined on this table, I pinky-swear! like so:
Settings.UpdateColumn = (Column column, Table table) =>
{
// ...
if( column.ParentTable.Name == "Memb" )
{
switch( column.Name )
{
case "EnrollNumb":
case "CubNumb":
case "SeqVal":
column.IsNullable = false; // PK columns cannot be NULLable.
column.IsPrimaryKey = true;
column.PrimaryKeyOrdinal = column.Ordinal;
column.ParentTable.HasPrimaryKey = true
break;
}
}
// ...
}
Run Model.tt
Assuming no errors happened, take a look at the folder with the generated .cs files, look for Memb.Configuration.cs. It should look something like this:
[System.CodeDom.Compiler.GeneratedCode("EF.Reverse.POCO.Generator", "2.37.1.0")]
internal class MembConfiguration : System.Data.Entity.ModelConfiguration.EntityTypeConfiguration<Memb>
{
public MembConfiguration( String schema )
{
ToTable( "Memb", schema );
HasKey( x => new { x.EnrollNumb, x.CubNumb, x.SeqVal } );
Property( x => x.EnrollNumb ).HasColumnName( "EnrollNumb" ).HasColumnType( "nvarchar" ).IsRequired().MaxLength(10);
Property( x => x.CubNumb ).HasColumnName( "CubNumb" ).HasColumnType( "nvarchar" ).IsRequired().MaxLength(50);
Property( x => x.SeqVal ).HasColumnName( "SeqVal" ).HasColumnType( "nvarchar" ).IsRequired().MaxLength(5);
// other columns here
}
}
And you should be able to build your project, then run it, and it should just work.
You can also define fake Foreign Key constraints and set-up other kinds of relationships and EF will believe you, which is handy for when you want EF to handle a VIEW as though it were a TABLE, especially as a VIEW in SQL Server cannot be imbued with PK and FK constraints (you can also get DML working provided your VIEW is updatable).
I have a table with PK (String) and SK (Integer) - e.g.
PK_id SK_version Data
-------------------------------------------------------
c3d4cfc8-8985-4e5... 1 First version
c3d4cfc8-8985-4e5... 2 Second version
I can do a conditional insert to ensure we don't overwrite the PK/SK pair using ConditionalExpression (in the GoLang SDK):
putWriteItem := dynamodb.Put{
TableName: "example_table",
Item: itemMap,
ConditionExpression: aws.String("attribute_not_exists(PK_id) AND attribute_not_exists(SK_version)"),
}
However I would also like to ensure that the SK_version is always consecutive but don't know how to write the expression. In pseudo-code this is:
putWriteItem := dynamodb.Put{
TableName: "example_table",
Item: itemMap,
ConditionExpression: aws.String("attribute_not_exists(PK_id) AND attribute_not_exists(SK_version) **AND attribute_exists(SK_version = :SK_prev_version)**"),
}
Can someone advise how I can write this?
in SQL I'd do something like:
INSERT INTO example_table (PK_id, SK_version, Data)
SELECT {pk}, {sk}, {data}
WHERE NOT EXISTS (
SELECT 1
FROM example_table
WHERE PK_id = {pk}
AND SK_version = {sk}
)
AND EXISTS (
SELECT 1
FROM example_table
WHERE PK_id = {pk}
AND SK_version = {sk} - 1
)
Thanks
A conditional check is applied to a single item. It cannot be spanned across multiple items. In other words, you simply need multiple conditional checks. DynamoDb has transactWriteItems API which performs multiple conditional checks, along with writes/deletes. The code below is in nodejs.
const previousVersionCheck = {
TableName: 'example_table',
Key: {
PK_id: 'prev_pk_id',
SK_version: 'prev_sk_version'
},
ConditionExpression: 'attribute_exists(PK_id)'
}
const newVersionPut = {
TableName: 'example_table',
Item: {
// your item data
},
ConditionExpression: 'attribute_not_exists(PK_id)'
}
await documentClient.transactWrite({
TransactItems: [
{ ConditionCheck: previousVersionCheck },
{ Put: newVersionPut }
]
}).promise()
The transaction has 2 operations: one is a validation against the previous version, and the other is an conditional write. Any of their conditional checks fails, the transaction fails.
You are hitting your head on some of the differences between a SQL and a no-SQL database. DynamoDB is, of course, a no-SQL database. It does not, out of the box, support optimistic locking. I see two straight forward options:
Use a software layer to give you locking on your DynamoDB table. This may or may not be feasible depending on how often updates are made to your table. How fast 'versions' are generated and the maximum time your application can be gated on the lock will likely tell you if this can work foryou. I am not familiar with Go, but the Java API supports this. Again, this isn't a built-in feature of DynamoDB. If there is no such Go API equivalent, you could use the technique described in the link to 'lock' the table for updates. Generally speaking, locking a no-SQL DB isn't a typical pattern as it isn't exactly what it was created to do (part of which is achieving large scale on unstructured documents to allow fast access to many consumers at once)
Stop using an incrementor to guarantee uniqueness. Typically, incrementors are frowned upon in DynamoDB, in part due to the lack of intrinsic support for it and in part because of how DynamoDB shards you don't want a lot of similarity between records. Using a UUID will solve the uniqueness problem, but if you are porting an existing application that means more changes to the elements that create that ID and updates to reading the ID (perhaps to include a creation-time field so you can tell which is the newest, or the prepending or appending of an epoch time to the UUID to do the same). Here is a pertinent link to a SO question explaining on why to use UUIDs instead of incrementing integers.
Based on Hung Tran's answer, here is a Go example:
checkItem := dynamodb.TransactWriteItem{
ConditionCheck: &dynamodb.ConditionCheck{
TableName: "example_table",
ConditionExpression: aws.String("attribute_exists(pk_id) AND attribute_exists(version)"),
Key: map[string]*dynamodb.AttributeValue{"pk_id": {S: id}, "version": {N: prevVer}},
},
}
putItem := dynamodb.TransactWriteItem{
Put: &dynamodb.Put{
TableName: "example_table",
ConditionExpression: aws.String("attribute_not_exists(pk_id) AND attribute_not_exists(version)"),
Item: data,
},
}
writeItems := []*dynamodb.TransactWriteItem{&checkItem, &putItem}
_, _ = db.TransactWriteItems(&dynamodb.TransactWriteItemsInput{TransactItems: writeItems})