As I see LIKE operator can optimize query if I switch PRAGMA case_sensitive_like=ON. I measured, it really worked, queries "LIKE someth%" becomes ten times faster on a compartively large binary indexed tables.
But the problem is that my library implemented as an add-on to my application, it maintains its own tables with any db it is connected. So the problems are
I can not read case_sensitive_like since it is only supported to be set, not read. So I can not temporarily read the state and return it after the query,
As an addon that should obey the main functionality of the db, I should not change the setting to my need for good since it can affect other routines.
As I see there's no Like(case-sensitive) internal equivalent for me to call optimizid query directly. For example use LIKECASESENSITIVE instead of LIKE
I can call sqlite3_create_function, but I don't know whether or not I can call LIKE(CASE SENSITIVE) internally.
I can not read case_sensitive_like
since it is only supported to be set,
not read. So I can not temporarily
read the state and return it after the
query
You can get the state of case_sensitive_like with a query like this one:
select case when 'a' like 'A' then 0 else 1 end
which will return 1 if case_sensitive_like = ON and 0 if it's OFF.
Related
I recognized that (insert/delete)-XQueries executed with the BaseX client always returning an empty string. I find this very confusing or unintuitive.
Is there a way to find out if the query was "successful" without querying the database again (and using potentially buggy "transitive" logic like "if I deleted a node, there must be 'oldNodeCount-1' nodes in the XML")?
XQuery Update statements do not return anything -- that's how they are defined. But you're not the only one who does not like those restrictions, and BaseX added two ways around this limitation:
Returning Results
By default, it is not possible to mix different types of expressions
in a query result. The outermost expression of a query must either be
a collection of updating or non-updating expressions. But there are
two ways out:
The BaseX-specific update:output() function bridges this gap: it caches the results of its arguments at runtime and returns them after
all updates have been processed. The following example performs an
update and returns a success message:
update:output("Update successful."), insert node <c/> into doc('factbook')/mondial
With the MIXUPDATES option, all updating constraints will be turned off. Returned nodes will be copied before they are modified by
updating expressions. An error is raised if items are returned within
a transform expression.
If you want to modify nodes in main memory, you can use the transform
expression.
The transform expression will not help you, as you seem to modify the data on disk. Enabling MIXUPDATES allows you to both update the document and return something at the same time, for example running something like
let $node := <c/>
return ($node, insert node $node into doc('factbook')/mondial)
MIXUPDATES allows you to return something which can be further processed. Results are copied before being returned, if you run multiple updates operations and do not get the expected results, make sure you got the concept of the pending update list.
The db:output() function intentionally breaks its interface contract: it is defined to be an updating function (not having any output), but at the same time it prints some information to the query info. You cannot further process these results, but the output can help you debugging some issues.
Pending Update List
Both ways, you will not be able to have an immediate result from the update, you have to add something on your own -- and be aware updates are not visible until the pending update list is applied, ie. after the query finished.
Compatibility
Obviously, these options are BaseX-specific. If you strongly require compatible and standard XQuery, you cannot use these expressions.
I recognized that (insert/delete)-XQueries executed with the BaseX client always returning an empty string. I find this very confusing or unintuitive.
Is there a way to find out if the query was "successful" without querying the database again (and using potentially buggy "transitive" logic like "if I deleted a node, there must be 'oldNodeCount-1' nodes in the XML")?
XQuery Update statements do not return anything -- that's how they are defined. But you're not the only one who does not like those restrictions, and BaseX added two ways around this limitation:
Returning Results
By default, it is not possible to mix different types of expressions
in a query result. The outermost expression of a query must either be
a collection of updating or non-updating expressions. But there are
two ways out:
The BaseX-specific update:output() function bridges this gap: it caches the results of its arguments at runtime and returns them after
all updates have been processed. The following example performs an
update and returns a success message:
update:output("Update successful."), insert node <c/> into doc('factbook')/mondial
With the MIXUPDATES option, all updating constraints will be turned off. Returned nodes will be copied before they are modified by
updating expressions. An error is raised if items are returned within
a transform expression.
If you want to modify nodes in main memory, you can use the transform
expression.
The transform expression will not help you, as you seem to modify the data on disk. Enabling MIXUPDATES allows you to both update the document and return something at the same time, for example running something like
let $node := <c/>
return ($node, insert node $node into doc('factbook')/mondial)
MIXUPDATES allows you to return something which can be further processed. Results are copied before being returned, if you run multiple updates operations and do not get the expected results, make sure you got the concept of the pending update list.
The db:output() function intentionally breaks its interface contract: it is defined to be an updating function (not having any output), but at the same time it prints some information to the query info. You cannot further process these results, but the output can help you debugging some issues.
Pending Update List
Both ways, you will not be able to have an immediate result from the update, you have to add something on your own -- and be aware updates are not visible until the pending update list is applied, ie. after the query finished.
Compatibility
Obviously, these options are BaseX-specific. If you strongly require compatible and standard XQuery, you cannot use these expressions.
I'm building in scheme a database using a wiredtiger key/value store.
To query a given table one needs to have a cursor over the table. The library recommends to re-use the cursor. The general behavior can be described by the following pseudo-code:
with db.cursor() as cursor:
cursor.get(key)
...
do_something(db)
...
During the extent of the with statment cursor can only be used in the current context. If do_something(db) needs a cursor, it must create/retrieve another cursor even if it's to query the same table. Otherwise the cursor loose its position and the continuation of do_something(db) doesn't expect.
You can work around it by always reseting the cursor, that's a waste. Instead it's preferable to keep a set of cursors ready to be used and when one can request via db.cursor() this will remove a cursor from the available cursors and return it. Once the "context" operation is finished, put it back.
The way I solve this in Python is by using a list. db.cursor() looks like:
def cursor(self):
cursor = self.cursors.pop()
yield cursor
self.cursors.append(cursor)
Which means, retrieve a cursor, send it to the current context, once the context is finished, put it back to the list of available cursors.
How can I avoid the mutation and use more functional approach?
Maybe you want parameters?
Lookup the exact construct used by your Scheme implementation.
Some implementations use:
http://srfi.schemers.org/srfi-39/srfi-39.html
As the Sqlite docs specify, unbound parameters in prepared statements are interpreted as NULL - my question is this then:
Is there a way to have Sqlite ensure that all parameters have been bound at least once, thereby ensuring none were missed by accident?
It is better to get an error and require calls to sqlite3_bind_null(statement_, col); then to get a subtle error because I forgot to call sqlite3_bind_* on the where clause of an update statement!
It is not possible to differentiate unbound parameters from parameters set to NULL using the current SQLite libraries.
If you have a look at the C source code for sqlite3_bind_null(), you will find that it simply calls the internal SQLite function that unbinds a parameter. Therefore there is no way to tell the two cases apart.
The only solution for this would be to wrap the SQLite C API functions with your own functions that will do a bit of book-keeping. You can bundle a bitmap with each sqlite3_stmt structure, with each bit set to true if and only if the corresponding parameter is bound. You would have to create wrappers for at least the sqlite3_bind_*() functions and sqlite3_clear_bindings().
I am using an sqlite database for a producer-consumer queue.
One or more producers INSERT one row at a time with a new autoincremented primary key.
There is one consumer (implemented in java, uses the sqlite-jdbc library) and I want it to read a batch of rows and delete them. It seems like I need transactions to do this but trying to use SQLite with transactions seems to not work right. Am I overthinking this?
If I do end up needing transactions, what's the right way to do this in Java?
Connection conn;
// assign here
boolean success = false;
try {
// do stuff
success = true;
}
finally
{
if (success)
conn.commit();
else
conn.rollback();
}
See this trail for an introduction on transaction handling with Java JDBC.
As for your use case, I think you should use transactions, especially if the consumer is complex. The tricky part is always to decide when a row has been consumed and when it should be considered again. For example, if you have an error before the consumer can actually do its job, you'll want a rollback. But if the row contains illegal data (like a text in a number field), then the rollback will turn into an infinite loop.
Normally, with SQLite there are explicit (not implicit!) transactions. So you need something like "START TRANSACTION" of course, it could be that your Java binding has this incorporated -- but good bindings don't.
So you might want to add the necessary transaction start (there might be a specialiced method in your binding).