The following query takes 5 seconds to execute:
SELECT DISTINCT(Product.Name) FROM Product WHERE (0=1 OR Product.Number="prod11");
While the following takes ONLY 15 milliseconds:
SELECT DISTINCT(Product.Name) FROM Product WHERE (Product.Number="prod11");
Interestingly the following also takes only 15 milliseconds:
SELECT DISTINCT(Product.Name) FROM Product WHERE (1=1 AND Product.Number="prod11");
The query plan shows that the first query uses a full table scan (for some unknown reason), while the second and third queries use an index (as expected).
For some reason it looks like Sqlite optimizes the "1=1 AND ..." but it doesn't optimize "0=1 OR ...".
What can I do to make Sqlite use the index for the first query as well?
The queries are built by NHibernate so it's kind of hard to change them...
Sqlite version is the latest for Windows.
SQLite's query optimizer is rather simple and does not support OR expressions very well.
For some reason, it can optimize this query if it can use a covering index, so try this:
CREATE INDEX TakeThatNHibernate ON Product(Number, Name)
1=1and 1=0 are SQL expressions used in some parts of the NHibernate framework to denote empty statements that won't alter the logic of the sql query. A Conjunction with no subcriterias generates an 1=1 expression, A Disjunction with no subcriterias generates an 1=0 expression. An In() generates an 1=0 expression if no values are provided.
To avoid such optimization, you could change the code that is creating those empty expressions and only use the criterions that have at least one subcriteria.
Related
I have an SQLITE Database with the following tables
items(item_id,item_name,....);
tags(tag_id,tag_name);
items_tags(item_id,tag_id);
I'm using QSqlRelationalTableModel class to for the items table.
this class have this function QSqlTableModel::setFilter(const QString &filter);
The filter is a SQL WHERE clause without the keyword WHERE
for example: setFilter("item_name=arg");
what I want to do is to use the setFilter function to select only items with a specific tag/tags.
since I can only use the setFilter(QString &Filter) function which is basically a WHERE clause
I found this solution which I think is a bad one
I'll use this filter
setFilter(FilterString);
FilterString="item_id=arg1 or item_id=arg2 or item_id=argX or......"
the IDs will be obtained from this query
Query.prepare("select item_id from items_tags where tag_id=?");
the FilterString will be generated and incremented by a (do while loop) using Query.next() function that will loop over the resulted Query IDs
FilterString+="item=Query.value(item_id).toString();
and use like 2 if statements to determine when to add the or clause to the FilterString
this solution will definitely work but I think it's stupid because if there are many tagged items(say 5000) the Query string will contain 4999 or clause and another 5000 "id=x" so the Query will be more than 10000 characters long.
QString object can contain a lot more characters but is this Query will have a big impact on performance ?
what are the other alternatives, Even if it requires to switch to another DB like ORACLE or Mysql since SQLITE doesn't have many internal features
My R workflow now involves dealing with a lot of queries (RPostgreSQL library). I really want to make code easy to maintain and manage in the future.
I started loading large queries from separate .SQL files (this helped) and it worked great.
Then I started using interpolated values (that helped) which means that I can write
SELECT * FROM table WHERE value = ?my_value;
and (after loading it into R) interpolate it using sqlInterpolate(ANSI(), query, value = "stackoverflow").
What happens now is I want to use something like this
SELECT count(*) FROM ?my_table;
but how can I make it work? sqlInterpolate() only interpolates safely by default. Is there a workaround?
Thanks
In ?DBI::SQL, you can read:
By default, any user supplied input to a query should be escaped using
either dbQuoteIdentifier() or dbQuoteString() depending on whether it
refers to a table or variable name, or is a literal string.
Also, on this page:
You may also need dbQuoteIdentifier() if you are creating tables or
relying on user input to choose which column to filter on.
So you can use:
sqlInterpolate(ANSI(),
"SELECT count(*) FROM ?my_table",
my_table = dbQuoteIdentifier(ANSI(), "table_name"))
# <SQL> SELECT count(*) FROM "table_name"
sqlInterpolate() is for substituting values only, not other components like table names. You could use other templating frameworks such as brew or whisker.
In AX programming best practice, which is the best way:
using a Query created from the AOT,
using a select statement with X++ code,
using a query created with X++ code the Query classe ...
And when to use each one of them?
First off, AX always uses queries internally, X++ selects are translated to query constructions calls, which are executed at run time. The query is translated to SQL at runtime on the the first queryRun.next() or datasource.executeQuery(). There is thus no performance difference using one or the other.
Forms also use queries, most often it is automatically constructed for you, because property AutoQuery has a Yes default value. You can use a X++ select in the executeQuery method, but I would consider that bad practice, as the user will have no filter or sorting options available. Always use queries in forms, prefer to use the auto queries. Add ranges or sorting in the init method using this.queryBuildDatasource() if needed. The exception being listpages which always use an AOT query.
In RunBase classes prefer to use queries, as the user will have the option to change the query. You may of cause use simple X++ select in the inner loop, but consider to include it in the prebuilt query, if possible.
Otherwise, your primary goal as a programmer (besides solving the problem) is to minimize the number of code lines.
Queries defined in the AOT start out with zero code lines, which count in their favor. Thus, if there are serveral statically defined ranges, links, or complex joins, use AOT queries. You cannot beat:
QueryRun qr = new QueryRun(queryStr(MyQuery))
qr.query().dataSourceTable(tableNum(MyTable)).findRange(fieldNum(MyTable,MyField)).value('myValue');
With:
Query q = new Query();
QueryRun qr = new QueryRun(q);
QueryBuildDataSource ds = q.addDataSource(tableNum(MyTable));
QueryBuildRange qbr = ds.addRange(fieldNum(MyTable,MyField));
qbr.value('myValue');
qbr.locked(true);
Thus in the static case prefer to use AOT queries, then change the query at runtime if needed. On the flip side, if your table is only known at runtime, you cannot use AOT queries, nor X++ selects, and you will need to build your query at runtime. The table browser is a good example of that.
What is left for X++?
Simple selects with small where clauses and with simple or no joins.
Cases where you cannot use queries (yet), delete_from, update_recordset and insert_recordset comes to mind.
Avoiding external dependencies (like AOT queries) may sometimes be more important.
Code readability of X++ queries is better than query construction.
The main techniques for selecting records in the database are as follows:
The select statement
A query
The techniques are essentially the same. They both deliver a set of records from the database in a table variable that can be accessed.
Use the select statement when:
The selection criteria are complex.
You are selecting a set of records from X++. The user is not going to change the selection criteria.
Use a query when:
The user can choose which records are selected.
The user can change the range of records to be selected.
The selection criteria are not more complex than the query can accommodate.
When you use queries, develop them in the Application Object Tree (AOT) or build them from scratch in your code. In both situations, the query can be modified in the code. A query built from scratch in code can be saved so that it occurs in the AOT. However, this should usually be avoided.
Build a query in the AOT when:
A specific query definition is being used in many places. (The query in the AOT can be reused.)
The query must contain code.
The query definition is more complex. The AOT provides a visual representation of the query.
I'm looking for difference between ax query and select (or while select)
In this example i don't see what's i can not do with statement select : the example of ax query in msdn
I think I misunderstood the syntax of ax query ranges :'(
A (while) select is a 'one use' statement, ie, you put it inline in your code and it is used only there.
A query can be setup to require parameters and can be used multiple times throughout your class or saved into the AOT for use in any class.
Generally I only use select statements for simple queries where its not worth the effort to create a query, for anything more complex I use queries.
Is there a way to determine if a MATCH query sent to an fts3 table in sqlite is valid? Currently, I don't find out if the expression is invalid until I try to run it, which makes it a little tricky. I'd like to report to the user that the syntax is invalid before even trying to run it.
I'm using sqlite with the C api.
Additionally, doing a search with the expression "NOT " will fail with a "SQLlite logic/database error". Googling seems to indicate that such a query is invalid. Is there a correct syntax to do the operation? I'm essentially trying to find entries that do NOT contain that term. Or do I have to drop back down to using LIKE and do a sequential scan and not use FTS?
Looks like the simplest way right now is to create a separate FTS table with no rows, and execute the query against it. It will be immediate since there's no data in the table, and will report and error if the query syntax is incorrect.