Using SQLite 3.19.2, I'm running into an odd situation.
One of the queries my application performs takes an enormously long time (100+ seconds) when running inside my app. Using the sqlite3 shell, the same query takes 0.5s.
I'm using a custom build of SQLite, statically linked into my app. The version of the shell is from my custom compilation, so this isn't an issue with the compilation.
I was using multiple threads however I've since managed to reproduce this issue single-threaded.
Using perf I've determined that the majority of the CPU time is spent in sqlite3VdbeExec and not in any of my code (for instance code that reads the fields of each returned row.)
The query is sqlite3_prepare_v2'd with bound parameters. I've reproduced the query below as well as a similar query that doesn't exhibit the performance problem.
Has anyone else seen anything like this?
SLOW QUERY (100+s in the app, 0.5s in the shell):
SELECT DISTINCT
Track.*
FROM
TrackGenres,
TrackFirstArtist,
Track
WHERE
TrackFirstArtist.id = Track.id AND
TrackGenres.id = Track.id AND
TrackGenres.genreID = 328
ORDER BY
(CASE WHEN 1 = 1 THEN TrackFirstArtist.artistName COLLATE ENGLISH END) ASC,
(CASE WHEN 1 != 1 THEN TrackFirstArtist.artistName COLLATE ENGLISH END) DESC
LIMIT 50
OFFSET 0;
QUERY PLAN (the plans are identical for whether run in the app or in the shell):
2 0 0 SEARCH TABLE TrackGenre USING COVERING INDEX sqlite_autoindex_TrackGenre_1 (genreID=?)
3 0 1 SEARCH TABLE WorkGenre USING COVERING INDEX sqlite_autoindex_WorkGenre_1 (genreID=?)
3 1 0 SEARCH TABLE TrackWork USING COVERING INDEX sqlite_autoindex_TrackWork_1 (workID=?)
1 0 0 COMPOUND SUBQUERIES 2 AND 3 USING TEMP B-TREE (UNION)
4 0 0 SCAN TABLE TrackArtist USING INDEX idx_TrackArtist_trackID
4 1 1 SEARCH TABLE Artist USING INTEGER PRIMARY KEY (rowid=?)
0 0 0 SCAN SUBQUERY 1
0 1 2 SEARCH TABLE Track USING INTEGER PRIMARY KEY (rowid=?)
0 2 1 SEARCH SUBQUERY 4 USING AUTOMATIC COVERING INDEX (id=?)
0 0 0 USE TEMP B-TREE FOR DISTINCT
0 0 0 USE TEMP B-TREE FOR ORDER BY
SIMILAR QUERY (0.5s in app, 0.5s in shell):
SELECT
COUNT (Track.id)
FROM
TrackGenres,
TrackFirstArtist,
Track
WHERE
TrackFirstArtist.id = Track.id AND
TrackGenres.id = Track.id AND
TrackGenres.genreID = 328 AND
( TrackFirstArtist.artistName >= 'a' AND TrackFirstArtist.artistName < 'b' );
I believe this is a bug in SQLite. I tracked the problem down to the CASE statements in the ORDER BY clause when using the query in a prepared statement.
When I removed one or the other of the statements, the query ran quickly again. The collation didn't make any difference.
This doesn't seem to be a general issue with prepared statements, I have other queries with a similar structure and they work properly. As such, I've been unable to create a simple, reproducible example that illustrates the problem.
Ultimately I had to solve this by replacing the variables in the query manually to avoid the use of a prepared statement.
Related
I'm making a flight tracking map that will need to pull live data from a sql lite db. I'm currently just using the sqlite executable to navigate the db and understand how to interact with it. Each aircraft is identified by a unique hex_ident. I want to get a list of all aircraft that have sent out a signal in the last minute as a way of identifying which aircraft are actually active right now. I tried
select distinct hex_ident, parsed_time
from squitters
where parsed_time >= Datetime('now','-1 minute')
I expected a list of 4 or 5 hex_idents only but I'm just getting a list of every entry (today's entries only) and some are outside the 1 minute bound. I'm new to sql so I don't really know how to do this yet. Here's what each entry looks like. The table is called squitters.
{
"message_type":"MSG",
"transmission_type":8,
"session_id":"111",
"aircraft_id":"11111",
"hex_ident":"A1B4FE",
"flight_id":"111111",
"generated_date":"2021/02/12",
"generated_time":"14:50:42.403",
"logged_date":"2021/02/12",
"logged_time":"14:50:42.385",
"callsign":"",
"altitude":"",
"ground_speed":"",
"track":"",
"lat":"",
"lon":"",
"vertical_rate":"",
"squawk":"",
"alert":"",
"emergency":"",
"spi":"",
"is_on_ground":0,
"parsed_time":"2021-02-12T19:50:42.413746"
}
Any ideas?
You must remove 'T' from the value of parsed_time or use datetime() for it also to make the comparison work:
where datetime(parsed_time) >= datetime('now', '-1 minute')
Note that datetime() function does not take into account microseconds, so if you need 100% accuracy, you must put them in the code with concatenation:
where replace(parsed_time, 'T', ' ') >=
datetime('now', '-1 minute') || substr(parsed_time, instr(parsed_time, '.'))
I have some trouble using CombiTimeTable.
I want to fill the table using a txt file that contains two columns, the first is the time and the second is the related value (a current sample). Furthermore, I add #1 in the first line as the manual says.
Moreover, I add the following parameters:
tableOnFile=true,
fileName="C:/Users/gg/Desktop/CurrentDrivingCycle.txt"
I also have to add the parameter tableName but I don't know how to define it. I tried to define it using the name of the file (i.e. CurrentDrivingCycle) but I got this error message at the end of the simulation:
Table matrix "CurrentDrivingCycle" not found on file "C:/Users/ggalli/Desktop/CurrentDrivingCycle.txt".
simulation terminated by an assertion at initialization
Simulation process failed. Exited with code -1.
Do you know how can I solve this issue?
Thank you in advance!
See the documentation:
https://build.openmodelica.org/Documentation/Modelica.Blocks.Sources.CombiTimeTable.html
The name tab1(6,2) in the example of the documentation is the tableName. So yours should look something like:
#1
double CurrentDrivingCycle(6,2) # comment line
0 0
1 0
1 1
2 4
3 9
4 16
I'm writing an SQLite select statement and want to pick out the first hit only that satisfy my criterion.
My problem is that I'm writing code inside a simulation framework that wraps my SQLite code before sending it to the database, and this wrapping already adds 'LIMIT 100' to the end of the code.
What I want to do:
SELECT x, y, z FROM myTable WHERE a = 0 ORDER BY y LIMIT 1
What happens when this simulation development framework has done its job:
SELECT x, y, z FROM myTable WHERE a = 0 ORDER BY y LIMIT 1 LIMIT 100
exec error near "LIMIT": syntax error
So my question is: How do I work around this limitation? Is there any way to still limit my results to give only one hit back despite that the statement will end in 'LIMIT 100'? I'm thinking something like creating a temporary table, add an index and filter on that, but my knowledge is limited to simple database queries.
I'm working in Access 2010. I want NULL values to return as 0 but I cannot get them to show up. The false values work fine. There should be 29 total rows, 20 returning 0 and 9 returning their value. Here is the code.
SELECT [QAT Export].Title, IIF(ISNULL(Count([QAT Export].[TaskID])),0,Count([QAT Export].[Task ID])) AS [Update Good]
FROM [QAT Export]
WHERE ((([QAT Export].[Task Status])<>"Closed" And ([QAT Export].[Task Status])<>"Cancelled") AND (([QAT Export].[Updated By]) <>"linker")AND((DateDiff("d",[Update Time],(Date())))<10))
GROUP BY [QAT Export].Title
ORDER BY [QAT Export].Title;
Your IIF() and IsNull() functions look fine.
Chances are: your WHERE clause is excluding the nulls through one of the tests in there.
What are the chances since [TaskID] is Null, [Task Status] is Null as well?
If that's the case, you'd need to test for IsNull([Task Status]) in your WHERE clause also.
Post your 29 records if you need more help. Most likely the Data is causing your WHERE clause to drop those records.
SQL. I have created 1 procedure but I am not getting the desired output. My procedure is below:
--/
CREATE OR REPLACE procedure Update_TB_INTERACTLOG
IS
BEGIN
FOR records in (select TNAME from tab where TNAME like 'TB_INTERACTLOG%' and TABTYPE = 'TABLE')
LOOP
dbms_output.put_line(records.TNAME||' modified');
END LOOP;
END;
/
There are 7 records I am getting from select query.
This I am getting in Log Output.
13:10:02 [CREATE - 0 row(s), 0.031 secs] Command processed. No rows were affected
... 1 statement(s) executed, 0 row(s) affected, exec/fetch time: 0.031/0.000 sec [0 successful, 1 warnings, 0 errors]
It looks as if you have created the procedure but not executed it. To execute it, run the following code:
exec Update_TB_INTERACTLOG;
Furthermore, you will need to turn on DBMS output in the tool you're using to run it (unless it's SQL*plus).
And please not that the procedure wasn't properly compiled (1 warnings). The procedure should probably end with:
END Update_TB_INTERACTLOG;
instead of:
END;