For the sake of argument, assume that I have a very simple database:
CREATE TABLE test(idx integer primary key autoincrement, count integer);
This has one row. The database is accessed by a CGI script, which is called by Apache. The script reads the current value of count, increments it, and writes it back. I can run the script as
curl http://localhost/cgi-bin/test
and it tells me what the new value of count is. The script is actually C++; the basic stripped-down code looks like this:
// 'callback' sets 'count' to the current value of count
sqlite3_exec(con, "select count from test where idx=1", callback, &count, 0);
++count;
command << "update test set count=" << count << " where idx=1";
sqlite3_exec(con, command.str().c_str(), 0, 0, 0);
If I write a bash script that runs 20 instances of curl in the background, then I get lots of messages that the database is locked, and the counter is only incremented to 2 or 3, instead of 20. Ok, that's not very surprising, but how do I fix this?
After some experimenting, I've put both sqlite3_exec statements inside an exclusive transaction:
while(true) {
rc = sqlite3_exec(con, "begin exclusive transaction", 0, 0, 0);
if(rc != SQLITE_BUSY)
break;
std::this_thread::sleep_for(std::chrono::milliseconds(100));
}
if(rc != SQLITE_OK)
error();
...the select and update code shown above, followed by:
sqlite3_exec(con, "end transaction", 0, 0, 0);
This appears to be rock-solid, but I can't make much sense of the relevant bits of the sqlite docs, and I'm not convinced. Is there anything else I need to think about? Note that I don't have any rollbacks, or any other sqlite3 calls, apart from sqlite_open_v2, sqlite3_errmsg, and sqlite_close, no WAL, and I only test for SQLITE_BUSY. For testing, I run the bash script below, with $1 set to 1000 (ie. 1000 curl instances all running the CGI code). This completes in 10 or 11 seconds, and every time I run it it shows the final value of count as 1000, so it appears to be working.
Test script:
#!/bin/bash
sqlite3 /var/www/cgi-bin/test.db <<EOF
update test set count=0 where idx=1;
EOF
for ((c=0; c<$1; c++ ))
do
curl http://localhost/cgi-bin/test > /dev/null 2>&1 &
done
wait
sqlite3 /var/www/cgi-bin/test.db <<EOF
select count from test where idx=1;
EOF
Related
I have a function/store proc in Maria DB
CREATE DEFINER=`root`#`localhost` PROCEDURE `test1`(var1 varchar(100))
BEGIN
select * from ttype where kode=var1;
END
I need to get a cursor from store proc, Howto get a cursor in vfp application, database in MariaDb/MySQL StoreProc?
I try with this in my Visual Foxpro :
Sqlexec(kon,"call test1 ('ABC')","test") --> not running
But when I use common select like this :
sqlexec(kon,"select * from ttype where kode='ABC'","test") --it's running well..
"Can you show me how to use aerror() in my case?"
You would use the AError() function always when any ODBC Remote action would fail, i.e. any of Vfp's SQL*() functions, like SqlStringConnect() for example or your proposed
sqlexec(kon,"select * from ttype where kode='ABC'","test") --it's running well
&& Actually you cannot know whether "it's running well" unless you are evaluating its return value like this:
Local lnResult, laSqlErrors[1], lcErrorMessage
lnResult = SqlExec(kon,"select * from ttype where kode='ABC'","test")
If m.lnResult = –1 && as documented in the F1 Help
AERROR(laSqlErrors)
lcErrorMessage = ;
TRANSFORM(laSqlErrors[1]) + ", " + ;
TRANSFORM(laSqlErrors[2])
&& now write a log and/or inform the user
ENDIF
&& to be continued
In an unit test, I need to verify that the program skip locked records when processing a table.
I have been unable to setup a locked records because the test can't lock itself which make a lot of sense.
Here is a sample of what I'm trying to achieve.
DEV VAR v_isCommitted AS LOGI NO-UNDO.
DEF VAR hl AS HANDLE NO-UNDO.
DEF BUFFER bufl FOR tablename.
hl = BUFFER bufl:HANDLE.
LOCKED_RECORDS:
DO TRANSACTION ON ERROR UNDO, LEAVE LOCKED_RECORDS:
/*Setup : Create record not committed yet*/
CREATE tablename.
ASSIGN tablename.fields = fieldsvalue.
/*ACT : Code I'm trying to test*/
/*...some code...*/
v_isCommitted = hl:FIND-BY-ROWID(ROWID(tablename), EXCLUSIVE-LOCK, NO-WAIT)
AND AVAILABLE(bufl)
AND NOT LOCKED(bufl).
/*...some code touching the record if it is commited...*/
/*ASSERT : program left new record tablename AS IS.*/
END.
The problem is that the record is available and not locked to the test because it was created by it.
Is there a way I could have the test lock a record from itself so the act part can actually skip the record like it was created by someone else?
Progress: 11.7.1
A session can not lock itself. So you will need to start a second session. For example:
/* code to set things up ... */
/* spawn a sub process to try to lock the record */
os-command silent value( substitute( '_progres -b -db &1 -p lockit.p -param "&2" && > logfile 2>&&1', dbname, "key" )).
In lockit.p use session:parameter to get the key for the record to test (or hard code it I suppose).
Or, as mentioned in the comments below:
/* locktest.p
*/
define variable lockStatus as character no-undo format "x(20)".
find first customer exclusive-lock.
input through value( "_progres /data/sports120/sports120 -b -p ./lockit.p" ).
repeat:
import unformatted lockStatus.
end.
display lockStatus.
and:
/* lockit.p
*/
find first customer exclusive-lock no-wait no-error.
if locked( customer ) then
put "locked".
else
put "not locked".
quit.
I'm using System.Data.SQLite ADO.NET provider for SQLite and the following Powershell code to execute queries (and nonqueries) against a Sqlite3 DB:
Function Invoke-SQLite ($DBFile,$Query) {
try {
Add-Type -Path ".\System.Data.SQLite.dll"
}
catch {
write-warning "Unable to load System.Data.SQLite.dll"
return
}
if (!$DBFile) {
throw "DB Not Found" R
Sleep 5
Exit
}
$conn = New-Object System.Data.SQLite.SQLiteConnection
$conn.ConnectionString="Data Source={0}" -f $DBFile
$conn.Open()
$cmd = $Conn.CreateCommand()
$cmd.CommandText = $Query
#$cmd.CommandTimeout = 10
$ds = New-Object system.Data.DataSet
$da = New-Object System.Data.SQLite.SQLiteDataAdapter($cmd)
[void]$da.fill($ds)
$cmd.Dispose()
$conn.Close()
write-host ("{0} Row(s) returned " -f ($ds.Tables[0].Rows|Measure-Object|Select -ExpandProperty Count))
return $ds.Tables[0]
}
The problem is: while it is trivial to know how many rows have been SELECTed in a query operation, the same is not true if the operation is an INSERT,DELETE or UPDATE (nonqueries)
I know I could use the ExecuteNonQuery method, but i need a generic wrapper which returns number of affected rows while being agnostic about the query it executed (as Invoke-SQLCmd would do, for example)
Is that possible?
Thanks!
A few comments before the answer:
System.data.Sqlite supports executing multiple SQL statements for one command, as long as the CommandText has each valid statements delimited by a semicolon (;). This means that there could be a mixture of queries and DML statements (i.e. INSERT, UPDATE, DELETE). The fact that you do not want to distinguish between the type of statement in $Query tells me that you are likely just passing statements blindly, so it could contain any combination of statements. Simply getting only one value (whether from a query or DML) seems too limiting.
Using a DataAdapter to fill a dataset just to get counts is inefficient. Instead, it may be better to just get a DataReader object and count the returned rows. This also allows a separate count for each query statement to be retrieved, something that gets obscured by using the DataAdapter object. (Perhaps enumerating all tables in the resultant dataset could get the same number, but I'm not certain that would always be equivalent.)
One good thing is that if you insist on using a DataAdapter, it will still execute DML statements (even though the expected result is query that returns rows). The dataset will not be changed (filled), but all statements in the command text will still affect changes in the database, so the following solution will still be useful.
Even if the code had works, I assume that the line which prints "{0} Rows returned" is meant to get a simple count, but $ds.Tables[0].Rows needs to be $ds.Tables[0].Rows.Count.
Notes about this particular solution:
The key is to call either of the sqlite SQL functions changes() or total_changes(). These can be retrieved using SQL: SELECT total_changes();. I recommend getting total_changes() before and after a command, then subtracting the difference. That will get changes for multiple statements executed by one command.
I'm not a PowerShell guru, so I tested everything in C#. Treat the code below more as pseudo code since it may need tweaking.
The code:
$conn = New-Object System.Data.SQLite.SQLiteConnection
try {
$conn.ConnectionString="Data Source={0}" -f $DBFile
$conn.Open()
$cmdCount = $Conn.CreateCommand()
$cmd = $Conn.CreateCommand()
try {
$cmdCount.CommandText = "SELECT total_changes();"
$beforeChanges = $cmdcount.ExecuteScalar()
$cmd.CommandText = $Query
$ds = New-Object System.Data.DataSet
$da = New-Object System.Data.SQLite.SQLiteDataAdapter($cmd)
$rows = 0
try {
[void]$da.fill($ds)
foreach ($tbl in $ds.Tables) {
$rows += $tbl.Rows.Count;
}
} catch {}
$afterChanges = $cmdcount.ExecuteScalar()
$DMLchanges = $afterChanges - $beforeChanges
$totalRowAndChanges = $rows + $DMLchanges
# $ds.Tables[0] may or may not be valid here.
# If query returned no data, no tables will exist.
} finally {
$cmdCount.Dispose()
$cmd.Dispose()
}
} finally {
$conn.Dispose()
}
Alternatively, you could eliminate the DataAdapter:
$cmd.CommandText = $Query
$rdr = $cmd.ExecuteReader()
$rows = 0
do {
while ($rdr.Read()) {
$rows++
}
} while ($rdr.NextResult())
$rdr.Close();
I have an sql command that depends on the results of other sql commands. There are five sql commands in this chain, but the problem occurs in the 4th.
The 5th command must save data to the archive table. When it has to run and take the completion_date value from the 4th command, it throws a null reference exception. Actually it says, that Reader4[0] can't be read because it has null value. That's wrong because it has value, because in the database these queries work fine and also because the condition if (Reader4.HasRows == true) is true, it means that WHERE statement is true in the 4th command, which also includes checking the completion_date! What's wrong with this asp.net?
Command4 = new SqlCommand("SELECT trade_date FROM Schedule WHERE traider_id=#traiderID AND good_id=#goodID AND position_id=#positionID AND status_id=1 AND completion_date IS NOT NULL", Connection4); //note the completion_date check
Command4.Parameters.Add("#traiderID", Convert.ToInt32(Reader3[0]));
Command4.Parameters.Add("#positionID", CurrentPosition);
Command4.Parameters.Add("#goodID", Convert.ToInt32(Reader1[0]));
Reader4 = Command4.ExecuteReader();
if (Reader4.HasRows == true) //this check is done successfully
{
Command5 = new SqlCommand("INSERT INTO Archive (traider_id, good_id, completion_date) VALUES (#traiderID, #goodID, #completionDate)", Connection5);
Command5.Parameters.Add("#traiderID", Convert.ToInt32(Reader3[0]));
Command5.Parameters.Add("#goodID", Convert.ToInt32(Reader1[0]));
Command5.Parameters.Add("#completionDate", Convert.ToDateTime(Reader4[0])); //Here is the problem
Command5.ExecuteNonQuery();
}
Reader4.Close();
Where are you startin to read from the reader?
i.e. where do you call
Reader4.Read();
after
Reader4 = Command4.ExecuteReader();
I am writing some fixed length records to a QUEUE in Berkeley DB, and get back the record number agter each PUT. So for example if I put 4 messages on the queue I am getting back 1, 2, 3, 4.
NOw I would like to retrieve a message from the queue based on it's KEY....
So if I try:
db_recno_t keyval;
DBT key, data;
memset(&key, 0, sizeof(DBT));
memset(&data, 0, sizeof(DBT));
keyval = 2;
key.data = &keyval;
key.ulen = sizeof(keyval);
ret = q->get(q, NULL, &key, &data, DB_CONSUME);
printf("Key peek = %i\n", keyval);
printf("Data peek = %s\n", data.data);
I keep getting back the first record in the queue, not the one I specify with the key (in this case "2")
I know the keys are 1,2,3,4 on the queue so I am wondering what stupid thing I am doing here?
Thanks for the help, much aprreciated ;-)
Lynton
Try some other database format than DB_QUEUE if you need random access.