Not able to store binary data in sqlite using QT - sqlite

Not able to store all binary data values into sqlite3 table using QT.
For this I have created a BLOB data column to store binary data table named as logTable.
I am trying to insert binary data to binary_data[] buffer, with values as 0x1 to 0xFF and 0x00 to 0xFF and so on till 1024 bytes. But when I execute the query to store the data, the table shows only 0x1 to 0xFF, but remaining character are not getting stored (since next immediate value is 0x00). I want to store all the binary data values?
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
conect_to_log_db("./log.db");
unsigned char binary_data[1024];
for(unsigned int i = 0, value = 1; i < 1024; i++, value++)
{
binary_data[i] = value;
}
store_to_log_db("01/02/2012,13:03:58", binary_data, 1024);
......
}
bool store_to_log_db(QString dateTime, unsigned char *data, unsigned int dataLength)
{
QSqlQuery objQuery(objLogsDB);
QString query = "CREATE TABLE IF NOT EXISTS logTable(logDateTime VARCHAR(19), packet BLOB, direction INTEGER)";
objQuery.exec(query);
QByteArray dataArr;
dataArr.resize(dataLength);
for(unsigned int i = 0; i < dataLength; i++)
{
dataArr[i] = data[i];
}
QVariant blobData = dataArr.data();
objQuery.prepare("INSERT INTO logTable VALUES(:logDateTime,:packet,:direction)");
objQuery.bindValue(":logDateTime",dateTime);
objQuery.bindValue(":packet",blobData,QSql::In | QSql::Binary);
objQuery.bindValue(":direction",1);
qDebug() << objQuery.exec();
return true;
}
after executing this code, the result of the table is till 254 characters when I output from sqlite using
$sqlite3 log.db
sqlite>.output try.txt
sqlite>select * from logTable;
$ls -l try.txt
size is 406 bytes

You must use .dump. The sqlite3 interactive client doesn't output BLOB columns.
$sqlite3 log.db
sqlite> .output try.txt
sqlite> .dump
sqlite> .quit
The code below is a self contained example of creating a simple blob-containing database.
// https://github.com/KubaO/stackoverflown/tree/master/questions/sqlite-blob-11062145
#include <QtSql>
int main()
{
QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE");
db.setDatabaseName("./log.db");
if (!db.open()) { qDebug() << "can't open the database"; return 1; }
QSqlQuery query{db};
query.exec("DROP TABLE log");
if (!query.exec("CREATE TABLE log(packet BLOB)"))
qDebug() << "create table failed";
QVariant data[2] = {QByteArray{1024, 1}, QByteArray{2048, 2}};
query.prepare("INSERT INTO log VALUES(:packet)");
query.bindValue(":packet", data[0], QSql::In | QSql::Binary);
if (!query.exec()) qDebug() << "insert failed";
query.bindValue(":packet", data[1], QSql::In | QSql::Binary);
if (!query.exec()) qDebug() << "insert failed";
db.close();
if (!db.open()) { qDebug() << "can't reopen the database"; return 2; }
query.prepare("SELECT (packet) FROM log");
if (!query.exec()) qDebug() << "select failed";
for (auto const & d : data) if (query.next()) {
qDebug() << query.value(0).toByteArray().size() << d.toByteArray().size();
if (d != query.value(0)) qDebug() << "mismatched readback value";
}
db.close();
}

Related

Why do I get false when I try to insert a record to a QSqlTableModel in QT?

I'm creating a QSqlRecord object and then I set the values to that QSqlRecord object. But even if I insert the QSqlRecord object to the QSqlTableModel object, the function of inserting records, returns false.
I have this C++ code and it create a QSqlRecord object and set the values. It setting the values in the correct indexed order as how the table was created.
/* Insert data */
int column_index = 0; /* Index 0 is the ID column */
QSqlRecord record;
qDebug() << CALIBRATION_COLUMNS.at(column_index).first;
record.setValue(CALIBRATION_COLUMNS.at(column_index++).first, 1); /* ID */
qDebug() << CALIBRATION_COLUMNS.at(column_index).first;
record.setValue(CALIBRATION_COLUMNS.at(column_index++).first, calibration_id);
qDebug() << CALIBRATION_COLUMNS.at(column_index).first;
record.setValue(CALIBRATION_COLUMNS.at(column_index++).first, calibration_comment);
qDebug() << CALIBRATION_COLUMNS.at(column_index).first;
record.setValue(CALIBRATION_COLUMNS.at(column_index++).first, calibration_date_time);
for(int i = 0; i < 12; i++){
record.setValue(CALIBRATION_COLUMNS.at(column_index++).first, min_adc[i]);
record.setValue(CALIBRATION_COLUMNS.at(column_index++).first, max_adc[i]);
record.setValue(CALIBRATION_COLUMNS.at(column_index++).first, bias_adc[i]);
}
for(int i = 0; i < 5; i++){
record.setValue(CALIBRATION_COLUMNS.at(column_index++).first, min_dadc[i]);
record.setValue(CALIBRATION_COLUMNS.at(column_index++).first, max_dadc[i]);
record.setValue(CALIBRATION_COLUMNS.at(column_index++).first, bias_dadc[i]);
}
for(int i = 0; i < 2; i++)
record.setValue(CALIBRATION_COLUMNS.at(column_index++).first, pulses_per_revolution_encoder[i]);
/* -1 means append record */
qDebug() << calibration_model->insertRecord(-1, record);
qDebug() << calibration_model->lastError().text();
qDebug() << "Submit:";
if(!calibration_model->submitAll()){
qDebug() << calibration_model->lastError().text();
return DATABASE_STATUS_COULD_NOT_INSERT_ROW;
}
return DATABASE_STATUS_OK;
But even if I insert the record, this function calibration_model->insertRecord(-1, record); returns false but the calibration_model->submitAll() returns true.
Output:
"ID"
"calibration_id"
"calibration_comment"
"calibration_date_time"
false
"No Fields to update"
Submit:
So tell me. What I'm I doing wrong here?
I'm getting the error No Fields to update, but what does that mean? I have an empty table and I just want to append with one row.
Not sure why you're getting that error. I have a small example for QSqlTableModel. Let me put it here. Maybe you could compare with your code.
main.cpp
#include <QApplication>
#include "mysqltablemodel.h"
int main(int argc, char *argv[])
{
QApplication app(argc, argv);
QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE");
db.setDatabaseName("mydb");
if(!db.open()) {
qDebug() << db.lastError().text();
return 0;
}
QSqlQuery query(db);
if(!query.exec("DROP TABLE IF EXISTS mytable")) {
qDebug() << "create table error: " << query.lastError().text();
return 0;
}
if(!query.exec("CREATE TABLE IF NOT EXISTS mytable \
(id integer primary key autoincrement, name varchar(15), salary integer)")) {
qDebug() << "create table error: " << query.lastError().text();
return 0;
}
MySqlTableModel *model = new MySqlTableModel(0, db);
model->setTable("mytable");
model->setEditStrategy(QSqlTableModel::OnManualSubmit);
model->select();
QSqlRecord rec = model->record();
rec.setValue(1, "peter");
rec.setValue(2, 100);
qDebug() << model->insertRecord(-1, rec);
rec.setValue(1, "luke");
rec.setValue(2, 200);
qDebug() << model->insertRecord(-1, rec);
if(model->submitAll()) {
model->database().commit();
} else {
model->database().rollback();
qDebug() << "database error: " << model->lastError().text();
}
query.exec("SELECT name, salary FROM mytable");
while (query.next()){
QString name = query.value(0).toString();
int salary = query.value(1).toInt();
qDebug() << name << salary;
}
return app.exec();
}
mysqltablemodel.h
#ifndef MYSQLTABLEMODEL_H
#define MYSQLTABLEMODEL_H
#include <QSqlTableModel>
#include <QSqlRecord>
#include <QSqlError>
#include <QSqlQuery>
#include <QDebug>
class MySqlTableModel : public QSqlTableModel
{
Q_OBJECT
public:
MySqlTableModel(QObject *parent = 0, QSqlDatabase db = QSqlDatabase());
QVariant data(const QModelIndex &index, int role=Qt::DisplayRole ) const;
protected:
QHash<int, QByteArray> roleNames() const;
private:
QHash<int, QByteArray> roles;
};
#endif // MYSQLTABLEMODEL_H
mysqltablemodel.cpp
#include "mysqltablemodel.h"
MySqlTableModel::MySqlTableModel(QObject *parent, QSqlDatabase db): QSqlTableModel(parent, db) {}
QVariant MySqlTableModel::data ( const QModelIndex & index, int role ) const
{
if(index.row() >= rowCount()) {
return QString("");
}
if(role < Qt::UserRole) {
return QSqlQueryModel::data(index, role);
}
else {
return QSqlQueryModel::data(this->index(index.row(), role - Qt::UserRole), Qt::DisplayRole);
}
}
QHash<int, QByteArray> MySqlTableModel::roleNames() const
{
QHash<int, QByteArray> roles;
roles[Qt::UserRole + 1] = "name";
roles[Qt::UserRole + 2] = "salary";
return roles;
}

Reading from character device with Qt

I'm not very good at character devices, so I need your help. A have a char device(let's call it /dev/my_light) which is a light sensor. I have to read the data from this file and transform it to the brightness value and then pass it to the brightness manager that changes the brightness of my screen. The problem is that when I read the value for some period of time I get old values from the file.I assume there is a buffer(again not sure how character devices exactly work). Whereas when I use cat /dev/my_light I see new data! Is it possible to get rid off the buffer and read new values that were written to the file just right now. Here is my code in Qt:
void MySensor::updateMySensor()
{
Packet packet;
packet.startByte = 0;
packet.mantissa = 0;
packet.exp = 0;
d->device = ::open(d->path.toStdString().c_str(), O_RDONLY);
if (d->device == -1)
{
qDebug() << Q_FUNC_INFO << "can't open the sensor";
return;
}
ssize_t size = ::read(d->device, &packet, sizeof(packet));
close(d->device);
if (size == -1)
{
qDebug() << errno;
return;
}
packet.exp &= 0x0F;
float illumination = pow(2, packet.exp) * packet.mantissa * 0.045;
if(d->singleShot) emit lightSensorIsRunning(true);
emit illuminationRead(illumination);
}
The mySensor function is called every second. I tried to call it each 200 msec but it didn't help. The value of illumination stays old for about 7 seconds(!) whereas the value that I get from cat is new just immediately.
Thank you in advance!
I can't test with your specific device, however, I'm using the keyboard as a read only device.
The program attempts to connect to keyboard and read all keys pressed inside and outside the window. It's a broad solution you'll have to adapt to meet your demands.
Note that I'm opening the file with O_RDONLY | O_NONBLOCK which means open in read only mode and no wait for the event be triggered(some notifier needed to know when data is ready!) respectively.
You'll need super user privilege to run this example!
#include <QtCore>
#include <fcntl.h>
#include <linux/input.h>
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
const char *device_name = "/dev/input/by-path/platform-i8042-serio-0-event-kbd";
int descriptor = open(device_name, O_RDONLY | O_NONBLOCK);
if (descriptor < 0)
{
qDebug() << "Error" << strerror(errno);
return a.exec();
}
QFile device;
if (!device.open(descriptor, QFile::ReadOnly))
{
qDebug() << "Error" << qPrintable(device.errorString());
return a.exec();
}
QSocketNotifier notifier(device.handle(), QSocketNotifier::Read);
QObject::connect(&notifier, &QSocketNotifier::activated, &notifier, [&](int socket){
Q_UNUSED(socket)
struct input_event ev;
QByteArray data = device.readAll();
qDebug() << "Event caught:"
<< "\n\nDATA SIZE" << data.size()
<< "\nSTRUCT COUNT" << data.size() / int(sizeof(input_event))
<< "\nSTRUCT SIZE" << sizeof(input_event);
qDebug() << ""; //New line
while (data.size() >= int(sizeof(input_event)))
{
memcpy(&ev, data.data(), sizeof(input_event));
data.remove(0, int(sizeof(input_event)));
qDebug() << "TYPE" << ev.type << "CODE" << ev.code << "VALUE" << ev.value << "TIME" << ev.time.tv_sec;
}
qDebug() << ""; //New line
});
return a.exec();
}

What is the defined behavior of SQLite when interleaving statements that affect each other?

In SQLite if I prepare a SELECT statement and begin stepping through it, then before the last row of the results is reached I execute another statement that has an effect on the SELECT statement that I am stepping through, what is the expected result?
I can't find anything in the SQLite documentation about what is supposed to happen but it seems like an extremely common case when programming in a multi-threaded environment.
Below is a c++ file that can be compiled and run on Windows to demonstrate the situation.
#include "stdafx.h"
#include "sqlite3.h"
#include <Windows.h>
#include <iostream>
#include <Knownfolders.h>
#include <Shlobj.h>
#include <wchar.h>
#include <comdef.h>
using namespace std;
int exec_sql(sqlite3 *db, const char* sql)
{
char *errmsg;
int result = sqlite3_exec(db, sql, NULL, NULL, &errmsg);
if (result != SQLITE_OK) {
cout << errmsg << endl;
return -1;
}
return 0;
}
int _tmain(int argc, _TCHAR* argv[])
{
cout << "Running jsqltst with SQLite version: ";
cout << sqlite3_libversion();
cout << endl;
PWSTR userhome;
if (!SUCCEEDED(SHGetKnownFolderPath(FOLDERID_Profile, NULL, NULL, &userhome))) {
cout << "Failed getting user home dir\n";
return -1;
}
wcout << "User home: " << userhome << endl;
wchar_t *ws1 = userhome, *ws2 = L"\\test.sqlite";
wstring dbpath_str(ws1);
dbpath_str += wstring(ws2);
_bstr_t dbpath(dbpath_str.c_str());
cout << "DB path: " << dbpath << endl;
sqlite3 *db;
int result = sqlite3_open_v2(dbpath, &db, SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE, NULL);
if (result != SQLITE_OK) {
cout << sqlite3_errmsg(db) << endl;
return -1;
}
const char * create_stmt = "CREATE TABLE IF NOT EXISTS atable (id INTEGER PRIMARY KEY, name TEXT, number INTEGER);";
if (exec_sql(db, create_stmt) != 0) {
return -1;
}
const char * delete_stmt = "DELETE FROM atable;";
if (exec_sql(db, delete_stmt) != 0) {
return -1;
}
const char * insert_stmt = "INSERT INTO atable (name,number) VALUES ('Beta',77),('Alpha',99);";
if (exec_sql(db, insert_stmt) != 0) {
return -1;
}
sqlite3_stmt* select_ss;
const char * select_stmt = "SELECT * FROM atable;";
result = sqlite3_prepare_v2(db, select_stmt, -1, &select_ss, NULL);
if (result != SQLITE_OK) {
cout << sqlite3_errmsg(db) << endl;
return -1;
}
int i = 0;
boolean gotrow;
do {
result = sqlite3_step(select_ss);
gotrow = result == SQLITE_ROW;
if (gotrow) {
i++;
cout << "I got a row!" << endl;
if (i == 1) {
if (exec_sql(db, insert_stmt) != 0) {
return -1;
}
}
}
} while (gotrow);
cout << "Last result: " << result << ", errstr: " << sqlite3_errstr(result) << endl;
result = sqlite3_finalize(select_ss);
if (result != SQLITE_OK) {
cout << sqlite3_errmsg(db) << endl;
return -1;
}
return 0;
}
SQLite's behaviour for concurrent statements in the same transaction is neither documented nor defined.
As you have seen, newly inserted records might be seen when a SELECT's cursor has not yet reached that part of the table.
However, if SQLite needed to create a temporary result table for sorting or grouping, later changes in the table will not appear in that result.
Whether you have a temporary table or not might depend on decisions made by the query optimizer, so this is often not predictable.
If multiple threads access the same connection, SQLite will lock the DB around each sqlite3_step call.
This prevent data corruption, but you will still have the problem that automatic transaction end when their last active statement ends, and that explicit transaction will fail the COMMIT if there is some other active statement.
Multi-threaded programs are better off using (at least) one connection per thread.

How to create a Queue of unsigned char array in Qt?

I am new in Queue (FIFO) and Qt. I want to create a Queue of unsigned char array in Qt. How to do it? Please help
unsigned char buffer[1024];
If you want to use the Qt API, then you can use the QQueue class -
QQueue<unsigned char> queue;
queue.enqueue(65);
queue.enqueue(66);
queue.enqueue(67);
while (!queue.isEmpty())
cout << queue.dequeue() << endl;
If you want to build the queue on your own, then I guess you can declare a Queue class like this -
class Queue
{
private:
enum{SIZE=1024, EMPTY=0};
unsigned char buffer[SIZE];
int readHead, writeHead;
public:
Queue()
{
readHead = writeHead = EMPTY;
}
void push(unsigned char data);
unsigned char pop();
unsigned char peek();
bool isEmpty();
};
void Queue::push(unsigned char data)
{
if((readHead - writeHead) >= SIZE)
{
// You should handle Queue overflow the way you want here.
return;
}
buffer[writeHead++ % SIZE] = data;
}
unsigned char Queue::pop()
{
unsigned char item = peek();
readHead++;
return item;
}
unsigned char Queue::peek()
{
if(isEmpty())
{
// You should handle Queue underflow the way you want here.
return;
}
return buffer[readHead % SIZE];
}
bool Queue::isEmpty()
{
return (readHead == writeHead);
}
If you want to maintain a Queue of unsigned char array, then you will have to maintain a queue of unsigned char pointers -
QQueue<unsigned char *> queue;
unsigned char *array1 = new unsigned char[10]; // array of 10 items
array1[0] = 65;
array1[1] = 66;
queue.enqueue(array1);
unsigned char *array2 = new unsigned char[20]; // an array of 20 items
queue.enqueue(array2);
unsigned char *arr = queue.dequeue();
qDebug() << arr[0] << ", " << arr[1];
Note: You should take care of the memory cleanup after you are done with this queue. IMHO, you better avoid this type of design though.

Pass struct to function through pthread

How do I pass a struct through the pthread_create to my function? The errors I keep getting are that lower and upper has not been defined which would lead me to believe that the struct was not passed properly. I have tried referencing lower as arg1.lower and my error back is saying im trying to request for the member of a non class type void*
void * mergeSortThread(void *arg1){
std::ostringstream ostr;
ostr << "begin: " << lower << " " << upper << endl;
string message = ostr.str();
write(2, message.data(), message.length());
}
int main(int argc, char** argv)
{
struct MergeSortThreadArg
{
int * array;
int * temp;
int lower, upper;
};
pthread_attr_t attr;
pthread_attr_init(&attr);
int arr[20];
int temp[20];
MergeSortThreadArg arg;
arg.array = arr;
arg.temp = temp;
arg.lower = 0;
arg.upper = 19;
pthread_create(th, &attr, mergeSortThread, &arg);
pthread_join(th[z], NULL);
return 0;
}
Define struct before mergeSortThread() and add casting of arg1 to struct MergeSortThreadArg * it in mergeSortThread():
void * mergeSortThread(void *arg1){
struct MergeSortThreadArg *p_MST = (struct MergeSortThreadArg *)arg1;
std::ostringstream ostr;
ostr << "begin: " << p_MST->lower << " " << p_MST->upper << endl;
...
Moreover, it's more correct/safe to allocate the struct and its fields (array and tmp) to avoid passing to new thread data located on stack of another thread.

Resources