SQLite on Embedded System - sqlite

I am trying to configure SQLite to run on an embedded system (ARM® Cortex®-M7). I have downloaded the amalgamation from the SQLite website, imported it into the project, and added the following symbols: SQLITE_THREADSAFE=0, SQLITE_OS_OTHER=1, SQLITE_OMIT_WAL=1 to allow it to compile.
I then downloaded test_onefile.c (available here: http://www.sqlite.org/vfs.html) which is supposed to allow SQLite to operate directly on embedded media without using an intermediate filesystem and imported it into the project (I was also sure to provide an sqlite3_os_init() function to register the VFS).
SQLITE_API int sqlite3_os_init(void)
{
extern int fs_register(void);
return fs_register();
}
In a separate file fs_register() looks like this:
/*
** This procedure registers the fs vfs with SQLite. If the argument is
** true, the fs vfs becomes the new default vfs. It is the only publicly
** available function in this file.
*/
int fs_register(void)
{
if (fs_vfs.pParent) return SQLITE_OK;
fs_vfs.pParent = sqlite3_vfs_find(0);
fs_vfs.base.mxPathname = fs_vfs.pParent->mxPathname;
fs_vfs.base.szOsFile = MAX(sizeof(tmp_file), sizeof(fs_file));
return sqlite3_vfs_register(&fs_vfs.base, 0);
}
I can successfully register a filesystem, open a database, and prepare SQL statements using sqlite3_register_vfs(), sqlite3_open(), and sqlite3_prepare().
When opening a database I am sure to use the ":memory:" string to create the database in memory rather than as a file.
static void TestSQLiteOpenDB(void)
{
/******** setup ********************************/
sqlite3 *db;
int rc;
/******** run element/component under test *****/
rc = sqlite3_open(":memory:", &db);
sqlite3_close(db);
/******** assertion test ***********************/
TEST_ASSERT_EQUAL_INT(SQLITE_OK, rc);
}
My issue is when trying to run sqlite3_exec(). The program crashes when the following piece of code from test_onefile.c is called:
/*
** Populate the buffer pointed to by zBufOut with nByte bytes of
** random data.
*/
static int fsRandomness(sqlite3_vfs *pVfs, int nByte, char *zBufOut)
{
sqlite3_vfs *pParent = ((fs_vfs_t *)pVfs)->pParent;
return pParent->xRandomness(pParent, nByte, zBufOut);
}
If I change this function to simply return 0, it appears to work. I can then create tables, insert data into tables etc...
My question is this: Is there a need in SQLite to populate this buffer with random data or is this workaround ok? I do not want to create further headaches for myself but it was a nightmare to track this down as the point of failure and I can not quite wrap my head around what is happening.

SQLite uses this randomness for temporary files, to force changes in journal/WAL files, to generate unique column names, and when autoincremented IDs overflow.
If the returned value is constant, some of these might go into an infinite loop, so you should attempt to get actual randomness. (It does not need to be cryptographically secure.)

Related

Returning object reference from C++ function

I have began learning C++ for Arduino and have run into some troubles.
I have some functions reading/writing to SPIFFS files.
Since the code for opening configuration files is common I would want to have a separate function to handle it.
I have come up with following function declaration
ConfigFileOpenStatus DeviceOpenConfigFile(const char *path, File *file);
The function accepts pointer to char array for the file path, and pointer to opened file.
I then tried to make following function definition
ConfigFileOpenStatus DeviceOpenConfigFile(const char *path, File *file)
{
if (SPIFFS.exists(path))
{
file = &SPIFFS.open(path, "r+");
return !file ? Failed : Opened;
}
else
{
file = &SPIFFS.open(path, "w+");
return !file ? Failed : Created;
}
}
That did not work as compiler complained with error error: taking address of temporary [-fpermissive]
As I understand this means that the file object will be disposed once DeviceOpenConfigFile function returns?
So my question is whether its possible to implement a function in a way where I can get File object reference and release it later?
SPIFFS.open apparently returns File, by value. The returned value will be a temporary variable available on that line. So taking the address of that one doesn't make any sense, for the same reason as int func (void); ... &func() doesn't make any sense. It has nothing to do with the surrounding DeviceOpenConfigFile function.
It doesn't make sense to assign a new address to pointer passed by parameter, for the same reason as void func (int x) { x = 0; } doesn't make sense - you change a local variable only, nothing on the caller side gets changed and nothing gets returned to the caller.
It would seem that the solution you are looking for is this:
ConfigFileOpenStatus DeviceOpenConfigFile(const char *path, File* file)
{
...
*file = SPIFFS.open(path, "r+");
where file is allocated on the caller-side.

Finding pointer with 'find out what writes to this address' strange offset

I'm trying to find a base pointer for UrbanTerror42.
My setup is as followed, I have a server with 2 players.
cheat-engine runs on client a.
I climb a ladder with client b and then scan for incease/decrease.
When I have found the values, I use find out what writes to this address.
But the offset are very high and point to empty memory.
I don't really know how to proceed
For the sake of clarity, I have looked up several other values and they have the same problem
I've already looked at a number of tutorials and forums, but that's always about values where the offsets are between 0 and 100 and not 80614.
I would really appreciate it if someone could tell me why this happened and what I have to do/learn to proceed.
thanks in advance
Urban Terror uses the Quake Engine. Early versions of this engine use the Quake Virtual Machine and the game logic is implemented as bytecode which is compiled into assembly by the Quake Virtual Machine. Custom allocation routines are used to load these modules into memory, relative and hardcoded offsets/addresses are created at runtime to accommodate these relocations and do not use the normal relocation table method of the portable executable file format. This is why you see these seemingly strange numbers that change every time you run the game.
The Quake Virtual Machines are file format .qvm and these qvms in memory are tracked in the QVM table. You must find the QVM table to uncover this mystery. Once you find the 2-3 QVMs and record their addresses, finding the table is easy, as you're simply doing a scan for pointers that point to these addresses and narrowing down your results by finding those which are close in memory to each other.
The QVM is defined like:
struct vmTable_t
{
vm_t vm[3];
};
struct vm_s {
// DO NOT MOVE OR CHANGE THESE WITHOUT CHANGING THE VM_OFFSET_* DEFINES
// USED BY THE ASM CODE
int programStack; // the vm may be recursively entered
intptr_t(*systemCall)(intptr_t *parms);
//------------------------------------
char name[MAX_QPATH];
// for dynamic linked modules
void *dllHandle;
intptr_t entryPoint; //(QDECL *entryPoint)(int callNum, ...);
void(*destroy)(vm_s* self);
// for interpreted modules
qboolean currentlyInterpreting;
qboolean compiled;
byte *codeBase;
int codeLength;
int *instructionPointers;
int instructionCount;
byte *dataBase;
int dataMask;
int stackBottom; // if programStack < stackBottom, error
int numSymbols;
struct vmSymbol_s *symbols;
int callLevel; // counts recursive VM_Call
int breakFunction; // increment breakCount on function entry to this
int breakCount;
BYTE *jumpTableTargets;
int numJumpTableTargets;
};
typedef struct vm_s vm_t;
The value in EAX in your original screenshot should be the same as either the codeBase or dataBase member variable of the QVM structure. The offsets are just relative to these addresses. Similarly to how you deal with ASLR, you must calculate the addresses at runtime.
Here is a truncated version of my code that does exactly this and additionally grabs important structures from memory, as an example:
void OA_t::GetVM()
{
cg = nullptr;
cgs = nullptr;
cgents = nullptr;
bLocalGame = false;
cgame = nullptr;
for (auto &vm : vmTable->vm)
{
if (strstr(vm.name, "qagame")) { bLocalGame = true; continue; }
if (strstr(vm.name, "cgame"))
{
cgame = &vm;
gamestatus = GSTAT_GAME;
//char* gamestring = Cvar_VariableString("fs_game");
switch (cgame->instructionCount)
{
case 136054: //version 88
cgents = (cg_entities*)(cgame->dataBase + 0x1649c);
cg = (cg_t*)(cgame->dataBase + 0xCC49C);
cgs = (cgs_t*)(cgame->dataBase + 0xf2720);
return;
Full source code for reference available at OpenArena Aimbot Source Code, it even includes a video overview of the code.
Full disclosure: that is a link to my website and the only viable resource I know of that covers this topic.

SQLite DB closed after prepare but stmt still works?

Also, the sqlite_master table for DB and registered functions still seem available. Is this just a case of the stmt accessing memory that hasn't been overwritten yet or does the prepare write details into the stmt that means it doesn't subsequently require the sqlite3* structure.
#include "sqlite3.h"
//---------------------------------------------------------------------------
void Odd(sqlite3_context *ctx,int nargs,sqlite3_value **values)
{
sqlite3_result_int(ctx,sqlite3_value_int(values[0])%2);
}
//---------------------------------------------------------------------------
int _tmain(int argc,_TCHAR* argv[])
{
sqlite3 *DB;
if (sqlite3_open_v2("c:/SQLiteData/MyDB.db",&DB,SQLITE_OPEN_READWRITE,NULL)!=SQLITE_OK)
return 1;
sqlite3_create_function_v2(DB,"Odd",-1,SQLITE_UTF16 | SQLITE_DETERMINISTIC,NULL,
&Odd,NULL,NULL,NULL);
sqlite3_stmt *stmt;
if (sqlite3_prepare16_v2(DB,L"select * from sqlite_master where Odd(rowid)",
-1,&stmt,NULL)!=SQLITE_OK) return 2;
if (sqlite3_close_v2(DB)!=SQLITE_OK) return 3;
int Count=0;
while (sqlite3_step(stmt)==SQLITE_ROW) Count++;
return 0;
}
The documentation says:
If the database connection is associated with unfinalized prepared statements … then sqlite3_close() will leave the database connection open and return SQLITE_BUSY. If sqlite3_close_v2() is called with unfinalized prepared statements …, then the database connection becomes an unusable "zombie" which will automatically be deallocated when the last prepared statement is finalized or the last sqlite3_backup is finished. The sqlite3_close_v2() interface is intended for use with host languages that are garbage collected, and where the order in which destructors are called is arbitrary.
But you are not using such a language.
You should not try to access the zombie; your application
should finalize all prepared statements … associated with the sqlite3 object prior to attempting to close the object.

variable in a header file shared between different projects

I have a solution which includes three projects. one is creating static library i.e .lib file. It contains one header file main.h and one main.cpp file. cpp file contains the definition of functions of header file.
second project is .exe project which includes the header file main.h and calls a function of header file.
third project is also a .exe project which includes the header file and uses a variable flag of header file.
Now both .exe projects are creating different instance of the variable. But I want to share same instance of the variable between the projects dynamically. as I have to map the value generated by one project into other project at the same instant.
Please help me as I am nearing my project deadline.
Thanks for the help.
Here are some part of the code.
main.cpp and main.h are files of .lib project
main.h
extern int flag;
extern int detect1(void);
main.cpp
#include<stdio.h>
#include"main.h"
#include <Windows.h>
#include <ShellAPI.h>
using namespace std;
using namespace cv;
int flag=0;
int detect1(void)
{
int Cx=0,Cy=0,Kx=20,Ky=20,Sx=0,Sy=0,j=0;
//create the cascade classifier object used for the face detection
CascadeClassifier face_cascade;
//use the haarcascade_frontalface_alt.xml library
face_cascade.load("E:\\haarcascade_frontalface_alt.xml");
//System::DateTime now = System::DateTime::Now;
//cout << now.Hour;
//WinExec("E:\\FallingBlock\\FallingBlock\\FallingBlock\\bin\\x86\\Debug\\FallingBlock.exe",SW_SHOW);
//setup video capture device and link it to the first capture device
VideoCapture captureDevice;
captureDevice.open(0);
//setup image files used in the capture process
Mat captureFrame;
Mat grayscaleFrame;
//create a window to present the results
namedWindow("capture", 1);
//create a loop to capture and find faces
while(true)
{
//capture a new image frame
captureDevice>>captureFrame;
//convert captured image to gray scale and equalize
cvtColor(captureFrame, grayscaleFrame, CV_BGR2GRAY);
equalizeHist(grayscaleFrame, grayscaleFrame);
//create a vector array to store the face found
std::vector<Rect> faces;
//find faces and store them in the vector array
face_cascade.detectMultiScale(grayscaleFrame, faces, 1.1, 3, CV_HAAR_FIND_BIGGEST_OBJECT|CV_HAAR_SCALE_IMAGE, Size(30,30));
//draw a rectangle for all found faces in the vector array on the original image
for(unsigned int i = 0; i < faces.size(); i++)
{
Point pt1(faces[i].x + faces[i].width, faces[i].y + faces[i].height);
Point pt2(faces[i].x, faces[i].y);
rectangle(captureFrame, pt1, pt2, cvScalar(0, 255, 0, 0), 1, 8, 0);
if(faces.size()>=1)
j++;
Cx = faces[i].x + (faces[i].width / 2);
Cy = faces[i].y + (faces[i].height / 2);
if(j==1)
{
Sx=Cx;
Sy=Cy;
flag=0;
}
}
if(Cx-Sx > Kx)
{
flag = 1;
printf("%d",flag);
}
else
{
if(Cx-Sx < -Kx)
{
flag = 2;
printf("%d",flag);
//update(2);
}
else
{
if(Cy-Sy > Ky)
{
flag = 3;
printf("%d",flag);
//update(3);
}
else
{
if(Cy-Sy < -Ky)
{
flag = 4;
printf("%d",flag);
//update(4);
}
else
if(abs(Cx-Sx) < Kx && abs(Cy-Sy)<Ky)
{
flag = 0;
printf("%d",flag);
//update(0);
}
}
}
}
2nd project's code
face.cpp
#include"main.h"
#include<stdio.h>
int main()
{
detect1();
}
3rd project's code
tetris.cpp
#include"main.h"
int key;
key = flag;
if(key==0)
{
MessageBox(hwnd,"Space2","TetRiX",0);
}
if(key==4)
{
tetris.handleInput(1);
tetris.drawScreen(2);
//MessageBox(hwnd,"Space2","TetRiX",0);
}
You need to look up how to do inter-process communication in the operating system under which your applications will run. (At this point I assume that the processes are running on the same computer.) It looks like you're using Windows (based on seeing a call to "MessageBox") so the simplest means would be for both processes to use RegisterWindowMessage create a commonly-understood message value, and then send the data via LPARAM using either PostMessage or SendMessage. (You'll need each of them to get the window handle of the other, which is reasonably easy.) You'll want to have some sort of exclusion mechanism (mutex or critical section) in both processes to ensure that the shared value can't be read and written at the same time. If both processes can do the "change and exchange" then you'll have an interesting problem to solve if both try to do that at the same time, because you'll have to deal with the possibility of deadlocks over that shared value.
You can also use shared memory, but that's a bit more involved.
If the processes are on different computers you'll need to do it via TCP/IP or a protocol on top of TCP/IP. You could use a pub-sub arrangement--or any number of things. Without an understanding of exactly what you're trying to accomplish, it's difficult to know what to recommend.
(For the record, there is almost no way in a multi-process/multi-threaded O/S to share something "at the same instant." You can get arbitrarily close, but computers don't work like that.)
Given the level of difficulty involved, is there some other design that might make this cleaner? Why do these processes have to exchange information this way? Must it be done using separate processes?

SQLite3: Insert BLOB with NULL characters in C++

I'm working on the development of a C++ API which uses custom-designed plugins
to interface with different database engines using their APIs and specific SQL
syntax.
Currently, I'm attempting to find a way of inserting BLOBs, but since NULL is
the terminating character in C/C++, the BLOB becomes truncated when constructing
the INSERT INTO query string. So far, I've worked with
//...
char* sql;
void* blob;
int len;
//...
blob = some_blob_already_in_memory;
len = length_of_blob_already_known;
sql = sqlite3_malloc(2*len+1);
sql = sqlite3_mprintf("INSERT INTO table VALUES (%Q)", (char*)blob);
//...
I expect that, if it is at all possible to do it in the SQLite3 interactive console, it should be possible to construct the query string with properly escaped NULL characters. Maybe there's a way to do this with standard SQL which is also supported by SQLite SQL syntax?
Surely someone must have faced the same situation before. I've googled and found some answers but were in other programming languages (Python).
Thank you in advance for your feedback.
Thank you all again for your feedback. This time I'm reporting how I solved the problem with the help of the indications provided here. Hopefully this will help others in the future.
As suggested by the first three posters, I did use prepared statements — additionally because I was also interested in getting the columns' data types, and a simple sqlite3_get_table() wouldn't do.
After preparing the SQL statement in the form of the following constant string:
INSERT INTO table VALUES(?,?,?,?);
it remains the binding of the corresponding values. This is done by issuing as many sqlite3_bind_blob() calls as the columns. (I also resorted to sqlite3_bind_text() for other "simple" data types because the API I'm working on can translate integers/doubles/etc into a string). So:
#include <stdio.h>
#include <string.h>
#include <sqlite3.h>
/* ... */
void* blobvalue[4] = { NULL, NULL, NULL, NULL };
int blobsize[4] = { 0, 0, 0, 0 };
const char* tail = NULL;
const char* sql = "INSERT INTO tabl VALUES(?,?,?,?)";
sqlite3_stmt* stmt = NULL;
sqlite3* db = NULL;
/* ... */
sqlite3_open("sqlite.db", &db);
sqlite3_prepare_v2(db,
sql, strlen(sql) + 1,
&stmt, &tail);
for(unsigned int i = 0; i < 4; i++) {
sqlite3_bind_blob(stmt,
i + 1, blobvalue[i], blobsize[i],
SQLITE_TRANSIENT);
}
if(sqlite3_step(stmt) != SQLITE_DONE) {
printf("Error message: %s\n", sqlite3_errmsg(db));
}
sqlite3_finalize(stmt);
sqlite3_close(db);
Note also that some functions (sqlite3_open_v2(), sqlite3_prepare_v2()) appear on the later SQLite versions (I suppose 3.5.x and later).
The SQLite table tabl in file sqlite.db can be created with (for example)
CREATE TABLE tabl(a TEXT PRIMARY KEY, b TEXT, c TEXT, d TEXT);
You'll want to use this function with a prepared statement.
int sqlite3_bind_blob(sqlite3_stmt*, int, const void*, int n, void(*)(void*));
In C/C++, the standard way of dealing with NULLs in strings is to either store the beginning of the string and a length, or store a pointer to the beginning of a string and one to the end of the string.
You want to precompile the statement sqlite_prepare_v2(), and then bind the blob in using sqlite3_bind_blob(). Note that the statement you bind in will be INSERT INTO table VALUES (?).

Resources