I am creating a C++ program which will output a series of SQL statements (create, insert, etc) and write them to a file. This file will be used to create and populate a SQLite3 database.
I need to ensure that and values inserted are properly escaped so they can fit within the double quoted string (in the insert statement). Since there is no SQLite database available (this program just writes to a text file), I cannot use prepare. Can someone tell me which characters need to be escaped and how?
So far I've only found that the ' character needs to be escaped with another '
Inside a string, the only character to be escaped is the quote ' itself.
As for table/column names, you need to quote them if they conflict with SQL keywords.
Related
So I have to get a table which is in a schema in a database. The schema name contains a backslash, e.g david\b.
I have my connection con so I use dbplyr
tabel <- dplyr::tbl(con, in_schema("david\\b", "some_tabel"))
But this does not work.
Every database I know would only possibly allow a backslash in a quoted identifier. So I think you need to include the double quotes as well as the (escaped) backslash:
in_schema('"david\\b"', "some_tabel")
If you click on the links in my comment, they all pretty much say identifiers (like table and schema names) can only include letters, numbers, _ and (sometimes) $ and #. Unless the identifier is quoted.
SELECT TRIM(BOTH 'a' FROM 'aaaaaaaaaaaaasdfasa');
works fine and returns sdfas
but I am trying to remove quotes, so I did
SELECT TRIM(BOTH ''' FROM (''2565','5216','5364'') ;
I get error - Query ends within a string or comment block. Please suggest how to do this
This answer assumes that the .NET application successfully inserts data and the issue is at the time of SELECT or UPDATE i.e. the issue is not in a prepared SQL statement used within the .NET application. This also assumes that data sanity checks for using such logic in WHERE .. IN are already in place.
Assume your data values are:
2565,5216,5364
'2565','5216','5364'
Your application coverts these into following:
'2565,5216,5364'
''2565','5216','5364''
For ease of query design for this answer, we can convert these to string values for SQL by escaping each single quote in the value with an additional single quote, and then putting the entire thing inside a pair of single quotes to make it a string; which gives us:
'''2565,5216,5364'''
'''''2565'',''5216'',''5364'''''
If you want to remove all single quotes in these, you can use
SELECT OREPLACE( '''2565,5216,5364''', '''', '');
SELECT OREPLACE( '''''2565'',''5216'',''5364''''', '''', '');
Which means replace all single quotes with empty strings and gives us:
2565,5216,5364
2565,5216,5364
This may be the way to go in case you are comparing with integer values.
Now if you want your data to be preserved and remove only the enclosing quotes put by the .NET application, (e.g. if the comparison is also with character data) then you can combine this with further logic. Let us use the second data value, since it present more value for such an operation
SELECT TRIM(BOTH '''' FROM '''''2565'',''5216'',''5364''''');
The query above will give you following results which removes the enclosing quotes from application but also removes the first and the last quotes entered by the user
2565','5216','5364
A better option, but with the assumption that you application always encloses the data in quotes will be
SELECT SUBSTR('''''2565'',''5216'',''5364''''',2,CHARACTER_LENGTH('''''2565'',''5216'',''5364''''')-2);
This will perform substring operation from second character in the string till length-2, and will thus ignore both quotes inserted by the application
'2565','5216','5364'
I have an SQLite database and I want to dump and read individual statements from the dump. The problem I am encountering is that the individual statements can span multiple lines. So, I can't just read the dump file line-by-line. Is there a way to change the default line separator for .dump/.output command?
The .dump command always use a new-line as separator.
Any combination of characters (except a single ') is valid inside a string, and semicolons are valid inside a trigger body. To be able to read individual statements, you'd have to implement an SQL parser.
I'm trying to perform a SQLite FTS query with untrusted user input. I do not want to give the user access to the query syntax, that is they will not be able to perform a match query like foo OR bar AND cats. If they tried to query with that string I would want to interpret it as something more like foo \OR bar \AND cats.
There doesn't seem to be anything built in to SQLite for this, so I'll probably end up building my own escaping function, but this seems dangerous and error-prone. Is there a preferred way to do this?
The FTS MATCH syntax is its own little language. For FTS5, verbatim string literals are well defined:
Within an FTS expression a string may be specified in one of two ways:
By enclosing it in double quotes ("). Within a string, any embedded double quote characters may be escaped SQL-style - by adding a second double-quote character.
(redacted special case)
It turns out that correctly escaping a string for an FTS query is simple enough to implement completely and reliably: Replace " with "" and enclose the result in " on both ends.
In my case it then works perfectly when I put it into a prepared statement such as SELECT stuff FROM fts_table WHERE fts_table MATCH ?. I would then .bind(fts_escape(user_input)) where fts_escape is the function I described above.
OK I've investigated further, and with some heavy magic you can access the actual tokenizer used by SQLite's FTS. The "simple" tokenizer takes your string, separates it on any character that is not in [A-Za-z0-0], and lowercases the remaining. If you perform this same operation you will get a nicely "escaped" string suitable for FTS.
You can write your own, but you can access SQLite's internal one as well. See this question for details on that: Automatic OR queries using SQLite FTS4
Can strings in SQLite 3 include NUL characters?
If the answer to 1 is "yes", how can they be written in SQL queries? SQLite doesn't seem to have chr or char functions.
In general, no - SQLite internally is not 8-bit clean, probably due to its Tcl heritage. While NULs do not cause corruption problems, SQLite typically stops processing strings at the first embedded NUL character.
This is true even for operators such as GLOB. For instance, you cannot match a BLOB column with GLOB when you have embedded NUL characters, e.g. this
select * from table where blobcol glob x'00022a';
will only match empty blob values: While you can use literal BLOB syntax (i.e. x'hexdigits') and use the resulting values where strings are used, SQLite typically only uses the part before the first NUL.
At least,. this is the state of affairs up to including SQLite 3.26.0.
Note that SQLite also has a BLOB type which can store embedded NULs without any issues, but there is very little functionality in SQLite to use them, and they are often harder to use from SQL interface libraries.
They also silently convert to strings in many contexts, at which point the embedded NULs start causing issues again.
Not sure from which version onwards it is supported, but you can do it:
create table foo (bar data);
insert into foo(bar) values (x'001122334400ff');
select length(bar),hex(bar),bar from foo;