In PHP, i reproduce the problem with this command as an example, introducing some junk chars in the data i want to store :
$my_val = 'blabla' . chr(0xC2) . chr(0x94) . 'blabla';
$query = "UPDATE my_table SET my_field='" . $my_val . "' WHERE id='1'";
my_database, my_table, and my_field are ALL set to utf8mb4 and this cannot be changed.
I would like MariaDb to accept the wrong chars instead of returning systematic 1366 (Incorrect string value: '\xC2\x94b ' on this query.)
This problem is new on this MariaDb version : everything was ok on previous version.
I tried to add the IGNORE command to my query, but it ignore the error without writing the data.
Thank you for your help !
Related
I am using Drupal 9.2.10 and would like to update a database entry that is not in the default database.
when using select and join i use:
$query = \Drupal::database()->select('default.table1', 'table1');
$query->leftJoin('db_name2.table2', 'table2', 'table2.id=table1.id);
which works fine
however,
when trying to update a row in the second db by \Drupal::database()->update('db_name2.table2') Drupal will add the prefix of the default db so that is will actually try to update default.db_name2.table2 which will fail
I could set the active db by \Drupal\Core\Database\Database::setActiveConnection('db_name2');
however would like to avoid it, and use the db name instead
I tried the following with no luck:
\Drupal::database()->update('`db_name2.table2`')`
\Drupal::database()->update('"db_name2.table2"')`
\Drupal::database()->update('{db_name2.table2}')`
Can you help?
In Drupal\Core\Database\Query\Update __toString(), Drupal forces adding the active table by wrapping the table name with curly brackets:
$query = $comments . 'UPDATE {' . $this->connection->escapeTable($this->table) . '} SET ' . implode(', ', $update_fields);
In Drupal\Core\Database\Query\Select toString(), Drupal recognizes the possibility of the table name being a complte db_name.table_name by building the query as follows:
if (strpos($table_string, '.') === FALSE) {
$table_string = '{' . $table_string . '}';
}
Not sure why it's different.
*** Edit ***
Please refer to issue at Drupal site:
https://www.drupal.org/project/drupal/issues/3253479#comment-14334898
This question already has an answer here:
UPSERT for "INSERT INTO tab SELECT * FROM another_tab"
(1 answer)
Closed 2 years ago.
I am trying to load a daily CSV load file into a main table, so that all new records are inserted and all existing records are updated. I am using UPSERT feature of Sqlite for this purpose.
Here is my shell script.
LOAD_FILE="/tmp/main.csv"
LOAD_TABLE="tbl_tmp_main"
MAIN_TABLE="tbl_main"
COLUMNS="t_id, t_col1, t_col2"
CREATE_TABLE_QUERY="DROP TABLE IF EXISTS $LOAD_TABLE; CREATE TABLE $LOAD_TABLE AS SELECT $COLUMNS FROM $MAIN_TABLE WHERE false"
LOAD_TABLE_QUERY=".separator ','\n.import '$LOAD_FILE' $LOAD_TABLE"
UPSERT_TABLE_QUERY="
INSERT INTO $MAIN_TABLE($COLUMNS) SELECT $COLUMNS FROM $LOAD_TABLE
ON CONFLICT(t_id) DO UPDATE
SET
t_col1 = excluded.t_col1,
t_col2 = excluded.t_col2
;
"
echo ""
echo "$CREATE_TABLE_QUERY" | sqlite3 mydatabase.sqlite3 # <-- This works
echo "[INFO] Temporary Table Created."
echo "$LOAD_TABLE_QUERY" | sqlite3 mydatabase.sqlite3 # <-- This works
echo "[INFO] Data Loaded into Temporary Table."
echo "$UPSERT_TABLE_QUERY" | sqlite3 mydatabase.sqlite3 # <-- This errors out
echo "[INFO] Records Inserted/Updated."
echo ""
Here is the error.
Error: near line 2: near "DO": syntax error
I tried to check the documentation, but I am not sure what is wrong in the query above. Can somebody point out why it isnt working.
Here is my sqlite version
-> sqlite3 --version
3.34.1 2021-01-20 14:10:07 10e20c0b43500cfb9bbc0eaa061c57514f715d87238f4d835880cd846b9ebd1f
The error is due to Parsing Ambiguity as mentioned in the documentation. You need to add WHERE true as part of the SELECT clause and that should fix it.
Here is your updated UPSERT query.
INSERT INTO $MAIN_TABLE($COLUMNS) SELECT $COLUMNS FROM $LOAD_TABLE WHERE true
-- ^^^^^^^^^^
ON CONFLICT(t_id) DO UPDATE
SET
t_col1 = excluded.t_col1,
t_col2 = excluded.t_col2
;
Here is the section in Docs.
Parsing Ambiguity
When the INSERT statement to which the UPSERT is attached takes its values from a SELECT statement, there is a potential parsing ambiguity. The parser might not be able to tell if the "ON" keyword is introducing the UPSERT or if it is the ON clause of a join. To work around this, the SELECT statement should always include a WHERE clause, even if that WHERE clause is just "WHERE true".
Ambiguous use of ON:
INSERT INTO t1 SELECT * FROM t2
ON CONFLICT(x) DO UPDATE SET y=excluded.y;
Ambiguity resolved using a WHERE clause:
INSERT INTO t1 SELECT * FROM t2 WHERE true
ON CONFLICT(x) DO UPDATE SET y=excluded.y;
I connected from R to the PostgreSQL and i am able to write a table by using timestamp as table name, But i am unable to extract the values.
I used the following code.
library(DBI)
con <- dbConnect(RPostgres::Postgres(),dbname = 'postgres',
host = 'hostname',
port = 5432,
user = 'username',
password = 'pwd')
tm<-paste0('job_status_',Sys.time())
dbWriteTable(con,tm,jbs)
dbGetQuery(con,paste0('select * from ',tm))
When I ran the select command, I got the following syntax error.
Error in result_create(conn#ptr, statement) :
Failed to prepare query: ERROR: syntax error at or near "-"
LINE 1: select * from job_status_2019-03-12 04:33:08
Can anyone help me to resolve this issue?!
As your table name contains characters - :, it needs to be quoted with " to be understood as a table name.
dbGetQuery(con,paste0('select * from "',tm, '"'))
BTW. It may be a good idea to avoid unusual characters in tables' names and limit yourself to just letters, digits and underscore (_). To achieve that you can utilize gsub().
tm<-gsub('-|:| ', '_', paste0('job_status_',Sys.time()))
dbWriteTable(con,tm,jbs)
dbGetQuery(con,paste0('select * from ',tm))
Using Vertica 7.1 with flyway 3.2.1 getting errors when creating functions even tried the example from the docs. Almost like it doesn't ignore the first ';' :
code in the .sql file
CREATE or replace FUNCTION default_to_hello(x VARCHAR) RETURN VARCHAR
AS BEGIN
-- The body of a SQL function in Vertica must be in the form 'RETURN expression'
RETURN (CASE WHEN (x IS NOT NULL) THEN x ELSE 'Hello' END);
END;
Error reported:
Current version of schema "public": 2
Migrating schema "public" to version 3 - test
ERROR: Migration of schema "public" to version 3 failed! Please restore backups and roll back database and code!
ERROR:
Migration V3__test.sql failed
-----------------------------
SQL State : 42601
Error Code : 4856
Message : [Vertica][VJDBC](4856) ERROR: Syntax error at or near "EOL"
Location : /opt/flywaydb/sql/V3__test.sql (/opt/flywaydb/sql/V3__test.sql)
Line : 1
Statement : CREATE or replace FUNCTION default_to_hello(x VARCHAR) RETURN VARCHAR
AS BEGIN
-- The body of a SQL function in Vertica must be in the form 'RETURN expression'
RETURN (CASE WHEN (x IS NOT NULL) THEN x ELSE 'Hello' END)
Has any one else had this happen or am I missing something?
Flyway's VerticaStatementBuilder does not support OR REPLACE. You can see the code here for how that parse (it leaves no room for deviation).
if (statementStart.startsWith("CREATE FUNCTION")) {
if (line.startsWith("BEGIN") || line.endsWith("BEGIN")) {
insideBeginEndBlock = true;
}
if (line.endsWith("END;")) {
insideBeginEndBlock = false;
}
}
I submitted a pull request for it and it was merged for Flyway 4.0.
Your function works, it is some hidden character or something like this.
i managed to reproduce the error while trying to execute it in DbVis.
But in vsql client on the Vertica nodes is working fine.
dbadmin=> CREATE or replace FUNCTION default_to_hello(x VARCHAR) RETURN VARCHAR
dbadmin-> AS BEGIN
dbadmin-> -- The body of a SQL function in Vertica must be in the form 'RETURN expression'
dbadmin-> RETURN (CASE WHEN (x IS NOT NULL) THEN x ELSE 'Hello' END);
dbadmin-> END;
CREATE FUNCTION
dbadmin=> select default_to_hello('Working');
default_to_hello
------------------
Working
(1 row)
dbadmin=> select default_to_hello(null);
default_to_hello
------------------
Hello
(1 row)
This looks like a parsing bug. Please file an issue in the Flyway issue tracker. Pull request also welcome.
I cannot get trailing spaces from varchar column in Informix database.
I created test table, filled it with field with some trailing spaces,
but they are not returned by SELECT while it seems they are stored in db.
CREATE TABLE tmptable (txt varchar(240));
INSERT INTO tmptable (txt) VALUES ('123 ');
SELECT txt, txt || '***', LENGTH(txt) FROM tmptable;
And I got fields:
1: '123' : no trailing spaces!!!
2: '123 ***' : it seems that spaces are stored!!!
3: 3 : LENGTH() do not count trailing spaces!!!
Other databases I tested: Oracle and PostgreSQL return varchar fields
with trailing spaces. I tried RPAD() but with no success. Is there any way to get
trailing spaces?
Server: IBM Informix Dynamic Server Version 11.50.TC2DE
Client: tested with both ISA (no spaces in HTML page source) and ODBC driver 3.50.TC3DE
EDIT
Simple Python test program (tested with ActivePytnon 2.6 on Windows, you must change connection string in the last lines)
import odbc
def test_varchar(db_alias, dbname):
print
print
arr = db_alias.split('/')
print '%s %s' % (arr[0], dbname)
print '--------------'
connection = odbc.odbc(db_alias)
try:
cursor = connection.cursor()
cursor.execute("DELETE FROM tmptable;")
cursor.execute("INSERT INTO tmptable (txt) VALUES (' %s ')" % (dbname))
#cursor.commit()
cursor.execute("SELECT txt, txt || '***', LENGTH(txt) FROM tmptable;")
for row in cursor.fetchall():
print '[%s]\t[%s]\t[%s]' % (row[0], row[1], row[2])
finally:
connection.close()
#test_varchar('database/user/passwd', 'DBproducer')
test_varchar('oracledb/usr/passwd', 'Oracle ')
test_varchar('informixdb/usr/passwd', 'Informix ')
test_varchar('postgresqldb/usr/passwd', 'PostgreSQL')
And results:
c:\tools\pyscripts\scripts\db_examples>test_odbc.py
oracledb Oracle
--------------
[ Oracle ] [ Oracle ***] [16]
informixdb Informix
--------------
[ Informix] [ Informix ***] [11]
postgresqldb PostgreSQL
--------------
[ PostgreSQL ] [ PostgreSQL ***] [16]
Similar program in Jython using JDBC:
works (do not trim trailing spaces)
with native JDBC driver
doesn't work
(trim trailing spacec) with JDBC-ODBC
bridge
Source:
# for Jython 2.5 invoke with --verify
# beacuse of bug: http://bugs.jython.org/issue1127
import sys
from com.ziclix.python.sql import zxJDBC
def test_varchar(driver, db_url, usr, passwd):
arr = db_url.split(':', 2)
dbname = arr[1]
if dbname == 'odbc':
dbname = db_url
print "\n\n%s\n--------------" % (dbname)
try:
connection = zxJDBC.connect(db_url, usr, passwd, driver)
except:
ex = sys.exc_info()
s = 'Exception: %s: %s\n%s' % (ex[0], ex[1], db_url)
print s
return
cursor = connection.cursor()
cursor.execute("SELECT txt, txt || '***', LENGTH(txt) FROM tmptable")
for row in cursor.fetchall():
print '[%s]\t[%s]\t[%s]' % (row[0], row[1], row[2])
#test_varchar(driver, db_url, usr, passwd)
test_varchar("org.postgresql.Driver", 'jdbc:postgresql://127.0.0.1/pg_testdb', 'postgres', 'postgres')
test_varchar("oracle.jdbc.driver.OracleDriver", 'jdbc:oracle:oci:#MNTEST', 'user', 'passwd')
test_varchar("com.informix.jdbc.IfxDriver", 'jdbc:informix-sqli://127:0:0:1:9088/test_td:informixserver=ol_mn;DB_LOCALE=pl_PL.CP1250;CLIENT_LOCALE=pl_PL.CP1250;charSet=CP1250', 'user', 'passwd')
# db_url = jdbc:odbc:[ODBC source name]
test_varchar("sun.jdbc.odbc.JdbcOdbcDriver", 'jdbc:odbc:inf_test_db_odbc', 'user', 'passwd')
test_varchar("sun.jdbc.odbc.JdbcOdbcDriver", 'jdbc:odbc:ora_testdb_odbc', 'user', 'passwd')
test_varchar("sun.jdbc.odbc.JdbcOdbcDriver", 'jdbc:odbc:pg_testdb_odbc', 'postgres', 'postgres')
Results (for Informix only):
C:\tools\pyscripts\scripts\db_examples>jython --verify test_jdbc2.py
informix-sqli
--------------
[ Informix ] [ Informix ***] [11]
jdbc:odbc:inf_test_db_odbc
--------------
[ Informix] [ Informix ***] [11]
In ESQL/C, it is most certainly possible to get the trailing spaces from a VARCHAR column; my SQLCMD program (available from the IIUG Software Archive) does it. But you have to be extremely careful to use the correct type for the variables that hold the result. By default, the various char types are treated as CHAR rather than VARCHAR, and the libraries strip trailing blanks from CHAR values unless you direct otherwise (and blank pad to full length when you do direct otherwise).
Regarding ISA: I don't know how you established what it returns. I'm not altogether surprised that it loses the trailing blanks. Similar comments would apply to DB-Access.
Regarding ODBC: can you show the code, please, because although it is possible that there's a bug in the code (thank you for including the version information - it helps, and reassures me you are effectively up-to-date with your system), it is more likely that there is something up with the code you wrote to use it.
Regarding LENGTH(): it is defined to remove trailing blanks before calculating the length; it always treats its argument as if it was a CHAR value rather than as a VARCHAR value.
Taking your code and using SQLCMD:
Black JL: sqlcmd -d stores - <<!
> CREATE TABLE tmptable (txt varchar(240));
> INSERT INTO tmptable (txt) VALUES ('123 ');
> SELECT txt, txt || '***', LENGTH(txt) FROM tmptable;
> !
123 |123 ***|3
Black JL:
'Black JL:' is my Unix prompt on machine 'black'; as you can see, I got the trailing blanks OK (but I wrote SQLCMD about 20 years ago, now, in part because DB-Access, or rather its predecessor ISQL, didn't do things carefully enough for my purposes).
For anyone that doesn't feel like guessing what the "correct type" is, I've gotten my esql program to work by specifying a "lvarchar" type. (at least one version of the esql/c guide implies that "varchar" should work, but it didn't for me)