I am trying to update a table with latin charset (in my target) from the source table which is unicode in charset.
I am getting an error like "string contains untranslatable characters".
I then deleted the table and changed the characters set to latin but the same error came while reinserting data in this.
What can i do in this scenario?
below is my update..
update AM1 from DB1.AM_7541 AM1,
(sel
distinct TRIM(SB.BUILDING_NAME) NEID,
trim( leading '0' from sb.MAN) LCDMNO,
AM.Netw_equip,
LAST_TRANSACTION_DATE,
DISCONNECT_DATE,
SEQUENCE_NUMBER
from DB1.PLTL SB
inner join DB1.AM_7541 AM
on trim( leading '0' from sb.MAN)=AM.AM_NUM
where (SB.LAST_TRANSACTION_DATE,trim(sb.MAN),SB.SEQUENCE_NUMBER)
in ( sel max(LAST_TRANSACTION_DATE),trim(MAN) lcdmno,max(cast(SEQUENCE_NUMBER as integer))
from DB1.PLTL
where DISCONNECT_DATE ='2500-01-01 00:00:00' and trim(lcdmno) not like ''
and lcdmno is not null
group by lcdmno
)
and TRIM(SB.BUILDING_NAME) not like ''
and DISCONNECT_DATE ='2500-01-01 00:00:00'
) der
set Netw_equip=der.NEID
where AM1.AM_NUM=der.lcdmno
My table DB1.PLTL is the source table with unicode character set and DB1.AM_7541 has latin character set.
Thanks.
Amit
Since your source is unicode, you would be best off using unicode for both tables. If you can't do that, you can use translate:
select
...
translate(<your latin column> USING UNICODE_TO_LATIN)
Keep in mind that not all unicode characters will translate to latin, so you may get errors. If you do, you can put some ugly case statement together using TRANSLATE_CHK. But you'd be much better off using unicode everywhere.
Related
how do I code this properly to work in Oracle SQL :
update table_name
set field_name =
replace(field_name, x'BF', x'00')
where condition expression ;
Not sure how to code the replace all occurrence of hex 'BF' with null value hex'00' contained in data field field_name.
You can use the unistr() function to provide a Unicode character. e.g.:
update table_name
set field_name = replace(field_name, unistr('\00bf'))
where condition expression ;
which would remove the ¿ character completely; or to replace it with a null character:
set field_name = replace(field_name, unistr('\00bf'), unistr('\0000'))
though I suspect sticking a null in there will confuse things even more later, when some other system tries to read that text and stops at the null.
Quick demo:
with t (str) as (
select 'A ¿ char' from dual
)
select str,
replace(str, unistr('\00bf')) as removed,
replace(str, unistr('\00bf'), unistr('\0000')) as replaced,
dump(replace(str, unistr('\00bf')), 16) as removed_hex,
dump(replace(str, unistr('\00bf'), unistr('\0000')), 16) as replaced_hex
from t;
STR REMOVED REPLACED REMOVED_HEX REPLACED_HEX
--------- --------- --------- ----------------------------------- -----------------------------------
A ¿ char A char A char Typ=1 Len=7: 41,20,20,63,68,61,72 Typ=1 Len=8: 41,20,0,20,63,68,61,72
(Just as an example of the problems you'll have - because of the null I couldn't copy and paste that from SQL Developer, and had to switch to SQL*Plus...)
The first dump shows the two spaces (hex 20) next to each other; the second shows a null character between them.
This should be an easy one for someone with more SQLite experience than myself.
I need a select statement to get the name out of the following example string:
{"email":"12345678#facebook.com","user_key":"FACEBOOK:12345678","name":"John Smith"}
The output I need is John Smith.
The number of characters before the name is not always the same so a simple substr command won't work. It needs to be dynamic so it can locate where the name starts and then spit it out. I think ltrim or rtrim may help, but even after researching those commands, I don't understand them very well. Also, SQLite doesn't offer instr or position, which might have been helpful, too!
Edit: the schema for this table is as follows:
CREATE TABLE messages (msg_id TEXT PRIMARY KEY, thread_id TEXT, action_id INTEGER, subject TEXT, text TEXT, sender TEXT, timestamp_ms INTEGER, timestamp_sent_ms INTEGER, attachments TEXT, shares TEXT, msg_type INTEGER, affected_users TEXT, coordinates TEXT, offline_threading_id TEXT, source TEXT, channel_source TEXT, is_non_authoritative INTEGER, pending_send_media_attachment STRING, handled_internally_time INTEGER, pending_shares STRING, pending_attachment_fbid STRING, client_tags TEXT, send_error STRING, send_error_message STRING, send_error_timestamp_ms INTEGER, publicity TEXT, tracking TEXT );
CREATE INDEX messages_offline_threading_id_index ON messages ( offline_threading_id );
CREATE INDEX messages_timestamp_index ON messages ( thread_id, timestamp_ms DESC );
CREATE INDEX messages_type_index ON messages ( thread_id, msg_type, timestamp_ms );
The string I have above that I'm working with is from the sender column.
I don't know the language you're using but most of them support user defined functions (http://www.sqlite.org/c3ref/create_function.html) and you could do
select jsonGetName(columnWithJsonText) from messages where ...
It's like a callback to your programming language where you defined which C/Java/PHP function gets called when using jsonGetName() in sqlite.
In that function (in your language of choice) you decode the json string and return the name property.
while inserting i can insert danish character in proper format in sqlite Db but while retrieving my query returns no result
String searchQuery= "SELECT * FROM article,product where article.ItemNo=product.ItemNo ";
if(searchText.length()>0)
{
searchQuery += " AND (article.itemNo like '"+ searchText +"%' OR product.Description like '"+ searchText +"%')";
}
in debug mode query is
`SELECT * FROM article,product where article.ItemNo=product.ItemNo AND (article.itemNo like '%ø%' OR product.Description like '%ø%')..`
No result returns
Proper query will be
SELECT * FROM article,product where article.ItemNo=product.ItemNo AND (article.itemNo like '%Ø%' OR product.Description like '%Ø%');
the desired description field value in Db is MØNTPUNG.
I am wondering is there any issue of case sensitivty?I am using UTF8 encoding for my raw file that will insert data to DB.
The documentation says:
SQLite only understands upper/lower case for ASCII characters by default. The LIKE operator is case sensitive by default for unicode characters that are beyond the ASCII range. For example, the expression 'a' LIKE 'A' is TRUE but 'æ' LIKE 'Æ' is FALSE.
To handle non-ASCII characters correctly, store an uppercase version of your string(s) in a separate column, and search in that with an uppercase search pattern.
I have a Sqllite query
SELECT * FROM m_table WHERE LOWER(fName) = LOWER('yui!"'':;/?') AND account = '100' ORDER BY fName COLLATE NOCASE ASC ;
Above returns 0 rows; But when I use the same as below , it Works
update m_table set fName = 'yui!"'':;/? renamed' where fname='yui!"'':;/?' AND account = '100';
Any clues ?
PS: I am using LOWER to ignore case sensitive. I am using this via an android client. Hence
I am also doing StringEscapeUtils.escapeSql("folderName")
This is most likely related to the fact, that your input string contains characters in a non-ASCII-charset. from the documentation of SQLlite:
lower(X) -- The lower(X) function returns a copy of string X with all
ASCII characters converted to lower case. The default built-in lower()
function works for ASCII characters only. To do case conversions on
non-ASCII characters, load the ICU extension.
http://www.sqlite.org/lang_corefunc.html
Try to run the following statement and see what it returns.
SELECT LOWER(fName) FROM m_table WHERE fname='yui!"'':;/?' AND account = '100';
I wrote the database schema (only one table so far), and the INSERT statements for that table in one file. Then I created the database as follows:
$ sqlite3 newdatabase.db
SQLite version 3.4.0
Enter ".help" for instructions
sqlite> .read ./schema.sql
SQL error near line 16: near "s": syntax error
Line 16 of my file looks something like this:
INSERT INTO table_name (field1, field2) VALUES (123, 'Hello there\'s');
The problem is the escape character for a single quote. I also tried double escaping the single quote (using \\\' instead of \'), but that didn't work either. What am I doing wrong?
Try doubling up the single quotes (many databases expect it that way), so it would be :
INSERT INTO table_name (field1, field2) VALUES (123, 'Hello there''s');
Relevant quote from the documentation:
A string constant is formed by enclosing the string in single quotes ('). A single quote within the string can be encoded by putting two single quotes in a row - as in Pascal. C-style escapes using the backslash character are not supported because they are not standard SQL. BLOB literals are string literals containing hexadecimal data and preceded by a single "x" or "X" character. ... A literal value can also be the token "NULL".
I believe you'd want to escape by doubling the single quote:
INSERT INTO table_name (field1, field2) VALUES (123, 'Hello there''s');
for replace all (') in your string, use
.replace(/\'/g,"''")
example:
sample = "St. Mary's and St. John's";
escapedSample = sample.replace(/\'/g,"''")
Just in case if you have a loop or a json string that need to insert in the database. Try to replace the string with a single quote . here is my solution. example if you have a string that contain's a single quote.
String mystring = "Sample's";
String myfinalstring = mystring.replace("'","''");
String query = "INSERT INTO "+table name+" ("+field1+") values ('"+myfinalstring+"')";
this works for me in c# and java
In C# you can use the following to replace the single quote with a double quote:
string sample = "St. Mary's";
string escapedSample = sample.Replace("'", "''");
And the output will be:
"St. Mary''s"
And, if you are working with Sqlite directly; you can work with object instead of string and catch special things like DBNull:
private static string MySqlEscape(Object usString)
{
if (usString is DBNull)
{
return "";
}
string sample = Convert.ToString(usString);
return sample.Replace("'", "''");
}
In bash scripts, I found that escaping double quotes around the value was necessary for values that could be null or contained characters that require escaping (like hyphens).
In this example, columnA's value could be null or contain hyphens.:
sqlite3 $db_name "insert into foo values (\"$columnA\", $columnB)";
Demonstration of single quoted string behavior where complexity or double quotes are not desired.
Test:
SELECT replace('SAMY''S','''''','''');
Output:
SAMY'S
SQLite version:
SELECT sqlite_version();
Output:
3.36.0