Case sensitive and insensitive like in SQLite - sqlite

In SQLite it is possible to change the case sensitive behaviour of 'LIKE' by using the commands:
PRAGMA case_sensitive_like=ON;
PRAGMA case_sensitive_like=OFF;
However in my situation I would like to execute a query, part of which is case sensitive and part of which isn't. For example:
SELECT * FROM mytable
WHERE caseSensitiveField like 'test%'
AND caseInsensitiveField like 'g2%'
Is this possible?

You can use the UPPER keyword on your case insensitive field then upper-case your like statement. e.g.
SELECT * FROM mytable
WHERE caseSensitiveField like 'test%'
AND UPPER(caseInsensitiveField) like 'G2%'

Use plain comparisons, which are case sensitive by default (unless you have declared the column COLLATE NOCASE):
SELECT *
FROM mytable
WHERE caseSensitiveField >= 'test'
AND caseSensitiveField < 'tesu'
AND caseInsensitiveField LIKE 'g2%'
This works only if the original LIKE is searching for a prefix, but allows using an index.

In SQLite you can use GLOB instead of LIKE for pattern search. For example:
SELECT * FROM mytable
WHERE caseSensitiveField GLOB 'test*'
AND caseInsensitiveField LIKE 'g2%'
With this approach you don't have to worry about PRAGMA.

I know this is an old question, but if you are coding in Java and have this problem this might be helpful. You can register a function that handles the like checking. I got the tip form this post: https://stackoverflow.com/a/29831950/1271573
The solution i dependent on sqlite jdbc: https://mvnrepository.com/artifact/org.xerial/sqlite-jdbc
In my case I only needed to see if a certain string existed as part of another string (like '%mystring%'), so I created a Contains function, but it should be possible to extend this to do a more sql-like check using regex or something.
To use the function in SQL to see if MyCol contains "searchstring" you would do:
select * from mytable where Contains(MyCol, 'searchstring')
Here is my Contains function:
public class Contains extends Function {
#Override
protected void xFunc() throws SQLException {
if (args() != 2) {
throw new SQLException("Contains(t1,t2): Invalid argument count. Requires 2, but found " + args());
}
String testValue = value_text(0).toLowerCase();
String isLike = value_text(1).toLowerCase();
if (testValue.contains(isLike)) {
result(1);
} else {
result(0);
}
}
}
To use this function you must first register it. When you are done with using it you can optionally destroy it. Here is how:
public static void registerContainsFunc(Connection con) throws SQLException {
Function.create(con, Contains.class.getSimpleName(), new Contains());
}
public static void destroyContainsFunc(Connection con) throws SQLException {
Function.destroy(con, Contains.class.getSimpleName());
}

I used a regular expression to do what I needed. I wanted to identify all the occurrences of the word "In" that was not all lower case.
select [COL] from [TABLE] where [COL] REGEXP '\bIn\b';
Example:
with x as (select 'in' Diff_Ins union select 'In' Diff_Ins)
select Diff_Ins from x where Diff_Ins REGEXP '\bIn\b';

As others mention, SQLite also offers the GLOB function which is case-sensitive.
Assume g2* is text entered by the user at the application-level. To simplify application-side grammar and make GLOB case-insensitive, the text needs to be normalised to a common case:
SELECT * FROM mytable WHERE LOWER(caseInsensitiveField) GLOB LOWER('g2*');
If UNICODE is required, carefully test LOWER and UPPER to confirm they operate as expected. GLOB is an extension function specific to SQLite. Building a general grammar engine supporting multiple database vendors is non-trivial.

Related

How to delete a row that has the same ID as a variable in an SQLite database

I've got a function that I'd like use to delete a row in my database. This is the only way I've used the DELETE statement to remove a row before but I want the 1 to be replaced by a variable called recID so that the value of recID is the row ID number which is deleted. So if recID = 6, I want the function to delete the row with ID = 6. I hope that makes sense.
'DELETE FROM MyRecords WHERE ID=1';
The notation I've been using is the following, if it helps or makes any difference.
db.transaction(function(transaction) {
transaction.executeSql( //DELETE STATEMENT HERE );
});
executeSql supports arguments (check definition).
Use it like:
db.transaction(function(transaction) {
transaction.executeSql("DELETE FROM MyRecords WHERE ID=?", [recId]);
});
If you're certain that your variable, recID, will only ever contain numbers, you can just use:
transaction.executeSql("DELETE FROM MyRecords WHERE ID=" + recID);
If recID comes from outside your application (user input) however, it either needs to be sanitized, or use a prepared statement and use the database API to set the parameter after the statement has been prepared. Otherwise you open yourself up to SQL injection attacks.
I don't know the details of your SQLite wrapper, or what version of SQLite it wraps, but creating a prepared statement using the SQLite3 C API would go something like this:
// sqlite3* db = ...
sqlite3_stmt* stmt;
sqlite3_prepare_v2(db, "DELETE FROM MyRecords WHERE ID=?", -1, &stmt, 0);
sqlite3_bind_int(stmt, 1, recID);
sqlite3_step();
// ...
sqlite3_finalize(stmt);
This simple example excludes all the error checking you'd want to do in a real application, but since you're using a wrapper that has different syntax anyway, you'd have to figure out how it wraps these functions anyway.

Hive Query with UDF

I have some encrypted data in an HDFS csv, that I've created a Hive table for, and I want to run a Hive query that first encrypts the query param, then does the lookup. I have a UDF that does encryption as follows:
public class ParamEncrypt extends UDF {
public Text evaluate(String name) throws Exception {
String result = new String();
if (name == null) { return null; }
result = ParamData.encrypt(name);
return new Text(result);
}
}
Then I run the Hive query as:
select * from cc_details where first_name = encrypt('Ann');
The problem is, it's running encrypt('Ann') across every single record in the table. I want it do the encryption once, then do the matchup. I've tried:
select * from cc_details where first_name in (select encrypt('Ann') from cc_details limit 1);
But Hive doesn't support IN or select queries in the where clause.
What can I do?
Can I do something like:
select encrypt('Ann') as ann from cc_details where first_name = ann;
That also doesn't work because the query parser throws an error saying ann is not a known column
Finally got it with a right outer join as
select * from cc_details ssn_tbl
right outer join ( select encrypt('850-37-8230','ssn') as ssn
from cc_details limit 1) ssn_tmp
on (ssn_tbl.ssn = ssn_tmp.ssn);
I think what you are looking for is an annotation #UDFType(deterministic = true) on your UDF. It's definitely available on the Generic UDFs, you can check if it's available for regular UDF like you have created. If not, just convert your UDF to GenericUDF. You can read about it on this blog post that I wrote a while back.
Another way to do it (and actually the way I ended up going with), is by caching the result of the encryption. It's actually faster this way, because with the join, you get a separate set of map-reduce jobs, which slows down the overall execution time.
it's like this:
private static String result = null;
public Text evaluate(String data) {
if (result == null) {
result = Data.encrypt(data);
}
return new Text(result);
}

Using prepared statements and full-text-search in SQLite

I'm using the SQLite C interface to write an application. Since I like security, I'm using prepared statements to query the database. In one such query, I'm selecting rows from a virtual database using the MATCH keyword for full-text-searching. Here's an example:
SELECT * FROM Emails
WHERE ( Subject LIKE ?001 OR ?001 IS NULL )
AND ( Author LIKE ?002 OR ?002 IS NULL )
AND ( Body MATCH ?003 OR ?003 IS NULL )
This allows the user to enter any terms (Subject, Author, or Body) individually or in any combination to do a search. Any term that isn't entered, I'll bind NULL to that parameter. The problem with that statement is that you can't use the OR keyword with the MATCH keyword. I'm looking for a statement I can use with the MATCH keyword to return all rows if not searching in the Body column. Is there such a statement?
I suggest the following:
SELECT * FROM emails
WHERE ...
AND ( CASE (SELECT COUNT(*) FROM emails WHERE body MATCH ?003)
WHEN 0 THEN 1
ELSE body MATCH ?003
END )
I ended up modifying the SQL statement at runtime to replace MATCH with LIKE '%'. Not very elegant, but it works for now.

How do I check in SQLite whether a table exists?

How do I, reliably, check in SQLite, whether a particular user table exists?
I am not asking for unreliable ways like checking if a "select *" on the table returned an error or not (is this even a good idea?).
The reason is like this:
In my program, I need to create and then populate some tables if they do not exist already.
If they do already exist, I need to update some tables.
Should I take some other path instead to signal that the tables in question have already been created - say for example, by creating/putting/setting a certain flag in my program initialization/settings file on disk or something?
Or does my approach make sense?
I missed that FAQ entry.
Anyway, for future reference, the complete query is:
SELECT name FROM sqlite_master WHERE type='table' AND name='{table_name}';
Where {table_name} is the name of the table to check.
Documentation section for reference: Database File Format. 2.6. Storage Of The SQL Database Schema
This will return a list of tables with the name specified; that is, the cursor will have a count of 0 (does not exist) or a count of 1 (does exist)
If you're using SQLite version 3.3+ you can easily create a table with:
create table if not exists TableName (col1 typ1, ..., colN typN)
In the same way, you can remove a table only if it exists by using:
drop table if exists TableName
A variation would be to use SELECT COUNT(*) instead of SELECT NAME, i.e.
SELECT count(*) FROM sqlite_master WHERE type='table' AND name='table_name';
This will return 0, if the table doesn't exist, 1 if it does. This is probably useful in your programming since a numerical result is quicker / easier to process. The following illustrates how you would do this in Android using SQLiteDatabase, Cursor, rawQuery with parameters.
boolean tableExists(SQLiteDatabase db, String tableName)
{
if (tableName == null || db == null || !db.isOpen())
{
return false;
}
Cursor cursor = db.rawQuery(
"SELECT COUNT(*) FROM sqlite_master WHERE type = ? AND name = ?",
new String[] {"table", tableName}
);
if (!cursor.moveToFirst())
{
cursor.close();
return false;
}
int count = cursor.getInt(0);
cursor.close();
return count > 0;
}
You could try:
SELECT name FROM sqlite_master WHERE name='table_name'
See (7) How do I list all tables/indices contained in an SQLite database in the SQLite FAQ:
SELECT name FROM sqlite_master
WHERE type='table'
ORDER BY name;
Use:
PRAGMA table_info(your_table_name)
If the resulting table is empty then your_table_name doesn't exist.
Documentation:
PRAGMA schema.table_info(table-name);
This pragma returns one row for each column in the named table. Columns in the result set include the column name, data type, whether or not the column can be NULL, and the default value for the column. The "pk" column in the result set is zero for columns that are not part of the primary key, and is the index of the column in the primary key for columns that are part of the primary key.
The table named in the table_info pragma can also be a view.
Example output:
cid|name|type|notnull|dflt_value|pk
0|id|INTEGER|0||1
1|json|JSON|0||0
2|name|TEXT|0||0
SQLite table names are case insensitive, but comparison is case sensitive by default. To make this work properly in all cases you need to add COLLATE NOCASE.
SELECT name FROM sqlite_master WHERE type='table' AND name='table_name' COLLATE NOCASE
If you are getting a "table already exists" error, make changes in the SQL string as below:
CREATE table IF NOT EXISTS table_name (para1,para2);
This way you can avoid the exceptions.
If you're using fmdb, I think you can just import FMDatabaseAdditions and use the bool function:
[yourfmdbDatabase tableExists:tableName].
The following code returns 1 if the table exists or 0 if the table does not exist.
SELECT CASE WHEN tbl_name = "name" THEN 1 ELSE 0 END FROM sqlite_master WHERE tbl_name = "name" AND type = "table"
Note that to check whether a table exists in the TEMP database, you must use sqlite_temp_master instead of sqlite_master:
SELECT name FROM sqlite_temp_master WHERE type='table' AND name='table_name';
Here's the function that I used:
Given an SQLDatabase Object = db
public boolean exists(String table) {
try {
db.query("SELECT * FROM " + table);
return true;
} catch (SQLException e) {
return false;
}
}
Use this code:
SELECT name FROM sqlite_master WHERE type='table' AND name='yourTableName';
If the returned array count is equal to 1 it means the table exists. Otherwise it does not exist.
class CPhoenixDatabase():
def __init__(self, dbname):
self.dbname = dbname
self.conn = sqlite3.connect(dbname)
def is_table(self, table_name):
""" This method seems to be working now"""
query = "SELECT name from sqlite_master WHERE type='table' AND name='{" + table_name + "}';"
cursor = self.conn.execute(query)
result = cursor.fetchone()
if result == None:
return False
else:
return True
Note: This is working now on my Mac with Python 3.7.1
You can write the following query to check the table existance.
SELECT name FROM sqlite_master WHERE name='table_name'
Here 'table_name' is your table name what you created. For example
CREATE TABLE IF NOT EXISTS country(country_id INTEGER PRIMARY KEY AUTOINCREMENT, country_code TEXT, country_name TEXT)"
and check
SELECT name FROM sqlite_master WHERE name='country'
Use
SELECT 1 FROM table LIMIT 1;
to prevent all records from being read.
Using a simple SELECT query is - in my opinion - quite reliable. Most of all it can check table existence in many different database types (SQLite / MySQL).
SELECT 1 FROM table;
It makes sense when you can use other reliable mechanism for determining if the query succeeded (for example, you query a database via QSqlQuery in Qt).
The most reliable way I have found in C# right now, using the latest sqlite-net-pcl nuget package (1.5.231) which is using SQLite 3, is as follows:
var result = database.GetTableInfo(tableName);
if ((result == null) || (result.Count == 0))
{
database.CreateTable<T>(CreateFlags.AllImplicit);
}
The function dbExistsTable() from R DBI package simplifies this problem for R programmers. See the example below:
library(DBI)
con <- dbConnect(RSQLite::SQLite(), ":memory:")
# let us check if table iris exists in the database
dbExistsTable(con, "iris")
### returns FALSE
# now let us create the table iris below,
dbCreateTable(con, "iris", iris)
# Again let us check if the table iris exists in the database,
dbExistsTable(con, "iris")
### returns TRUE
I thought I'd put my 2 cents to this discussion, even if it's rather old one..
This query returns scalar 1 if the table exists and 0 otherwise.
select
case when exists
(select 1 from sqlite_master WHERE type='table' and name = 'your_table')
then 1
else 0
end as TableExists
My preferred approach:
SELECT "name" FROM pragma_table_info("table_name") LIMIT 1;
If you get a row result, the table exists. This is better (for me) then checking with sqlite_master, as it will also check attached and temp databases.
This is my code for SQLite Cordova:
get_columnNames('LastUpdate', function (data) {
if (data.length > 0) { // In data you also have columnNames
console.log("Table full");
}
else {
console.log("Table empty");
}
});
And the other one:
function get_columnNames(tableName, callback) {
myDb.transaction(function (transaction) {
var query_exec = "SELECT name, sql FROM sqlite_master WHERE type='table' AND name ='" + tableName + "'";
transaction.executeSql(query_exec, [], function (tx, results) {
var columnNames = [];
var len = results.rows.length;
if (len>0){
var columnParts = results.rows.item(0).sql.replace(/^[^\(]+\(([^\)]+)\)/g, '$1').split(','); ///// RegEx
for (i in columnParts) {
if (typeof columnParts[i] === 'string')
columnNames.push(columnParts[i].split(" ")[0]);
};
callback(columnNames);
}
else callback(columnNames);
});
});
}
Table exists or not in database in swift
func tableExists(_ tableName:String) -> Bool {
sqlStatement = "SELECT name FROM sqlite_master WHERE type='table' AND name='\(tableName)'"
if sqlite3_prepare_v2(database, sqlStatement,-1, &compiledStatement, nil) == SQLITE_OK {
if sqlite3_step(compiledStatement) == SQLITE_ROW {
return true
}
else {
return false
}
}
else {
return false
}
sqlite3_finalize(compiledStatement)
}
c++ function checks db and all attached databases for existance of table and (optionally) column.
bool exists(sqlite3 *db, string tbl, string col="1")
{
sqlite3_stmt *stmt;
bool b = sqlite3_prepare_v2(db, ("select "+col+" from "+tbl).c_str(),
-1, &stmt, 0) == SQLITE_OK;
sqlite3_finalize(stmt);
return b;
}
Edit: Recently discovered the sqlite3_table_column_metadata function. Hence
bool exists(sqlite3* db,const char *tbl,const char *col=0)
{return sqlite3_table_column_metadata(db,0,tbl,col,0,0,0,0,0)==SQLITE_OK;}
You can also use db metadata to check if the table exists.
DatabaseMetaData md = connection.getMetaData();
ResultSet resultSet = md.getTables(null, null, tableName, null);
if (resultSet.next()) {
return true;
}
If you are running it with the python file and using sqlite3 obviously. Open command prompt or bash whatever you are using use
python3 file_name.py first in which your sql code is written.
Then Run sqlite3 file_name.db.
.table this command will give tables if they exist.
I wanted to add on Diego Vélez answer regarding the PRAGMA statement.
From https://sqlite.org/pragma.html we get some useful functions that can can return information about our database.
Here I quote the following:
For example, information about the columns in an index can be read using the index_info pragma as follows:
PRAGMA index_info('idx52');
Or, the same content can be read using:
SELECT * FROM pragma_index_info('idx52');
The advantage of the table-valued function format is that the query can return just a subset of the PRAGMA columns, can include a WHERE clause, can use aggregate functions, and the table-valued function can be just one of several data sources in a join...
Diego's answer gave PRAGMA table_info(table_name) like an option, but this won't be of much use in your other queries.
So, to answer the OPs question and to improve Diegos answer, you can do
SELECT * FROM pragma_table_info('table_name');
or even better,
SELECT name FROM pragma_table_list('table_name');
if you want to mimic PoorLuzers top-voted answer.
If you deal with Big Table, I made a simple hack with Python and Sqlite and you can make the similar idea with any other language
Step 1: Don't use (if not exists) in your create table command
you may know that this if you run this command that will have an exception if you already created the table before, and want to create it again, but this will lead us to the 2nd step.
Step 2: use try and except (or try and catch for other languages) to handle the last exception
here if you didn't create the table before, the try case will continue, but if you already did, you can put do your process at except case and you will know that you already created the table.
Here is the code:
def create_table():
con = sqlite3.connect("lists.db")
cur = con.cursor()
try:
cur.execute('''CREATE TABLE UNSELECTED(
ID INTEGER PRIMARY KEY)''')
print('the table is created Now')
except sqlite3.OperationalError:
print('you already created the table before')
con.commit()
cur.close()
You can use a simple way, i use this method in C# and Xamarin,
public class LoginService : ILoginService
{
private SQLiteConnection dbconn;
}
in login service class, i have many methods for acces to the data in sqlite, i stored the data into a table, and the login page
it only shows when the user is not logged in.
for this purpose I only need to know if the table exists, in this case if it exists it is because it has data
public int ExisteSesion()
{
var rs = dbconn.GetTableInfo("Sesion");
return rs.Count;
}
if the table does not exist, it only returns a 0, if the table exists it is because it has data and it returns the total number of rows it has.
In the model I have specified the name that the table must receive to ensure its correct operation.
[Table("Sesion")]
public class Sesion
{
[PrimaryKey]
public int Id { get; set; }
public string Token { get; set; }
public string Usuario { get; set; }
}
Look into the "try - throw - catch" construct in C++. Most other programming languages have a similar construct for handling errors.

HTML Encoding in T-SQL?

Is there any function to encode HTML strings in T-SQL? I have a legacy database which contains dodgey characters such as '<', '>' etc. I can write a function to replace the characters but is there a better way?
I have an ASP.Net application and when it returns a string it contains characters which cause an error. The ASP.Net application is reading the data from a database table. It does not write to the table itself.
We have a legacy system that uses a trigger and dbmail to send HTML encoded email when a table is entered, so we require encoding within the email generation. I noticed that Leo's version has a slight bug that encodes the & in < and > I use this version:
CREATE FUNCTION HtmlEncode
(
#UnEncoded as varchar(500)
)
RETURNS varchar(500)
AS
BEGIN
DECLARE #Encoded as varchar(500)
--order is important here. Replace the amp first, then the lt and gt.
--otherwise the &lt will become &lt;
SELECT #Encoded =
Replace(
Replace(
Replace(#UnEncoded,'&','&'),
'<', '<'),
'>', '>')
RETURN #Encoded
END
GO
It's a bit late, but anyway, here the proper ways:
HTML-Encode (HTML encoding = XML encoding):
DECLARE #s NVARCHAR(100)
SET #s = '<html>unsafe & safe Utf8CharsDon''tGetEncoded ÄöÜ - "Conex"<html>'
SELECT (SELECT #s FOR XML PATH(''))
HTML-encode in a query:
SELECT
FIELD_NAME
,(SELECT FIELD_NAME AS [text()] FOR XML PATH('')) AS FIELD_NAME_HtmlENcoded
FROM TABLE_NAME
HTML-Decode:
SELECT CAST('<root>' + '<root>Test&123' + '</root>' AS XML).value(N'(root)[1]', N'varchar(max)');
If you want to do it properly, you can use a CLR-stored procedure.
However, it gets a bit complicated, because you can't use the System.Web-Assembly in CLR-stored-procedures (so you can't do System.Web.HttpUtility.HtmlDecode(htmlEncodedStr);). So you have to write your own HttpUtility class, which I wouldn't recommend, especially for decoding.
Fortunately, you can rip System.Web.HttpUtility out of the mono sourcecode (.NET for Linux). Then you can use HttpUtility without referencing system.web.
Then you write this CLR-Stored-Procedure:
using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.SqlServer.Server;
using System.Data.SqlTypes;
//using Microsoft.SqlServer.Types;
namespace ClrFunctionsLibrary
{
public class Test
{
[Microsoft.SqlServer.Server.SqlFunction]
public static SqlString HtmlEncode(SqlString sqlstrTextThatNeedsEncoding)
{
string strHtmlEncoded = System.Web.HttpUtility.HtmlEncode(sqlstrTextThatNeedsEncoding.Value);
SqlString sqlstrReturnValue = new SqlString(strHtmlEncoded);
return sqlstrReturnValue;
}
[Microsoft.SqlServer.Server.SqlFunction]
public static SqlString HtmlDecode(SqlString sqlstrHtmlEncodedText)
{
string strHtmlDecoded = System.Web.HttpUtility.HtmlDecode(sqlstrHtmlEncodedText.Value);
SqlString sqlstrReturnValue = new SqlString(strHtmlDecoded);
return sqlstrReturnValue;
}
// ClrFunctionsLibrary.Test.GetPassword
//[Microsoft.SqlServer.Server.SqlFunction]
//public static SqlString GetPassword(SqlString sqlstrEncryptedPassword)
//{
// string strDecryptedPassword = libPortalSecurity.AperturePortal.DecryptPassword(sqlstrEncryptedPassword.Value);
// SqlString sqlstrReturnValue = new SqlString(sqlstrEncryptedPassword.Value + "hello");
// return sqlstrReturnValue;
//}
public const double SALES_TAX = .086;
// http://msdn.microsoft.com/en-us/library/w2kae45k(v=vs.80).aspx
[SqlFunction()]
public static SqlDouble addTax(SqlDouble originalAmount)
{
SqlDouble taxAmount = originalAmount * SALES_TAX;
return originalAmount + taxAmount;
}
} // End Class Test
} // End Namespace ClrFunctionsLibrary
And register it:
GO
/*
--http://stackoverflow.com/questions/72281/error-running-clr-stored-proc
-- For unsafe permission
EXEC sp_changedbowner 'sa'
ALTER DATABASE YOUR_DB_NAME SET TRUSTWORTHY ON
GO
*/
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[HtmlEncode]') AND type in (N'FN', N'IF', N'TF', N'FS', N'FT'))
DROP FUNCTION [dbo].[HtmlEncode]
GO
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[HtmlDecode]') AND type in (N'FN', N'IF', N'TF', N'FS', N'FT'))
DROP FUNCTION [dbo].[HtmlDecode]
GO
IF EXISTS (SELECT * FROM sys.assemblies asms WHERE asms.name = N'ClrFunctionsLibrary' and is_user_defined = 1)
DROP ASSEMBLY [ClrFunctionsLibrary]
GO
--http://msdn.microsoft.com/en-us/library/ms345101.aspx
CREATE ASSEMBLY [ClrFunctionsLibrary]
AUTHORIZATION [dbo]
FROM 'D:\username\documents\visual studio 2010\Projects\ClrFunctionsLibrary\ClrFunctionsLibrary\bin\Debug\ClrFunctionsLibrary.dll'
WITH PERMISSION_SET = UNSAFE --EXTERNAL_ACCESS --SAFE
;
GO
CREATE FUNCTION [dbo].[HtmlDecode](#value [nvarchar](max))
RETURNS [nvarchar](max) WITH EXECUTE AS CALLER
AS
-- [AssemblyName].[Namespace.Class].[FunctionName]
EXTERNAL NAME [ClrFunctionsLibrary].[ClrFunctionsLibrary.Test].[HtmlDecode]
GO
CREATE FUNCTION [dbo].[HtmlEncode](#value [nvarchar](max))
RETURNS [nvarchar](max) WITH EXECUTE AS CALLER
AS
-- [AssemblyName].[Namespace.Class].[FunctionName]
EXTERNAL NAME [ClrFunctionsLibrary].[ClrFunctionsLibrary.Test].[HtmlEncode]
GO
/*
EXEC sp_CONFIGURE 'show advanced options' , '1';
GO
RECONFIGURE;
GO
EXEC sp_CONFIGURE 'clr enabled' , '1'
GO
RECONFIGURE;
GO
EXEC sp_CONFIGURE 'show advanced options' , '0';
GO
RECONFIGURE;
*/
Afterwards, you can use it like normal functions:
SELECT
dbo.HtmlEncode('helloäÖühello123') AS Encoded
,dbo.HtmlDecode('helloäÖühello123') AS Decoded
Anybody who just copy-pastes, please note that for efficiency reasons, you would use
public const double SALES_TAX = 1.086;
// http://msdn.microsoft.com/en-us/library/w2kae45k(v=vs.80).aspx
[SqlFunction()]
public static SqlDouble addTax(SqlDouble originalAmount)
{
return originalAmount * SALES_TAX;
}
if you'd use this function in production.
See here for the edited mono classes:
http://pastebin.com/pXi57iZ3
http://pastebin.com/2bfGKBte
You need to define NET_2_0 in the build options
You shouldn't fix the string in SQL. A better way is to use a function in ASP.net called HtmlEncode, this will cook the special characters that cause the issues you're seeing see the example below. I hope this helps.
string htmlEncodedStr = System.Web.HttpUtility.HtmlEncode(yourRawStringVariableHere);
string decodedRawStr = System.Web.HttpUtility.HtmlDecode(htmlEncodedStr);
Edit:
Since you're data binding this from a datatable. Use an inline expression to call HTMLEncode in the markup of the GridView or whatever control your using and this will still satisfy your data binding requirement. See example below. Alternativly you can loop every record in the data table object and update each cell with the html encoded string prior to data binding.
<%# System.Web.HttpUtility.HtmlEncode(Eval("YourColumnNameHere")) %>
I don't think data in a database should know or care about the user interface. Display issues should be handled by the presentation layer. I wouldn't want to see any HTML mingled into the database.
You can simply use 'XML PATH in your query'. For example;
DECLARE #encodedString VARCHAR(MAX)
SET #encodedString = 'give your html string you want to encode'
SELECT #encodedString
SELECT (SELECT #encodedString FOR XML PATH(''))
Now as your wish you can you this in your own sql function. Hope this will help.
If you're displaying a string on the web, you can encode it with Server.HTMLEncode().
If you're storing a string in the database, make sure the database field is "nchar", instead of "char". That will allow it to store unicode strings.
If you can't control the database, you can "flatten" the string to ASCII with Encoding.ASCII.GetString.
I haven't tried this solution myself but what I would try is utilise the sql server / .NET CLR integration and actually call the C# HTMLEncode function from the T-SQL.
This may be inefficient but I suspect it would give you the most accurate result.
My starting point for working out how to do this would be http://msdn.microsoft.com/en-us/library/ms254498%28VS.80%29.aspx
I've been trying to do this today in T-SQL, mostly for fun at this point since my requirements changed, but i figured one way out. You can use a table of unicode characters, built from the NCHAR() function or just import it, iterating from 0 to 65535 (or less if you just need the first 512 or something). Then rebuild the string. There are probably better ways to rebuild the string, but this works in a pinch.
---store unicode chars into a table so you can replace those characters withthe decimal value
`
CREATE TABLE #UnicodeCharacters(
DecimalValue INT,
UnicodeCharacter NCHAR
)
;
--loop from 0 to highest unicode value you want and dump to the table you created
DECLARE #x INT = 0;
WHILE #x <= 65535
BEGIN
BEGIN
INSERT INTO #UnicodeCharacters(DecimalValue, UnicodeCharacter)
SELECT #x,NCHAR(#x)
END
;
SET #x = #x + 1
;
END
;
--index for fast retrieval
CREATE CLUSTERED INDEX CX_UnicodeCharacter_DecimalValue ON #UnicodeCharacters(UnicodeCharacter, DecimalValue);
--this is the string that you want to html-encode...
DECLARE #String NVARCHAR(100) = N'人This is a test - Ñ';
--other vars
DECLARE #NewString NVARCHAR(100) = '';
DECLARE #Word TABLE(Character NCHAR(1));
DECLARE #Pos INT = 1;
--run through the string and check each character to see if it is outside the regex expression
WHILE #Pos <= LEN(#String)
BEGIN
DECLARE #Letter NCHAR(1) = SUBSTRING(#String,#Pos,1);
PRINT #Letter;
--rebuild the string replacing each unicode character outside the regex with &#[unicode value];
SELECT #NewString = #NewString +
CASE
WHEN #Letter LIKE N'%[0-9abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ-!##$%^&*()_+-= ]%' THEN #Letter
ELSE '&#' + CAST(uc.DecimalValue AS VARCHAR(10)) + ';'
END
FROM #UnicodeCharacters uc
WHERE #Letter = uc.UnicodeCharacter COLLATE JAPANESE_UNICODE_BIN
SET #Pos += 1
END
--end result
SELECT #NewString
;
`
I know typically you would use [0-9A-Za-z], but for some reason, it considered accented characters within the scope of that expression when I did that. So I explicitly used every character that i didn't want to convert to Unicode in the expression.
Last note, I had to use a different collation to do matches on Unicode characters, because the default LATIN collation (CI or otherwise) seemed to incorrectly match on accented characters, much like the regex in the LIKE.
assign it to Text Property of label, it will be auto encoded by .NET
OK here is what I did. I created a simple function to handle it. Its far from complete but at least handles the standard <>& characters. I'll just add to it as I go along.
CREATE FUNCTION HtmlEncode
(
#UnEncoded as varchar(500)
)
RETURNS varchar(500)
AS
BEGIN
DECLARE #Encoded as varchar(500)
SELECT #Encoded = Replace(#UnEncoded,'<','<')
SELECT #Encoded = Replace(#Encoded,'>','>')
SELECT #Encoded = Replace(#Encoded,'&','&')
RETURN #Encoded
END
I can then use:
Select Ref,dbo.HtmlEncode(RecID) from Customers
This gives me a HTML safe Record ID. There is probably a built in function but I can't find it.

Resources