I was using tdstats.UDFCONCAT to aggregate result of the query,
This function trims the resulting output, upon looked at the definition of this function, it accepts VARCHAR(128) .
REPLACE FUNCTION tdstats.UDFCONCAT
(aVarchar VARCHAR(128) CHARACTER SET UNICODE)
RETURNS VARCHAR(10000) CHARACTER SET UNICODE
CLASS AGGREGATE (20000)
SPECIFIC udfConcat
LANGUAGE C
NO SQL
NO EXTERNAL DATA
PARAMETER STYLE SQL
NOT DETERMINISTIC
CALLED ON NULL INPUT
EXTERNAL NAME 'SL!staudf!F!udf_concatvarchar'
Can anybody tell if there is any specific reason to keep it limited to 128?
Note: I have duplicated the function and increased the size from 128 to 256 and it worked in my case. If anybody wants to know my use case then I can update here but question is regarding the default character limit mentioned by TD in the in-built function, so have not added my use case here.
Related
In this, the function sqlite3_column_type can tell me whether the initial data type of the result is text or not, but it will not tell whether it is UTF-8 or UTF-16. Is there a way to know that?
Thanks
If you have a brand new empty database, before any tables are created, you can set the internal encoding used for Unicode text with the encoding pragma, and later use it to see the encoding being used (It defaults to UTF-8).
When storing or retrieving TEXT values, sqlite will automatically convert if needed between UTF-8 and UTF-16, so it doesn't matter too much which one is being used internally unless you're trying to get every last tiny bit of performance out of it.
In the link you provided it says explicitely:
const unsigned char sqlite3_column_text(sqlite3_stmt, int iCol);
const void sqlite3_column_text16(sqlite3_stmt, int iCol);
sqlite3_column_text → UTF-8 TEXT result sqlite3_column_text16 → UTF-16
TEXT result
These routines return information about a single column of the current
result row of a query. In every case the first argument is a pointer
to the prepared statement that is being evaluated (the sqlite3_stmt*
that was returned from sqlite3_prepare_v2() or one of its variants)
and the second argument is the index of the column for which
information should be returned. The leftmost column of the result set
has the index 0. The number of columns in the result can be determined
using sqlite3_column_count().
How performant is the SQLite3 REGEXP operator?
For simplicity, assume a simple table with a single column pattern and an index
CREATE TABLE `foobar` (`pattern` TEXT);
CREATE UNIQUE INDEX `foobar_index` ON `foobar`(`pattern`);
and a query like
SELECT * FROM `foobar` WHERE `pattern` REGEXP 'foo.*'
I have been trying to compare and understand the output from EXPLAIN and it seems to be similar to using LIKE except it will be using regexp for matching. However, I am not fully sure how to read the output from EXPLAIN and I'm not getting a grasp of how performant it will be.
I understand it will be slow compared to a indexed WHERE `pattern` = 'foo' query but is it slower/similar to LIKE?
sqlite does not optimize WHERE ... REGEXP ... to use indexes. x REGEXP y is simply a function call; it's equivalent to regexp(x,y). Also note that not all installations of sqlite have a regexp function defined so using it (or the REGEXP operator) is not very portable. LIKE/GLOB on the other hand can take advantage of indexes for prefix queries provided that some additional conditions are met:
The right-hand side of the LIKE or GLOB must be either a string literal or a parameter bound to a string literal that does not begin with a wildcard character.
It must not be possible to make the LIKE or GLOB operator true by having a numeric value (instead of a string or blob) on the left-hand side. This means that either:
the left-hand side of the LIKE or GLOB operator is the name of an indexed column with TEXT affinity, or
the right-hand side pattern argument does not begin with a minus sign ("-") or a digit.
This constraint arises from the fact that numbers do not sort in lexicographical order. For example: 9<10 but '9'>'10'.
The built-in functions used to implement LIKE and GLOB must not have been overloaded using the sqlite3_create_function() API.
For the GLOB operator, the column must be indexed using the built-in BINARY collating sequence.
For the LIKE operator, if case_sensitive_like mode is enabled then the column must indexed using BINARY collating sequence, or if case_sensitive_like mode is disabled then the column must indexed using built-in NOCASE collating sequence.
If the ESCAPE option is used, the ESCAPE character must be ASCII, or a single-byte character in UTF-8.
I need to use cast function with length of column in teradata.
say I have a table with following data ,
id | name
1|dhawal
2|bhaskar
I need to use cast operation something like
select cast(name as CHAR(<length of column>) from table
how can i do that?
thanks
Dhawal
You have to find the length by looking at the table definition - either manually (show table) or by writing dynamic SQL that queries dbc.ColumnsV.
update
You can find the maximum length of the actual data using
select max(length(cast(... as varchar(<large enough value>))) from TABLE
But if this is for FastExport I think casting as varchar(large-enough-value) and postprocessing to remove the 2-byte length info FastExport includes is a better solution (since exporting a CHAR() will results in a fixed-length output file with lots of spaces in it).
You may know this already, but just in case: Teradata usually recommends switching to TPT instead of the legacy fexp.
I would like to query an SQLite table that contains directory paths to find all the paths under some hierarchy. Here's an example of the contents of the column:
/alpha/papa/
/alpha/papa/tango/
/alpha/quebec/
/bravo/papa/
/bravo/papa/uniform/
/charlie/quebec/tango/
If I search for everything under /bravo/papa/, I would like to get:
/bravo/papa/
/bravo/papa/uniform/
I am currently trying to do this like so (see below for the long story of why I can't use more simple methods):
SELECT * FROM Files WHERE Path >= '/bravo/papa/' AND Path < '/bravo/papa0';
This works. It looks a bit weird, but it works for this example. '0' is the unicode code point 1 greater than '/'. When ordered lexicographically, all the paths starting with '/bravo/papa/' compare greater than it and less than 'bravo/papa0'. However, in my tests, I find that this breaks down when we try this:
SELECT * FROM Files WHERE Path >= '/' AND Path < '0';
This returns no results, but it should return every row. As far as I can tell, the problem is that SQLite is treating '0' as a number, not a string. If I use '0Z' instead of '0', for example, I do get results, but I introduce a risk of getting false positives. (For example, if there actually was an entry '0'.)
The simple version of my question is: is there some way to get SQLite to treat '0' in such a query as the length-1 string containing the unicode character '0' (which should sort strings such as '!', '*' and '/', but before '1', '=' and 'A') instead of the integer 0 (which SQLite sorts before all strings)?
I think in this case I can actually get away with special-casing a search for everything under '/', since all my entries will always start with '/', but I'd really like to know how to avoid this sort of thing in general, as it's unpleasantly surprising in all the same ways as Javascript's "==" operator.
First approach
A more natural approach would be to use the LIKE or GLOB operator. For example:
SELECT * FROM Files WHERE Path LIKE #prefix || '%';
But I want to support all valid path characters, so I would need to use ESCAPE for the '_' and '%' symbols. Apparently this prevents SQLite from using an index on Path. (See http://www.sqlite.org/optoverview.html#like_opt ) I really want to be able to benefit from an index here, and it sounds like that's impossible using either LIKE or GLOB unless I can guarantee that none of their special characters will occur in the directory name, and POSIX allows anything other than NUL and '/', even GLOB's '*' and '?' characters.
I'm providing this for context. I'm interested in other approaches to solve the underlying problem, but I'd prefer to accept an answer that directly addresses the ambiguity of strings-that-look-like-numbers in SQLite.
Similar questions
How do I prevent sqlite from evaluating a string as a math expression?
In that question, the values weren't quoted. I get these results even when the values are quoted or passed in as parameters.
EDIT - See my answer below. The column was created with the invalid type "STRING", which SQLite treated as NUMERIC.
* Groan *. The column had NUMERIC affinity because it had accidentally been specified as "STRING" instead of "TEXT". Since SQLite didn't recognize the type name, it made it NUMERIC, and because SQLite doesn't enforce column types, everything else worked as expected, except that any time a number-like string is inserted into that column it is converted into a numeric type.
TALES expression is new to me. Can I get some good reference for the same? Actually I wish to define a content rule for numeric input field using ploneformgen. Something like:
python: request.form.get('amt', False) <= 5000
then apply the rule.
Here 'amt' is a numeric/whole number field on the input form.
For reference, you should look at the official TALES specification, or refer to the TALES section of the Zope Page Templates reference.
In this case, you are using a plain python expression, and thus the normal rules of python code apply.
The expression request.form.get('amt', False) would return the request parameter 'amt' from the request, and if that's missing, return the boolean False, which you then compare to an integer value.
There are 2 things wrong with that expression: first of all you assume that the 'amt' parameter is an integer value. Even a PFG integer field however, is still a string in the request object. As such you'll need to convert in to an integer first before you can compare it.
Also, you fall back to a boolean, which in integer comparisons will be regarded as the equivalent of 0, better be explicit and use that instead:
python: int(request.form.get('amt', 0)) <= 5000
Note that for a PFG condition, you can also return a string error message instead of boolean True:
python: int(request.form.get('amt', 0)) <= 5000 or 'Amount must be not be greater than 5000'
Usually form parameters are passed in as strings if they are not defined on the application level otherwise e.g.
Zope will under the hood use the fieldname amt:int in order to convert the value to an integer.
So you may want to try to put an int(....) around the first expression.