Using query_parameters, how can I:
specify a result column name (ex: summarize ResultColumnName = count())
specify the value of a bin, when value is actually the name of a column in the table
This is easiest to summarize with an example:
let myTable = datatable (Timestamp:datetime)
[datetime(1910-06-11),
datetime(1930-01-01),
datetime(1997-06-25),
datetime(1997-06-25)];
let UntrustedUserInput_ColumnName = "MyCount"; // actually from query_parameters
let UntrustedUserInput_BinValue = "Timestamp"; // actually from query_parameters
let UntrustedUserInput_BinRoundTo = "365d"; // actually from query_parameters
// the query I really want to perform
myTable
| summarize MyCount=count() by bin(todatetime(Timestamp), totimespan(365d));
// what the query looks like if I use query_parameters
myTable
| summarize UntrustedUserInput_ColumnName=count() by bin(todatetime(UntrustedUserInput_BinValue), totimespan(UntrustedUserInput_BinRoundTo));
Results:
Timestamp MyCount
--------- -------
1909-09-26T00:00:00Z 1
1929-09-21T00:00:00Z 1
1996-09-04T00:00:00Z 2
Column1 UntrustedUserInput_ColumnName
------- -----------------------------
4
I can't find a solution to #1.
It appears #2 can almost be solved by using column_ifexists, but I don't have a "default" to fall back on, I'd rather just fail if the column doesn't exist.
Treating column names as variables is not possible since columns names are part of the result schema coming out of each operator (with the exception of the "evaluate" operator, see specifically the pivot plugin).
There actually is a way to set variable names to a column, using a hacky trick:
let VariableColumnName = "TestColumn"; // the new column name that you want
range i from 1 to 5 step 1 // this is just a sample query
| project pack(VariableColumnName, i) // this created a JSON
| evaluate bag_unpack(Column1) // unpacking the JSON creates a column with a dynamic name
This will return a column named TestColumn, which is set in VariableColumnName.
I want to select a column from a JuliDB database. The problem is that I can't do it with string (not a symbol that starts with :), for example:
db = loadtable("table.dat")
#This table has 3 columns named position_1, position2, position_3
pos_num = 3
column_name = "position_$pos_num"
select(db,column_name)
If I do that, then the following error appears:
column position_3 not found.
Any suggestion?
You can create a symbol directly Symbol("position_$pos_num") or from your string with Symbol(column_name).
I have a string like this ('car, bus, train')
I want to convert it to be used in an in-clause. Basically I want to convert it to
('car','bus','train'). Please how do I do this in Teradata
I don't know how you are getting data like that, but if you have no control over that, you can use STRTOK_SPLIT_TO_TABLE.
select t.* from table (strtok_split_to_table(1,'car, bus, train',',')
returns (outkey integer,tokennum integer,resultstring varchar(25))) as t
Run by itself, that gives you:
outkey tokennum resultstring
1 1 car
1 2 bus
1 3 train
You can use that as a derived table and join it to the table you want to filter by. Something like:
select
<your table>.*
from
<your table>
inner join (select t.* from table (strtok_split_to_table(1,'car, bus, train',',')
returns (outkey integer,tokennum integer,resultstring varchar(25))) as t) dt
on yourtable.yourcolumn = dt.resultstring
here is the another way of spliting the input for n number of commas and use IN clause.
SELECT regexp_substr('car,bus,train','[^,]+',1,day_of_calendar) fields
FROM sys_calendar.calendar
WHERE day_of_calendar <= (CHAR('car,bus,train') - CHAR(oreplace('car,bus,train',',','')))+1;
Output of the Query
fields
~~~~~~~~
bus
car
train
Here is the systax to use in where clause
SELECT * FROM <your table>
WHERE yourtable.requiredColumn in
(
SELECT regexp_substr('car,bus,train','[^,]+',1,day_of_calendar) fields
FROM sys_calendar.calendar
WHERE
day_of_calendar <= (CHAR('car,bus,train') - CHAR(oreplace('car,bus,train',',','')))+1
);
Basically what we are doing here is splitting the string for each comma and below function is calculating number of commas in the string
(CHAR('car,bus,train') - CHAR(oreplace('car,bus,train',',','')))+1
i search for a special kind of query in SQLite
to sort a notes table.
The result from the query should be like this:
id oid
1 1
2 1,1
5 1,1,a
6 1,1,a,1
3 1,1,A
4 1,1,A,1
But with the folling code I receive this:
CREATE TABLE note (
id INTEGER PRIMARY KEY AUTOINCREMENT,
created DATETIME DEFAULT CURRENT_TIMESTAMP,
oid VARCHAR unique,
tit VARCHAR,
dsc VARCHAR
);
select id, oid from note
order by oid collate NOCASE
Result:
id oid
1 1
2 1,1
5 1,1,a
3 1,1,A
6 1,1,a,1
4 1,1,A,1
Any suggestions?
Thanks
--jonah
The following transforms the sort keys so that the normal case sensitive ordering yields the requested result:
If there was a togglecase() function, the function would uppercase lowercase and lowercase uppercase (for example Hello => hELLO), one could ORDER BY togglecase(oid) and the result would be in the order requested.
You could define such a function and expose it to SQLite as a UDF. It could also be possible to write this function using builtin SQLite functions but I don't know them well enough to give an answer using them. The following is an example of such a function in Python:
def togglecase(s):
def toggle(l):
if l.isupper():
return l.lower()
if l.islower():
return l.upper()
return l
return ''.join(toggle(l)
for l in s)
Note that for proper Unicode support it needs to iterate over graphemes. Not over code points.
See that this does what I described it to do:
>>> togglecase("1,1,A")
'1,1,a'
>>> togglecase("1,1,a")
'1,1,A'
It is possible to test if this sorts correctly in Python:
>>> sorted(["1", "1,1", "1,1,a", "1,1,a,1", "1,1,A", "1,1,A,1"])
['1', '1,1', '1,1,A', '1,1,A,1', '1,1,a', '1,1,a,1']
See how the uppercase follows the lowercase:
>> sorted(["1", "1,1", "1,1,a", "1,1,a,1", "1,1,A", "1,1,A,1"], key=togglecase)
['1', '1,1', '1,1,a', '1,1,a,1', '1,1,A', '1,1,A,1']
Now if you use it in SQLite like:
SELECT id, oid
FROM note
ORDER BY togglecase(oid)
This should result in:
1 "1"
2 "1,1"
3 "1,1,a"
4 "1,1,a,1"
5 "1,1,A"
6 "1,1,A,1"
The code is untested except for the togglecase function.
You are getting that result because the sorting is pecified to be NOCASE. That means that "a" and "A" are equals. So, first the rows with "a/A" and nothing after, and then rows with "a/A" and data after.
If you make the query CASE SENSITIVE, you will get a different result. BUT "A" comes befores "a" in case sensitive sort:
SELECT id, oid
FROM note
ORDER by oid
Results:
1 "1"
2 "1,1"
5 "1,1,A"
6 "1,1,A,1"
3 "1,1,a"
4 "1,1,a,1"