I am trying to convert a hexadecimal value to a binary string within a Kusto query, but cannot find any method of doing this in the Kusto documentation / StackOverflow. I see that SQL has this method, but in my case I must use Kusto.
Kusto's scalar data types do not include binary, bytearray etc.
If I get your intention correctly, you are looking for something like this:
print hex_str = '48656c6c6f20576f726c64'
| mv-apply c = extract_all('(..)', hex_str) on (summarize str = make_string(make_list(toint(strcat('0x', c)))))
hex_str
str
48656c6c6f20576f726c64
Hello World
Fiddle
Related
I have a table in a SQLite database that contains about 15,000 single-page document scans stored as base64 strings. If I understand correctly, converting these to binary would reduce the size of the table by 25%.
Is it correct that it is not possible to convert the images to binary in SQLite directly but the base64 strings need to be converted to images first and then to binary? If so, will creating an image in Tcl from each base64 string and converting to binary suffice? And are there any tricky items that a novice is likely to overlook in attempting to do so?
When the test code below is executed, it appears that img_binary is binary data, but is this the correct approach?
Thank you.
set db "database_name"
sqlite3 dbws $db
#Base64 strings in database are prefixed with "data:image/gif;charset=utf-8;base64,"
set l [expr {[string length {data:image/gif;charset=utf-8;base64,}] -1}]
dbws eval { select img_base64 from lexi_raw where img_no = $nbr } {
image create photo ::img::lexi -data [string replace $img_base64 0 $l]
set img_binary [::img::lexi data -format png]; #Does this return binary to be written to SQLite?
puts $img_binary
}
SQLite doesn't have a built-in base64 decoder, but you can add one.
Try this:
package require sqlite3
sqlite3 db :memory:
db function base64decode -argcount 1 -deterministic -returntype blob {binary decode base64}
db eval {SELECT base64decode('SGVsbG8sIFdvcmxk') AS message}
The trick is the function method, which creates a new function (called base64decode) that is implemented by the given Tcl script fragment (binary decode base64; the argument is appended as a word to that). I'm passing -argcount 1 because we only ever want to pass a single argument here, -deterministic because the result is always the same for the same input, and -returntype blob because we know the result is binary.
If you want to do more complex processing (such as stripping a prefix as well) then it's best to implement by calling a procedure:
db function base64decode -argcount 1 -deterministic -returntype blob myDecoder
proc myDecoder value {
# Strip a leading prefix
regsub {^data:image/gif;charset=utf-8;base64,} $value "" value
# Decode the rest
return [binary decode base64 $value]
}
I am using Robot framework in python and I am running below query:
${queryResults1}= query select count(*) from table
It gives me below result:
${queryResults2} = [(91,)]
How to convert above result as String?
The response format is a list of tuples - as can also be seen by the sample you've pasted. Thus get the 1st column of the first row (in python/Robot Framework the lists are 0-based), and pass that to Convert To String:
${the value as string}= Convert To String ${queryResults1[0][0]}
I am trying the following query in marklogic-9:
cts:element-value-match(xs:QName("cd:modificationDate"), "[Y0001]-[M01]-[D01]T[H01]:[m01]:[s01].000Z", ("type=dateTime","timezone=TZ"))
to achieve this, but this gives me the following error:
[1.0-ml] XDMP-ARG: cts:element-value-match(xs:QName("cd:modificationDate"), "[Y0001]-[M01]-[D01]T[H01]:[m01]:[s01].000Z", ("type=dateTime", "timezone=TZ")) -- arg2 is invalid
What i want to do is to find out all those documents which conforms to this specific pattern of dateTime. We have a date-range index on this element - modificationDate.
How best can we do this using marklogic and xquery api.
cts:element-value-match is really only useful on string range indexes and even there it only takes simple wildcards (* and ?), not general regular expressions or date formats.
If your range index is a dateTime range index, then every value must conform to the proper xs:dateTime format, so this query would tell you nothing.
This will give you a list of all the URIs where you have a valid dateTime in that element:
cts:uris("", (),
cts:element-range-query(xs:QName("modificationDate"), ">", xs:dateTime("0001-01-01T00:00:00"))
)
I am trying to convert from a string representing hex data to the textual encoding (ASCII, UTF-8, etc.) of the hex data using purely SQLite language. Essentially I want the functionality of the X'[hex]' syntax, but applied to a programmatically derived hex string.
I want, for example, select X(hex_data_string) from ..., which is not legal SQLite syntax.
Obviously in the above snippet, I would not necessarily be able to output the data if it was not in a valid textual encoding. That is, if hex_data_string contains control chars, etc., the X() should fail in some way. If this is possible, there would have to be a default character encoding or the desired character encoding would have to be specified somehow.
I am not asking about how to retrieve the hex data string value from the SQLite database and then use C or some other facility to convert it. I am trying to perform this conversion in pure SQLite because I have queries that I check which return a text representation of hex characters representing binary data. Most of the binary data is ASCII, so I want to be able to quickly view the content of the binary data in my query output when applicable.
Intuitively, I figured this could be accomplished by casting the hex data string to a blob and using hex() but that still returns the hex data string.
Any ideas?
Possible duplicates:
SQLite X'...' notation with column data
sqlite char, ascii function
This is quite old but I was looking for the same thing so I thought I'd post:
It seems you can just cast hex data as a varchar to convert it to ascii.
ie:
select cast(data as varchar) from some_table will return a string representation of a binary field (data).
This is not possible in pure SQLite.
As an embedded database, SQLite is designed to provide only pure database functions, and leave the program logic to the the application.
You are supposed to retrieve the hex data string value from the SQLite database and then use C or some other facility to convert it.
If you control the program you're running the queries in, you could install a user-defined function that does this conversion.
This will not be fully useful since it can't really be used inline, but I have done this in the past when I just wanted to convert one row of data and not create/find another utility.
WITH RECURSIVE test(c,cur) as (
select '','686F77647921'
UNION ALL
select c || char((case substr(cur,1,1) when 'A' then 10 when 'B' then 11 when 'C' then 12 when 'D' then 13 when 'E' then 14 when 'F' then 15 else substr(cur,1,1) end)*16
+ (case substr(cur,2,1) when 'A' then 10 when 'B' then 11 when 'C' then 12 when 'D' then 13 when 'E' then 14 when 'F' then 15 else substr(cur,2,1) end)),
substr(cur,3)
from test where length(cur)>0
)
select * from test
I don't find any functions to convert string to binary in Teradata Database manual-SQL Reference -Functions and Operators.
Cast string to byte does not work too.
SELECT C1, C2 FROM table WHERE C1 < (cast( ('QREPI.\k'||'00'XC||'00'XC||'00'XC||'00'XC||'00'
XC||'00'XC||'00'XC||'00'XC||'00'XC||'00'XC||'00'XC||'00'XC||'00'XC||'
..') as byte(24)) ));
*** Failure 3532 Conversion between BYTE data and other types is illegal.
Statement# 1, Info =0
Anybody knows if Teradata provides a way for the conversion?
Any comments are highly appreciated.
If all you need is a literal, you can get a binary equivalent of your string like this:
SELECT C1, C2 FROM table
WHERE C1 < '51524550492e5c6b000000000000000000000000'xb
Otherwise, for data that is stored in your tables in hex it could be done within Teradata by writing a new UDF. Or you could export it to a file, transform it with a program, and load it back.
According to their blog you should be able to do it with implicit casting (but not explicit). I take that to mean something like:
SELECT C1, C2
FROM table
WHERE C1 < ('QREPI.\k'||'00'XC||'00'XC||'00'XC||'00'XC||'00'XC||
'00'XC||'00'XC||'00'XC||'00'XC||'00'XC||'00'XC||'00'XC||'00'XC||' ..');
Have you tried that?