I have a TERADATA dataset that resembles the below :
'Project: Hercules IssueType: Improvement Components: core AffectsVersions: 2.4.1 Priority: Minor Time: 15:25:23 04/06/2020'
I want to extract tag value from the above based on the key.
Ex:
with comm as
(
select 'Project: Hercules IssueType: Improvement Components: core AffectsVersions: 2.4.1 Priority: Minor' as text
)
select regexp_substr(comm.text,'[^: ]+',1,4)
from comm where regexp_substr(comm.text,'[^: ]+',1,3) = 'IssueType';
Is there a way to query without having to change the position arguments for every tag.
Also I am finding the last field a little tricky with date & time fields.
Any help is appreciated.
Thank you.
There's the NVP function to access Name/Value-pair data, but to split into multiple rows you need either strtok_split_to_table or regexp_split_to_table. The tricky part in your case are the delimiters, would be easier if they were unique instead of ' 'and ':':
WITH comm AS
(
SELECT 1 as keycol, -- should be a key column in your table, either numeric or varchar
'Project: Hercules IssueType: Improvement Components: core AffectsVersions: 2.4.1 Priority: Minor Time: 15:25:23 04/06/2020' AS text
)
SELECT id, tokennum, token,
-- get the key
StrTok(token,':', 1) AS "Key",
-- get the value (can't use StrTok because of ':' delimiter)
Substring(token From Position(': ' IN token)+2) AS "Value"
FROM TABLE
( RegExp_Split_To_Table(comm.keycol
,comm.text
,'( )(?=[^ ]+: )' -- assuming names don't contain spaces: split at the last space before ': '
, 'c')
RETURNS (id INT , tokennum INTEGER, token VARCHAR(1000) CHARACTER SET Latin)) AS dt
Related
I have MariaDB 10.1. - I can't use JSON functions - JSON_EXTRACT etc.).
In the database I have a table CONTRACTS and a column data, which contains JSON (data type TEXT):
{"879": "Test", "880": "15255", "881": "2021-10-22"}
And I need to find all records that have a key value of "880" in some range, eg greater than 10000 and less than 20000, ie. in this case, a record with a value of 15255.
Thanks for advice.
Maybe something like this:
SELECT
TRIM(BOTH '"' FROM
REGEXP_SUBSTR(REGEXP_SUBSTR(CONTRACTS.`data`, '"880": "[0-9]+"'), '"[0-9]+"$')
) * 1 BETWEEN 10000 AND 20000
FROM
(SELECT
'{"879": "Test", "880": "15255", "881": "2021-10-22"}' AS `data`
) AS CONTRACTS
So the most internal regexp gives you the key + value. The outer regexp takes that result and extracts the value in quotes. Trim the quotes and test the value. You could use the entire TRIM(...) as a criterium .
I have a sqlLite database which contain in some field sql statement, this sql statement are needed to create some chart in an IOS Application.
Now I need to display this chart in a web application which use Postgresql
I need to find a script who convert from sqllite syntax to postgresql syntax
things like printf, current date etc...
I am asking if there is some ready to use script for this kind of conversion
PS : I am using Symfony for the back-end so PHP
Example:
SELECT
r.agent AS gebiet,
r.invoiceno AS rechnung,
r.infotext AS auftrag,
c.companyno AS kundenr,
c.companyname AS kunde,
r.itemno AS artikelnr,
r.itemtext AS artikel,
SUM(r.qty) || ' ' || r.unit AS menge,
printf('%.2f', SUM(r.turnover) / SUM(r.qty)) AS preis,
printf('%.2f', SUM(r.turnover)) || ' ' || r.currency AS gesamt,
'2' AS 'sys_align9',
'2' AS 'sys_align10',
'2' AS 'sys_align11'
FROM
invoices r
INNER JOIN
company c
ON
r.companyno = c.companyno
WHERE
r.agent = ?
Disclaimer: I don't know SQLite ;)
The query structure itself looks good, but two things will not work with Postgres:
I assume printf() formats the output to two decimals, so to_char() is probably what you are looking for
identifiers need to be enclosed in double quotes, not single quotes. So AS 'sys_align11' should be `AS "sys_align11" but the quotes are not required to begin with.
SELECT
r.agent AS gebiet,
r.invoiceno AS rechnung,
r.infotext AS auftrag,
c.companyno AS kundenr,
c.companyname AS kunde,
r.itemno AS artikelnr,
r.itemtext AS artikel,
SUM(r.qty) || ' ' || r.unit AS menge,
to_char(SUM(r.turnover) / SUM(r.qty), '0.00') AS preis,
to_char(SUM(r.turnover),'0.00') || ' ' || r.currency AS gesamt,
'2' AS sys_align9,
'2' AS sys_align10,
'2' AS sys_align11
FROM
invoices r
INNER JOIN
company c
ON
r.companyno = c.companyno
WHERE
r.agent = ?
If turnover is an integer (or bigin) you need to cast it to numeric, otherwise the division is done as an integer division, e.g. SUM(r.turnover)::numeric
As you are using an aggregat function you will need some kind of group by in Postgres - otherwise the result wouldn't be deterministic.
If invoiceno is the primary key of the invoices table, a `group by r.invoiceno' should be enough.
how do I code this properly to work in Oracle SQL :
update table_name
set field_name =
replace(field_name, x'BF', x'00')
where condition expression ;
Not sure how to code the replace all occurrence of hex 'BF' with null value hex'00' contained in data field field_name.
You can use the unistr() function to provide a Unicode character. e.g.:
update table_name
set field_name = replace(field_name, unistr('\00bf'))
where condition expression ;
which would remove the ¿ character completely; or to replace it with a null character:
set field_name = replace(field_name, unistr('\00bf'), unistr('\0000'))
though I suspect sticking a null in there will confuse things even more later, when some other system tries to read that text and stops at the null.
Quick demo:
with t (str) as (
select 'A ¿ char' from dual
)
select str,
replace(str, unistr('\00bf')) as removed,
replace(str, unistr('\00bf'), unistr('\0000')) as replaced,
dump(replace(str, unistr('\00bf')), 16) as removed_hex,
dump(replace(str, unistr('\00bf'), unistr('\0000')), 16) as replaced_hex
from t;
STR REMOVED REPLACED REMOVED_HEX REPLACED_HEX
--------- --------- --------- ----------------------------------- -----------------------------------
A ¿ char A char A char Typ=1 Len=7: 41,20,20,63,68,61,72 Typ=1 Len=8: 41,20,0,20,63,68,61,72
(Just as an example of the problems you'll have - because of the null I couldn't copy and paste that from SQL Developer, and had to switch to SQL*Plus...)
The first dump shows the two spaces (hex 20) next to each other; the second shows a null character between them.
Ok, so I have a basic table called "ledger", it contains fields of various types, integers, varchar, etc.
In my program, I used to use a query with no "from" predicate to collect all of the rows, which of course works fine. But... I changed my code to allow selecting one row at a time using "where acctno = x" (where X is the account number I want to select at the time).
I thought this must be a bug in the client library for my programming language, so I tested it in the SQLite command-line client - and it still doesn't work!
I am relatively new to SQLite, but I have been using Oracle, MS SQL Server, etc. for years and never seen this type of issue before.
Other things I can tell you:
* Queries using other integer fields also don't work
* Queries on char fields work
* Querying it as a string (with the account number on quotes) still doesn't work. (I thought maybe the numbers were stored as a string inadvertently).
* Accessing rows by rowid works fine - which is why I can edit the database with GUI tools with no noticeable problem.
Examples:
Query with no WHERE (works fine):
1|0|0|JPY|8|Paid-In Capital|C|X|0|X|0|0||||0|0|0|
0|0|0|JPY|11|Root Account|P|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
3|0|0|JPY|13|Mitsubishi Bank Futsuu|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
4|0|0|JPY|14|Japan Post Bank|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
...
Query with WHERE clause: (no results)
sqlite> select * from ledger where acctno=1;
sqlite>
putting quotes around the 1 above changes nothing.
Interestingly enough, "select * from ledger where acctno > 1" returns results! However since it returns ALL results, it's not terrible useful.
I'm sure someone will ask about the table structure, so here goes:
sqlite> .schema ledger
CREATE TABLE "LEDGER" (
"ACCTNO" integer(10,0) NOT NULL,
"drbal" integer(20,0) NOT NULL,
"crbal" integer(20,0) NOT NULL,
"CURRKEY" char(3,0) NOT NULL,
"TEXTKEY" integer(10,0),
"TEXT" VARCHAR(64,0),
"ACCTYPECD" CHAR(1,0) NOT NULL,
"ACCSTCD" CHAR(1,0),
"PACCTNO" number(10,0) NOT NULL,
"CATCD" number(10,0),
"TRANSNO" number(10,0) NOT NULL,
"extrefno" number(10,0),
"UPDATEUSER" VARCHAR(32,0),
"UPDATEDATE" text(8,0),
"UPDATETIME" TEXT(6,0),
"PAYEECD" number(10,0) NOT NULL,
"drbal2" number(10,0) NOT NULL,
"crbal2" number(10,0) NOT NULL,
"delind" boolean,
PRIMARY KEY("ACCTNO"),
CONSTRAINT "fk_curr" FOREIGN KEY ("CURRKEY") REFERENCES "CURRENCY" ("CUR
RKEY") ON DELETE RESTRICT ON UPDATE CASCADE
);
The strangest thing is that I have other similar tables where this works fine!
sqlite> select * from journalhdr where transno=13;
13|Test transaction ATM Withdrawel 20130213|20130223||20130223||
TransNo in that table is also integer (10,0) NOT NULL - this is what makes me thing it is something to do with the values.
Another clue is that the sort order seems to be based on ascii, not numeric:
sqlite> select * from ledger order by acctno;
0|0|0|JPY|11|Root Account|P|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
1|0|0|JPY|8|Paid-In Capital|C|X|0|X|0|0||||0|0|0|
10|0|0|USD|20|Sallie Mae|L|X|0|X|0|0|SYSTEM|20121209|153900|0|0|0|
21|0|0|USD|21|Skrill|A|X|0|X|0|0|SYSTEM|20121209|154000|0|0|0|
22|0|0|USD|22|AES|L|X|0|X|0|0|SYSTEM|20121209|154200|0|0|0|
23|0|0|JPY|23|Marui|L|X|0|X|0|0|SYSTEM|20121209|154400|0|0|0|
24|0|0|JPY|24|Amex JP|L|X|0|X|0|0|SYSTEM|20121209|154500|0|0|0|
3|0|0|JPY|13|Mitsubishi Bank Futsuu|A|X|0|X|0|0|SYSTEM|20121209|150000|0|0|0|
Of course the sort order on journalhdr (where the select works properly) is numeric.
Solved! (sort-of)
The data can be fixed like this:
sqlite> update ledger set acctno = 23 where rowid = 13;
sqlite> select * from ledger where acctno = 25;
25|0|0|JPY|0|Test|L|X|0|X|0|0|SYSTEM|20130224|132500|0|0|0|
Still, if it was stored as strings, then that leave a few questions:
1. Why couldn't I select it as a string using the quotes?
2. How did it get stored as a string since it is a valid integer?
3. How would you go about detecting this problem normally besides noticing bizzarre symptoms?
Although the data would normally be entered by my program, some of it was created by hand using Navicat, so I assume the problem must lie there.
You are victim of SQLite dynamic typing.
Even though SQLite defines system of type affinity, which sets some rules on how input strings or numbers will be converted to actual internal values, but it does NOT prevent software that is using prepared statements to explicitly set any type (and data value) for the column (and this can be different per row!).
This can be shown by this simple example:
CREATE TABLE ledger (acctno INTEGER, name VARCHAR(16));
INSERT INTO ledger VALUES(1, 'John'); -- INTEGER '1'
INSERT INTO ledger VALUES(2 || X'00', 'Zack'); -- BLOB '2\0'
I have inserted second row not as INTEGER, but as binary string containing embedded zero byte. This reproduces your issue exactly, see this SQLFiddle, step by step. You can also execute these commands in sqlite3, you will get the same result.
Below is Perl script that also reproduces this issue
This script creates just 2 rows with acctno having values of integer 1 for first, and "2\0" for second row. "2\0" means string consisting of 2 bytes: first is digit 2, and second is 0 (zero) byte.
Of course, it is very difficult to visually tell "2\0" from just "2", but this is what script below demonstrates:
#!/usr/bin/perl -w
use strict;
use warnings;
use DBI qw(:sql_types);
my $dbh = DBI->connect("dbi:SQLite:test.db") or die DBI::errstr();
$dbh->do("DROP TABLE IF EXISTS ledger");
$dbh->do("CREATE TABLE ledger (acctno INTEGER, name VARCHAR(16))");
my $sth = $dbh->prepare(
"INSERT INTO ledger (acctno, name) VALUES (?, ?)");
$sth->bind_param(1, "1", SQL_INTEGER);
$sth->bind_param(2, "John");
$sth->execute();
$sth->bind_param(1, "2\0", SQL_BLOB);
$sth->bind_param(2, "Zack");
$sth->execute();
$sth = $dbh->prepare(
"SELECT count(*) FROM ledger WHERE acctno = ?");
$sth->bind_param(1, "1");
$sth->execute();
my ($num1) = $sth->fetchrow_array();
print "Number of rows matching id '1' is $num1\n";
$sth->bind_param(1, "2");
$sth->execute();
my ($num2) = $sth->fetchrow_array();
print "Number of rows matching id '2' is $num2\n";
$sth->bind_param(1, "2\0", SQL_BLOB);
$sth->execute();
my ($num3) = $sth->fetchrow_array();
print "Number of rows matching id '2<0>' is $num3\n";
Output of this script is:
Number of rows matching id '1' is 1
Number of rows matching id '2' is 0
Number of rows matching id '2<0>' is 1
If you were to look at resultant table using any SQLite tool (including sqlite3), it will print 2 for second row - they all get confused by trailing 0 inside a BLOB when it gets coerced to string or number.
Note that I had to use custom param binding to coerce type to BLOB and permit null bytes stored:
$sth->bind_param(1, "2\0", SQL_BLOB);
Long story short, it is either some of your client programs, or some of client tools like Navicat which screwed it up.
I'm preety new to SQLite.
I have a preety basic question.. Why can't I select rows where specific column equals zero?
The is_unwanted column is type TINYINT (which I see in SQLite basically means INTEGER)
So, I have only one record in the database (for testing).
When I try
SELECT is_unwanted FROM 'urls'
I get a result of "0" (zero), which is fine because that column contains the actual number 0.
I tried =>
SELECT * FROM 'urls' WHERE is_unwanted = 0
And got NO result, but
SELECT * FROM 'urls' WHERE is_unwanted <> 0
gives me result.
What am I doing wrong??
Try running
select '{' || is_unwanted || '}' from urls
to see if the value in the database is really a string containing spaces.
SQLite is a dynamically typed database; when you specify TINYINT is is a hint (SQLite uses the term "affinity") for the column. You can use
select is_unwanted, typeof(is_unwanted) from urls
to see the values with their types.
You could try:
SELECT * FROM urls WHERE coalesce(is_unwanted,'') = ''