I have a column field: location. I need to extract the string between the first and second delimeter ('/').
I already have a column name where I ltrim to the first '/'. I've tried to create a similar query with a combination of rtrim, replace, substr as my source column to no avail. Here is what my data looks like. I want to extract AML, for example. Right now, there are only three options (value1, value2, value3) between the first and second delimiters, but there could be more later.
Attribute data
----------+--------------------------------------------------------------------------------------------------------------------
Field | First value
----------+--------------------------------------------------------------------------------------------------------------------
location | './AML/Counties/*****************kyaml_20190416_transparent_mosaic_group1.tif'
name | 'kyaml_20190416_transparent_mosaic_group1.tif'
----------+--------------------------------------------------------------------------------------------------------------------
What is the best way of creating my column source with the value from location?
Output should be like this:
Attribute data
----------+--------------------------------------------------------------------------------------------------------------------
Field | First value
----------+--------------------------------------------------------------------------------------------------------------------
location | './AML/Counties/****************kyaml_20190416_transparent_mosaic_group1.tif'
name | 'kyaml_20190416_transparent_mosaic_group1.tif'
source | 'AML'
----------+--------------------------------------------------------------------------------------------------------------------
With substr() and instr():
select *,
substr(
substr(location, instr(location, '/') + 1),
1,
instr(substr(location, instr(location, '/') + 1), '/') - 1
) as source
from data
See the demo.
I used forpas query to modify my query. Here is my final query
ogrinfo box_tiles.shp -dialect SQLITE -sql \
"UPDATE box_tiles SET source = \
substr(\
substr(location, instr(location, '/') + 1), 1, \
instr(substr(location, instr(location, '/') + 1), '/') - 1)"
Related
Using query_parameters, how can I:
specify a result column name (ex: summarize ResultColumnName = count())
specify the value of a bin, when value is actually the name of a column in the table
This is easiest to summarize with an example:
let myTable = datatable (Timestamp:datetime)
[datetime(1910-06-11),
datetime(1930-01-01),
datetime(1997-06-25),
datetime(1997-06-25)];
let UntrustedUserInput_ColumnName = "MyCount"; // actually from query_parameters
let UntrustedUserInput_BinValue = "Timestamp"; // actually from query_parameters
let UntrustedUserInput_BinRoundTo = "365d"; // actually from query_parameters
// the query I really want to perform
myTable
| summarize MyCount=count() by bin(todatetime(Timestamp), totimespan(365d));
// what the query looks like if I use query_parameters
myTable
| summarize UntrustedUserInput_ColumnName=count() by bin(todatetime(UntrustedUserInput_BinValue), totimespan(UntrustedUserInput_BinRoundTo));
Results:
Timestamp MyCount
--------- -------
1909-09-26T00:00:00Z 1
1929-09-21T00:00:00Z 1
1996-09-04T00:00:00Z 2
Column1 UntrustedUserInput_ColumnName
------- -----------------------------
4
I can't find a solution to #1.
It appears #2 can almost be solved by using column_ifexists, but I don't have a "default" to fall back on, I'd rather just fail if the column doesn't exist.
Treating column names as variables is not possible since columns names are part of the result schema coming out of each operator (with the exception of the "evaluate" operator, see specifically the pivot plugin).
There actually is a way to set variable names to a column, using a hacky trick:
let VariableColumnName = "TestColumn"; // the new column name that you want
range i from 1 to 5 step 1 // this is just a sample query
| project pack(VariableColumnName, i) // this created a JSON
| evaluate bag_unpack(Column1) // unpacking the JSON creates a column with a dynamic name
This will return a column named TestColumn, which is set in VariableColumnName.
There is a column.
And it has values like
'/abc/def/ghi/w1.xyz'
'/jkl/mno/r.stuv'
(it's path data and the number of '/'s in each value is not fixed.)
how can i get substring column which has values like
'/abc/def/ghi/'
'/jkl/mno/'
(extracting only the directory part. removing the file part.)
i read about substr(X,Y), substr(X,Y,Z), instr(X,Y).
but it's not easy to apply them because the number of '/'s in each value is not fixed and instr(X,Y) seems to find the first occurrence from the left.
With a recursive CTE:
create table tablename(col text);
insert into tablename(col) values
('/abc/def/ghi/w1.xyz'),
('/jkl/mno/r.stuv');
with recursive cte(col, pos, rest) as (
select col, instr(col, '/') pos, substr(col, instr(col, '/') + 1) rest
from tablename
union all
select col, instr(rest, '/'), substr(rest, instr(rest, '/') + 1)
from cte
where instr(rest, '/') > 0
)
select col, substr(col, 1, sum(pos)) path
from cte
group by col
See the demo.
Results:
| col | path |
| ------------------- | ------------- |
| /abc/def/ghi/w1.xyz | /abc/def/ghi/ |
| /jkl/mno/r.stuv | /jkl/mno/ |
If you are on Unix/Linux environment, you could do something like this (let's take an example).
Let's say test.db was your SQLite3 database with a table like so:
create table test (dataset text);
insert into test (dataset) values ('/abc/def/ghi/w1.xyz');
insert into test (dataset) values ('/jkl/mno/r.stuv');
From command line, you can run:
sqlite3 test.db -batch -noheader "select dataset from test"
That'll give you this as the output (I know that's not the output you want, but just read on):
/abc/def/ghi/w1.xyz
/jkl/mno/r.stuv
Explanation
the -batch and -noheader switches suppress all output except for the data resulting from the SQL statement
sqlite3 test.db -batch -noheader "sql statement" runs the SQL statement you provided and the output is dumped on the screen (stdout)
Solution
Now, we'll use awk with it to get what you want like so:
sqlite3 test.db -batch -noheader "select dataset from test" | awk -F'/' 'BEGIN{OFS="/"} {$NF = ""; print $0}'
Result will be:
/abc/def/ghi/
/jkl/mno/
Crude explanation
awk works on every line output by your sqlite3 command
the line is split by / character with -F'/' switch
since we want the output to contain the delimiters as is, we set the output field separator to be '/' as well using OFS='/'
we set number of fields (NF) in a way that the last item is ignored and then we print the rest of the data
as the other split fields are printed, OFS inserts / between those fields
I would like to repeat this command as many times as there is still sometextin the field note (several rows from the table itemNotes could have one or more sometext in the field note):
UPDATE itemNotes
SET
note = SUBSTR(note, 0, INSTR(LOWER(note), 'sometext')) || 'abc' || SUBSTR(note, INSTR(LOWER(note), 'sometext')+sometext_len)
WHERE
INSTR(LOWER(note), 'sometext') >= 0;
So a proto-code would be :
While (SELECT * FROM itemNotes WHERE note like "%sometext%") >1
UPDATE itemNotes
SET
note = SUBSTR(note, 0, INSTR(LOWER(note), 'sometext')) || 'abc' || SUBSTR(note, INSTR(LOWER(note), 'sometext')+sometext_len)
WHERE
INSTR(LOWER(note), 'sometext') >= 0;
END
But apparently Sqlite3 doesn't support While loop or for loop. They can be emulated with something like this but I have difficulties integrating what I want with this query:
WITH b(x,y) AS
(
SELECT 1,2
UNION ALL
SELECT x+ 1, y + 1
FROM b
WHERE x < 20
) SELECT * FROM b;
Any idea how to do this?
PS: I don't use replace because I want to replace all the case combinations of sometext (e.g. sometext, SOMEtext, SOmeText...) cf this question
Current input and desired output:
For a single row, a note field could look like (and many rows in the table itemNotescould look like this one):
There is SOmetext and also somETExt and more SOMETEXT and even more sometext
The query should output:
There is abc and also abc and more abc and even more abc
I am doing it on the zotero.sqlite, which is created by this file (line 85). The table is created by this query
CREATE TABLE itemNotes (
itemID INTEGER PRIMARY KEY,
parentItemID INT,
note TEXT,
title TEXT,
FOREIGN KEY (itemID) REFERENCES items(itemID) ON DELETE CASCADE,
FOREIGN KEY (parentItemID) REFERENCES items(itemID) ON DELETE CASCADE
);
You just have your answer in your query:
UPDATE itemNotes
SET
note = SUBSTR(note, 0, INSTR(LOWER(note), 'sometext')) || 'abc' || SUBSTR(note, INSTR(LOWER(note), 'sometext')+sometext_len)
WHERE
note LIKE "%sometext%";
It will update all rows that contain sometext in the note field
UPDATE
If you want to update the field which has multiple occurrences in different cases and maintain the rest of the text the simplest solution imo is to use regex and for that you need an extension
UPDATE itemNotes
SET
note = regex_replace('\bsometext\b',note,'abc')
WHERE
note LIKE "%sometext%";
As recommended by Stephan in his last comment, I used python to do this.
Here is my code :
import sqlite3
import re
keyword = "sometext"
replacement = "abc"
db = sqlite3.connect(path_to_sqlite)
cursor = db.cursor()
cursor.execute(f'SELECT * FROM itemNotes WHERE note like "%{keyword}%"')
for row in cursor.fetchall():
row_regex = re.compile(re.escape(keyword), re.IGNORECASE)
row_regex_replaced = row_regex.sub(replacement, row[2])
rowID = row[0]
sql = "REPLACE INTO itemNotes (itemID,note) VALUES (?,?)"
data = (rowID, row_regex_replaced)
cursor.execute(sql, data)
db.commit()
it is possible do a SELECT CASE, decode, nvl or another query function when I need verify if the return of a select query is empty or has a value?
For example, I have this:
Record | type | id_customer
-------+--------+-------------
1 | T | cus1
2 | A | cus2
3 | T | cus3
4 | | cus4
If I do this:
select decode(type,'T','Main','A','Adicional','none') from table where record=1;
I get Main.
If I fo this:
select decode(type,'T','Main','A','Adicional','none') from table where record=4;
I get none.
But if I do this:
select decode(type,'T','Main','A','Aditional','none') from table where record=5;
I get nothing, and is logic. So, I need get the decode value when the row exist and a text if the rows no exist.
So, I tried with SELECT CASE but is not posible get a value using COUNT. For example like this:
SELECT
CASE
WHEN count(1)>0 THEN decode(type,'T','Main','A','Aditional','none')
ELSE '-'
END
FROM TABLE WHERE record=5;
And get a ' - ', or the same if the record is 2, get 'Aditional'
Thanks a lot.
You can use aggregate functions min or max outside expression:
select max(decode(type,'T','Main','A','Aditional','none'))
from table
where record=5;
If query returns one row, you get value of that row. If query returns 0 rows, you get NULL.
Then you can replace NULL using nvl:
select nvl(max(decode(type,'T','Main','A','Aditional','none')), ' - ')
from table
where record=5;
EDIT
Also, if you need to choose one string from several:
select decode(max(decode(type,'T', 2, 'A', 1, 0)), 0, 'none', 1, 'Additional', 2, 'Main', null, ' - ')
from table
where record=5;
This is an option:
select decode(type,'T','Main','A','Aditional','none')
from table
where record = 5
union all
select '-'
from dual
where not exists (select 1 from table where record = 5);
It selects records with record = 5 and unifies them with '-', if no records exits with record = 5. Check out this Fiddle.
I am looking for a method to return the results of an SQLite query as a single string for use in an internal trigger.
Something like Python's 'somestring'.join() method.
Table: foo
id | name
1 | "foo"
2 | "bar"
3 | "bro"
Then a select statement:
MAGIC_STRING_CONCAT_FUNCTION(SELECT id FROM foo,",");
To return
"1,2,3"
You're looking for the group_concat function:
group_concat((SELECT id FROM foo), ",");
The following is the description of the function group_concat from the documentation:
group_concat(X) group_concat(X,Y)
The group_concat() function returns a string which is the
concatenation of all non-NULL values of X. If parameter Y is present
then it is used as the separator between instances of X. A comma (",")
is used as the separator if Y is omitted. The order of the
concatenated elements is arbitrary.