Repeat a command while true or x times (equivalent of while/for loop) - sqlite

I would like to repeat this command as many times as there is still sometextin the field note (several rows from the table itemNotes could have one or more sometext in the field note):
UPDATE itemNotes
SET
note = SUBSTR(note, 0, INSTR(LOWER(note), 'sometext')) || 'abc' || SUBSTR(note, INSTR(LOWER(note), 'sometext')+sometext_len)
WHERE
INSTR(LOWER(note), 'sometext') >= 0;
So a proto-code would be :
While (SELECT * FROM itemNotes WHERE note like "%sometext%") >1
UPDATE itemNotes
SET
note = SUBSTR(note, 0, INSTR(LOWER(note), 'sometext')) || 'abc' || SUBSTR(note, INSTR(LOWER(note), 'sometext')+sometext_len)
WHERE
INSTR(LOWER(note), 'sometext') >= 0;
END
But apparently Sqlite3 doesn't support While loop or for loop. They can be emulated with something like this but I have difficulties integrating what I want with this query:
WITH b(x,y) AS
(
SELECT 1,2
UNION ALL
SELECT x+ 1, y + 1
FROM b
WHERE x < 20
) SELECT * FROM b;
Any idea how to do this?
PS: I don't use replace because I want to replace all the case combinations of sometext (e.g. sometext, SOMEtext, SOmeText...) cf this question
Current input and desired output:
For a single row, a note field could look like (and many rows in the table itemNotescould look like this one):
There is SOmetext and also somETExt and more SOMETEXT and even more sometext
The query should output:
There is abc and also abc and more abc and even more abc
I am doing it on the zotero.sqlite, which is created by this file (line 85). The table is created by this query
CREATE TABLE itemNotes (
itemID INTEGER PRIMARY KEY,
parentItemID INT,
note TEXT,
title TEXT,
FOREIGN KEY (itemID) REFERENCES items(itemID) ON DELETE CASCADE,
FOREIGN KEY (parentItemID) REFERENCES items(itemID) ON DELETE CASCADE
);

You just have your answer in your query:
UPDATE itemNotes
SET
note = SUBSTR(note, 0, INSTR(LOWER(note), 'sometext')) || 'abc' || SUBSTR(note, INSTR(LOWER(note), 'sometext')+sometext_len)
WHERE
note LIKE "%sometext%";
It will update all rows that contain sometext in the note field
UPDATE
If you want to update the field which has multiple occurrences in different cases and maintain the rest of the text the simplest solution imo is to use regex and for that you need an extension
UPDATE itemNotes
SET
note = regex_replace('\bsometext\b',note,'abc')
WHERE
note LIKE "%sometext%";

As recommended by Stephan in his last comment, I used python to do this.
Here is my code :
import sqlite3
import re
keyword = "sometext"
replacement = "abc"
db = sqlite3.connect(path_to_sqlite)
cursor = db.cursor()
cursor.execute(f'SELECT * FROM itemNotes WHERE note like "%{keyword}%"')
for row in cursor.fetchall():
row_regex = re.compile(re.escape(keyword), re.IGNORECASE)
row_regex_replaced = row_regex.sub(replacement, row[2])
rowID = row[0]
sql = "REPLACE INTO itemNotes (itemID,note) VALUES (?,?)"
data = (rowID, row_regex_replaced)
cursor.execute(sql, data)
db.commit()

Related

SQLite prepending zeros

Does anyone know how to prepend leading zeros to a column in sqlite?
I am getting columns like 1, 2 and for our data import they need to be 3 character codes with leading zeros.
I've tried this
update LockBox
set CustomField2 = case
when CustomField2 = '1' then '001'
end
WHERE CustomField2 = '1';
but does not give me the correct result. I'm probably doing this wrong. I'm more of a sql server guy not a sqlite person
I would do this in SQL Server
SELECT RIGHT('000'+CAST(field AS VARCHAR(3)),3)
How can I accomplish this in SQLite. I want to update the table then export to CSV to import into our business application.
Use substr:
update LockBox
set CustomField2 = substr('000' || CustomField2, -3, 3);
provided that all values in CustomField2 are no longer than 3 digits.
Define the column as text?
$ sqlite3 types
sqlite> create table T( i integer, c text );
sqlite> insert into T values ( '001', '001' );
sqlite> select * from T;
i c
---------- ----------
1 001

Change phone number format

I have phone number field in database. It has already data.
I want to change my phone number format to "XXX-XXX-XXXX"
Current database has no any phone format.
So there may be garbage data. I have already applied validation for new records but now I want to change my existing data also.
Is there any specific way through that I can change my existing data. And make all phone numbers to follow this format.
Please advice.
Create function to remove the non-numeric data and do the formatting
CREATE FUNCTION [UDF_STRIP_NONNUMERIC_DATA](#str VARCHAR(8000))
RETURNS VARCHAR(8000)
AS
BEGIN
WHILE Patindex('%[^0-9]%', #str) > 0
BEGIN
SET #str = Stuff(#str, Patindex('%[^0-9]%', #str), 1, '')
END
RETURN #str
END
You can use STUFF function to inset the - between phone number
Select left(Stuff(Stuff(dbo.[UDF_STRIP_NONNUMERIC_DATA](Phone),4,0,'-'),8,0,'-'),12)
From yourtable
If you are using SQL SERVER 2012+ use can use FORMAT function (thanks to LukStorms, who mentioned it in comment)
SELECT Format(Cast(dbo.[Udf_strip_nonnumeric_data](Phone) AS BIGINT), '###-###-####')
FROM yourtable
To update
Update yourtable
SET phone = left(Stuff(Stuff(dbo.[UDF_STRIP_NONNUMERIC_DATA](Phone),4,0,'-'),8,0,'-'),12)
Demo
declare #str varchar(100)= '9225-123-4567'
select left(Stuff(Stuff(dbo.[UDF_STRIP_NONNUMERIC_DATA](#str),4,0,'-'),8,0,'-'),12)
Result : 922-512-3456
declare #phone varchar(24)
set #phone = '(334)789-4532'
--set #phone = '314789-4532'
--set #phone = '3457894532'
--set #phone = '534-789-4532'
SELECT
LEFT(N,3) + '-' + SUBSTRING(N,4,3) + '-' + RIGHT(N,4)
FROM
(SELECT CAST(CAST((
SELECT SUBSTRING(#phone, Number, 1)
FROM master..spt_values
WHERE Type='p' AND Number <= LEN(#phone) AND
SUBSTRING(#phone, Number, 1) LIKE '[0-9]' FOR XML Path(''))
AS xml) AS varchar(MAX)) as N) as N
Ok, to replace all non-numeric characters, look at this.
Here is a sample script (copied from that link) to show you how it works (You'll need to modify this to fit your table name and column names:
-- Step 1: creates table to use to hold every char in every phone number
if object_id('dbo.tally') is not null drop table dbo.tally
select top 10000 --change to fit max length of phone number
identity(int,1,1) as n
into dbo.tally
from master.dbo.syscolumns sc1,
master.dbo.syscolumns sc2
-- add pk to maximize performance
alter table dbo.tally
add constraint pk_tally_n
primary key clustered (n) with fillfactor = 100
-- Step 2: Create temporary table holding three bad phone numbers
declare #phonetable table
(uniqueid int identity(1,1),
phone_number varchar(500))
insert into #phonetable (phone_number)
select '01234-567-890' union
select '012345 6789ext' union
select 'n/a' union select '...12345.....';
-- Step 3: identify, for every character, whether it is a number or not,
and remove the non-numeric ones
with cte (uniqueid, phone_number, goodchar, badchar) as
( select uniqueid, phone_number,
case when substring(phone_number,N,1) not like '%[^0-9]%'
then substring(phone_number,N,1) end as goodchar,
case when substring(phone_number,N,1) like '%[^0-9]%'
then substring(phone_number,N,1) end as badchar
from #phonetable , Tally
where phone_number like '%[^0-9]%' and N <= len(phone_number) )
select distinct phone_number,
isnull( stuff (
( SELECT '' + goodchar
FROM cte t1
where t1.UniqueID = t2.UniqueID
FOR XML PATH ( '' ) ) , 1 , 0 , '' ) ,'')
as clean_phone_number from cte t2
to display the numbers with formatting, just extract the appropriate pieces and re-concatenate them with the dashes.
Select case len(phone)
When 10 then left(phone, 3) + '-' +
substring(phone, 4,3) + '-' +
substring(phone, 7,4)`
When 7 then left(phone, 3) + '-' +
substring(phone, 4,4)
Else '' end
To create a computed column
Alter table Add Column FormattedPhone as
case len(phone)
When 10 then left(phone, 3) + '-' +
substring(phone, 4,3) + '-' +
substring(phone, 7,4)`
When 7 then left(phone, 3) + '-' +
substring(phone, 4,4)
Else '' end
If you don't mind a UDF
Select [dbo].[udf-Str-Format-Phone]('334)789-4532')
Returns
334-789-4532
The UDF
CREATE FUNCTION [dbo].[udf-Str-Format-Phone] (#S varchar(max))
Returns varchar(25)
AS
Begin
Declare #Return varchar(25)
;with cte0(N) As (Select 1 From (Values(1),(1),(1),(1),(1)) N(N))
, cteN(N) As (Select Top (Len(#S)) Row_Number() over (Order By (Select NULL)) From cte0 N1, cte0 N2)
, cteS(S) As (Select Substring(#S,N,1) From cteN Where Substring(#S, N, 1) LIKE '[0-9]' FOR XML Path(''))
Select #Return = IIf(Len(S)>=10,Stuff(stuff(S,4,0,'-'),8,0,'-'),Stuff(S,4,0,'-')) From cteS
Return #Return
End
-- Syntax : Select [dbo].[udf-Str-Format-Phone]('(334)789-4532') -- Returns 334-789-4532
-- Syntax : Select [dbo].[udf-Str-Format-Phone]('Phone:7894532') -- Returns 789-4532

Select Case, when no data return

it is possible do a SELECT CASE, decode, nvl or another query function when I need verify if the return of a select query is empty or has a value?
For example, I have this:
Record | type | id_customer
-------+--------+-------------
1 | T | cus1
2 | A | cus2
3 | T | cus3
4 | | cus4
If I do this:
select decode(type,'T','Main','A','Adicional','none') from table where record=1;
I get Main.
If I fo this:
select decode(type,'T','Main','A','Adicional','none') from table where record=4;
I get none.
But if I do this:
select decode(type,'T','Main','A','Aditional','none') from table where record=5;
I get nothing, and is logic. So, I need get the decode value when the row exist and a text if the rows no exist.
So, I tried with SELECT CASE but is not posible get a value using COUNT. For example like this:
SELECT
CASE
WHEN count(1)>0 THEN decode(type,'T','Main','A','Aditional','none')
ELSE '-'
END
FROM TABLE WHERE record=5;
And get a ' - ', or the same if the record is 2, get 'Aditional'
Thanks a lot.
You can use aggregate functions min or max outside expression:
select max(decode(type,'T','Main','A','Aditional','none'))
from table
where record=5;
If query returns one row, you get value of that row. If query returns 0 rows, you get NULL.
Then you can replace NULL using nvl:
select nvl(max(decode(type,'T','Main','A','Aditional','none')), ' - ')
from table
where record=5;
EDIT
Also, if you need to choose one string from several:
select decode(max(decode(type,'T', 2, 'A', 1, 0)), 0, 'none', 1, 'Additional', 2, 'Main', null, ' - ')
from table
where record=5;
This is an option:
select decode(type,'T','Main','A','Aditional','none')
from table
where record = 5
union all
select '-'
from dual
where not exists (select 1 from table where record = 5);
It selects records with record = 5 and unifies them with '-', if no records exits with record = 5. Check out this Fiddle.

sqlite - how do I get a one row result back? (luaSQLite3)

How can I get a single row result (e.g. in form of a table/array) back from a sql statement. Using Lua Sqlite (LuaSQLite3). For example this one:
SELECT * FROM sqlite_master WHERE name ='myTable';
So far I note:
using "nrows"/"rows" it gives an iterator back
using "exec" it doesn't seem to give a result back(?)
Specific questions are then:
Q1 - How to get a single row (say first row) result back?
Q2 - How to get row count? (e.g. num_rows_returned = db:XXXX(sql))
In order to get a single row use the db:first_row method. Like so.
row = db:first_row("SELECT `id` FROM `table`")
print(row.id)
In order to get the row count use the SQL COUNT statement. Like so.
row = db:first_row("SELECT COUNT(`id`) AS count FROM `table`")
print(row.count)
EDIT: Ah, sorry for that. Here are some methods that should work.
You can also use db:nrows. Like so.
rows = db:nrows("SELECT `id` FROM `table`")
row = rows[1]
print(row.id)
We can also modify this to get the number of rows.
rows = db:nrows("SELECT COUNT(`id`) AS count FROM `table`")
row = rows[1]
print(row.count)
Here is a demo of getting the returned count:
> require "lsqlite3"
> db = sqlite3.open":memory:"
> db:exec "create table foo (x,y,z);"
> for x in db:urows "select count(*) from foo" do print(x) end
0
> db:exec "insert into foo values (10,11,12);"
> for x in db:urows "select count(*) from foo" do print(x) end
1
>
Just loop over the iterator you get back from the rows or whichever function you use. Except you put a break at the end, so you only iterate once.
Getting the count is all about using SQL. You compute it with the SELECT statement:
SELECT count(*) FROM ...
This will return one row containing a single value: the number of rows in the query.
This is similar to what I'm using in my project and works well for me.
local query = "SELECT content FROM playerData WHERE name = 'myTable' LIMIT 1"
local queryResultTable = {}
local queryFunction = function(userData, numberOfColumns, columnValues, columnTitles)
for i = 1, numberOfColumns do
queryResultTable[columnTitles[i]] = columnValues[i]
end
end
db:exec(query, queryFunction)
for k,v in pairs(queryResultTable) do
print(k,v)
end
You can even concatenate values into the query to place inside a generic method/function.
local query = "SELECT * FROM ZQuestionTable WHERE ConceptNumber = "..conceptNumber.." AND QuestionNumber = "..questionNumber.." LIMIT 1"

issue in sql Query

I have an column in table where this column name is items it contains value like this
itemID items
1 school,college
2 place, country
3 college,cricket
4 School,us,college
5 cricket,country,place
6 football,tennis,place
7 names,tennis,cricket
8 sports,tennis
Now I need to write a search query
Ex: if the user types 'cricket' into a textbox and clicks the button I need to check in the column items for cricket.
In the table I have 3 rows with cricket in the items column (ItemId = 3, 5, 7)
If the user types in tennis,cricket then I need to get the records that match either one. So I need to get 5 row (ItemId = 3, 5, 6, 7, 8)
How do I write a query for this requirement?
You need to start by redesigning your database as this is is a very bad structure. You NEVER store a comma delimited list in a field. First think about waht fields you need and then design a proper database.
The very bad structure of this table (holding multiple values in one column) is the reason you are facing this issue. Your best option is to normalize the table.
But if you can't, then you can use the "Like" operator, with a wildcard
Select * From Table
Where items Like '%cricket%'
or
Select * From Table
Where items Like '%cricket%'
or items Like '%tenis%'
You will need to dynamically construct these sql queries from the inputs the user makes. The other alternative is to write code on the server to turn the comma delimited list of parameters into a table variable or temp table and then join to it..
Delimited values in columns is almost always a bad table design. Fix your table structure.
If for some reason you are unable to do that, the best you can hope for is this:
SELECT * FROM [MyTable] WHERE items LIKE '%CRICKET%'
This is still very bad, for two important reasons:
Correctness. It would return values that only contain the word cricket. Using your tennis example, what if you also had a "tennis shoes" item?
Performance. It's not sargable, which means the query won't work with any indexes you may have on that column. That means your query will probably be incredibly slow.
If you need help fixing this structure, the solution is to add another table — we'll call it TableItems — with a column for your ItemID that will be a foreign key to your original table and an item field (singular) for each of your item values. Then you can join to that table and match a column value exactly. If these items work more like categories, where you want to rows with the "Cricket" item to match the same cricket item, you also want a third table to be an intersection between your original table and the other one I just had you create.
For a single item:
SELECT itemID, items FROM MyTable WHERE items LIKE '%cricket%'
For multiple items:
SELECT itemID, items FROM MyTable WHERE items LIKE '%tennis%' or items LIKE '%cricket%'
You'll need to parse the input and split them up and add each item to the query:
items LIKE '%item1%' or items LIKE '%item2%' or items LIKE '%item3%' ...
I think that in the interest of validity of data, it should be normalized so that you split the Items into a separate table with an item on each row.
In either case, here is a working sample that uses a user defined function to split the incoming string into a Table Variable and then uses JOIN with a LIKE
CREATE FUNCTION dbo.udf_ItemParse
(
#Input VARCHAR(8000),
#Delimeter char(1)='|'
)
RETURNS #ItemList TABLE
(
Item VARCHAR(50) ,
Pos int
)
AS
BEGIN
DECLARE #Item varchar(50)
DECLARE #StartPos int, #Length int
DECLARE #Pos int
SET #Pos = 0
WHILE LEN(#Input) > 0
BEGIN
SET #StartPos = CHARINDEX(#Delimeter, #Input)
IF #StartPos < 0 SET #StartPos = 0
SET #Length = LEN(#Input) - #StartPos - 1
IF #Length < 0 SET #Length = 0
IF #StartPos > 0
BEGIN
SET #Pos = #Pos + 1
SET #Item = SUBSTRING(#Input, 1, #StartPos - 1)
SET #Input = SUBSTRING(#Input, #StartPos + 1, LEN(#Input) - #StartPos)
END
ELSE
BEGIN
SET #Pos = #Pos+1
SET #Item = #Input
SET #Input = ''
END
INSERT #ItemList (Item, Pos) VALUES(#Item, #Pos)
END
RETURN
END
GO
DECLARE #Itemstable TABLE
(
ItemId INT,
Items VarChar (1000)
)
INSERT INTO #Itemstable
SELECT 1 itemID, 'school,college' items UNION
SELECT 2, 'place, country' UNION
SELECT 3, 'college,cricket' UNION
SELECT 4, 'School,us,college' UNION
SELECT 5, 'cricket,country,place' UNION
SELECT 6, 'footbal,tenis,place' UNION
SELECT 7, 'names,tenis,cricket' UNION
SELECT 8, 'sports,tenis'
DECLARE #SearchParameter VarChar (100)
SET #SearchParameter = 'cricket'
SELECT DISTINCT ItemsTable.*
FROM #Itemstable ItemsTable
INNER JOIN udf_ItemParse (#SearchParameter, ',') udf
ON ItemsTable.Items LIKE '%' + udf.Item + '%'
SET #SearchParameter = 'cricket,tenis'
SELECT DISTINCT ItemsTable.*
FROM #Itemstable ItemsTable
INNER JOIN udf_ItemParse (#SearchParameter, ',') udf
ON ItemsTable.Items LIKE '%' + udf.Item + '%'
Why exactly are you using a database in the first place?
I mean : you are clearly not using it's potential. If you like using comma separated stuff, try a file.
In MySQL, create a fulltext index on your table:
CREATE FULLTEXT INDEX fx_mytable_items ON mytable (items)
and issue this query:
SELECT *
FROM mytable
WHERE MATCH(items) AGAINST ('cricket tennis' IN BOOLEAN MODE)

Resources