How to replace None values in Python SQL Lite? - sqlite

I am working on the 8 Week SQL Challenge with the following data. I am using Python and SQL Lite.
c.execute('DROP TABLE IF EXISTS runner_orders')
c.execute('CREATE TABLE runner_orders (order_id INTEGER, runner_id INTEGER, pickup_time VARCHAR(19), distance VARCHAR(7), duration VARCHAR(10), cancellation VARCHAR(23))')
more_order2 = [ ('1', '1', '2020-01-01 18:15:34', '20km', '32 minutes', ''),
('2', '1', '2020-01-01 19:10:54', '20km', '27 minutes', ''),
('3', '1', '2020-01-03 00:12:37', '13.4km', '20 mins', 'NULL'),
('4', '2', '2020-01-04 13:53:03', '23.4', '40', 'NULL'),
('5', '3', '2020-01-08 21:10:57', '10', '15', 'NULL'),
('6', '3', 'null', 'null', 'null', 'Restaurant Cancellation'),
('7', '2', '2020-01-08 21:30:45', '25km', '25mins', 'null'),
('8', '2', '2020-01-10 00:15:02', '23.4 km', '15 minute', 'null'),
('9', '2', 'null', 'null', 'null', 'Customer Cancellation'),
('10', '1', '2020-01-11 18:50:20', '10km', '10minutes', 'null')]
c.executemany('INSERT INTO runner_orders VALUES (?,?,?,?,?,?)', more_order2)
c.execute(query)
conn.commit()
I am trying to replace the '' values with null with this code:
query = '''
UPDATE runner_orders
SET cancellation = null
WHERE cancellation IN ('NULL', '')
'''
c.execute(query)
query2 = '''
SELECT * FROM runner_orders
'''
for match in c.execute(query2):
print(match)
For some reason, when I output the values, I got None in all the NULL spaces and it does not replace them with null. How do I fix this?

null is not a string. SQL uses trinary logic: true, false, and null. SQL null roughly means "unknown" or "no value".
If you look at your table in the SQLite client you should see the value is null. Python is translating SQL null into the Python equivalent of None.

Related

Join values of two tables that represent the missing values between those tables in SQLITE

I have the following table
CREATE TABLE holes (`tournament_id` INTEGER, `year` INTEGER, `course_id` INTEGER, `round` INTEGER, `hole` INTEGER, `front` INTEGER, `side` INTEGER, `region` INTEGER);
With the following data sample
INSERT INTO holes (`tournament_id`, `year`, `course_id`, `round`, `hole`, `front`, `side`, `region`) VALUES
('33', '2016', '895', '1', '1', '12', '5', 'L'),
('33', '2016', '895', '1', '2', '18', '10', 'R'),
('33', '2016', '895', '1', '3', '15', '7', 'R'),
('33', '2016', '895', '1', '4', '11', '7', 'R'),
('33', '2016', '895', '1', '5', '18', '7', 'L'),
('33', '2016', '895', '1', '6', '28', '5', 'L'),
('33', '2016', '895', '1', '7', '21', '12', 'R'));
In addition, I have another table tournaments
CREATE TABLE tournaments (`tournament_id` INTEGER, `year` INTEGER, `R1` INTEGER, `R2` INTEGER, `R3` INTEGER, `R4` INTEGER);
With data
INSERT INTO tournaments VALUES
(33, 2016, 715, 715, 895, 400);
The values for R1, R2, R3 and R4 present ids of the courses.
I want the columns tournament_id, year and course_id that are missing in table holes based on all the possible values of table tournaments.
With the help of this answer I tried the following:
WITH h AS (
SELECT DISTINCT tournament_id, year, course_id
FROM holes)
SELECT t.tournament_id, t.year
FROM tournaments t
WHERE NOT EXISTS (
SELECT *
FROM h
WHERE h.tournament_id = t.tournament_id
AND h.year = t.year
AND h.course_id IN (t.R1, t.R2, t.R3, t.R4)
);
demo
The above goes a long way but I also want the h.course_id that is/are missing. Desired result:
33 2016 715
33 2016 400
These combinations of tournament_id, year and course_id are not present in holes. However, they do exists because they are present in tournaments.
For this requirement you need a resultset consisting of all the values of the Rx columns which you can get with UNION in a CTE.
Then you can use NOT EXISTS to get all the combinations of id, year and course that do not exist in holes:
WITH cte AS (
SELECT id, year, R1 AS course FROM tournaments
UNION
SELECT id, year, R2 FROM tournaments
UNION
SELECT id, year, R3 FROM tournaments
UNION
SELECT id, year, R4 FROM tournaments
)
SELECT c.*
FROM cte c
WHERE NOT EXISTS (
SELECT *
FROM holes h
WHERE (h.id, h.year, h.course) = (c.id, c.year, c.course)
);
See the demo.

MariaDB problem with bit column, check constraint and JSON - check constraint is violated

I have the following table on 10.3.35 MariaDB server:
CREATE TABLE `_boat_product` (
`id` INT(11) NOT NULL AUTO_INCREMENT,
`standard` BIT(1) NOT NULL DEFAULT b'0',
`selected_by_default` BIT(1) NOT NULL DEFAULT b'0',
`date_created` DATETIME NULL DEFAULT current_timestamp(),
PRIMARY KEY (`id`) USING BTREE,
CONSTRAINT `CC1` CHECK (cast(`standard` as signed) + cast(`selected_by_default` as signed) <= 1)
)
COLLATE='utf8mb4_unicode_ci'
ENGINE=InnoDB
AUTO_INCREMENT=1486
;
I fetch data like this (PHP):
function select($order) {
$query = $this->db->query("SELECT id, "
. "standard,"
. "selected_by_default,"
. "DATE_FORMAT(date_created, '%Y-%m-%d %H:%i') date_created "
. "FROM _boat_product "
. $order);
return ['result' => 'OK', 'data' => $query->result()];
}
echo json_encode($this->select($order));
I have a tabulator.js table set like this (excerpt):
columns: [
{title: "standard", field: "standard", editor: "tickCross", minWidth: 50, maxWidth: 80, headerFilter: true, hozAlign: "center", headerVertical: true,
editorParams: {
trueValue: "1",
falseValue: "0"
}, formatter: "tickCross",
formatterParams: {
tickElement: "yes",
crossElement: "no"
}
},
{title: "default", field: "selected_by_default", editor: "tickCross", minWidth: 50, maxWidth: 80, headerFilter: true, hozAlign: "center", headerVertical: true,
editorParams: {
trueValue: "1",
falseValue: "0"
}, formatter: "tickCross",
formatterParams: {
tickElement: "yes",
crossElement: "no"
}
},...
Data that I am sending to the server to update records looks like this:
{
"data": {
"data": [
{
"id": "483",
"selected_by_default": "1",
"standard": "0"
}
],
"function": "insertUpdate"
}
}
and the update statement is like this:
UPDATE `_boat_product` SET `standard` = '0', `selected_by_default` = '1'
WHERE `id` = '483'
But I am getting an error that check constraint is violated:
Error Number: 4025</p><p>CONSTRAINT `CC1` failed for...
Why that is happening?
The problem seems to be with the conversion from VARCHAR to BIT(1).
UPDATE `_boat_product` SET `standard` = '0', `selected_by_default` = '1'
WHERE `id` = '483'
would work only like this:
UPDATE `_boat_product` SET `standard` = b'0', `selected_by_default` = b'1'
WHERE `id` = '483'
but because I can't complicate it too much so the solution was to change BIT(1) column type to TINYINT(1).
It is a common misconception that BIT(1) is the appropriate data type to store true/false values:
BIT(1) requires 1 byte storage (bytes = (bit_length + 7)/8 as does TINYINT. In contrast to fixed length TINYINT, BIT requires additional metadata to store the number of bits: In the binary log protocol, 2 additional bytes must be used for this in table_map event, on the client side it must be checked whether the value must be stored in a 1,2,4 or 8 byte integer.
BIT(1) values ​​are 0x00 and 0x01 binary values ​​and cannot be displayed without prior conversion, e.g. in the command line client (bin+0,hex(bin),oct(bin),bin(bin))
BIT values always have to be converted to a number or binary before you can modify it, while a TINYINT value can be passed as a number or string.
I would suggest to change the table definition from BIT(1) to TINYINT,if this is not possible you need to unset strict_mode:
MariaDB [test]> create table bit_test(a bit(1));
Query OK, 0 rows affected (0,016 sec)
MariaDB [test]> insert into bit_test values ("1");
ERROR 1406 (22001): Data too long for column 'a' at row 1
MariaDB [test]> set sql_mode=0;
Query OK, 0 rows affected (0,001 sec)
MariaDB [test]> insert into bit_test values ("1");
Query OK, 1 row affected, 1 warning (0,011 sec)

Kusto summarize unique occurrences of the value in the column

I have following dataset:
let t1 = datatable(id:string, col1:string, col2:string)
[
'1', 'ValueA', 'AT',
'2', 'ValueC', 'AT',
'3', 'ValueA', 'AT',
'4', 'ValueB', 'AT',
'1', 'ValueC', 'v-username',
];
t1
| summarize (Id) by col1
My goal is to count occurrences of values in col1 per Id. Because ID=1 occurs twice, I need to decide whether to take ValueA or ValueC. This is decided by value of col2. If col2 startswith "v-" then take Value from this row.
When I use "summarize (Id) by col1" I am getting:
ValueA,2
ValueC,2
ValueB,1
ValueD,1
Total:6
Expected result is:
ValueA,1
ValueC,2
ValueB,1
ValueD,1
Total:5
Is it possible to achieve with Kusto?
a. when you run ... | summarize (id) by col1" you should get a semantic error as there's no aggregation function specified (e.g. you could have run... | summarize dcount(id) by col1`)
b. it's not clear where ValueD, 1, in your expected result, come from. as your datatable expression includes no record with ValueD
c. if i had to guess the solution to your question, despite a and b, this would be my guess:
let t1 = datatable(id:string, col1:string, col2:string)
[
'1', 'ValueA', 'AT',
'2', 'ValueC', 'AT',
'3', 'ValueD', 'AT',
'4', 'ValueB', 'AT',
'1', 'ValueC', 'v-username',
];
t1
| summarize c = dcount(id) by col1
| as T
| union (T | summarize c = sum(c) by col1 = "Total")

perform a where in query in bookshelf.js

I want to perform a WHERE - IN query/operation but normal where gives error.
I want this
select * from `calendar_event_rsvp` where `event_id` in ('1', '2', '3')
But below code leads to
select * from `calendar_event_rsvp` where `event_id in` = '1', '2', '3'
Code
CalendarEventRSVP.forge()
.where({
"event_id": event_ids
})
How do i do this in bookshelf.js
Try to add the operator:
CalendarEventRSVP.forge()
.where('event_id', 'in', event_ids)
Or use knex's whereIn:
CalendarEventRSVP.forge()
.query({whereIn: {event_id: event_ids}})
try query() function on your model.
CalendarEventRSVP.query(function(qb){
qb.where('event_id' , 'in' , [1,2,3,4]) ;
})
.fetchAll()
.then();

Inserting values with specific date and time format in Oracle

I tried to insert values into SURVEY.YSG table ( used in the insert query )
INSERT INTO SURVEY.YSG(YSG_ID,YSG_MSN_DTM,TAX_ID,TAX_NM,TYPE_CD,USER_ID,
FRST_NM,LAST_NM,SHL_TXT,NPR_DT, PHONE_NBR,FAX_NBR,
EMAIL_ADRS_TXT,YSG_IMD,MDFD_ID,YSG_VRFCTN_CD,
YSG_VRFCTN_DTM,LOG_KEY,PLS_DTM,LOAD_KEY,UPDTD_LOAD_KEY )
VALUES ( '103','2011-08-11 13:34:36.000000','656002075',
'STG HEALTH SYSTEM','null','OPLINKS4','UNK','UNK','UNK',
'12/31/8888','UNK','UNK','UNK','X','UNK','Y','8888-12-31 00:00:00.000000',
0,'2011-07-20 12:00:00.000000',0,0)
Error report:
SQL Error: ORA-01843: not a valid month
01843. 00000 - "not a valid month"
*Cause:
*Action:
Below are the 4 columns created with date and timestamp in SURVEY.YSG table.
YSG_MSN_DTM - TIMESTAMP(6)
NPR_DT - DATE
YSG_VRFCTN_DTM - TIMESTAMP(6)
PLS_DTM - TIMESTAMP(6)
tried with TO_DATE() function:
TO_DATE('2011-08-11 13:34:36','YYYY-MM-DD HH24:MI:SS')
Got output like: 11-AUG-11 01.34.36.000000000 PM.
Expected Output:
column: YSG_MSN_DTM -> 2011-08-11 13:34:36.000000
and for column NPR_DT ->12/31/8888 in another column.
is there any other way to achieve this in Oracle?
Any of your help is appreciated.
Thanks
Try this:
INSERT INTO
SURVEY.YSG ( YSG_ID,
YSG_MSN_DTM,
TAX_ID,
TAX_NM,
TYPE_CD,
USER_ID,
FRST_NM,
LAST_NM,
SHL_TXT,
NPR_DT,
PHONE_NBR,
FAX_NBR,
EMAIL_ADRS_TXT,
YSG_IMD,
MDFD_ID,
YSG_VRFCTN_CD,
YSG_VRFCTN_DTM,
LOG_KEY,
PLS_DTM,
LOAD_KEY,
UPDTD_LOAD_KEY )
VALUES
( '103',
TO_TIMESTAMP ( '2011-08-11 13:34:36.000000',
'YYYY-MM-DD HH24:MI:SS.FF' ),
'656002075',
'STG HEALTH SYSTEM',
'null',
'OPLINKS4',
'UNK',
'UNK',
'UNK',
TO_DATE ( '12/31/8888',
'MM/DD/YYYY' ),
'UNK',
'UNK',
'UNK',
'X',
'UNK',
'Y',
TO_TIMESTAMP ( '8888-12-31 00:00:00.000000',
'YYYY-MM-DD HH24:MI:SS.FF' ),
0,
TO_TIMESTAMP ( '2011-07-20 12:00:00.000000',
'YYYY-MM-DD HH24:MI:SS.FF' ),
0,
0 );

Resources