sqlite: how does order by in recursive queries work? - recursion

Question

https://www.sqlite.org/lang_with.html#withorderby shows an example of how order-by on a recursion level can change Depth-First to Breadth-First ordering of a tree. Richard Hipp in
http://sqlite.1065341.n5.nabble.com/why-does-the-recursive-example-sort-alphabetically-td80404.html
says that order-by in recursive part has a special meaning: determines the order of recursion.
I've played with order-by on fields other than recursion level and ... I cannot predict the results. :( Could someone please explain how it works?
Basically, let's look at an example similar to Alice/Bob/Cindy from the sqlite docs (the first link above), but with names mixed up a bit (not alphabetically ordered within the tree, and inserted in a random order) and then with "order by name" (instead of recursion level) in the recursive part.
CREATE TABLE child_parent(
name TEXT PRIMARY KEY,
parent TEXT
);
INSERT INTO child_parent VALUES('KKK','HHH');
INSERT INTO child_parent VALUES('HHH','LLL');
INSERT INTO child_parent VALUES('MMM','CCC');
INSERT INTO child_parent VALUES('CCC','QQQ');
INSERT INTO child_parent VALUES('QQQ','LLL');
INSERT INTO child_parent VALUES('TTT','QQQ');
INSERT INTO child_parent VALUES('AAA','HHH');
INSERT INTO child_parent VALUES('EEE','TTT');
INSERT INTO child_parent VALUES('UUU','CCC');
INSERT INTO child_parent VALUES('FFF','MMM');
INSERT INTO child_parent VALUES('LLL',NULL);
The child_parent structure is:
-- LLL -> QQQ
-- -> TTT
-- -> EEE
-- -> CCC
-- -> UUU
-- -> MMM
-- -> FFF
-- -> HHH
-- -> AAA
-- -> KKK
First select - with "ORDER BY new_name asc"
WITH RECURSIVE
tree(name,level) AS (
SELECT cp.name, 0
FROM child_parent cp
WHERE cp.parent IS NULL
UNION ALL
SELECT ch.name AS new_name, tree.level+1 AS new_lvl
FROM child_parent ch
INNER JOIN tree ON ch.parent=tree.name
ORDER BY new_name ASC
)
SELECT *, substr('...................',1,level*3) || name FROM tree;
-- name level substr('...................',1,level*3) || name
-- ---------- ---------- -----------------------------------------------
-- LLL 0 LLL
-- HHH 1 ...HHH
-- AAA 2 ......AAA
-- KKK 2 ......KKK
-- QQQ 1 ...QQQ
-- CCC 2 ......CCC
-- MMM 3 .........MMM
-- FFF 4 ............FFF
-- TTT 2 ......TTT
-- EEE 3 .........EEE
-- UUU 3 .........UUU
First select - with "ORDER BY new_name desc"
WITH RECURSIVE
tree(name,level) AS (
SELECT cp.name, 0
FROM child_parent cp
WHERE cp.parent IS NULL
UNION ALL
SELECT ch.name AS new_name, tree.level+1 AS new_lvl
FROM child_parent ch
INNER JOIN tree ON ch.parent=tree.name
ORDER BY new_name DESC
)
SELECT *, substr('...................',1,level*3) || name FROM tree;
-- name level substr('...................',1,level*3) || name
-- ---------- ---------- -----------------------------------------------
-- LLL 0 LLL
-- QQQ 1 ...QQQ
-- TTT 2 ......TTT
-- HHH 1 ...HHH
-- KKK 2 ......KKK
-- EEE 3 .........EEE
-- CCC 2 ......CCC
-- UUU 3 .........UUU
-- MMM 3 .........MMM
-- FFF 4 ............FFF
-- AAA 2 ......AAA
Basically the question is: how to think about db behaviour to predict that the result of the last two queries would be as above. Could you describe what happens step-by-step, e.g. at each recursion level?
SQLite 3.15.2.0.37

Answer 1

The documentation says:
The basic algorithm for computing the content of the recursive table is as follows:
Run the initial-select and add the results to a queue.
While the queue is not empty:
a. Extract a single row from the queue.
b. Insert that single row into the recursive table
c. Pretend that the single row just extracted is the only row in the recursive table and run the recursive-select, adding all results to the queue.
[…]
If an ORDER BY clause is present, it determines the order in which rows are extracted from the queue in step 2a.
When the ORDER BY is applied to the tree level column, the result is a breadth-first or depth-first search. (With ASC, the 'older' rows are extracted first; with DESC, the most recent row at the lowest level is extracted first.)
But when applied to some other column, the order is no longer related to the structure of the tree.

Answer 2

Answer
Oh, I see, so the queue contains only a list of single rows and there are two points that I have missed:
the queue gets re-ordered after EVERY small STEP and NOT after every recursion level
the results are added to the results at the moment when the row is processed from the queue, and not when they are added to the queue.
Step-by-step example
Example: so in the last case above (ORDER BY new_name DESC):
initial select => result: empty, queue (LLL)
the second part of recursive select based on the first record of the queue
result: (LLL), queue empty
we get 2 records: QQQ, HHH
they both get added to the queue in order of ORDER BY: queue {QQQ, HHH}
first record from the queue gets picked up
result: (LLL), (QQQ); queue {HHH}
first based on QQQ we get 2 new records: TTT and AAA
they get added to the queue BUT at this moment (!) the queue wll get ordered and it will be in fact {TTT, HHH, AAA}
first record from the queue gets picked up
result: (LLL), (QQQ), (TTT); queue {HHH, AAA}
select based on TTT does not add anything to the queue
first record from the queue gets picked up:
result: (LLL), (QQQ), (TTT), (HHH); queue {AAA}
query based on HHH leads to two new rows: KKK and CCC, and the queue after this step is {KKK, CCC, AAA}
and so on
Summary
Yep, we do get the results as from the above selects
and in the case of ordering by recursion level these steps do lead to Depth-First or Breadth-First.
Thanks!