Obtain the same cross join with merge() and sqldf::sqldf() - r

Question

I have two data frames: Sales and Clients. I want to perform cross joins on these data frames using sqldf::sqldf() and also using merge() and obtain the exact same result with both methods.
So far I´ve only been able to obtain two data frames with the rows ordered differently.
This is the code to generate the Sales and Clients data frames:
set.seed(1)
Sales <- data.frame(
Product = sample(c("Toaster", "Radio", "TV"), size = 7, replace = TRUE),
CustomerID = c(rep("1_2019", 2), paste(2:3, "2019", sep = "_"), paste(1:3, "2020", sep = "_"))
)
Sales$Price <- round(ifelse(Sales$Product == "TV", rnorm(1, 400, 20),
ifelse(Sales$Product == "Toaster", rnorm(1, 40, 2),
rnorm(1, 35, 2))))
Clients <- data.frame(
CustomerID = c(paste(2:4, "2019", sep = "_"), paste(1:2, "2020", sep = "_")),
State = sample(c("CA", "AZ", "IL", "MA"), size = 5, replace = TRUE)
)
This is what I got:
library(sqldf)
# cross join with base R
out1 <- merge(x = Sales, y = Clients, by = NULL)
# cross join with sqldf
out2 <- sqldf("SELECT *
FROM Sales
CROSS JOIN Clients")
out1 and out2 have different row orderings. How can I tweak the sqldf() call in order for out1 and out2 to be exactly the same?
This is the closest I got:
merge(x = Sales, y = Clients, by = NULL)
sqldf("SELECT *
FROM Sales
CROSS JOIN Clients
ORDER BY State DESC, Clients.CustomerID")

Answer 1

I think including ORDER BY in sqldf is important, since it drives home the fact that in SQL, ordering is never guaranteed unless explicitly directed.
If you were doing simple ORDER BY with just "increasing" on both variables, then the translation to order in R would be direct. However, since one variable is decreasing and one is increasing, order by itself doesn't deal with that. However, as suggested by https://stackoverflow.com/a/3316719, we can do the same with xtfrm.
out1 <- merge(x = Sales, y = Clients, by = NULL)
out1 <- out1[order(-xtfrm(out1$State), out1$CustomerID.y),]
out2 <- sqldf::sqldf(
"SELECT *
FROM Sales
CROSS JOIN Clients
ORDER BY State DESC, Clients.CustomerID")
### proof they are identical
all(unlist(Map(`==`, out1, out2)))
# [1] TRUE
The xtfrm helper function here allows us to negate the "values" of a column for the purposes of sorting. From ?xtfrm:
A generic auxiliary function that produces a numeric vector which will sort in the same order as 'x'.
If the field were already numeric, we could merely do order(-State, CustomerID.y), but the fact that it is character requires a further step. Argo xtfrm.
Edit: in comments, it's determined that the OP wants to mimic the sort-order of merge in the SQL statement. Unfortunately, because this is a cartesian product of the two frames, no sorting is applied: merge merely cbinds all rows of the first frame against the first row of the second frame, then repeats with each row of the second.
This can be demonstrated by using some code from merge:
nx <- nrow(x) # Sales
ny <- nrow(y) # Clients
expand.grid(seq_len(nx), seq_len(ny))
# Var1 Var2
# 1 1 1
# 2 2 1
# 3 3 1
# 4 4 1
# 5 5 1
# 6 1 2
# ...
# 33 3 7
# 34 4 7
# 35 5 7
where each number is a row from the respective frames (x for Var1, y for Var2). If the original data is:
## Sales ## Clients
Product CustomerID Price CustomerID State
1 Toaster 1_2019 37 1 2_2019 AZ
2 Radio 1_2019 33 2 3_2019 MA
3 Radio 2_2019 33 3 4_2019 AZ
4 TV 3_2019 408 4 1_2020 IL
5 Toaster 1_2020 37 5 2_2020 MA
6 TV 2_2020 408
7 TV 3_2020 408
then this results in
out1
# Product CustomerID.x Price CustomerID.y State
# 1 Toaster 1_2019 37 2_2019 AZ
# 2 Radio 1_2019 33 2_2019 AZ
# 3 Radio 2_2019 33 2_2019 AZ
# 4 TV 3_2019 408 2_2019 AZ
# 5 Toaster 1_2020 37 2_2019 AZ
# 6 TV 2_2020 408 2_2019 AZ
# 7 TV 3_2020 408 2_2019 AZ
# 8 Toaster 1_2019 37 3_2019 MA
# ...
# 33 Toaster 1_2020 37 2_2020 MA
# 34 TV 2_2020 408 2_2020 MA
# 35 TV 3_2020 408 2_2020 MA
which will very much destroy any sorting present in x (Sales), even if y (Clients) comes pre-sorted (which it does).
Because of this, if you want congruity between R and SQL cross-join solutions, I suggest the most transparent/clear way would be to merge in R and then apply post-merge ordering in a fashion that is similar to SQL. In fact, from a pedagogic perspective, ask the question: *"What ordering makes sense to humans?" If you assert during the lesson plan that ordering may not be assured until explicitly strong-armed into the process (via dplyr::arrange, x[order(...),], or SQL's ORDER BY clause). Find the intuitive ordering of the data and then demonstrate that in both R and SQL.
Side notes:
Your sqldf query results in same-named columns, this results in some errors post-sqldf if you start playing with columns. This can be mitigated with select ... as ... field-naming.
Lexicographic sorting of your data is unfortunately counter-intuitive at the moment: having year at the end of a customer id suggests (yes, I'm inferring) a timeline of customer onboarding, yet they will sort first by the leading number. Similar to how "2020-05-04" sorts correctly even as a string, while "05/04/2020" does not, it might support more intuitive sorting to have the most-significant portion be the leading part of id strings. Or make them integers. Or UUIDs (v4, of course), those are always fun.