Optimize the performance of product scoped query - google-analytics

Question

I would like to build the following table every day, to store some aggregate data on page performance of a website. However, each days worth of data is over 15 million rows.
What steps can I take to improve performance? I am intending to save them as sharded tables, but I would like to improve further, could I nest the data within each table to improve performance further? What would be the best way to do this?
SELECT
device.devicecategory AS device,
hits_product.productListName AS list_name,
UPPER(hits_product.productSKU) AS SKU,
AVG(hits_product.productListPosition) AS avg_plp_position
FROM `mindful-agency-136314.43786551.ga_sessions_20*` AS t
CROSS JOIN UNNEST(hits) AS hits
CROSS JOIN UNNEST(hits.product) AS hits_product
WHERE parse_date('%y%m%d', _table_suffix) between
DATE_sub(current_date(), interval 1 day) and
DATE_sub(current_date(), interval 1 day)
AND hits_product.productListName != "(not set)"
GROUP BY
device,
list_name,
SKU

Answer 1

Since you're using productSku and productListName as dimensions/groups there is no way around cross joining with product array.
You're also cross joining with product which can be dangerous because sometimes this array is missing and you destroy the whole row - typically you'd use a left join. But in this case, it's fine because you're only interested in product fields.
You should, however, be clear about whether you want to see list clicks or list impressions using hits.product.isImpression and hits.product.isClick. Atm I don't see a distinction there. Maybe filter for WHERE hits_product.isImpression in case of list views?
Instead of shards you might want to consider adding a date field and PARTITION BY date as well as CLUSTER BY list_name. See INSERT Statement for updating
and DDL Syntax to start the table. This is more performant than shards when it comes to querying the table later.
Starting the table could look something like this:
CREATE TABLE `x.y.z`
PARTITION BY date
CLUSTER BY list_name
AS (
SELECT
PARSE_DATE('%Y%m%d',date) AS date,
device.devicecategory AS device,
hits_product.productListName AS list_name,
UPPER(hits_product.productSKU) AS SKU,
AVG(IF(hits_product.isClick, hits_product.productListPosition, NULL)) AS avg_plp_click_position,
AVG(IF(hits_product.isImpression, hits_product.productListPosition, NULL)) AS avg_plp_view_position
FROM `bigquery-public-data.google_analytics_sample.ga_sessions_20*` AS t
CROSS JOIN UNNEST(hits) AS hits
CROSS JOIN UNNEST(hits.product) AS hits_product
WHERE
parse_date('%y%m%d', _table_suffix)
between
DATE_sub(current_date(), interval 1 day)
and DATE_sub(current_date(), interval 1 day)
AND hits_product.productListName != "(not set)"
GROUP BY
date,
device,
list_name,
SKU
)
Inserting new records is quite similar, you just need to mention the fields upfront as described in the documentation.