R WFS overcome server maximum 3D BAG - r

Question

For species niche modeling I am trying to fetch building heights from the briljant 3D BAG data of the TU Delft https://3dbag.nl/nl/download. I want to do this for the city of Haarlem. It is possible to select and download tiles manually. But this is quite labor-intensive and prone to errors (a missing tile), and I want to repeat this action for more cities. So I am trying to use the AFS service to download features. I created a bounding box of Haarlem with a 1.2 extent for the AFS request. However, the maximum record the server delivers is 5000. Despite many alternative attempts I have failed so far to overcome this number. This is partly caused by my confusion in the AWF semantics, when I check with GetCapabilities it is hard to find out the name space, featureTypes and individual attributes (or properties). What I have tried:
Add pagination. But all the tutorials I have read so far need the number of actual/maximum features beside the server maximum (resultType = "hits"). And I was not able to easily retrieve this maximum for the limits of the bounding box.
Select tiles. I figured it should be possible to extract the tile ids that match with the bounding box, using the tile_id, an attribute from the BAG3D_v2:bag_tiles_3k layer, and then somehow build an apply or loop to extract the features per tile. But I already failed to create a cql_filter to select an individual tile.
Create tiles. Since I am not entirely sure whether the individual tiles from the 3D BAG service already exceed the 5000 feature limit, an alternative approach could be to split the bounding box is many small tiles using the R package slippymath, and then extract the features per tile. But then the challenge of filtering remains the same.
Any help with this would be appreciated. The basic code I used in many different ways:
library(httr)
url <- parse_url("https://data.3dbag.nl/api/BAG3D_v2/wfs")
url$query <- list(service = "WFS",
version = "2.0.0",
request = "GetFeature",
typename = "BAG3D_v2:lod22",
#cql_filter = "BAG3D_v2:tile_id ='4199'",
bbox = "100768.4,482708.5,107923.1,494670.4",
startindex = 10000,
sortBy = "gid")
request <- build_url(url)
test <- st_read(request)
qtm(test)

Answer 1

One solution is to loop over startindex 5000 by 5000. Then stop the code when the shape returned contained less than 5000 which mean it's done (unless the total number of features is a multiple of 5000...)
Below a piece of code adapted from happign package.
library(httr)
library(sf)
# function for building url
build_3DBAG_url <- function(startindex){
url <- parse_url("https://data.3dbag.nl/api/BAG3D_v2/wfs")
url$query <- list(service = "WFS",
version = "2.0.0",
request = "GetFeature",
typename = "BAG3D_v2:lod22",
#cql_filter = "BAG3D_v2:tile_id ='4199'",
bbox = "100768.4,482708.5,107923.1,494670.4",
startindex = startindex,
count = 5000,
sortBy = "gid")
url <- build_url(url)
return(url)
}
# initialize first request
resp <- read_sf(build_3DBAG_url(startindex = 0))
message("Features downloaded : ", nrow(resp), appendLF = F)
# loop until returned shape is less than 5000
i <- 5000
temp <- resp
while(nrow(temp) == 5000){
message("...", appendLF = F)
temp <- read_sf(build_3DBAG_url(startindex = i))
resp <- rbind(resp, temp)
message(nrow(resp), appendLF = F)
i <- i + 5000
}