R- Excluding random numbers that have already been generated - r

Question

So I'm working on a webscraping script in R and because the particular website I'm scraping doesn't take too kindly to people who scrape their data in large volumes, I have broken down my loop to handle only 10 links at a time. I still want to go through all the links, however, just in a random and slow manner.
productLink # A list of all the links that I'll be scraping
x<- length(productLink)
randomNum <- sample(1:x, 10)
library(rvest)
for(i in 1:10){
url <- productLink[randomNum[i]]
specs <- url %>%
html() %>%
html_nodes("h5") %>%
html_text()
specs
message<- "\n Temporarily unavailable\n "
if(specs == message){
print("Item unavailable")
}
else{
print("Item available")
}
}
Now the next time I run this for-loop I want to exclude all the random numbered indices that have already been tried in the previous running of the loop. That way this for loop runs through 10 new links each time until all the links have been used. There is another aspect to this that I'd like some input on. Since I can raise alarm flags by brute force scraping the particular company's website, is there any way I can slow down this loop so that it only runs every couple of minutes? I'm thinking of a timeout function or such where the code runs the for-loop once, waits a few minutes then runs it again (with new links each time as mentioned above). Any ideas?

Answer 1

Use something like this. Loop over all the product index randomly.
for (i in sample(1:x)){
<Your code here>
# Sleep for 120 seconds
Sys.sleep(120)
}
And if you want to do 10 at a time. Sleep for 120 seconds every 10 executions.
n = 1
for (i in sample(1:x)){
# Sleep for 120 seconds every 10 runs
if (n == 10) {Sys.sleep(120); n = 0}
n = n+1
<Your code here>
}