Efficient loan repayment calculation - r

Question

I have a table of loan issuance and repayments by customers that I have preprocessed like this
customerID | balanceChange | trxDate | TYPE
242105 | 500 | 20170605 | loan
242105 | 1500 | 20170605 | loan
242105 | -1000 | 20170607 | payment
242111 | 500 | 20170605 | loan
242111 | -500 | 20170606 | payment
242111 | 500 | 20170607 | loan
242111 | -500 | 20170609 | payment
242151 | 500 | 20170605 | loan
What I would like to do is to (1) count for each of the loans issued every day, how many of them have been paid back in full, and (2) how many days did it take the customer to pay them.
The rule of the repayment is of course FIFO (First In First Out), so the oldest loan gets paid back first.
In the example above, the solution would be
trxDate | nRepayments | timeGap(days)
20170605 | 2 | 1.5
20170606 | 0 | 0
20170607 | 1 | 2
So, the explanation on why the solution is like that is on 20170605, there are 4 loans issued (2 to customerID 242105, and he other two to 242111 and 242151), but only 2 of those loans were paid back (the 500 given to 242105 and the 500 given to 242111). The timeGap is the average of sum of how many days did it took every customers to pay them back (242105 paid back on 20170607 - 2 day, and 242111 paid back on 20170606 - 1 day), so (2+1)/2 = 1.5.
I have tried to calculate the nRepayments (I figured if I did this the timeGap should be a piece of cake) with the following R script.
#Recoveries
data_loans_rec <- data_loans %>% arrange(customerID, trxDate) %>% as.data.table()
data_loans_rec[is.na(data_loans_rec)] <- 0
data_loans_rec <- data_loans_rec[, index := seq_len(.N), by = customerID][!(index == 1 & TYPE == "payment")][, index := seq_len(.N), by = customerID]
n_loans_given <- data_loans[TYPE == "loan", ][, .(nloans = .N), .(payment)][order(payment)]
n_loans_rec <- copy(n_loans_given)
n_loans_rec[, nloans:=0]
unique_cust <- unique(data_loans_rec$customerID)
#Check repayment for every customer================
for (i in 1:length(unique_cust)) {
cur_cust <- unique_cust[i]
list_loan <- as.vector(data_loans_rec[customerID == cur_cust & TYPE == "loan", .(balanceChange)] )
list_loan_time <- as.vector(data_loans_rec[customerID == cur_cust & TYPE == "loan", .(trxDate) ])
list_pay <- as.vector(data_loans_rec[customerID == cur_cust & TYPE == "payment", .(balanceChange) ])
if (dim(list_pay)[1] == 0) { #if there are no payments
list_pay <- c(0)
}
sum_paid <- sum(abs(list_pay))
i_paid_until <- 0
for (i_loantime in 1:(dim(list_loan_time)[1])) {
#if there is only one loan
if (i_loantime == 0) {
i_loantime <- 1
}
loan_curr <- list_loan[i_loantime]
loan_left <- loan_curr - sum_paid
if (loan_left <= 0) {
n_loans_rec[trxDate == list_loan_time[i_loantime], nloans:=nloans+1]
sum_paid <- sum_paid - loan_curr
print (paste(i_loantime, list_loan_time[i_loantime], n_loans_rec[trxDate == list_loan_time[i_loantime], .(nloans)]))
# break
} else {
break
}
}
print (i)
}
The idea is that for every customer, make a list of loans, time of loan, and payments. The best case scenario is if the customer's total amount of loan is equal or less (due to dirty data) the total amount of payment (full payment). Then the number of repayments equals the number of loans issued to that customer. The average case is when customer's make a partial payment. In which case, I sum the total amount of payments, and I iterate through each loan the customer made whilst summing the total amount of loans as I iterate. If the amount of loan finally exceeds the amount of payments, then I count how many loans have actually been covered by the customer's payments.
The problem is I have millions of customers, and each of them have made loans and payments at least 5 times. So, since I am using a nested loop, it would take hours to complete.
So, I am asking here if anyone has ever come across this problem and/or have a better, more efficient solution.
Thanks in advance!

Answer 1

Your logic is quite complicated and with this answer I don't attempt to replicate it fully; my intention is just to give you some ideas on how to optimise.
Also, as mentioned in comments, you could try to parallelise, or maybe use another programming language.
Anyway, as your setup is already with data.table, you can try to use global operations to the full set, as much as you can, which will usually go faster than your big loop. Something for example like this.
I first calculate, per customer id, the balance and the sum of payments done:
data_loans_rec <- data_loans_rec[, balance := sum(balanceChange), by = customerID]
data_loans_rec <- data_loans_rec[, sumPayments := sum(balanceChange[TYPE == "payment"]), by = customerID]
With this, you already know that every customer with balance 0 has repaid everything:
data_loans_rec <- data_loans_rec[TYPE == "loan" & balance == 0, repaid := TRUE, by = list(customerID, index)]
These operations of course read a lot of data if you have millions of customers, but I'd say that data.table should handle them pretty quickly.
For the rest of the customers, but only for those registers that are a loan and you don't know yet if they have been repaid, you can use a data.table function.
setRepaid <- function(balanceChange, sumPayments) {
# note that here you get a vector for all the loans of a customer
sumPay <- (-1) * sumPayments[1]
if (sumPay == 0)
return(rep(FALSE, length(balanceChange)))
number_of_loans_paid <- 0
for (i in 1:length(balanceChange)) {
if (sum(balanceChange[1:i]) > sumPay)
break
number_of_loans_paid <- number_of_loans_paid + 1
}
return(c(rep(TRUE, number_of_loans_paid), rep(FALSE, length(balanceChange)-number_of_loans_paid)))
}
data_loans_rec <- data_loans_rec[TYPE == "loan" & is.na(repaid), repaid := setRepaid(balanceChange, sumPayments), by = list(customerID) ]
With that you get the desired result, at least for your example.
customerID balanceChange trxDate TYPE index balance sumPayments repaid
1: 242105 500 20170605 loan 1 1000 -1000 TRUE
2: 242105 1500 20170605 loan 2 1000 -1000 FALSE
3: 242105 -1000 20170607 payment 3 1000 -1000 NA
4: 242111 500 20170605 loan 1 0 -1000 TRUE
5: 242111 -500 20170606 payment 2 0 -1000 NA
6: 242111 500 20170607 loan 3 0 -1000 TRUE
7: 242111 -500 20170609 payment 4 0 -1000 NA
8: 242151 500 20170605 loan 1 500 0 FALSE
Advantages being: the final loop works over much less customers, you have some stuff already precalculated, and you rely on data.table for actually substituting your loop. Hopefully this approach will give you an improvement. I think it is work a try.