Merge by multiple criteria and split out duplicates into separate columns? - r

Question

I'm quite sure this has been asked and answered at some point but I'm a novice and really lack the vocabulary to effectively find the question and solution. I have a simple task that I can't perform in Excel because of the internal memory limitations, but I don't know enough about SQL or R to figure out how to do it in either of those platforms.
I have two tables, one has unique entries with unique ID numbers, the other has multiple duplicates of those ID numbers, each showing a different number (representing each new salary over the course of a career). I'm trying to map each of the salaries to the original unique ID table, creating new columns for every possible change (Salary1:Salary50). Ultimately I'll also need to map on the dates and differences of each change for analysis. Here's an example:
This is the unique ID table:
Table 1
ID Salary1 Salary2 Salary3 Salary4 Salary5
1 ? ? ? ? ?
2 ? ? ? ? ?
3 ? ? ? ? ?
4 ? ? ? ? ?
5 ? ? ? ? ?
Here's the salary table with duplicate IDs and the info I want:
Table2
ID Salary SalaryDate
1 10 1/1/2014
1 11 1/1/2015
1 12 1/1/2016
2 12 1/1/2015
2 13 1/1/2016
3 10 1/1/2016
4 10 1/1/2014
4 12 1/1/2015
4 14 1/1/2016
5 10 1/1/2016
And the end state should look like this:
Table3
ID Salary1 Salary2 Salary3 Salary4 Salary5
1 10 11 12 0 0
2 12 13 0 0 0
3 10 0 0 0 0
4 10 12 0 0 0
5 10 0 0 0 0
I built a multiple-criteria Vlookup to pull everything into the right columns but the dataset has well over 100,000 rows to check so it can't complete it memory-wise. Can anyone advise on how I can do the same thing in Access, R, SPSS or if there is some efficient Excel-VBA code I can use?
Thanks for any help!

Answer 1

I have no idea what a "Vlookup" is, but apparently you are looking for something like this:
DF <- read.table(text = "ID Salary SalaryDate
1 10 1/1/2014
1 11 1/1/2015
1 12 1/1/2016
2 12 1/1/2015
2 13 1/1/2016
3 10 1/1/2016
4 10 1/1/2014
4 12 1/1/2015
4 14 1/1/2016
5 10 1/1/2016", header = TRUE)
#years of employment assuming the table is sorted by dates
DF$y <- ave(DF$ID, DF$ID, FUN = seq_along)
#reshape
library(reshape2)
dcast(DF, ID ~ y, value.var = "Salary", fill = 0)
# ID 1 2 3
#1 1 10 11 12
#2 2 12 13 0
#3 3 10 0 0
#4 4 10 12 14
#5 5 10 0 0
Note that this is not a very useful data format in R. Your original data format seems much more useful for further analyses.

Answer 2

Assume that the IDs in Table1 are a subset of the IDs in Table2 and we want just those. Also we want the first Salary for any ID in the Salary1 result column, the second salary in the Salary2 result column and so on. First compute Seq which is 1 for the first date in any ID, 2 for the second and so on. Then create a factor out of those sequence numbers whose levels are labelled by the Salary columns in Table1. In the last statement subset Table2 to the ID values of Table1 (in the case of the data shown they are the same so it does not have any effect) and reshape from long to wide form using xtabs. No packages are used.
Seq <- ave(1:nrow(Table2), Table2$ID, FUN = seq_along)
Table0 <- Table1[-1] # Table0 is Table1 without ID column
Table2$SalaryNo <- factor(Seq, levels = 1:ncol(Table0), labels = colnames(Table0))
xtabs(Salary ~ ID + SalaryNo, data = subset(Table2, ID %in% Table1$ID))
giving:
Salary_No
ID Salary1 Salary2 Salary3 Salary4 Salary5
1 10 11 12 0 0
2 12 13 0 0 0
3 10 0 0 0 0
4 10 12 14 0 0
5 10 0 0 0 0
Note: The tables were not provided in reproducible form and the solution may depend on specifically what they are so we have assumed this:
Lines1 <- "
ID Salary1 Salary2 Salary3 Salary4 Salary5
1 ? ? ? ? ?
2 ? ? ? ? ?
3 ? ? ? ? ?
4 ? ? ? ? ?
5 ? ? ? ? ?"
Table1 <- read.table(text = Lines1, header = TRUE)
Lines2 <- "
ID Salary SalaryDate
1 10 1/1/2014
1 11 1/1/2015
1 12 1/1/2016
2 12 1/1/2015
2 13 1/1/2016
3 10 1/1/2016
4 10 1/1/2014
4 12 1/1/2015
4 14 1/1/2016
5 10 1/1/2016"
Table2 <- read.table(text = Lines2, header = TRUE)
Update: Changed assumptions and code correspondingly. Also fixed an error (that did not affect the data shown but could affect other data).