Selecting a subset of IDs - r

Question

I have a data.table dt with ids on the column idnum and a data.table ids that contains a list of ids in the column idnum (all of which exist on dt)
I want to get
The intersection: dt where dt.idnum ==ids.idnum`
The complement to the intersection: dt where dt.idnum not in ids.idnum
I got the first one with ease using
setkey(dt, idnum)
setkey(ids, idnum)
dt[ids]
However, Im stuck getting the second one. My approach was
dt[is.element(idnum, ids[, idnum]) == FALSE]
However, the row numbers of the two groups do not add up to nrow(dt). I suspect the second command. What can I do instead / Where am I going wrong? Is there perhaps a more efficient way of computing the second group given that it's the complement to the first group and I already have that one?
Update
I tried the approach given in the answer, but my numbers don't add up:
> nrow(x[J(ids$idnum)])
[1] 148
> nrow(x[!J(ids$idnum)])
[1] 52730
> nrow(x)
[1] 52863
While, the first two numbers added yield 52878. That is, I have 15 rows too many. My data contains duplicates in adj, could that be the reason?
Here's some description of the data I used:
> str(x)
Classes 'data.table' and 'data.frame': 52863 obs. of 1 variable:
$ idnum: int 6 6 11 21 22 22 22 22 27 27 ...
- attr(*, ".internal.selfref")=<externalptr>
- attr(*, "sorted")= chr "idnum"
> head(x)
idnum
1: 6
2: 6
3: 11
4: 21
5: 22
6: 22
> str(ids)
Classes 'data.table' and 'data.frame': 46 obs. of 1 variable:
$ idnum: int 2909 5012 5031 5033 5478 6289 6405 6519 7923 7940 ...
- attr(*, ".internal.selfref")=<externalptr>
- attr(*, "sorted")= chr "idnum"
> head(ids)
idnum
1: 2909
2: 5012
3: 5031
4: 5033
5: 5478
6: 6289
and here is
> sessionInfo()
R version 3.1.1 (2014-07-10)
Platform: x86_64-apple-darwin10.8.0 (64-bit)
locale:
[1] C/C/C/C/C/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods
[7] base
other attached packages:
[1] yaml_2.1.13 ggplot2_1.0.0 mFilter_0.1-3
[4] data.table_1.9.4 foreign_0.8-61
loaded via a namespace (and not attached):
[1] MASS_7.3-35 Rcpp_0.11.3 chron_2.3-45
[4] colorspace_1.2-4 digest_0.6.4 grid_3.1.1
[7] gtable_0.1.2 labeling_0.3 munsell_0.4.2
[10] plyr_1.8.1 proto_0.3-10 reshape2_1.4
[13] scales_0.2.4 stringr_0.6.2 tools_3.1.1

Answer 1

Here is one way:
library(data.table)
set.seed(1) # for reproducible example
dt <- data.table(idnum=1:1e5,x=rnorm(1e5)) # 10,000 rows, unique ids
ids <- data.table(idnum=sample(1:1e5,10)) # 10 random ids
setkey(dt,idnum)
result.1 <- dt[J(ids$idnum)] # inclusive set (records with common ids)
result.2 <- dt[!J(ids$idnum)] # exclusive set (records from dt with ids$idnum excluded
any(result.2$idnum %in% result.1$isnum)
# [1] FALSE
EDIT: Response to OPs comment.
Comparing the number of rows is not meaningful. The join will return rows corresponding to all matches. So if a given idnum is present twice in dt and three times in ids, you will get 2 X 3 = 6 rows in the result. The important test is the one I did: are any of the ids in result.1 also present in result.2? If so, then there's something wrong.
If you have duplicated ids$idnum, try:
result.1 <- dt[J(unique(ids$idnum))] # inclusive set (records with common ids)