Replace expressions in a source file from another source file in R - r

Question

Hello I have the following problem
Say I have a file base.R
x <- 1
# comment
y <- Y ~ X1 +
X2
# comment 2
z <- function(x) {
x + 1
}
t <- z(x)
and another file override.R
x <- 2
y <- Y ~ X1 + X3
my goal would be to create another file new.R which is essentially base.R overriden by override.R
x <- 2
# comment
y <- Y ~ X1 + X3
# comment 2
z <- function(x) {
x + 1
}
t <- z(x)
Obviously if all expressions in base.R were 1 liners I would be able to use sed but unfortunately it's not the case.
Note that I only need it to work for assignations lhs <- rhs either if ideally lhs = rhs would work as well.
EDIT: the above is a minimization of my actual problem

Answer 1

Sometime a difficult problem is best made easier by redefining the problem itself. In the following we suggest a number of approaches some of which have particularly simple implementations. In (7) we provide code that does what the question asks but you may prefer to change the problem slightly and use simpler code in one of the other solutions we provide.
1) omit first few lines in base.R & concatenate files We will assume that override.R should override everything in base.R up to the last statement to be overriden in base.R. Comments in base.R after the overridden statements will be kept as well any comments in override.R . Thus in the example comments 2 will be kept and comments will be overridden but could be replicated in override.R if desired which seems reasonable since you can't assume that a comment that applies to the assignment in base.R also applies in override.R .
Determine the number of statements n in override.R. Then parse base.R and find the last line number prior to the first line not to be overridden, ix. Then in the lines ending in that line number find the last non-comment line number, mx. Now write out override.R followed by all but the first mx lines of base.R . In the code below replace stdout() with the desired name of the output file, e.g. "outfile.R" .
library(utils)
n <- length(parse("override.R"))
g <- getParseData(parse("base.R"))
ix <- g$line1[grep("^0", g$parent)][n + 1] - 1
baseLines <- readLines("base.R")
is_comment <- grepl("^\\s*#", head(baseLines, ix))
mx <- max(which(!is_comment))
overrideLines <- readLines("override.R")
writeLines(c(overrideLines, tail(baseLines, -mx)), stdout())
giving:
x <- 2
y <- Y ~ X1 + X3
# comment 2
z <- function(x) {
x + 1
}
t <- z(x)
2) comment out rather thqan omit One alternative would be to comment out the overridden lines rather than omitting them. We can readily do that by replacing the writeLines statement with the statement below. This will allow one to see both the comments in base.R, if any, and the comments in override.R .
writeLines(c(overrideLines, sub("^", "# ", head(baseLines, mx)),
tail(baseLines, -mx)), stdout())
giving:
x <- 2
y <- Y ~ X1 + X3
# x <- 1
# # comment
# y <- Y ~ X1 +
# X2
# comment 2
z <- function(x) {
x + 1
}
t <- z(x)
3) separator If you control base.R then a simpler approach is to mark the end of the portion to be overriden. Suppose we put #--- on a line by itself in base.R between the portion to override and the rest. Then we have the following which is simpler:
overrideLines <- readLines("override.R")
baseLines <- readLines("base.R")
ix <- grep("#---", baseLines)[1]
writeLines(c(overrideLines, tail(baseLines, -ix)), stdout())
4) exists or possibly, in base.R, check if x has already been defined and only define it if not. Ditto for y. Then it is just a matter of concatenating the two files or sourcing one after the other.
if (!exists("x")) x <- ...whatever...
if (!exists("y")) y <- ...whatever...
5) function Yet another possibility is to define a function whose defaults are the current values of x and y in base.R. Then we can call it as f() to get the defaults or specify them.
f <- function(x = ..., y = ...) { ...base.R code except x and y ...}
6) Omit definitions from base.R Perhaps the simplest alternative is just to omit the definitions from base.R and for each run have a override.R that is sourced first or concatenated.
7) Keep base.R comments This one does what the question asked but it is a bit complex and you may prefer one of the other solutions.
library(codetools)
library(utils)
baseLines <- readLines("base.R")
overrideLines <- readLines("override.R")
p_o <- parse("override.R")
g_o <- getParseData(p_o)
locals_o <- findLocalsList(p_o)
ipos <- sapply(locals_o, function(x) which(g_o$text == x)[1]-1)
DFo <- cbind(g_o[ipos, ], var = names(ipos))
p_b <- parse("base.R")
g_b <- getParseData(p_b)
ipos <- sapply(locals_o, function(x) which(g_b$text == x)[1]-1)
DFb <- data.frame(g_b[ipos, ], var = names(ipos), row.names = NULL)
o <- order(-DFb$line1)
DFb <- DFb[o, ]
newLines <- baseLines
for(i in 1:nrow(DFb)) {
j <- match(DFb$var[i], DFo$var)
newLines <- append(newLines,
overrideLines[DFo$line1[j]:DFo$line2[j]], DFb$line2[i])
newLines <- newLines[-(DFb$line1[i]:DFb$line2[i])]
}
writeLines(newLines, stdout())
giving:
x <- 2
# comment
y <- Y ~ X1 + X3
# comment 2
z <- function(x) {
x + 1
}
t <- z(x)

Answer 2

If you can accept comments being stripped, then this might suffice for you:
Starting with base.R:
x <- 1
# comment
y <- Y ~ X1 +
X2
# comment 2
z <- function(x) {
x + 1
}
t <- z(x)
and override.R:
x <- 2
y <- Y ~ X1 + X3
We can run:
base <- parse("base.R")
override <- parse("override.R")
base_assignment <-
sapply(base, function(z) as.character(z[[1]]) %in% c("<-", "="))
base_lhs <- mapply(function(assigned, z) as.character(z[[2]]),
base_assignment, base)
override_assignment <-
sapply(override, function(z) as.character(z[[1]]) %in% c("<-", "="))
override_lhs <- mapply(function(assigned, z) as.character(z[[2]]),
override_assignment, override)
matches <- match(base_lhs, override_lhs)
base[which(!is.na(matches))] <- override[na.omit(matches)]
writeLines(paste(do.call(c, lapply(base, deparse)), collapse = "\n"), "new.R")
and now we have new.R with
x <- 2
y <- Y ~ X1 + X3
z <- function(x) {
x + 1
}
t <- z(x)
For conversation, in order to retain comments we'd likely need to use getParseData:
iterate over $parent and $id so that our $line1 references can be combined, store this reduced line1 into a new variable (since we'll need to remove the originals from getParseData(base);
find all references to $token == "SYMBOL" where there exists $token == "LEFT_ASSIGN" later in each expression. This starts to hobble it a little in the instance we have "EQ_ASSIGN" or, more of a challege, "RIGHT_ASSIGN" (since the presumed order of symbols changes);
step 2 helps us find object names to which assignments occur, which we use to compare between base/override processing;
replace the subset of each versions' parsed frame;
find a way to recombine the resulting parsed frame into a source file.
I ran out of time trying to get this to work elegantly/robustly, so I offer it as an example of effort-required in order to retain comments.
I suggest that if your intent is to allow a single source file of overriding expressions, it makes sense to keep the base.R untouched (as in your question) and create a temporary new.R that is used and sourced and discarded, in which case its comments are tangential.

Answer 3

This would be very challenging with sed, but you could try using awk; this works with the example data:
awk 'BEGIN{FS="<-|="} NR==FNR{a[$1]=$0; next}; {if($1 in a){c=1} else if (/[#<=]/){c++}; if(c == 1){print a[$1]} else {print $0}}' override.R base.R
x <- 2
# comment
y <- Y ~ X1 + X3
# comment 2
z <- function(x) {
x + 1
}
t <- z(x)
Basically, if the LHS from override.R is found in base.R the 'counter' is set to 1, then if any of the "#<=" characters are encountered in the following lines the counter is incremented. Then all lines in base.R with counter == 1 are replaced with the corresponding line from override.R. I can't think of cases where this would fail, but I'd be interested to see if it holds up on more complicated examples.
Formatted with further explanation:
awk 'BEGIN{FS="<-|="} # set the field separator to either "<-" or "="
NR==FNR{a[$1]=$0; next} # load override.R into an array, key = 1st field (i.e. the LHS)
{
if($1 in a){c=1} # if the LHS from override.R is found, set "c" (counter) to 1
else if(/[#<=]/){c++} # if the line contains "#", "<" or "=", increment counter
if(c==1){print a[$1]} # if c equals 1 (i.e. the LHS is present) print the override.R line
else {print $0} # else print the base.R line
}' override.R base.R