This code is about inverting an index using clusters.
Unfortunately I do not understand the line with recognize<-...
I know that the function Vectorize applies the inner function element-wise, but I do not understand the inner function here.
The parameters (uniq, test) are not defined, how can we apply which then? Also why is there a "uniq" as text right after?
slots <- as.integer(Sys.getenv("NSLOTS"))
cl <- makeCluster(slots, type = "PSOCK")
inverted_index4<-function(x){
y <- unique(x)
recognize <- Vectorize(function(uniq,text) which(text %in% uniq),"uniq",SIMPLIFY = F)
y2 <- parLapply(cl, y, recognize, x)
unlist(y2,recursive=FALSE)
}
The
Vectorise()
function is just making a new element wise, vectorised function of the custom function
function(uniq,text) which(text %in% uniq).
The 'uniq' string is the argument of that function that you must specify you want to iterate over. Such that now you can pass a vector of length greater than one for uniq, and get returned a list with an element for the output of the function evaluated for every element of the input vector uniq.
I would suggest the author make the code a little clearer, better commented etc. the vectorise function doesn't need to be inside the function call necessarily.
Note
ParLapply()
isn't a function I recognise. But the x will be passed to the recognise function and the second argument text should presumably be defined earlier on, in the global environment, .GlobalEnv().
I am reading Advanced R by Hadley Wickham where some very good exercises are provided. One of them asks for description of this function:
f1 <- function(x = {y <- 1; 2}, y = 0) {
x + y
}
f1()
Can someone help me to understand why it returns 3? I know there is something called lazy evaluation of the input arguments, and e.g. another exercise asks for description of this function
f2 <- function(x = z) {
z <- 100
x
}
f2()
and I correctly predicted to be 100; x gets value of z which is evaluated inside a function, and then x is returned. I cannot figure out what happens in f1(), though.
Thanks.
See this from https://cran.r-project.org/doc/manuals/r-patched/R-lang.html#Evaluation:
When a function is called or invoked a new evaluation frame is
created. In this frame the formal arguments are matched with the
supplied arguments according to the rules given in Argument matching.
The statements in the body of the function are evaluated sequentially
in this environment frame.
...
R has a form of lazy evaluation of function arguments. Arguments are not evaluated until needed.
and this from https://cran.r-project.org/doc/manuals/r-patched/R-lang.html#Arguments:
Default values for arguments can be specified using the special form
‘name = expression’. In this case, if the user does not specify a
value for the argument when the function is invoked the expression
will be associated with the corresponding symbol. When a value is
needed the expression is evaluated in the evaluation frame of the
function.
In summary, if the parameter does not have user-specified value, its default value will be evaluated in the function's evaluation frame. So y is not evalulated at first. When the default of x is evaluated in the function's evaluation frame, y will be modified to 1, then x will be set to 2. As y is already found, the default argument has no change to be evaluated. if you try f1(y = 1) and f1(y = 2), the results are still 3.
Here is the setup.
Below are two functions that are used to create a special object that stores a numeric vector and cache's its mean.
The first function, makeVector creates a special "vector", which is really a list containing a function to
set the value of the vector
get the value of the vector
set the value of the mean
get the value of the mean
makeVector <- function(x = numeric()) {
m <- NULL
set <- function(y) {
x <<- y
m <<- NULL
}
get <- function() x
setmean <- function(mean) m <<- mean
getmean <- function() m
list(set = set, get = get,
setmean = setmean,
getmean = getmean)
}
The following function calculates the mean of the special "vector" created with the above function. However, it first checks to see if the mean has already been calculated. If so, it gets the mean from the cache and skips the computation. Otherwise, it calculates the mean of the data and sets the value of the mean in the cache via the setmean function.
cachemean <- function(x, ...) {
m <- x$getmean()
if(!is.null(m)) {
message("getting cached data")
return(m)
}
data <- x$get()
m <- mean(data, ...)
x$setmean(m)
m
}
Ok, I was thinking about this 2 function for 2 days. I tried to understand how it work because I will need to write similar function for different purpose.
After all this thinking I could not understand one thing - why the heck we need set() function in makeVector and what the heck does it do?
Below is the description of what I managed to understand and how I understand the logic (taken from my comment on Coursera):
makeVector works without set() function?
set() is never called in cachemean. And no any function in makeVector is calling set() as well. What is really happening as I understand it:
cachemean calls getmean() function and assigns it to variable m. m here is local variable in cachemean scope
What getmean() does it returns m which is already from makeVector scope! In makeVector scope m is NULL at the moment of execution
cachemean checks if m is NULL or not. At the first call it is NULL so we skip if condition
Now we create a local variable data and assign it to a call of get() function. What get() does it returns the value of x which is in fact formal parameter of makeVector. In other words it returns a vector which we initially passed to makeVector function as an argument. After this we have that data is our numeric vector
Next step is rather simple. We reassign cahchemean scope m to mean() of data. That is we find mean of our numeric vector we passed as an argument to makeVector.
Now comes the step which allows us to cache result! we call setmean() function and pass mean of our numeric vector - m - to it as an argument. If you look at makeVector you will see that what setmean() does is just assigns the argument passed to it to m BUT in makeVector scope!.
Finally cachemean returns m.
The interesting things start to happen when we call cachemean on the same vector once again.
The first line of cachemean is the key line here. We assign getmean() to m in cachemean scope. But what getmean() does in this case? If you look at the makeVector, getmean() just returns the value of m BUT from makeVector scope!. And the value of m in that scope is our mean of numeric vector.
After that it is simple. We check if m is NULL, it is obviously not. So we print a message and return m which is simply the mean we have found during the first call of cachemean.
Now comes the question which confuses me a lot. Why the heck we need to define set() function?
We do not call it from cachemean.
Moreover if you delete set() from makeVector, it still works and cachemean still works and returns cached value of mean of a numeric vector.
Moreover-moreover if I look closer to the set() function, this function does not really make sense. It assigns formal parameter y to parent environment variable x. But why we need to do this? If to get an initial numeric vector, then we do it by get().
And then it assigns NULL to m which also does not make sense. Just a line before we have done the same thing.
So can please someone explain me:
Do I understand the logic correct?
What set() function is doing and why do we need it at all?
Thanks a lot in advance!
P.S. I am new to R and I am stupid, but I want to understand, what I am missing here.
EDIT: Really sorry for long read. But everyone on this site is asking what I have already done/tried, so I tried to describe it :-)
I am learning R and came across some code as part of the practice assignment.
makeVector <- function(x = numeric()) {
m <- NULL
set <- function(y) {
x <<- y
m <<- NULL
}
get <- function() x
setmean <- function(mean) m <<- mean
getmean <- function() m
list(set = set, get = get,
setmean = setmean,
getmean = getmean)
}
The documentation says:
The function, makeVector creates a special "vector", which is
really a list containing a function to
set the value of the vector
get the value of the vector
set the value of the mean
get the value of the mean
But i can not understand how the function works except for the point that it is assigning mean value to the variable m in that particular environment.
m <- NULL begins by setting the mean to NULL as a placeholder for a future value
set <- function(y) {x <<- y; m <<- NULL} defines a function to set the vector, x, to a new
vector, y, and resets the mean, m, to NULL
get <- function() x returns the vector, x
setmean <- function(mean) m <<- mean sets the mean, m, to mean
getmean <- function() m returns the mean, m
list(set = set, get = get,setmean = setmean,getmean = getmean) returns the 'special
vector' containing all of the functions just defined
This answer is an excerpt from an article I originally wrote in 2016 as a Community Mentor in the Johns Hopkins University R Programming course: Demystifying makeVector().
Overall Design of makeVector() and cachemean()
The cachemean.R file contains two functions, makeVector() and cachemean(). The first function in the file, makeVector() creates an R object that stores a vector and its mean. The second function, cachemean() requires an argument that is returned by makeVector() in order to retrieve the mean from the cached value that is stored in the makeVector() object's environment.
What's going on in makeVector()?
The key concept to understand in makeVector() is that it builds a set of functions and returns the functions within a list to the parent environment. That is,
myVector <- makeVector(1:15)
results in an object, myVector, that contains four functions: set(), get(), setmean(), and getmean(). It also includes the two data objects, x and m.
Due to lexical scoping, myVector contains a complete copy of the environment for makeVector(), including any objects that are defined within makeVector() at design time (i.e., when it was coded). A diagram of the environment hierarchy makes it clear what is accessible within myVector.
Illustrated as a hierarchy, the global environment contains the makeVector() environment. All other content is present in the makeVector() environment, as illustrated below.
Since each function has its own environment in R, the hierarchy illustrates that the objects x and m are siblings of the four functions, get(), set(), getmean(), and setmean().
Once the function is run and an object of type makeVector() is instantiated (that is, created), the environment containing myVector looks like:
Notice that the object x contains the vector 1:15, even though myVector$set() has not been executed. This is the case because the value 1:15 was passed as an argument into the makeVector() function. What explains this behavior?
When an R function returns an object that contains functions to its parent environment (as is the case with a call like myVector <- makeVector(1:15)), not only does myVector have access to the specific functions in its list, but it also retains access to the entire environment defined by makeVector(), including the original argument used to start the function.
Why is this the case? myVector contains pointers to functions that are within the makeVector() environment after the function ends, so these pointers prevent the memory consumed by makeVector() from being released by the garbage collector. Therefore, the entire makeVector() environment stays in memory, and myVector can access its functions as well as any data in that environment that is referenced in its functions.
This feature explains why x (the argument initialized on the original function call) is accessible by subsequent calls to functions on myVector such as myVector$get(), and it also explains why the code works without having to explicitly issue myVector$set() to set the value of x.
makeVector() step by step
Now, let's break the behavior of the function down, step by step.
Step 1: Initialize objects
The first thing that occurs in the function is the initialization of two objects, x and m.
makeVector(x = numeric()) {
m <- NULL
...
}
Notice that x is initialized as a function argument, so no further initialization is required within the function. m is set to NULL, initializing it as an object within the makeVector() environment to be used by later code in the function.
Furthermore, the formals part of the function declaration define the default value of x as an empty numeric vector. Initialization of the vector with a default value is important because without a default value, data <- x$get() generates the following error message.
Error in x$get() : argument "x" is missing, with no default
Step 2: Define the "behaviors" or functions for objects of type makeVector()
After initializing key objects that store key information within makeVector(), the code provides four basic behaviors that are typical for data elements within an object-oriented program. They're called "getters and settters," and more formally known as mutator and accessor methods. As one might expect, "getters" are program modules that retrieve (access) data within an object, and "setters" are program modules that set (mutate) the data values within an object.
First makeVector() defines the set() function. Most of the "magic" in makeVector() takes place in the set() function.
set <- function(y) {
x <<- y
m <<- NULL
}
set() takes an argument that is named as y. It is assumed that this value is a numeric vector, but is not stated directly in the function formals. For the purposes of the set() function, it doesn't matter whether this argument is called y, aVector or any object name other than x. Why? Since there is an x object already defined in the makeVector() environment, using the same object name would make the code more difficult to understand.
Within set() we use the <<- form of the assignment operator, which assigns the value on the right side of the operator to an object in the parent environment named by the object on the left side of the operator.
When set() is executed, it does two things:
Assign the input argument to the x object in the parent environment, and
Assign the value of NULL to the m object in the parent environment. This line of code clears any value of m that had been cached by a prior execution of cachemean().
Therefore, if there is already a valid mean cached in m, whenever x is reset, the value of m cached in the memory of the object is cleared, forcing subsequent calls to cachemean() to recalculate the mean rather than retrieving the wrong value from cache.
Notice that the two lines of code in set() do exactly the same thing as the first two lines in the main function: set the value of x, and NULL the value of m.
Second, makeVector() defines the getter for the vector x.
get <- function() x
Again, this function takes advantage of the lexical scoping features in R. Since the symbol x is not defined within get(), R retrieves it from the parent environment of makeVector().
Third, makeVector() defines the setter for the mean m.
setmean <- function(mean) m <<- mean
Since m is defined in the parent environment and we need to access it after setmean() completes, the code uses the <<- form of the assignment operator to assign the input argument to the value of m in the parent environment.
Finally, makeVector() defines the getter for the mean m. Just like the getter for x, R takes advantage of lexical scoping to find the correct symbol m to retrieve its value.
getmean <- function() m
At this point we have getters and setters defined for both of the data objects within our makeVector() object.
Step 3: Create a new object by returning a list()
Here is the other part of the "magic" in the operations of the makeVector() function. The last section of code assigns each of these functions as an element within a list(), and returns it to the parent environment.
list(set = set, get = get,
setmean = setmean,
getmean = getmean)
When the function ends, it returns a fully formed object of type makeVector() to be used by downstream R code. One other important subtlety about this code is that each element in the list is named. That is, each element in the list is created with a elementName = value syntax, as follows:
list(set = set, # gives the name 'set' to the set() function defined above
get = get, # gives the name 'get' to the get() function defined above
setmean = setmean, # gives the name 'setmean' to the setmean() function defined above
getmean = getmean) # gives the name 'getmean' to the getmean() function defined above
Naming the list elements is what allows us to use the $ form of the extract operator to access the functions by name rather than using the [[ form of the extract operator, as in myVector[[2]](), to get the contents of the vector.
Here it's important to note that the cachemean() function REQUIRES an input argument of type makeVector(). If one passes a regular vector to the function, as in
aResult <- cachemean(1:15)
the function call will fail with an error explaining that cachemean() was unable to access $getmean() on the input argument because $ does not work with atomic vectors. This is accurate, because a primitive vector is not a list, nor does it contain a $getmean() function, as illustrated below.
> aVector <- 1:10
> cachemean(aVector)
Error in x$getmean : $ operator is invalid for atomic vectors
Explaining cachemean()
Without cachemean(), the makeVector() function is incomplete. Why? As designed, cachemean() is required to populate or retrieve the mean from an object of type makeVector().
cachemean <- function(x, ...) {
...
Like makeVector(), cachemean() starts with a single argument, x, and an ellipsis that allows the caller to pass additional arguments into the function.
Next, the function attempts to retrieve a mean from the object passed in as the argument. First, it calls the getmean() function on the input object.
m <- x$getmean()
Then it checks to see whether the result is NULL. Since makeVector() sets the cached mean to NULL whenever a new vector is set into the object, if the value here is not equal to NULL, we have a valid, cached mean and can return it to the parent environment
if(!is.null(m)) {
message("getting cached data")
return(m)
}
If the result of !is.null(m) is FALSE, cachemean() gets the vector from the input object, calculates a mean(), uses the setmean() function on the input object to set the mean in the input object, and then returns the value of the mean to the parent environment by printing the mean object.
data <- x$get()
m <- mean(data, ...)
x$setmean(m)
m
Note that cachemean() is the only place where the mean() function is executed, which is why makeVector() is incomplete without cachemean().
Putting the Pieces Together: How the functions work at runtime
Now that we've explained the design of each of these functions, here is an illustration of how they work when used in an R script.
aVector <- makeVector(1:10)
aVector$get() # retrieve the value of x
aVector$getmean() # retrieve the value of m, which should be NULL
aVector$set(30:50) # reset value with a new vector
cachemean(aVector) # notice mean calculated is mean of 30:50, not 1:10
aVector$getmean() # retrieve it directly, now that it has been cached
Conclusion: what makes cachemean() work?
To summarize, the lexical scoping assignment in R Programming takes advantage of lexical scoping and the fact that functions that return objects of type list() also allow access to any other objects defined in the environment of the original function. In the specific instance of makeVector() this means that subsequent code can access the values of x or m through the use of getters and setters. This is how cachemean() is able to calculate and store the mean for the input argument if it is of type makeVector(). Because list elements in makeVector() are defined with names, we can access these functions with the $ form of the extract operator.
For additional commentary that explains how the assignment uses features of the S3 object system, please review makeCacheMatrix() as an Object.
Appendix A: What's the Point of this Assignment?
Once students get through the assignment, they frequently ask questions about its value and purpose. A good article explaining the value of lexical scoping in statistical computing is Lexical Scoping and Statistical Computing, written by Robert Gentleman and Ross Ihaka at the University of Auckland.
Appendix B: cachemean.R
Here is the entire listing for cachemean.R.
makeVector <- function(x = numeric()) {
m <- NULL
set <- function(y) {
x <<- y
m <<- NULL
}
get <- function() x
setmean <- function(mean) m <<- mean
getmean <- function() m
list(set = set, get = get,
setmean = setmean,
getmean = getmean)
}
cachemean <- function(x, ...) {
m <- x$getmean()
if(!is.null(m)) {
message("getting cached data")
return(m)
}
data <- x$get()
m <- mean(data, ...)
x$setmean(m)
m
}
Appendix C: Frequently Asked Questions
Q: Why doesn't cachemean() return the cached value? My code looks like:
cachemean(makeVector(1:100))
cachemean(makeVector(1:100))
A: Code written this way creates two different objects of type makeVector(), so the two calls to cachemean() initialize the means of each instance, rather than caching and retrieving from a single instance. Another way of illustrating how the above code operates is as follows.
Notice how the first call to cachemean() sets the cache, and the second call retrieves data from it.
Q: Why is set() never used in the code?
A: set() is included so that once an object of type makeVector() is created, its value can be changed without initializing another instance of the object. It is unnecessary the first time an object of type makeVector() is instantiated. Why? First, the value of x is set as a function argument, as in makeVector(1:30). Then, the first line of code in the function sets m <- NULL, simultaneously allocating memory for m and setting it to NULL. When a reference to this object is passed to the parent environment when the function ends, both x and m are available to be accessed by their respective get and set functions.
The following code illustrates the use of set().
Q: Why is x set with a default value in makeVector()?
A: Since x is an argument, the only place where one can set a default for it is in the formals. The type of error returned by cachemean() when a default value is not set,
Error in x$get() : argument "x" is missing, with no default
is undesirable. Our code should directly handle error conditions rather than relying on the underlying error handling in R.
It's perfectly valid to create an object of type makeVector() without populating its value during initialization. makeVector() includes a setter function so one can set its value after the object is created. However, the object must have valid data, a numeric vector, prior to executing cachemean().
Ideally, cachemean() would include logic to validate that x is not empty prior to calculating a mean. The default setting of x enables cachemean() to return NaN, which is a reasonable result.
References
Chi, Yau -- R-Tutor Named List Members, retrieved July 20, 2016.
Wickham, Hadley -- Advanced-R Functions, retrieved July 17, 2016.
Wickham, Hadley -- Advanced-R Scoping Issues, retrieved July 17, 2016.
I think that one good way to understand this example is trying the follow:
First check that when you use the function make_Vector now you have of four different setting
> mvec <- makeVector()
> x <- 1:4
> mvec$set(x)
> mvec$get()
> [1] 1 2 3 4
> mvec$getmean()
> NULL
> mvec$setmean(3.4)
> mvec$getmean()
> 3.4
3.4 It's not the correct mean, I put these number then you can check that you can set whatever number that you want.
The second part of the assignment is the follow:
cachemean <- function(x, ...) {
m <- x$getmean()
if(!is.null(m)) {
message("getting cached data")
return(m)
}
data <- x$get()
m <- mean(data, ...)
x$setmean(m)
m
}
These part or code check if you have the mean of the vector of interest. If these exist then you don't need calculate and you can use the cache variable.

I put a wrong number for the mean, then you can see that already I set the mean value as follow:
> cachemean(mvec)
> 3.4
You must pass the original mvec list used in the example
I have defined a couple of functions of arity 1, say func1(-) and func2(-). I have tested them and seen that they actually do what they are supposed to.
I wish to define a third function, say func3(-), that outputs the difference of func1(-) and func2(-). This is what I do
func3(k) = {j=func1(k)-func2(k); print(j)}
Nevertheless, it doesn't return what it ought to. Let us suppose that func1(5) outputs 10 and func2(5) outputs 2. Then, func3(5) ought to output an 8, right? It returns instead the output of func1(5) in one row, the output of func2(2) in another row, and then a zero (even though the difference of the corresponding outputs is not 0).
Do you know what's wrong with the definition of func3(-)?
A GP user function returns the last evaluated value. Here, it's the resut of
the 'print(j)' command, which prints j (side effect) and returns 'void',
which is typecast to 0 when it must be given a value, as here.
f1(x) = 10
f2(x) = 2
f3(x) = f1(x) - f2(x)
correctly returns 8. You didn't give the code for your func1 / func2
functions, but I expect you included a 'print' statement, maybe expecting it
to return a value. That's why you get outputs on different rows, before the 0.
If you don't like this 'return-last-evaluation-result' behaviour, you can use
explicit 'return (result)' statements.