How to bind two lists with same structure? - r

Introduction
I have two nested lists with the same structure that I'd like to combine (in the c() sense).
There might already exist a concept for what I mean by same structure in graph theory, or in computer science, for this relationship but I am not aware.
So here is my attempt to clarify what I mean by same structure:
Elements of a list at some level are either all named or none is named;
When we have named elements there are never duplicated names at that level;
Parent-child node relationships are the same for the two lists, when the nodes are named elements themselves.
So I am wondering if there is already a solution for this problem which I feel might be rather general and common...(?) Any solution involving:
Using base rapply;
Tidyverse solution with some combination of purrr functions;
Functions from the rlist package
would be great!
Example
foo and bar are two example lists with same structure.
wonderful is the desired list that results from combining foo and bar (done manually).
I hope it is clear enough!
# Input lists: foo and bar
foo <- list(a = list(a1 = 1:3, a2 = rep('a', 3)), b = list(b1 = list(b11 = c(4,5,6), b12 = rep('b', 3)), b2 = list(b21 = list(b31 = c(0, 1, 2)))), c = list(list(c21 = 1:3), list(c21 = 4:6), list(c21 = 7:9)))
bar <- list(a = list(a1 = 1:3, a2 = rep('z', 3)), b = list(b1 = list(b11 = c(-1,2,5), b12 = rep('b', 3)), b2 = list(b21 = list(b31 = -c(1,2,3)))), c = list(list(c21 = 3:1), list(c21 = 5:3)))
# wonderful: desired list (result from combining foo and bar)
wonderful <- list(
a = list(
a1 = c(foo$a$a1, bar$a$a1),
a2 = c(foo$a$a2, bar$a$a2)
),
b = list(
b1 = list(
b11 = c(foo$b$b1$b11, bar$b$b1$b11),
b12 = c(foo$b$b1$b12, bar$b$b1$b12)
),
b2 = list(
b21 = list(
b31 = c(foo$b$b2$b21$b31, bar$b$b2$b21$b31)
)
)
),
c = c(foo$c, bar$c)
)
str(foo)
#> List of 3
#> $ a:List of 2
#> ..$ a1: int [1:3] 1 2 3
#> ..$ a2: chr [1:3] "a" "a" "a"
#> $ b:List of 2
#> ..$ b1:List of 2
#> .. ..$ b11: num [1:3] 4 5 6
#> .. ..$ b12: chr [1:3] "b" "b" "b"
#> ..$ b2:List of 1
#> .. ..$ b21:List of 1
#> .. .. ..$ b31: num [1:3] 0 1 2
#> $ c:List of 3
#> ..$ :List of 1
#> .. ..$ c21: int [1:3] 1 2 3
#> ..$ :List of 1
#> .. ..$ c21: int [1:3] 4 5 6
#> ..$ :List of 1
#> .. ..$ c21: int [1:3] 7 8 9
str(bar)
#> List of 3
#> $ a:List of 2
#> ..$ a1: int [1:3] 1 2 3
#> ..$ a2: chr [1:3] "z" "z" "z"
#> $ b:List of 2
#> ..$ b1:List of 2
#> .. ..$ b11: num [1:3] -1 2 5
#> .. ..$ b12: chr [1:3] "b" "b" "b"
#> ..$ b2:List of 1
#> .. ..$ b21:List of 1
#> .. .. ..$ b31: num [1:3] -1 -2 -3
#> $ c:List of 2
#> ..$ :List of 1
#> .. ..$ c21: int [1:3] 3 2 1
#> ..$ :List of 1
#> .. ..$ c21: int [1:3] 5 4 3
str(wonderful)
#> List of 3
#> $ a:List of 2
#> ..$ a1: int [1:6] 1 2 3 1 2 3
#> ..$ a2: chr [1:6] "a" "a" "a" "z" ...
#> $ b:List of 2
#> ..$ b1:List of 2
#> .. ..$ b11: num [1:6] 4 5 6 -1 2 5
#> .. ..$ b12: chr [1:6] "b" "b" "b" "b" ...
#> ..$ b2:List of 1
#> .. ..$ b21:List of 1
#> .. .. ..$ b31: num [1:6] 0 1 2 -1 -2 -3
#> $ c:List of 5
#> ..$ :List of 1
#> .. ..$ c21: int [1:3] 1 2 3
#> ..$ :List of 1
#> .. ..$ c21: int [1:3] 4 5 6
#> ..$ :List of 1
#> .. ..$ c21: int [1:3] 7 8 9
#> ..$ :List of 1
#> .. ..$ c21: int [1:3] 3 2 1
#> ..$ :List of 1
#> .. ..$ c21: int [1:3] 5 4 3

Here's a go at it:
library(purrr)
rec_map <- function(fizz, buzz) {
if(is.atomic(fizz) | is.null(names(fizz))){
c(fizz, buzz)
} else {
imap(fizz,
~rec_map(fizz[[.y]], buzz[[.y]]))
}
}
temp <- rec_map(foo, bar)
all.equal(temp, wonderful)
#> [1] TRUE
I'm by no means a computer scientist, so take the solution with a grain of salt. I am not certain about the behavior desired when there are no names for one level, but then one level down there are names (e.g., foo$c). So I just combined the results (c()) if we encountered a level without names.
edit to take a number of lists:
prec_map <- function(...){
dots <- list(...)
first_el = dots[[1]]
if(is.atomic(first_el) | is.null(names(first_el))){
do.call(c, dots)
} else {
imap(first_el,
function(el, nme){
one_level_down <- map(dots, nme)
do.call(prec_map, one_level_down)
})
}
}
temp <- prec_map(foo, bar)
all.equal(temp, wonderful)
[1] TRUE
I haven't tested it out thoroughly, but light testing looks like it gets the job done.

list_merge does something close to the requirements:
library(purrr)
res <- list_merge(foo, !!! bar)
all.equal(wonderful, list_merge(foo, !!! bar))
# [1] "Component “c”: Length mismatch: comparison on first 3 components"
# [2] "Component “c”: Component 1: Component 1: Numeric: lengths (3, 6) differ"
# [3] "Component “c”: Component 2: Component 1: Numeric: lengths (3, 6) differ"
The only difference seems to be for elements that are unnamed lists (e.g. foo$c and bar$c), the elements of which are concatenated by position (foo$c[[1]] with bar$c[[1]], foo$c[[2]] with bar$c[[2]], and foo$c[[3]] left alone since there is no bar$c[[3]]... rather than c(foo$c, bar$c)).
And a parallel version could be:
plist_merge <- function(.l) {
reduce(.l, ~ list_merge(.x, !!! .y))
}
all.equal(
plist_merge(list(foo, bar)),
list_merge(foo, !!! bar)
)
# [1] TRUE

After thinking a bit more about this problem in general... and after some inspiration from dplyr's joins, here's three joins for lists for my own future reference:
lst_left_join
lst_right_join
lst_inner_join
library(purrr)
#
# Inspired by dplyr's joins: https://r4ds.had.co.nz/relational-data.html#inner-join
# Here's some (more or less equivalent) list joins
#
lst_left_join <- function(lst_x, lst_y) {
if(is.atomic(lst_x) || is.null(names(lst_x))){
c(lst_x, lst_y)
} else {
imap(lst_x, ~lst_left_join(lst_x[[.y]], lst_y[[.y]]))
}
}
plst_left_join <- function(.l) reduce(.l, lst_left_join)
lst_right_join <- function(lst_x, lst_y) {
if(is.atomic(lst_y) || is.null(names(lst_y))){
c(lst_x, lst_y)
} else {
imap(lst_y, ~lst_right_join(lst_x[[.y]], lst_y[[.y]]))
}
}
plst_right_join <- function(.l) reduce(.l, lst_right_join)
lst_inner_join <- function(lst_x, lst_y) {
if(is.atomic(lst_y) || is.null(names(lst_y))){
c(lst_x, lst_y)
} else {
common_names <- intersect(names(lst_x), names(lst_y))
names(common_names) <- common_names # so that map preserves names
map(common_names, ~lst_inner_join(lst_x[[.x]], lst_y[[.x]]))
}
}
plst_inner_join <- function(.l) reduce(.l, lst_inner_join)
# Input lists: foo and bar.
foo <- list(x1 = 1:2, x3 = 30+5:6)
bar <- list(x1 = 10+1:2, x2 = 10+3:4)
# Output lists: r1, r2 and r3.
r1 <- lst_left_join(foo, bar)
r2 <- lst_right_join(foo, bar)
r3 <- lst_inner_join(foo, bar)
str(r1)
#> List of 2
#> $ x1: num [1:4] 1 2 11 12
#> $ x3: num [1:2] 35 36
str(r2)
#> List of 2
#> $ x1: num [1:4] 1 2 11 12
#> $ x2: num [1:2] 13 14
str(r3)
#> List of 1
#> $ x1: num [1:4] 1 2 11 12

Related

How to modify elements on a multi-level nested R list?

What I need to do is replacing all values of type vectors on the 4th level of the nested list a by the corresponding ones on the transcodification tibble and keeping the same structure for the rest of the list :
a = list(
a1 = list(
b1 = list(
c1 = list(
type = c(1,3),
attribute1 = runif(3,0,1),
attribute2 = list(d = rpois(1,1))
),
c2 = list(
type = c(2,3,6),
attribute1 = runif(3,0,1),
attribute2 = list(d = rpois(1,1))
)
),
b2 = list("foo")
),
a2 = list(
b1 = list(
c3 = list(
type = c(5),
attribute1 = runif(3,0,1),
attribute2 = list(d = rpois(1,1))
),
c4 = list(
type = c(2,3,6),
attribute1 = runif(3,0,1),
attribute2 = list(d = rpois(1,1))
)
),
b2 = list("foo")
),
a3 = list(
b1 = list(
c5 = list(
type = c(6),
attribute1 = runif(3,0,1),
attribute2 = list(d = rpois(1,1))
),
c6 = list(
type = c(1,2,3,5),
attribute1 = runif(3,0,1),
attribute2 = list(d = rpois(1,1))
)
),
b2 = list("foo")
)
)
transcodification = tibble(origin = c(1,2,3,4,5,6),
replacement = c("Peter","Jake","Matthew","Suzan","Christina","Margot"))
Is it possible to do using purrr functions ?
You can start with purrr's modify function
modify_depth(a, 3, ~map(., ~str_replace_all(., transcodification %>% pull(2) %>% set_names(1:length(.)))))
$a1
$a1$b1
$a1$b1$c1
$a1$b1$c1$type
[1] "Peter" "Matthew"
$a1$b1$c1$attribute1
character(0)
$a1$b1$c1$attribute2
character(0)
$a1$b1$c2
$a1$b1$c2$type
[1] "Jake" "Matthew" "Margot"
$a1$b1$c2$attribute1
character(0)
$a1$b1$c2$attribute2
character(0)
$a1$b2
$a1$b2[[1]]
$a1$b2[[1]][[1]]
[1] "foo"
But this will introduce additional list layers in b2, respectively.
If "type" is always on the first tree, than you can try without any further transformations
modify_depth(a, 3, ~modify_at(.,1, ~str_replace_all(., transcodification %>% pull(2) %>% set_names(1:length(.)))))
Or on each numeric vector
modify_depth(a, 3, ~modify_if(., is.numeric, ~str_replace_all(., transcodification %>% pull(2) %>% set_names(1:length(.)))))
For the replacement we will use stringr's str_replace_all while the replacement is done using a named vector like this:
transcodification %>% pull(2) %>% set_names(1:length(.))
1 2 3 4 5 6
"Peter" "Jake" "Matthew" "Suzan" "Christina" "Margot"
Another approach is to use rrapply() in the rrapply-package (an extension of base rapply()).
The list elements with name "type" that need to be replaced are specified in the condition argument and the replacement function is specified in the f argument:
library(rrapply)
ans <- rrapply(
object = a,
condition = function(x, .xname) .xname == "type",
f = function(x) transcodification$replacement[x],
how = "replace"
)
str(ans)
#> List of 3
#> $ a1:List of 2
#> ..$ b1:List of 2
#> .. ..$ c1:List of 3
#> .. .. ..$ type : chr [1:2] "Peter" "Matthew"
#> .. .. ..$ attribute1: num [1:3] 0.37 0.685 0.783
#> .. .. ..$ attribute2:List of 1
#> .. .. .. ..$ d: int 2
#> .. ..$ c2:List of 3
#> .. .. ..$ type : chr [1:3] "Jake" "Matthew" "Margot"
#> .. .. ..$ attribute1: num [1:3] 0.251 0.613 0.301
#> .. .. ..$ attribute2:List of 1
#> .. .. .. ..$ d: int 1
#> ..$ b2:List of 1
#> .. ..$ : chr "foo"
#> $ a2:List of 2
#> ..$ b1:List of 2
#> .. ..$ c3:List of 3
#> .. .. ..$ type : chr "Christina"
#> .. .. ..$ attribute1: num [1:3] 0.548 0.233 0.623
#> .. .. ..$ attribute2:List of 1
#> .. .. .. ..$ d: int 2
#> .. ..$ c4:List of 3
#> .. .. ..$ type : chr [1:3] "Jake" "Matthew" "Margot"
#> .. .. ..$ attribute1: num [1:3] 0.618 0.828 0.685
#> .. .. ..$ attribute2:List of 1
#> .. .. .. ..$ d: int 0
#> ..$ b2:List of 1
#> .. ..$ : chr "foo"
#> $ a3:List of 2
#> ..$ b1:List of 2
#> .. ..$ c5:List of 3
#> .. .. ..$ type : chr "Margot"
#> .. .. ..$ attribute1: num [1:3] 0.424 0.156 0.79
#> .. .. ..$ attribute2:List of 1
#> .. .. .. ..$ d: int 0
#> .. ..$ c6:List of 3
#> .. .. ..$ type : chr [1:4] "Peter" "Jake" "Matthew" "Christina"
#> .. .. ..$ attribute1: num [1:3] 0.941 0.16 0.649
#> .. .. ..$ attribute2:List of 1
#> .. .. .. ..$ d: int 1
#> ..$ b2:List of 1
#> .. ..$ : chr "foo"
NB: if the name "type" also occurs on other list levels, the condition can be made more precise by evaluating only the "type" elements at the fourth level of the list:
ans <- rrapply(
object = a,
condition = function(x, .xname, .xpos) .xname == "type" && length(.xpos) == 4L,
f = function(x) transcodification$replacement[x],
how = "replace"
)
#Joris C. solution is a cleaner rrapply approach but here is another take that I think gets you what you want. I was thinking would an unlist/relist type option work here:
library(rrapply)
library(tidyverse)
#restructure transcodification
transcodification_named <- transcodification$origin %>% setNames(transcodification$replacement)
#unlist list into dataframe (instead of using base::unlist)
a_unlist <- rrapply(a, how = "melt")
Then replace type values in the dataframe as discussed here:
a_unlist <- a_unlist %>%
mutate(value = map2(value, L4, ~ if(.y %in% 'type')
unname(coalesce(setNames(names(transcodification_named),
transcodification_named)[.x], as.character(.x))) else .x))
#then reconvert to a list (instead of base::relist which needs list skeleton object)
a_relist <- rrapply(a_unlist, how = "unmelt")
a_relist

Exclude elements from a list of named numbers based on entries within a character list using purrr

My data has 1,000 entries and here is the str of the first 2 elements:
> str(my_boots[1:2])
List of 2
$ :List of 4
..$ result : Named num [1:10] 0.118 0.948 4.317 1.226 1.028 ...
.. ..- attr(*, "names")= chr [1:10] "(Intercept)" "pvi2" "freqchal" "sexexp" ...
..$ output : chr "list()"
..$ warnings: chr(0)
..$ messages: chr(0)
$ :List of 4
..$ result : Named num [1:10] 0.202 0.995 2.512 1.057 0.5 ...
.. ..- attr(*, "names")= chr [1:10] "(Intercept)" "pvi2" "freqchal" "sexexp" ...
..$ output : chr "list()"
..$ warnings: chr(0)
..$ messages: chr(0)
The fields of interest are $result and $warnings; I want to return a tibble with the columns based on the names within the named list result where warning == "" (where no warning).
I'm new to purrr but I was able to get most of the way there using map_dfr(my_boots[1:2],"result") - this returns a tibble with the column names from the named numbers list but I would like to only return the ones where the entry under warnings is blank.
I wasn't sure how to create this structure manually but was able to create a single element of my_boots:
test <- list(
list("warnings" = c("blah")),
list("result" = c("alpha" = 1.1, "beta" = 2.1, "theta" =3.1, "blah" = 4.1))
)
Also: I'm using the tidyverse - thank you.
Starting with some dummy data.
library(tidyverse)
l <- list(
list(
result = 1:10,
warnings = character(0)
),
list(
result = 2:20,
warnings = "warn"
),
list(
result = 3:30,
warnings = character(0)
),
list(
result = 4:40,
warnings = "warn"
)
)
Use keep to keep only elements without warnings. map("result") pulls the result element out of each list.
l %>%
keep(~is_empty(.$warnings)) %>%
map("result")
#> [[1]]
#> [1] 1 2 3 4 5 6 7 8 9 10
#>
#> [[2]]
#> [1] 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
#> [22] 24 25 26 27 28 29 30

How can I pass a list of names to list.select?

Assume I want to use list.select function from rlist package to select two fields.
x <- list(p1 = list(type='A',score=list(c1=10,c2=8)),
p2 = list(type='B',score=list(c1=9,c2=9)),
p3 = list(type='B',score=list(c1=9,c2=7)))
rather than using this syntax:
list.select(x, type, score)
I want to use something list this, but it doesn't work:
param <- c("type", "score")
list.select(x, param)
Not sure how to do it using list.select, but here is a purrr solution:
library(purrr)
param <- c("type", "score")
map(x, `[`, param)
this obviously also works with lapply:
lapply(x, `[`, param)
but if you have a deeper nested list of lists, use modify_depth:
modify_depth(x, 1, `[`, param)
the .depth argument can be adjusted to go deeper down the hierarchy.
Output:
$p1
$p1$type
[1] "A"
$p1$score
$p1$score$c1
[1] 10
$p1$score$c2
[1] 8
$p2
$p2$type
[1] "B"
$p2$score
$p2$score$c1
[1] 9
$p2$score$c2
[1] 9
$p3
$p3$type
[1] "B"
$p3$score
$p3$score$c1
[1] 9
$p3$score$c2
[1] 7
This is a hackish way using eval(parse(.)) but the result is not identical to your solution. The pieces are there, though.
> str(list.select(x, do.call(c, sapply(param, FUN = function(x) eval(parse(text = x))))))
List of 3
$ p1:List of 1
..$ :List of 3
.. ..$ type : chr "A"
.. ..$ score.c1: num 10
.. ..$ score.c2: num 8
$ p2:List of 1
..$ :List of 3
.. ..$ type : chr "B"
.. ..$ score.c1: num 9
.. ..$ score.c2: num 9
$ p3:List of 1
..$ :List of 3
.. ..$ type : chr "B"
.. ..$ score.c1: num 9
.. ..$ score.c2: num 7
> str(list.select(x, type, score))
List of 3
$ p1:List of 2
..$ type : chr "A"
..$ score:List of 2
.. ..$ c1: num 10
.. ..$ c2: num 8
$ p2:List of 2
..$ type : chr "B"
..$ score:List of 2
.. ..$ c1: num 9
.. ..$ c2: num 9
$ p3:List of 2
..$ type : chr "B"
..$ score:List of 2
.. ..$ c1: num 9
.. ..$ c2: num 7

Apply over nested list names: Sub out character in nested list names

I have lists of unknown structure (nesting) that always terminate with a named vector. I want to substitute all the periods in the list or atomic vector names for an underscore. There's rapply to apply functios to list elements but how do I apply over the list/atomic vector's names? I am after a base R solution but please share all solutions for others.
MWE
x <- list(
urban = list(
cars = c('volvo', 'ford'),
food.dining = list(
local.business = c('carls'),
chain.business = c('dennys', 'panera')
)
),
rural = list(
land.use = list(
farming =list(
dairy = c('cows'),
vegie.plan = c('carrots')
)
),
social.rec = list(
community.center = c('town.square')
),
people.type = c('good', 'bad', 'in.between')
),
other.locales = c('suburban'),
missing = list(
unknown = c(),
known = c()
),
end = c('wow')
)
Desired Outcome
## $urban
## $urban$cars
## [1] "volvo" "ford"
##
## $urban$food_dining
## $urban$food_dining$local_business
## [1] "carls"
##
## $urban$food_dining$chain_business
## [1] "dennys" "panera"
##
##
##
## $rural
## $rural$land_use
## $rural$land_use$farming
## $rural$land_use$farming$dairy
## [1] "cows"
##
## $rural$land_use$farming$vegie_plan
## [1] "carrots"
##
##
##
## $rural$social_rec
## $rural$social_rec$community_center
## [1] "town.square"
##
##
## $rural$people_type
## [1] "good" "bad" "in.between"
##
##
## $other_locales
## [1] "suburban"
##
## $missing
## $missing$unknown
## NULL
##
## $missing$known
## NULL
##
##
## $end
## [1] "wow"
Here is an idea for a recursive function. It first substitutes the periods in the names with underscores. It then checks if the class of an element is list, and if yes, it applies the function on that element. Otherwise, if the class is character, it substitutes the periods in its elements with underscores. Note that this will not work if there are for example data.frames in the list, that would have to be an extension defined in the function as well. Hope this helps!
Function:
my_func <- function(x)
{
names(x) <- gsub('\\.','_',names(x) )
for(i in 1:length(x))
{
if(any(class(x[[i]])=='list'))
{
x[[i]] <- my_func(x[[i]])
}
}
return(x)
}
y <- my_func(x)
Data:
x <- list(
urban = list(
cars = c('volvo', 'ford'),
food.dining = list(
local.business = c('carls'),
chain.business = c('dennys', 'panera')
)
),
rural = list(
land.use = list(
farming =list(
dairy = c('cows'),
vegie.plan = c('carrots')
)
),
social.rec = list(
community.center = c('town.square')
),
people.type = c('good', 'bad', 'in.between')
),
other.locales = c('suburban'),
missing = list(
unknown = c(),
known = c()
),
end = c('wow')
)
Output:
str(y)
List of 5
$ urban :List of 2
..$ cars : chr [1:2] "volvo" "ford"
..$ food_dining:List of 2
.. ..$ local_business: chr "carls"
.. ..$ chain_business: chr [1:2] "dennys" "panera"
$ rural :List of 3
..$ land_use :List of 1
.. ..$ farming:List of 2
.. .. ..$ dairy : chr "cows"
.. .. ..$ vegie_plan: chr "carrots"
..$ social_rec :List of 1
.. ..$ community_center: chr "town.square"
..$ people_type: chr [1:3] "good" "bad" "in.between"
$ other_locales: chr "suburban"
$ missing :List of 2
..$ unknown: NULL
..$ known : NULL
$ end : chr "wow"
For list objects, it will rename the list and recursively call the same function for each of its elements. For character objects, it will just return the character.
library('purrr')
fix_names.list <- function(v) {
names(v) <- gsub('\\.', '_', names(v))
map(v, fix_names)
}
fix_names.default <- function(v) v
fix_names <- function(v) UseMethod('fix_names')
fix_names(x) %>% str
# List of 5
# $ urban :List of 2
# ..$ cars : chr [1:2] "volvo" "ford"
# ..$ food_dining:List of 2
# .. ..$ local_business: chr "carls"
# .. ..$ chain_business: chr [1:2] "dennys" "panera"
# $ rural :List of 3
# ..$ land_use :List of 1
# .. ..$ farming:List of 2
# .. .. ..$ dairy : chr "cows"
# .. .. ..$ vegie_plan: chr "carrots"
# ..$ social_rec :List of 1
# .. ..$ community_center: chr "town.square"
# ..$ people_type: chr [1:3] "good" "bad" "in.between"
# $ other_locales: chr "suburban"
# $ missing :List of 2
# ..$ unknown: NULL
# ..$ known : NULL
# $ end : chr "wow"
Not a base-R approach, but might still be relevant as this can be done out-of-the-box with rrapply in the rrapply-package (an extension of base-rapply):
x1 <- rrapply::rrapply(
x, ## nested list
f = function(x, .xname) gsub("\\.", "_", .xname), ## new names
how = "names" ## replace names instead of content
)
str(x1)
#> List of 5
#> $ urban :List of 2
#> ..$ cars : chr [1:2] "volvo" "ford"
#> ..$ food_dining:List of 2
#> .. ..$ local_business: chr "carls"
#> .. ..$ chain_business: chr [1:2] "dennys" "panera"
#> $ rural :List of 3
#> ..$ land_use :List of 1
#> .. ..$ farming:List of 2
#> .. .. ..$ dairy : chr "cows"
#> .. .. ..$ vegie_plan: chr "carrots"
#> ..$ social_rec :List of 1
#> .. ..$ community_center: chr "town.square"
#> ..$ people_type: chr [1:3] "good" "bad" "in.between"
#> $ other_locales: chr "suburban"
#> $ missing :List of 2
#> ..$ unknown: NULL
#> ..$ known : NULL
#> $ end : chr "wow"

NA to replace NULL in list/for loop

I am trying to replace NULL values with NAs in a list pulled from an API, but the lengths are different and therefore can't be replaced.
I have tried using the nullToNA function in the toxboot package (found here), but it won't locate the function in R when I try to call it (I don't know if there have been changes to the package which I can't locate or whether it is because the list is not pulled from a MongoDB). I have also tried all the function call checks here . My code is below. Any help?
library(httr)
library(toxboot)
library(RJSONIO)
library(lubridate)
library(xlsx)
library(reshape2)
resUrl <- "http://api.eia.gov/series/?api_key=2B5239FA427673D22505DBF45664B12E&series_id=NG.N3010CO3.M"
comUrl <- "http://api.eia.gov/series/?api_key=2B5239FA427673D22505DBF45664B12E&series_id=NG.N3020CO3.M"
indUrl <- "http://api.eia.gov/series/?api_key=2B5239FA427673D22505DBF45664B12E&series_id=NG.N3035CO3.M"
apiList <- list(resUrl, comUrl, indUrl)
results <- vector("list", length(apiList))
for(i in length(apiList)){
raw <- GET(url = as.character(apiList[i]))
char <- rawToChar(raw$content)
list <- fromJSON(char)
for (j in length(list$series[[1]]$data)){
if (is.null(list$series[[1]]$data[[j]][[2]])== TRUE)
##nullToNA(list$series[[1]]$data[[j]][[2]])
##list$series[1]$data[[j]][[2]] <- NA
else
next
}
##seriesData <- list$series[[1]]$data
unlistResult <- lapply(list, unlist)
##unlistResult <- lapply(seriesData, unlist)
##unlist2 <- lapply(unlistResult,unlist)
##results[[i]] <- unlistResult
results[[i]] <- unlistResult
}
My hashtags have some of the things that I have tried. But there are a few other methods I haven't tried.
I have seen lapply(list, function(x) ifelse (x == "NULL", NA, x)) but haven't had any luck with that eiter.
Try this:
library(httr)
resUrl <- "http://api.eia.gov/series/?api_key=2B5239FA427673D22505DBF45664B12E&series_id=NG.N3010CO3.M"
x <- GET(resUrl)
y <- content(x)
str(head(y$series[[1]]$data))
# List of 6
# $ :List of 2
# ..$ : chr "201701"
# ..$ : NULL
# $ :List of 2
# ..$ : chr "201612"
# ..$ : num 6.48
# $ :List of 2
# ..$ : chr "201611"
# ..$ : num 7.42
# $ :List of 2
# ..$ : chr "201610"
# ..$ : num 9.75
# $ :List of 2
# ..$ : chr "201609"
# ..$ : num 12.1
# $ :List of 2
# ..$ : chr "201608"
# ..$ : num 14.3
In this first URL, only the first within $series[[1]]$data contained a NULL. BTW: be clear to distinguish between NULL (the literal) and "NULL" (a character string with 4 letters).
Here are some ways (with various data types) to check for NULLs:
is.null(NULL)
# [1] TRUE
length(NULL)
# [1] 0
Simple enough so far, let's try to list with NULLs:
l <- list(NULL, 1)
is.null(l)
# [1] FALSE
sapply(l, is.null)
# [1] TRUE FALSE
length(l)
# [1] 2
lengths(l)
# [1] 0 1
sapply(l, length)
# [1] 0 1
(The "0" lengths indicate NULLs.) I'll use lengths here:
y$series[[1]]$data <- lapply(y$series[[1]]$data, function(z) { z[ lengths(z) == 0 ] <- NA; z; })
str(head(y$series[[1]]$data))
# List of 6
# $ :List of 2
# ..$ : chr "201701"
# ..$ : logi NA
# $ :List of 2
# ..$ : chr "201612"
# ..$ : num 6.48
# $ :List of 2
# ..$ : chr "201611"
# ..$ : num 7.42
# $ :List of 2
# ..$ : chr "201610"
# ..$ : num 9.75
# $ :List of 2
# ..$ : chr "201609"
# ..$ : num 12.1
# $ :List of 2
# ..$ : chr "201608"
# ..$ : num 14.3

Resources