I'm using Vis.js timeline first time I want timeline year wise instead of combined year timeline. I tried groups options in Vis.js Instead of items but didn't work.
After page refresh i'm getting timeline like this:
But I want timeline like this:
Can you please help me out from this problem?
Thanks
Code:
var container = document.getElementById('visualization');
// Create a DataSet (allows two way data-binding)
var items = new vis.DataSet(
[
{
"content": "Application 31 August 2004 - 0.0 ",
"start": "2004-08-31",
"id": 0
},
{
"content": "cricket 10 October 2007 - 3.11 Years",
"start": "2007-10-10",
"id": 1
},
{
"content": "Inter 09 January 2008 - 3.36 Years",
"start": "2008-01-09",
"id": 2
},
{
"content": "Final 09 April 2008 - 3.61 Years",
"start": "2008-04-09",
"id": 3
},
{
"content": "exam 07 July 2008 - 3.85 Years",
"start": "2008-07-07",
"id": 4
},
{
"content": "asf 18 July 2008 - 3.88 Years",
"start": "2008-07-18",
"code": "all",
"id": 5
},
{
"content": "pal 01 August 2008 - 3.92 Years",
"start": "2008-08-01",
"id": 6
},
{
"content": "Final 08 January 2009 - 4.36 Years",
"start": "2009-01-08",
"id": 7
},
{
"content": "App 01 June 2009 - 4.75 Years",
"start": "2009-06-01",
"id": 8
},
{
"content": "N 31 August 2009 - 5.0 Years",
"start": "2009-08-31",
"id": 9
},
{
"content": "Fl 09 March 2010 - 5.52 Years",
"start": "2010-03-09",
"id": 10
},
{
"content": "Request 10 June 2010 - 5.78 Years",
"start": "2010-06-10",
"id": 11
},
{
"content": "Abn 15 June 2010 - 5.79 Years",
"start": "2010-06-15",
"id": 12
},
{
"content": "Non-Final 17 November 2010 - 6.22 Years",
"start": "2010-11-17",
"id": 13
},
{
"content": "Final R13 April 2011 - 6.62 Years",
"start": "2011-04-13",
"id": 14
},
{
"content": "App 07 September 2011 - 7.02 Years",
"start": "2011-09-07",
"id": 15,
}
]
);
// Configuration for the Timeline
var options = {
min: new Date(2000, 1, 5),
max: new Date(209,3,2),
// autoResize: false,
height: '200px'
};
// Create a Timeline
var timeline = new vis.Timeline(container, items, options);
You can use options as {stack:false} for making as same.
Related
I am trying to present linear regressions of two datasets on the same plot.
Bird data vs year
Signy data vs year
As they are exclusive to each other (count data from two islands) I don't want to plot a multiple regression, but I am not sure the code in R to produce both regressions on the same plot.
PlasticMass.data
ï..Year |Bird.Plastic.Mass Signy.Plastic.Mass
1 1991 | NA | 2.384
2 1992 | NA | 8.340
3 1993 | NA | 2.680
4 1994 | NA | 1.450
5 1995 | NA | 1.940
6 1996 | 6.43 | 0.570
7 1997 | 19.86| 1.170
8 1998 | 4.89 | 2.010
9 1999 | 2.97 | 1.410
10 2000 | 3.10 | 1.690
11 2001 | 3.30 | 0.350
12 2002 | 4.45 | 9.280
13 2003 | 4.05 | 16.750
14 2004 | 2.18 | 4.330
15 2005 | 4.88 | 0.260
16 2006 | 4.39 | 13.500
17 2007 | 4.27 | 6.270
18 2008 | 4.40 | 9.030
19 2009 | 1.63 | 3.860
20 2010 | 1.70 | 22.100
21 2011 | 1.64 | 1.150
22 2012 | 2.16 | 13.080
23 2013 | 3.05 | 0.140
24 2014 | 1.34 | 0.010
25 2015 | 3.66 | 0.000
26 2016 | 0.87 | 0.000
27 2017 | 1.10 | 7.010
28 2018 | 2.29 | 1.740
29 2019 | 1.44 | 80.790
R code to plot individual regressions:
Plastic by mass linear regressions
PlasticMass.data <-read.csv("Plastic by Mass.csv", header = T)
print(Plastic.Mass.data)
modelPB <-lm(Bird.Plastic.Mass ~ï..Year, data= PlasticMass.data)
modelPS <-lm(Signy.Plastic.Mass ~ï..Year, data = PlasticMass.data)
ggplot(PlasticMass.data, aes(ï..Year, Bird.Plastic.Mass))+
geom_point()+
geom_smooth(method = "lm")+
labs(x="Year", y="Bird Island Total Debris Count")
ggplot(PlasticMass.data, aes(ï..Year, Signy.Plastic.Mass))+
geom_point()+
geom_smooth(method = "lm", colour ="lightgreen")+
labs(x="Year", y="Signy Island Total Debris Count")
Here is a link to show the regression plot I made on excel (where both datasets are plotted and the separate linear regressions are shown).
Regressions in excel
Approach
Pivot to longer, use a group mapping to map pivoted group to lm
Code
library(dplyr)
library(tidyr)
library(ggplot2)
df %>%
mutate(Bird.Plastic.Mass = as.numeric(trimws(Bird.Plastic.Mass)),
Year = factor(Year))%>%
na.omit() %>%
pivot_longer(cols = Bird.Plastic.Mass:Signy.Plastic.Mass, names_to = "var", values_to="val") %>%
ggplot(aes(Year, val, col=var, group=var))+
geom_point() +
geom_smooth(method="lm")
Result (not exactly as Excel plot, may be due to less data)
Data
df <- structure(list(Year = c("1 1991 ", "2 1992 ", "3 1993 ",
"4 1994 ", "5 1995 ", "6 1996 ", "7 1997 ", "8 1998 ",
"9 1999 ", "10 2000 ", "11 2001 ", "12 2002 ", "13 2003 ",
"14 2004 ", "15 2005 ", "16 2006 ", "17 2007 ", "18 2008 ",
"19 2009 ", "20 2010 ", "21 2011 ", "22 2012 ", "23 2013 ",
"24 2014 ", "25 2015 ", "26 2016 ", "27 2017 ", "28 2018 ",
"29 2019 "), Bird.Plastic.Mass = c(" NA ", " NA ",
" NA ", " NA ", " NA ",
" 6.43 ", " 19.86", " 4.89 ",
" 2.97 ", " 3.10 ", " 3.30 ",
" 4.45 ", " 4.05 ", " 2.18 ",
" 4.88 ", " 4.39 ", " 4.27 ",
" 4.40 ", " 1.63 ", " 1.70 ",
" 1.64 ", " 2.16 ", " 3.05 ",
" 1.34 ", " 3.66 ", " 0.87 ",
" 1.10 ", " 2.29 ", " 1.44 "
), Signy.Plastic.Mass = c(2.384, 8.34, 2.68, 1.45, 1.94, 0.57,
1.17, 2.01, 1.41, 1.69, 0.35, 9.28, 16.75, 4.33, 0.26, 13.5,
6.27, 9.03, 3.86, 22.1, 1.15, 13.08, 0.14, 0.01, 0, 0, 7.01,
1.74, 80.79)), class = "data.frame", row.names = c(NA, -29L))
I have a pretty good understanding of R but am new to JSON file types and best practices for parsing. I'm having difficulties building a data frame from a raw JSON file. The JSON file (data below) is made up of repeated measure data that has multiple observations per user.
When the raw file is read into r
jdata<-read_json("./raw.json")
It comes in as a "List of 1" with that list being user_ids. Within each user_id are further lists, like so -
jdata$user_id$`sjohnson`$date$`2020-09-25`$city
The very last position actually splits into two options - $city or $zip. At the highest level, there are about 89 users in the complete file.
My goal would be to end up with a rectangular data frame or multiple data frames that I can merge together like this - where I don't actually need the zip code.
example table
I've tried jsonlite along with tidyverse and the farthest I seem to get is a data frame with one variable at the smallest level - cities and zip codes alternating rows
using this
df <- as.data.frame(matrix(unlist(jdata), nrow=length(unlist(jdata["users"]))))
Any help/suggestions to get closer to the table above would be much appreciated. I have a feeling I'm failing at looping it back through the different levels.
Here is an example of the raw json file structure:
{
"user_id": {
"sjohnson": {
"date": {
"2020-09-25": {
"city": "Denver",
"zip": "80014"
},
"2020-10-01": {
"city": "Atlanta",
"zip": "30301"
},
"2020-11-04": {
"city": "Jacksonville",
"zip": "14001"
}
},
"asmith: {
"date": {
"2020-10-16": {
"city": "Cleavland",
"zip": "34321"
},
"2020-11-10": {
"City": "Elmhurst",
"zip": "00013
},
"2020-11-10 08:49:36": {
"location": null,
"timestamp": 1605016176013
}
}
Another (straightforward) solution doing the heavy-lifting with rrapply() in the rrapply-package:
library(rrapply)
library(dplyr)
rrapply(jdata, how = "melt") %>%
filter(L5 == "city") %>%
select(user_id = L2, date = L4, city = value)
#> user_id date city
#> 1 sjohnson 2020-09-25 Denver
#> 2 sjohnson 2020-10-01 Atlanta
#> 3 sjohnson 2020-11-04 Jacksonville
#> 4 asmith 2020-10-16 Cleavland
#> 5 asmith 2020-11-10 Elmhurst
Data
jdata <- jsonlite::fromJSON('{
"user_id": {
"sjohnson": {
"date": {
"2020-09-25": {
"city": "Denver",
"zip": "80014"
},
"2020-10-01": {
"city": "Atlanta",
"zip": "30301"
},
"2020-11-04": {
"city": "Jacksonville",
"zip": "14001"
}
}
},
"asmith": {
"date": {
"2020-10-16": {
"city": "Cleavland",
"zip": "34321"
},
"2020-11-10": {
"city": "Elmhurst",
"zip": "00013"
},
"2020-11-10 08:49:36": {
"location": null,
"timestamp": 1605016176013
}
}
}
}
}')
We can build our desired structure step by step:
library(jsonlite)
library(tidyverse)
df <- fromJSON('{
"user_id": {
"sjohnson": {
"date": {
"2020-09-25": {
"city": "Denver",
"zip": "80014"
},
"2020-10-01": {
"city": "Atlanta",
"zip": "30301"
},
"2020-11-04": {
"city": "Jacksonville",
"zip": "14001"
}
}
},
"asmith": {
"date": {
"2020-10-16": {
"city": "Cleavland",
"zip": "34321"
},
"2020-11-10": {
"city": "Elmhurst",
"zip": "00013"
},
"2020-11-10 08:49:36": {
"location": null,
"timestamp": 1605016176013
}
}
}
}
}')
df %>%
bind_rows() %>%
pivot_longer(everything(), names_to = 'user_id') %>%
unnest_longer(value, indices_to = 'date') %>%
unnest_longer(value, indices_to = 'var') %>%
mutate(city = unlist(value)) %>%
filter(var == 'city') %>%
select(-var, -value)
which gives:
# A tibble: 5 x 3
user_id date city
<chr> <chr> <chr>
1 sjohnson 2020-09-25 Denver
2 sjohnson 2020-10-01 Atlanta
3 sjohnson 2020-11-04 Jacksonville
4 asmith 2020-10-16 Cleavland
5 asmith 2020-11-10 Elmhurst
Alternative solution inspired by #Greg where we change the last two rows:
df %>%
bind_rows() %>%
pivot_longer(everything(), names_to = 'user_id') %>%
unnest_longer(value, indices_to = 'date') %>%
unnest_longer(value, indices_to = 'var') %>%
mutate(value = unlist(value)) %>%
pivot_wider(names_from = "var") %>%
select(user_id, date, city)
This gives almost the same results with the exception of one additional case where city is NA:
# A tibble: 6 x 3
user_id date city
<chr> <chr> <chr>
1 sjohnson 2020-09-25 Denver
2 sjohnson 2020-10-01 Atlanta
3 sjohnson 2020-11-04 Jacksonville
4 asmith 2020-10-16 Cleavland
5 asmith 2020-11-10 Elmhurst
6 asmith 2020-11-10 08:49:36 NA
Here's a solution in the tidyverse: a custom function unnestable() designed to recursively unnest into a table the contents of a list like you describe. See Details for particulars regarding the format of such a list and its table.
Solution
First ensure the necessary libraries are present:
library(jsonlite)
library(tidyverse)
Then define the unnestable() function as follows:
unnestable <- function(v) {
# If we've reached the bottommost list, simply treat it as a table...
if(all(sapply(
X = v,
# Check that each element is a single value (or NULL).
FUN = function(x) {
is.null(x) || purrr::is_scalar_atomic(x)
},
simplify = TRUE
))) {
v %>%
# Replace any NULLs with NAs to preserve blank fields...
sapply(
FUN = function(x) {
if(is.null(x))
NA
else
x
},
simplify = FALSE
) %>%
# ...and convert this bottommost list into a table.
tidyr::as_tibble()
}
# ...but if this list contains another nested list, then recursively unnest its
# contents and combine their tabular results.
else if(purrr::is_scalar_list(v)) {
# Take the contents within the nested list...
v[[1]] %>%
# ...apply this 'unnestable()' function to them recursively...
sapply(
FUN = unnestable,
simplify = FALSE,
USE.NAMES = TRUE
) %>%
# ...and stack their results.
dplyr::bind_rows(.id = names(v)[1])
}
# Otherwise, the format is unrecognized and yields no results.
else {
NULL
}
}
Finally, process the JSON data as follows:
# Read the JSON file into an R list.
jdata <- jsonlite::read_json("./raw.json")
# Flatten the R list into a table, via 'unnestable()'
flat_data <- unnestable(jdata)
# View the raw table.
flat_data
Naturally, you can reformat this table however you desire:
library(lubridate)
flat_data <- flat_data %>%
dplyr::transmute(
user_id = as.character(user_id),
date = lubridate::as_datetime(date),
city = as.character(city)
) %>%
dplyr::distinct()
# View the reformatted table.
flat_data
Results
Given a raw.json file like that sampled here
{
"user_id": {
"sjohnson": {
"date": {
"2020-09-25": {
"city": "Denver",
"zip": "80014"
},
"2020-10-01": {
"city": "Atlanta",
"zip": "30301"
},
"2020-11-04": {
"city": "Jacksonville",
"zip": "14001"
}
}
},
"asmith": {
"date": {
"2020-10-16": {
"city": "Cleavland",
"zip": "34321"
},
"2020-11-10": {
"city": "Elmhurst",
"zip": "00013"
},
"2020-11-10 08:49:36": {
"location": null,
"timestamp": 1605016176013
}
}
}
}
}
then unnestable() will yield a tibble like this
# A tibble: 6 x 6
user_id date city zip location timestamp
<chr> <chr> <chr> <chr> <lgl> <dbl>
1 sjohnson 2020-09-25 Denver 80014 NA NA
2 sjohnson 2020-10-01 Atlanta 30301 NA NA
3 sjohnson 2020-11-04 Jacksonville 14001 NA NA
4 asmith 2020-10-16 Cleavland 34321 NA NA
5 asmith 2020-11-10 Elmhurst 00013 NA NA
6 asmith 2020-11-10 08:49:36 NA NA NA 1605016176013
which dplyr will format into the result below:
# A tibble: 6 x 3
user_id date city
<chr> <dttm> <chr>
1 sjohnson 2020-09-25 00:00:00 Denver
2 sjohnson 2020-10-01 00:00:00 Atlanta
3 sjohnson 2020-11-04 00:00:00 Jacksonville
4 asmith 2020-10-16 00:00:00 Cleavland
5 asmith 2020-11-10 00:00:00 Elmhurst
6 asmith 2020-11-10 08:49:36 NA
Details
List Format
To be precise, the list represents nested groupings by the fields {group_1, group_2, ..., group_n}, and it must be of the form:
list(
group_1 = list(
"value_1" = list(
group_2 = list(
"value_1.1" = list(
# .
# .
# .
group_n = list(
"value_1.1.….n.1" = list(
field_a = 1,
field_b = TRUE
),
"value_1.1.….n.2" = list(
field_a = 2,
field_c = "2"
)
# ...
)
),
"value_1.2" = list(
# .
# .
# .
)
# ...
)
),
"value_2" = list(
group_2 = list(
"value_2.1" = list(
# .
# .
# .
group_n = list(
"value_2.1.….n.1" = list(
field_a = 3,
field_d = 3.0
)
# ...
)
),
"value_2.2" = list(
# .
# .
# .
)
# ...
)
)
# ...
)
)
Table Format
Given a list of this form, unnestable() will flatten it into a table of the following form:
# A tibble: … x …
group_1 group_2 ... group_n field_a field_b field_c field_d
<chr> <chr> ... <chr> <dbl> <lgl> <chr> <dbl>
1 value_1 value_1.1 ... value_1.1.….n.1 1 TRUE NA NA
2 value_1 value_1.1 ... value_1.1.….n.2 2 NA 2 NA
3 value_1 value_1.2 ... value_1.2.….n.1 ... ... ... ...
⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮
j value_2 value_2.1 ... value_2.1.….n.1 3 NA NA 3
⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮
k value_2 value_2.2 ... value_2.2.….n.1 ... ... ... ...
⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮
This question already has answers here:
How to import JSON into R and convert it to table?
(3 answers)
Closed 3 years ago.
Current Data
Sr. Genres
1 [{"id": 28, "name": "Action"}, {"id": 12, "name": "Adventure"}, {"id": 14, "name": "Fantasy"}, {"id": 878, "name": "Science Fiction"}]
2 [{"id": 12, "name": "Adventure"}, {"id": 14, "name": "Fantasy"}, {"id": 28, "name": "Action"}]
3 [{"id": 28, "name": "Action"}, {"id": 12, "name": "Adventure"}, {"id": 80, "name": "Crime"}]
4 [{"id": 28, "name": "Action"}, {"id": 80, "name": "Crime"}, {"id": 18, "name": "Drama"}, {"id": 53, "name": "Thriller"}]
5 [{"id": 28, "name": "Action"}, {"id": 12, "name": "Adventure"}, {"id": 878, "name": "Science Fiction"}]
Required Output
Sr. id name id name id name id name
1 28 Action 12 Adventure 14 Fantasy 878 Science Fiction
2 12 Adventure 14 Fantasy 28 Action
3 28 Action 12 Adventure 80 Crime 53 Thriller
4 28 Action 80 Crime 18 Drama
5 28 Action 12 Adventure 878 Science Fiction
Any help how to do this?
Just use the RJSON package. ...
#First install JSON Package:
install.packages("rjson")
# Load the package to your workspace
library(rjson)
# read your input file, assuming this is called input.json
result <- fromJSON(file = "input.json")
# Print the result.
out <- as.data.frame(result)
# Then export it
write.table(out,file="out.txt",sep="\t",col.names=T,row.names=F)
I have a tibble and a list which I would like to write to a json file.
# A tibble: 2 x 12
i n c x
<chr> <chr> <chr> <chr>
1 NYC New York City United States LON,271;BOS,201
2 LON London United Kingdom NYC,270
I would like to replace the 'x' column with a list.
When I try to merge by the 'i' column with the element of the list, a lot of data is duplicated... :/
sample list:
$NYC
d p
1: LON 271
2: BOS 201
$LON
d p
1: NYC 270
I would like to end up with something that looks like this:
[
{
"i": "NYC",
"n": "New York City",
"c": "United States",
"C": "US",
"r": "Northern America",
"F": 66.256,
"L": -166.063,
"b": 94.42,
"s": 0.752,
"q": 4417,
"t": "0,0,0,0,0",
"x": [{
"d": "LON",
"p": 271
},
{
"d": "BOS",
"p": 201
}]
}
...
]
I'm thinking there should be a way to write the json file without merging the list and the tibble, or maybe there is a way to merge them in a ragged way ?
ah. I just had another idea. maybe I can convert my dataframe to a list then use Reduce to combine the lists...
http://www.sharecsv.com/s/2e1dc764430c6fe746d2299f71879c2e/routes-before-split.csv
http://www.sharecsv.com/s/b114e2cc6236bd22b23298035fb7e042/tibble.csv
We may do the following:
tbl
# A tibble: 1 x 13
# X i n c C r F L b s q t x
# <int> <fct> <fct> <fct> <fct> <fct> <dbl> <dbl> <dbl> <dbl> <int> <fct> <fct>
# 1 1 LON London United Kingd… GB Northern Eur… 51.5 -0.127 55.4 1.25 2088 0,0,1,3… AAL,15;AAR,15;A…
require(tidyverse)
tbl$x <- map(tbl$x, ~ strsplit(., ";|,")[[1]] %>%
{data.frame(d = .[c(T, F)], p = as.numeric(.[c(F, T)]))})
The latter two lines are a shortened version of this base R equivalent:
tbl$x <- lapply(tbl$x, function(r) {
tmp <- strsplit(r, ";|,")[[1]]
data.frame(d = tmp[seq(1, length(tmp), 2)],
p = as.numeric(tmp[seq(2, length(tmp), 2)]))
})
We go over the x column, split its elements by ; and , whenever possible, and then use the fact that the resulting odd elements will correspond do the d column in the desired outcome, and the even elements to the p column.
Output:
toJSON(tbl, pretty = TRUE)
[
{
"X": 1,
"i": "LON",
"n": "London",
"c": "United Kingdom",
"C": "GB",
"r": "Northern Europe",
"F": 51.508,
"L": -0.127,
"b": 55.43,
"s": 1.25,
"q": 2088,
"t": "0,0,1,3,1",
"x": [
{
"d": "AAL",
"p": 15
},
{
"d": "AAR",
"p": 15
},
{
"d": "ABZ",
"p": 48
}
]
}
]
I made data scraping like table below, but I can't find solution to clean up this table vith GSUB. Namely I tried code like :
populous_table$Tax_GDP <- gsub("[:punct:]","",populous_table$Tax_GDP )
but this code can't clean brackets [] for number 7 Australia.
Can anyone help me ?
1 Afghanistan 6.4
2 Albania 22.9
3 Algeria 7.7
4 Angola 5.7
5 Argentina 37.2
6 Armenia 22.0
7 Australia 34.3 [2]
8 Austria 43.4
You may use
populous_table$Tax_GDP <- gsub("\\s*\\[\\d+]","", populous_table$Tax_GDP )
Or, if that [digits] substring is always at the end, add $:
populous_table$Tax_GDP <- gsub("\\s*\\[\\d+]$", "", populous_table$Tax_GDP )
The \s*\[\d+] pattern means
\s* - 0+ whitespaces
\[ - a [ char
\d+ - 1+ digits
] - a ] char.
See R demo:
x <- c("1 Afghanistan 6.4", "2 Albania 22.9", "3 Algeria 7.7", "4 Angola 5.7", "5 Argentina 37.2", "Armenia 22.0", "7 Australia 34.3 [2]", "8 Austria 43.4")
gsub("\\s*\\[\\d+]", "", x)
## => [1] "1 Afghanistan 6.4" "2 Albania 22.9" "3 Algeria 7.7"
[4] "4 Angola 5.7" "5 Argentina 37.2" "Armenia 22.0"
[7] "7 Australia 34.3" "8 Austria 43.4"