Not able to add "DateTime" package in Julia 1.0. How to sort this issue ??;
using Pkg
julia> Pkg.add("DateTime")
ERROR: The following package names could not be resolved:
* dateTime (not found in project, manifest or registry)
Please specify by known `name=uuid`.
Stacktrace:
[1] pkgerror(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/Types.jl:120
[2] #ensure_resolved#72(::Bool, ::Function, ::Pkg.Types.EnvCache, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/Types.jl:1010
[3] #ensure_resolved at ./none:0 [inlined]
[4] #add_or_develop#15(::Symbol, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:49
[5] #add_or_develop at ./none:0 [inlined]
[6] #add_or_develop#14 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:31 [inlined]
[7] #add_or_develop at ./none:0 [inlined]
[8] #add_or_develop#13 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:29 [inlined]
[9] #add_or_develop at ./none:0 [inlined]
[10] #add_or_develop#12(::Base.Iterators.Pairs{Symbol,Symbol,Tuple{Symbol},NamedTuple{(:mode,),Tuple{Symbol}}}, ::Function, ::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:28
[11] #add_or_develop at ./none:0 [inlined]
[12] #add#20 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:59 [inlined]
[13] add(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/API.jl:59
[14] top-level scope at none:0
julia>
The documentation of DateTime.jl reads:
This project is now obsolete. Starting with version 0.4-dev, date and
time functionality has been added to Julia base.
What is the functionality that you need? Normally, when working with date use the in-built Dates package
using Dates
EDIT
Since you have asked for an example here it is:
julia> using DataFrames
julia> d = DataFrame( col1 = ["2018-01-01","2018-02-10","2018-03-20"])
3×1 DataFrame
│ Row │ col1 │
│ │ String │
├─────┼────────────┤
│ 1 │ 2018-01-01 │
│ 2 │ 2018-02-10 │
│ 3 │ 2018-03-20 │
julia> using Dates
julia> d.col1 = Date.(d.col1);
julia> d
3×1 DataFrame
│ Row │ col1 │
│ │ Date │
├─────┼────────────┤
│ 1 │ 2018-01-01 │
│ 2 │ 2018-02-10 │
│ 3 │ 2018-03-20 │
Note how the dot operator . has been used to vectorize the operation.
Related
I have a tibble with dates in one column I would like to round to the nearest quarter. Using lubridate::round_date() each date seems to just round down to the nearest quarter. I am trying to solve so that some dates round down and some dates round up.
library(tidyverse)
library(lubridate)
my_tibble <- tibble(my_dates = seq(ymd('2022-01-01'), ymd('2022-03-31'), by = 'days'))
my_tibble <-
my_tibble %>%
mutate(qtr_date = round_date(my_dates, unit = "quarter"))
The early dates should round down, which they do:
> head(my_tibble)
# A tibble: 6 × 2
my_dates qtr_date
<date> <dttm>
1 2022-01-01 2022-01-01 00:00:00
2 2022-01-02 2022-01-01 00:00:00
3 2022-01-03 2022-01-01 00:00:00
4 2022-01-04 2022-01-01 00:00:00
5 2022-01-05 2022-01-01 00:00:00
6 2022-01-06 2022-01-01 00:00:00
But the later dates also round down:
> tail(my_tibble)
# A tibble: 6 × 2
my_dates qtr_date
<date> <dttm>
1 2022-03-26 2022-01-01 00:00:00
2 2022-03-27 2022-01-01 00:00:00
3 2022-03-28 2022-01-01 00:00:00
4 2022-03-29 2022-01-01 00:00:00
5 2022-03-30 2022-01-01 00:00:00
6 2022-03-31 2022-01-01 00:00:00
I was expecting the dates after the midpoint (2022-02-15) to round up to second quarter date.
If I wanted the dates in the quarter to always round up or round down I would have used cieling_date() or floor_date().
Is there someway to modify round_date() so that it actually rounds up or down?
Here is my info from sessioninfo()
> sessionInfo()
R version 4.2.0 (2022-04-22 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22621)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.utf8 LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] lubridate_1.8.0 forcats_0.5.1 stringr_1.4.0 dplyr_1.0.9 purrr_0.3.4
[6] readr_2.1.2 tidyr_1.2.0 tibble_3.1.6 ggplot2_3.3.6 tidyverse_1.3.1
loaded via a namespace (and not attached):
[1] cellranger_1.1.0 pillar_1.7.0 compiler_4.2.0 dbplyr_2.1.1 tools_4.2.0
[6] jsonlite_1.8.0 lifecycle_1.0.1 gtable_0.3.0 pkgconfig_2.0.3 rlang_1.0.3
[11] reprex_2.0.1 DBI_1.1.2 cli_3.3.0 rstudioapi_0.13 haven_2.5.0
[16] xml2_1.3.3 withr_2.5.0 httr_1.4.3 fs_1.5.2 generics_0.1.2
[21] vctrs_0.4.1 hms_1.1.1 grid_4.2.0 tidyselect_1.1.2 glue_1.6.2
[26] R6_2.5.1 fansi_1.0.3 readxl_1.4.0 tzdb_0.3.0 modelr_0.1.8
[31] magrittr_2.0.3 backports_1.4.1 scales_1.2.0 ellipsis_0.3.2 rvest_1.0.2
[36] assertthat_0.2.1 colorspace_2.0-3 utf8_1.2.2 stringi_1.7.6 munsell_0.5.0
[41] broom_0.8.0 crayon_1.5.1
I could replicate this issue in lubridate v1.8.0. If you look at the source for round_date() you will see that this function has been completely refactored. The work is now done by the line:
timechange::time_round(x, unit = unit, week_start = as_week_start(week_start))
round_date() was previously calling floor_date() and ceiling_date() and finding which was nearest. We can see that this change was made in the commit on November 4th 2022 (line 174).
This does not entirely explain why your code did not work, but knowing round_date() is now calculated differently, I updated lubridate to the latest CRAN version (1.9.0) with:
install.packages("lubridate")
It is also possible to install a specific version of a package as described here.
Updating to 1.9.0 also installed timechange as a dependency (v.0.1.1), which fixed the problem.
Please have a look at the reprex at the end of the post.
I need to read a column as a string, perform several manipulations and then save convert it to a numerical column.
The blanks ("") in the string column give me a headache because arrow does not convert them to numerical missing values NA.
Does anybody know how to achieve that?
Many thanks
library(tidyverse)
library(arrow)
#> Some features are not enabled in this build of Arrow. Run `arrow_info()` for more information.
#>
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#>
#> timestamp
df <- tibble(x=rep(c("4000 -", "6000 -", "", "8000 - "), 10),
y=seq(1,10, length=40))
write_csv(df, "test_string.csv")
data <- open_dataset("test_string.csv",
format="csv",
skip=1,
schema=schema(x=string(), y=double()))
data2 <- data |>
mutate(x= sub(" -.*", "", x) ) |>
mutate(x2=as.numeric(x)) |>
collect() ## how to convert the blank to a numeric NA ?
#> Error in `collect()`:
#> ! Invalid: Failed to parse string: '' as a scalar of type double
#> Backtrace:
#> ▆
#> 1. ├─dplyr::collect(mutate(mutate(data, x = sub(" -.*", "", x)), x2 = as.numeric(x)))
#> 2. └─arrow:::collect.arrow_dplyr_query(mutate(mutate(data, x = sub(" -.*", "", x)), x2 = as.numeric(x)))
#> 3. └─base::tryCatch(...)
#> 4. └─base (local) tryCatchList(expr, classes, parentenv, handlers)
#> 5. └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
#> 6. └─value[[3L]](cond)
#> 7. └─arrow:::augment_io_error_msg(e, call, schema = x$.data$schema)
#> 8. └─rlang::abort(msg, call = call)
sessionInfo()
#> R version 4.2.2 (2022-10-31)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: Debian GNU/Linux 11 (bullseye)
#>
#> Matrix products: default
#> BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
#> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.13.so
#>
#> locale:
#> [1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C
#> [3] LC_TIME=en_GB.UTF-8 LC_COLLATE=en_GB.UTF-8
#> [5] LC_MONETARY=en_GB.UTF-8 LC_MESSAGES=en_GB.UTF-8
#> [7] LC_PAPER=en_GB.UTF-8 LC_NAME=C
#> [9] LC_ADDRESS=C LC_TELEPHONE=C
#> [11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C
#>
#> attached base packages:
#> [1] stats graphics grDevices utils datasets methods base
#>
#> other attached packages:
#> [1] arrow_10.0.0 forcats_0.5.2 stringr_1.4.1 dplyr_1.0.10
#> [5] purrr_0.3.5 readr_2.1.3 tidyr_1.2.1 tibble_3.1.8
#> [9] ggplot2_3.4.0 tidyverse_1.3.2
#>
#> loaded via a namespace (and not attached):
#> [1] lubridate_1.9.0 assertthat_0.2.1 digest_0.6.30
#> [4] utf8_1.2.2 R6_2.5.1 cellranger_1.1.0
#> [7] backports_1.4.1 reprex_2.0.2 evaluate_0.17
#> [10] httr_1.4.4 highr_0.9 pillar_1.8.1
#> [13] rlang_1.0.6 googlesheets4_1.0.1 readxl_1.4.1
#> [16] R.utils_2.12.1 R.oo_1.25.0 rmarkdown_2.17
#> [19] styler_1.8.0 googledrive_2.0.0 bit_4.0.4
#> [22] munsell_0.5.0 broom_1.0.1 compiler_4.2.2
#> [25] modelr_0.1.9 xfun_0.34 pkgconfig_2.0.3
#> [28] htmltools_0.5.3 tidyselect_1.2.0 fansi_1.0.3
#> [31] crayon_1.5.2 tzdb_0.3.0 dbplyr_2.2.1
#> [34] withr_2.5.0 R.methodsS3_1.8.2 grid_4.2.2
#> [37] jsonlite_1.8.3 gtable_0.3.1 lifecycle_1.0.3
#> [40] DBI_1.1.3 magrittr_2.0.3 scales_1.2.1
#> [43] vroom_1.6.0 cli_3.4.1 stringi_1.7.8
#> [46] fs_1.5.2 xml2_1.3.3 ellipsis_0.3.2
#> [49] generics_0.1.3 vctrs_0.5.0 tools_4.2.2
#> [52] bit64_4.0.5 R.cache_0.16.0 glue_1.6.2
#> [55] hms_1.1.2 parallel_4.2.2 fastmap_1.1.0
#> [58] yaml_2.3.6 timechange_0.1.1 colorspace_2.0-3
#> [61] gargle_1.2.1 rvest_1.0.3 knitr_1.40
#> [64] haven_2.5.1
Created on 2022-11-07 with reprex v2.0.2
ifelse works here when all classes are correct (and not double()); if_else enforces this already, so we can use either.
data |>
mutate(x = sub(" -.*", "", x)) |>
mutate(
x = ifelse(x == "", NA_character_, x), # also if_else works
x2 = as.numeric(x)
) |>
collect()
# # A tibble: 40 x 3
# x y x2
# <chr> <dbl> <dbl>
# 1 4000 1 4000
# 2 6000 1.23 6000
# 3 NA 1.46 NA
# 4 8000 1.69 8000
# 5 4000 1.92 4000
# 6 6000 2.15 6000
# 7 NA 2.38 NA
# 8 8000 2.62 8000
# 9 4000 2.85 4000
# 10 6000 3.08 6000
# # ... with 30 more rows
Try using the read_csv instead of open_dataset
library(readr)
data <- read_csv("test_string.csv")
I am working with data that has significant digits (i.e. digits after the "."). These digits appear when viewing my data both as a variable in base R, and also when the data is stored in a dataframe. However, they do not appear when I view the data in a tibble.
I need to view these significant digits for my work. Is there a way to make them appear when using tibbles?
Here is a reproducible example:
x has 5 significant digits, and 3 are displayed when using base R:
x = 1234.56789
x
[1] 1234.568
Within a data.frame, 3 significant digits are also displayed:
df = data.frame(x=x)
df
x
1 1234.568
Within a tibble, though, 0 significant digits are displayed:
library(tibble)
df = tibble(x=x)
df
# A tibble: 1 x 1
x
<dbl>
1 1235.
Again, I am looking for a way to make more than 0 significant digits appear whening viewing my data in a tibble.
Here is the result of my sessionInfo():
R version 3.5.1 (2018-07-02)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.6
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods
[7] base
other attached packages:
[1] tibble_1.4.2 readr_1.1.1 choroplethr_3.6.2
[4] acs_2.1.3 XML_3.98-1.12 stringr_1.3.1
loaded via a namespace (and not attached):
[1] httr_1.3.1 maps_3.3.0 splines_3.5.1
[4] Formula_1.2-3 assertthat_0.2.0 sp_1.3-1
[7] latticeExtra_0.6-28 yaml_2.2.0 pillar_1.3.0
[10] backports_1.1.2 lattice_0.20-35 glue_1.3.0
[13] uuid_0.1-2 digest_0.6.15 RColorBrewer_1.1-2
[16] checkmate_1.8.5 colorspace_1.3-2 htmltools_0.3.6
[19] Matrix_1.2-14 plyr_1.8.4 pkgconfig_2.0.1
[22] WDI_2.5 purrr_0.2.5 scales_0.5.0
[25] jpeg_0.1-8 tigris_0.7 ggmap_2.6.1
[28] htmlTable_1.12 ggplot2_3.0.0 nnet_7.3-12
[31] lazyeval_0.2.1 cli_1.0.0 proto_1.0.0
[34] survival_2.42-6 RJSONIO_1.3-0 magrittr_1.5
[37] crayon_1.3.4 maptools_0.9-2 fansi_0.2.3
[40] foreign_0.8-71 class_7.3-14 tools_3.5.1
[43] data.table_1.11.4 hms_0.4.2 geosphere_1.5-7
[46] RgoogleMaps_1.4.2 munsell_0.5.0 cluster_2.0.7-1
[49] bindrcpp_0.2.2 compiler_3.5.1 e1071_1.7-0
[52] rlang_0.2.1 classInt_0.2-3 units_0.6-0
[55] grid_3.5.1 rstudioapi_0.7 rjson_0.2.20
[58] rappdirs_0.3.1 htmlwidgets_1.2 base64enc_0.1-3
[61] gtable_0.2.0 curl_3.2 DBI_1.0.0
[64] reshape2_1.4.3 R6_2.2.2 gridExtra_2.3
[67] knitr_1.20 dplyr_0.7.6 rgdal_1.3-3
[70] utf8_1.1.4 bindr_0.1.1 Hmisc_4.1-1
[73] stringi_1.2.4 Rcpp_0.12.18 mapproj_1.2.6
[76] sf_0.6-3 rpart_4.1-13 acepack_1.4.1
[79] png_0.1-7 spData_0.2.9.0 tidyselect_0.2.4
you can set the option pillar.sigfig
options(pillar.sigfig = 1)
as_tibble(iris)
# # A tibble: 150 x 5
# Sepal.Length Sepal.Width Petal.Length Petal.Width Species
# <dbl> <dbl> <dbl> <dbl> <fct>
# 1 5. 4. 1. 0.2 setosa
# 2 5. 3 1. 0.2 setosa
# 3 5. 3. 1. 0.2 setosa
# 4 5. 3. 2. 0.2 setosa
# 5 5 4. 1. 0.2 setosa
# 6 5. 4. 2. 0.4 setosa
# 7 5. 3. 1. 0.3 setosa
# 8 5 3. 2. 0.2 setosa
# 9 4. 3. 1. 0.2 setosa
# 10 5. 3. 2. 0.1 setosa
options(pillar.sigfig = 7)
tb = tibble(x=x)
tb
# # A tibble: 1 x 1
# x
# <dbl>
# 1 1234.568
See also:
?`tibble-options`
or online:
https://www.rdocumentation.org/packages/tibble/versions/1.4.2/topics/tibble-options
If I pass a grouped data frame to a function, then change the name of the grouped variable, the grouping of the original data frame gets changed to the new name. When the function returns (I am not returning the altered data frame), the names of the original data frame are unchanged but the grouping is changed to the non-existent name.
# test scoping of group_by() which appears to change groups
library(dplyr)
muck_up_group<-function(mydf){
mydf<-mydf %>% rename(UhOh=Species)
}
dont_muck_up_group<-function(mydf){
mydf<-mydf %>% ungroup()
mydf<-mydf %>% rename(UhOh=Species)
}
data("iris")
iris<-as_tibble(iris) %>% group_by(Species)
iris
# A tibble: 150 x 5
# Groups: Species [3]
# Sepal.Length Sepal.Width Petal.Length Petal.Width Species
# <dbl> <dbl> <dbl> <dbl> <fctr>
# 1 5.1 3.5 1.4 0.2 setosa
muck_up_group(iris) # original grouping changed to column name that doesn't exist
iris
# A tibble: 150 x 5
# Groups: UhOh [3]
# Sepal.Length Sepal.Width Petal.Length Petal.Width Species
# <dbl> <dbl> <dbl> <dbl> <fctr>
# 1 5.1 3.5 1.4 0.2 setosa
#restore original state
iris<-as_tibble(iris) %>% group_by(Species)
dont_muck_up_group(iris) # original grouping preserved
iris
# A tibble: 150 x 5
# Groups: Species [3]
# Sepal.Length Sepal.Width Petal.Length Petal.Width Species
# <dbl> <dbl> <dbl> <dbl> <fctr>
# 1 5.1 3.5 1.4 0.2 setosa
I can understand why it might be bad practice to change the name of a grouping variable but it is permissible. This seems to be an example of an attribute of a variable being passed by reference when the content is being passed by value (as we understand R does normally).
> sessionInfo()
R version 3.4.0 (2017-04-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] graphics grDevices utils datasets stats methods base
other attached packages:
[1] lubridate_1.6.0 bindrcpp_0.2 mFilter_0.1-3
[4] ggrepel_0.6.5 reshape2_1.4.2 scales_0.4.1
[7] purrr_0.2.3 readr_1.1.1 tidyr_0.7.0
[10] tibble_1.3.4 tidyverse_1.1.1 knitr_1.17
[13] Rblpapi_0.3.6 stringr_1.2.0 rvest_0.3.2
[16] xml2_1.1.1 devtools_1.13.3 dplyr_0.7.2
[19] plyr_1.8.4 ggplot2_2.2.1 PerformanceAnalytics_1.4.3541
[22] xts_0.10-0 zoo_1.8-0
loaded via a namespace (and not attached):
[1] Rcpp_0.12.12 lattice_0.20-35 assertthat_0.2.0 rprojroot_1.2 digest_0.6.12
[6] psych_1.7.5 R6_2.2.2 cellranger_1.1.0 backports_1.1.0 evaluate_0.10.1
[11] httr_1.3.1 highr_0.6 rlang_0.1.2 curl_2.8.1 lazyeval_0.2.0
[16] readxl_1.0.0 TTR_0.23-2 tidyquant_0.5.3 rmarkdown_1.6 labeling_0.3
[21] foreign_0.8-67 munsell_0.4.3 broom_0.4.2 compiler_3.4.0 modelr_0.1.1
[26] pkgconfig_2.0.1 base64enc_0.1-3 mnormt_1.5-5 htmltools_0.3.6 tidyselect_0.1.1
[31] withr_2.0.0 Quandl_2.8.0 grid_3.4.0 nlme_3.1-131 jsonlite_1.5
[36] gtable_0.2.0 magrittr_1.5 quantmod_0.4-10 stringi_1.1.5 RColorBrewer_1.1-2
[41] tools_3.4.0 forcats_0.2.0 glue_1.1.1 hms_0.3 rsconnect_0.8.5
[46] parallel_3.4.0 yaml_2.1.14 colorspace_1.3-2 memoise_1.1.0 bindr_0.1
[51] haven_1.1.0
>
Bug? Thanks.
See #aosmith’s comment above. Dplyr closed issue.
I'm having problems with using dcast withoud "id" variables. The expected result is basically a transposition -- creating a 1-row data frame with as many columns as there are rows in the original data frame.
I've tried different approaches, but only "hacks" seem to work for now. Before filing a bug, I wanted to double-check if I'm missing something.
d <- data.frame(variable=letters[1:3], value=1:3)
d
## variable value
## 1 a 1
## 2 b 2
## 3 c 3
reshape2::dcast(d, ...~variable)
## . a b c
## 1 . 1 2 3
reshape2::dcast(d, .~variable)
## . a b c
## 1 . 1 2 3
reshape2::dcast(d, ~variable)
## Error: subscript out of bounds
reshape2::dcast(d, 0~variable)
## 0 a b c
## 1 0 1 2 3
sessionInfo()
## R version 3.1.1 (2014-07-10)
## Platform: x86_64-pc-linux-gnu (64-bit)
##
## locale:
## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
## [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
## [9] LC_ADDRESS=C LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
##
## attached base packages:
## [1] stats graphics grDevices utils datasets base
##
## loaded via a namespace (and not attached):
## [1] evaluate_0.5.6 formatR_1.0 knitr_1.6.18 methods_3.1.1
## [5] plyr_1.8.1 Rcpp_0.11.2 reshape2_1.4 stringr_0.6.2
## [9] tools_3.1.1 ulimit_0.0-2
What am I doing wrong? Why is dcast creating the odd and useless . column when using ...~variable or .~variable as formula?