I am generating scatterplots with some acoustics data. I am finding the way the data is mapped in ggplot I am unable to reorder the legend in the traditional way with renaming the levels.
Here is the toy example for you:
tickpointer$stt
[1] 0.000 0.166 0.324 0.515 0.734 0.909 1.059 1.191 1.319 1.445 1.561 1.683
tickpointer$highhz
[1] 7338 7073 7312 7948 7975 7709 7895 7736 8293 8373 8081 9920
tickpointer$peakhz
[1] 4969 4969 4969 4969 5250 5438 3938 4125 4594 4875 5531 7125
tickpointer$lowhz
[1] 2351 2404 2510 2351 2802 2908 2882 3174 3598 4022 4580 5587
looking like: Highhz,peakhz,lowhz,stt in a dataframe
I plotted each of these in a ggplot:
require(ggplot2)
tickpointer
ggplot(tickpointer,aes(stt))+#fileoff is choosen as the x value to have a standardized starttime
geom_line(aes(y=highhz,col="High"))+
geom_point(aes(y=highhz,col="High"))+
geom_line(aes(y=peakhz,col="Peak"))+
geom_point(aes(y=peakhz,col="Peak"))+
geom_line(aes(y=lowhz,col="Low"))+
geom_point(aes(y=lowhz,col="Low"))
notice the legend is in the order: High, Peak, Low.
I would like it to be: High, Low, Peak.
Any leads or help trying to figure this out would be great.
Cheers
Related
I have data of class xtabs in R that looks like bellow:
Last Update 1Y 2Y 3Y 4Y 5Y 7Y 10Y 15Y 20Y 25Y
2011-04-15 0.666 1.315 2.105 2.780 3.355 4.180 4.807 5.233 5.411 5.504
2011-04-18 0.653 1.280 2.053 2.727 3.311 4.142 4.785 5.206 5.395 5.491
2011-04-19 0.652 1.273 2.053 2.730 3.312 4.143 4.771 5.201 5.380 5.468
2011-04-20 0.655 1.293 2.092 2.766 3.356 4.181 4.796 5.227 5.402 5.490
2011-04-21 0.644 1.281 2.079 2.772 3.337 4.171 4.805 5.231 5.409 5.485
2011-04-25 0.635 1.261 2.047 2.734 3.314 4.141 4.762 5.181 5.361 5.449
Tenor
Last Update 30Y
2011-04-15 5.504
2011-04-18 5.486
2011-04-19 5.461
2011-04-20 5.503
2011-04-21 5.503
2011-04-25 5.494
The function that I am trying to use is Nelson.Siegel from package YieldCurve. The input structure of the function should be as I have but in xts format. I tried as.xts() but it is not working. How can I change the type from xtabs to xts maintaining the same structure?
Apparently there is not much you can do about converting xtabs to xts format. I reconstructed the structure using reshape(), converted it into a data frame and at last used xts() to change the type to xts format.
MWE: a dataframe shown below.
"TVF" "minEuler" "E_avg"
29.63 -12 3.65
30.00 -26 3.311
30.24 -72 8.247
30.82 -47 13.02
20.25 -6 2.085
14.53 -32 2.154
14.66 -1 1.44
24.12 -82 6.784
21.25 -35 4.309
25.82 -17 2.697
26.15 -42 2.832
18.38 -29 2.038
18.51 -80 5.514
14.67 -58 1.91
20.40 -166 2.786
26.50 -69 6.794
40.97 -12 24.852
18.98 -102 3.107
19.15 -75 3.85
40.54 -48 16.347
35.09 -41 18.847
28.15 -73 5.963
40.25 -94 21.82
24.28 -53 6.245
30.34 -42 25.578
CODE:
plot_ly(x = S_TVF_Euler_E_axes$TVF, y = S_TVF_Euler_E_axes$minEuler,
z = S_TVF_Euler_E_axes$E_avg, type = 'surface')
OUTPUT:
GOAL: Fit a surface to the scatter plot (or directly plot a surface). I could plot the scatterplot using plot_ly but the surface isn't working.
I have tried also this, which did not work for me.
I am new to 3D plots and any help or hint is greatly appreciated!
As explained by s_t, we need to have a matrix for plotting the surface. I tried a little different way, using the interp function from the package akima. I got the idea from the answer by "mschilli" here.
> S_TVF_Euler_E_axes_plotly <- with(S_TVF_Euler_E_axes, interp(TVF,
minEuler, E_avg, duplicate = "mean"))
> plot_ly(x = S_TVF_Euler_E_axes_plotly$x, y = S_TVF_Euler_E_axes_plotly$y,
z = S_TVF_Euler_E_axes_plotly$z, type = 'surface')
Now, I am worried if my surface is right or wrong, it doesn't match at all to the solution provided by s_t!
I have the following dataset:
Col1 Col2 Col3 Col4 Col5 Col6
4439.5 6.5211 50.0182 29.4709 -0.0207 0.0888
4453 25.1186 46.5586 34.1279 -0.0529 0.082
4453.5 24.2974 46.6291 30.6281 -0.057 0.0809
4457.5 25.3257 49.6885 26.2664 -0.0357 0.0837
4465 7.1077 53.516 32.5077 -0.0398 0.1099
4465.5 7.5892 53.0884 33.1582 -0.0395 0.1128
4898.5 8.8296 55.0611 40.3813 -0.0123 0.1389
4899 9.2469 54.4799 37.1927 -0.0061 0.1354
4900 13.4119 50.8334 28.9441 -0.0272 0.1071
4900.5 21.8415 50.1127 24.2351 -0.0375 0.0882
4905 11.3824 52.4024 37.2646 -0.0324 0.1215
4918.5 6.2601 49.9454 27.715 0.0101 0.1444
4919 7.4157 49.7412 25.6159 -0.0164 0.1038
4932 25.737 46.2825 38.6334 -0.0425 0.0717
5008.5 13.641 49.7868 18.0337 -0.0213 0.111
5010.5 13.5935 49.5352 23.9319 -0.0518 0.0979
5012 16.6945 48.0672 25.2408 -0.0446 0.0985
5014.5 14.1303 49.6361 23.1816 -0.0455 0.1056
5040 7.6895 49.8688 31.562 -0.0138 0.126
5044 12.594 60.822 52.4569 0.0481 0.1877
5045.5 10.3719 56.443 43.3782 0.0076 0.1403
5046 8.1382 54.5388 46.2675 0.01 0.1443
5051.5 29.0142 46.8052 43.3224 -0.0465 0.0917
5052 32.3053 46.4278 32.9387 -0.0509 0.0868
5052.5 38.4807 45.3555 24.4187 -0.0619 0.0774
5053 38.8954 43.8459 21.8487 -0.0688 0.0681
5055 19.69 50.9335 46.9419 -0.0527 0.0897
5055.5 11.7398 51.8329 59.5443 -0.0307 0.1083
5056 13.3196 51.8329 55.4419 -0.0276 0.1262
5056.5 18.3702 51.7003 39.232 -0.0408 0.1105
5057.5 14.0531 50.1129 24.4546 -0.0444 0.0921
5058 15.292 49.8805 23.0938 -0.0347 0.0925
5059 20.5135 49.52 21.6173 -0.0333 0.1006
5060 14.5151 47.5836 27.0685 -0.0156 0.1062
5060.5 14.5188 48.2506 27.9704 -0.0363 0.1018
5228 1.2168 54.2009 17.4351 0.0583 0.1794
5229 3.5896 51.7649 26.1107 -0.0033 0.1362
5232.5 2.7404 53.5941 38.6852 0.0646 0.194
5233 3.6694 53.9483 36.674 0.0633 0.204
5234 1.3789 53.8741 18.5804 0.0693 0.1958
5234.5 0.8592 53.6052 18.1654 0.0742 0.1982
5237 2.6951 52.3763 24.8098 0.0549 0.1923
I am trying to create an R visual that will break out each Column into facets, using Col1 as the identity column.
To do this I am using this (faulty) code:
library(reshape2)
library(plotly)
plot.data <- dataset
melted <- melt(dataset, id.vars="Col1")
sp <- ggplot(melted, aes(x=Col1, y=value)) + geom_line()
# Divide by variable in the vertical direction
sp + facet_grid(variable~.)
ggplotly()
However, I am receiving an error saying:
Faceting variables must have at least one value
I know this is an unlikely solution, but did you make sure all your filters are correct / not filtering out values somehow? I find that filter are often a source of mistakes for me so if it works in R, that could be the problem.
I had the same error and it was my filtering:
Example:
I did this data <- data[data$symbol == geneId,] instead of data <- data[data$symbol %in% geneId,]
Suppose I have many data frames, that have varying row numbers (of data) but Date as common among them. e.g. :
DF1:
Date Index Change
05-04-17 29911.55 0
03-04-17 29910.22 0.0098
31-03-17 29620.5 -0.0009
30-03-17 29647.42 0.0039
29-03-17 29531.43 0.0041
28-03-17 29409.52 0.0059
27-03-17 29237.15 -0.0063
24-03-17 29421.4 0.003
And
DF2:
Date NG NG_Change
05-04-17 213.8 0.0047
04-04-17 212.8 0.0421
03-04-17 204.2 -0.0078
31-03-17 205.8 -0.0068
30-03-17 207.2 -0.0166
29-03-17 210.7 0.0483
28-03-17 201 0.005
27-03-17 200 -0.0015
24-03-17 200.3 0.0137
And another one:
DF3:
Date TI_Price TI_Change
05-04-17 51.39 0.0071
04-04-17 51.03 0.0157
03-04-17 50.24 -0.0071
31-03-17 50.6 0.005
30-03-17 50.35 0.017
29-03-17 49.51 0.0236
28-03-17 48.37 0.0134
I wanted to combine them, using Dates column "as common variable", in a way that there are only those rows in the final for which Dates are common. Such as:
Date TI_Price TI_Change NG NG_Change TI_Price TI_Change
05-04-17 51.39 0.0071 213.8 0.0047 51.39 0.0071
04-04-17 51.03 0.0157 212.8 0.0421 51.03 0.0157
03-04-17 50.24 -0.0071 204.2 -0.0078 50.24 -0.0071
31-03-17 50.6 0.005 205.8 -0.0068 50.6 0.005
30-03-17 50.35 0.017 207.2 -0.0166 50.35 0.017
29-03-17 49.51 0.0236 210.7 0.0483 49.51 0.0236
28-03-17 48.37 0.0134 201 0.005 48.37 0.0134
I am just wondering if there is any method so that I could merge them in one go and not like the merge() function which takes DF2 and DF2 at a time, merge and then the result is merged with DF3.
What I used and tweaked around (but waste):
myfulldata = merge(DF1, DF2, all.x=T)
I have a data (large data 125000 rows, ~20 MB) in which some of the rows with certain string need to be deleted and some columns need to be selected during the reading process.
Firstly, I discovered that grepl function does not work properly since fread makes the data as one column indicated also in this question.
The example data can be found here (by following #akrun advice) and header of the data like this
head(sum_data)
TRIAL : 1 3331 9091
TRIAL : 2 1384786531 278055555
2 0.10 0.000E+00 -0.0047 -0.0168 -0.9938 -0.0087 -0.0105 -0.9709 0.0035 0.0079 -0.9754 0.0081 0.0023 0.9997 -0.135324E-09 0.278754E-01
2 0.20 0.000E+00 -0.0121 0.0002 -0.9898 -0.0364 -0.0027 -0.9925 -0.0242 -0.0050 -0.9929 0.0029 -0.0023 0.9998 -0.133521E-09 0.425567E-01
2 0.30 0.000E+00 0.0193 -0.0068 -0.9884 0.0040 0.0139 -0.9782 -0.0158 0.0150 -0.9814 0.0054 -0.0008 0.9997 -0.134103E-09 0.255356E-01
2 0.40 0.000E+00 -0.0157 0.0183 -0.9879 -0.0315 -0.0311 -0.9908 -0.0314 -0.0160 -0.9929 0.0040 0.0010 0.9998 -0.134819E-09 0.257300E-01
2 0.50 0.000E+00 -0.0402 0.0300 -0.9832 -0.0093 0.0269 -0.9781 -0.0326 0.0247 -0.9802 0.0044 -0.0010 0.9997 -0.131515E-09 0.440350E-01
I attempted to read the data with fread and used grepl for removing the rows;
files <-dir(pattern = "*sum.txt",full.names = FALSE)
library(data.table)
fread_files <- function(files){
sum_data_read <- fread(files,skip=2, sep="\t", ) #seperation is tab.
df_grep <- sum_vgm_read [!grepl("TRI",sum_vgm_read$V1),] # for removing the lines that contain "TRIAL" letter in V1 column. But so far there is no V1 column is recognized!!
df <- bind_rows(df_grep) #binding rows after removing
write.table(as.data.table(df),file = gsub("(.*)(\\..*)", "\\1_new\\2", files),row.names = FALSE,col.names = TRUE)
}
and finally lapply
lapply(files, fread_files)
when I perfom this, only one row of data is created as an output which is something going on but I dont know what.
Thanks for help in advance!
Firstly, I discovered that grepl function does not work properly since
fread makes the data as one column indicated also in this question.
But that question's accepted answer says that problem was fixed in v1.9.6. Which version are you using? That's why we ask you to please state the version number up front, to save time answering.
It is a great example file and the question is great.
I would not try to reinvent the wheel as operations like these have long been implemented as command line tools, which you can use together with fread directly. The advantage is that you won't churn through R memory, you can leave the filtering to the command tool and that can be much more efficient. For example, if you load all the lines as lines into R, those strings will be cached in R's global string cache (at least temporarily). Doing that filter outside R first will save that cost.
I downloaded your great file and tested the following which works.
> fread("grep -v TRIAL sum_data.txt")
V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 V16 V17
1: 2 0.1 0 -0.0047 -0.0168 -0.9938 -0.0087 -0.0105 -0.9709 0.0035 0.0079 -0.9754 0.0081 0.0023 0.9997 -1.35324e-10 0.0278754
2: 2 0.2 0 -0.0121 0.0002 -0.9898 -0.0364 -0.0027 -0.9925 -0.0242 -0.0050 -0.9929 0.0029 -0.0023 0.9998 -1.33521e-10 0.0425567
3: 2 0.3 0 0.0193 -0.0068 -0.9884 0.0040 0.0139 -0.9782 -0.0158 0.0150 -0.9814 0.0054 -0.0008 0.9997 -1.34103e-10 0.0255356
4: 2 0.4 0 -0.0157 0.0183 -0.9879 -0.0315 -0.0311 -0.9908 -0.0314 -0.0160 -0.9929 0.0040 0.0010 0.9998 -1.34819e-10 0.0257300
5: 2 0.5 0 -0.0402 0.0300 -0.9832 -0.0093 0.0269 -0.9781 -0.0326 0.0247 -0.9802 0.0044 -0.0010 0.9997 -1.31515e-10 0.0440350
---
124247: 250 49.5 0 -0.0040 0.0141 0.9802 -0.0152 0.0203 -0.9877 -0.0015 0.0123 -0.9901 0.0069 0.0003 0.9997 -1.30220e-10 0.0213215
124248: 250 49.6 0 -0.0006 0.0284 0.9819 0.0021 0.0248 -0.9920 0.0264 0.0408 -0.9919 0.0028 -0.0028 0.9997 -1.30295e-10 0.0284142
124249: 250 49.7 0 0.0378 0.0305 0.9779 -0.0261 0.0232 -0.9897 -0.0236 0.0137 -0.9928 0.0102 -0.0023 0.9997 -1.29890e-10 0.0410760
124250: 250 49.8 0 0.0569 -0.0203 0.9800 -0.0028 -0.0009 -0.9906 -0.0139 -0.0169 -0.9918 0.0039 -0.0017 0.9997 -1.31555e-10 0.0513482
124251: 250 49.9 0 0.0234 -0.0358 0.9840 -0.0340 0.0114 -0.9873 -0.0255 0.0134 -0.9888 0.0006 0.0009 0.9997 -1.30862e-10 0.0334976
>
The -v makes grep return all lines except lines containing the string TRIAL. Given the number of high quality engineers that have looked at the command tool grep over the years, it is most likely that it is as fast as you can get, as well as being correct, convenient, well documented online, easy to learn and search for solutions for specific tasks. If you need to do more complicated string filters (e.g. strings at the beginning or the end of the lines, etc) then grep syntax is very powerful. Learning its syntax is a transferable skill to other languages and environments.
For further examples on the use of command line tools in fread, you may check the article Convenience features of fread. Please note that "On Windows we recommend Cygwin (run one .exe to install) which includes the command line tools such as grep".
In order to read a file and remove row based on a string criteria, you could use readLines function, and filter the result.
I use stringr package for string manipulation.
library(stringr)
# Read your file by lines
DT <- readLines("sum_data")
length(DT)
#> [1] 124501
# detect which lines contains trial
trial_lines <- str_detect(DT, "TRI")
head(trial_lines)
#> [1] TRUE TRUE FALSE FALSE FALSE FALSE
# Remove those lines
DT <- DT[!trial_lines]
length(DT)
#> [1] 124251
# Rewrite your file by line
writeLines(DT, "new_file")
If you have performance issues, you could try read_lines from package readr instead of base readLines