I read eachline of a csv file and save the first element of each line in a list, then I want to run FFT on this list, but I get this error:
Error in fft(x) : non-numeric argument
in my Example hier I read 4 rows:
con<-file("C:\\bla\\test.csv","r")
datalist<-list()
m<-list()
for(i in 1:4)
{
line<-readLines(con,n=1,warn=FALSE)
m<-list(as.integer(unlist(strsplit(line,split=","))))
datalist<-c(datalist,sapply(m,"[[",1))
}
datalist
close(con)
fftfun<- function(x) {fft(x)}
fft_amplitude <- function(x) {sqrt((Re(fft(x)))^2+(Im(fft(x)))^2)} }
apply(as.matrix(datalist),2,FUN=fftfun)
what should I do to solve this problem?
EDIT
My rows in csv file:
12,85,365,145,23
13,84,364,144,21
14,86,366,143,24
15,83,363,146,22
16,85,365,145,23
17,80,361,142,21
Your code seems overly complicated. Why don't you just do something like this :
df <- read.csv("test.csv", header=FALSE)
x <- df[,1]
fft(x)
Or, if you really want to read line by line :
con <- file("test.csv","r")
data <- NULL
for (i in 1:4) {
line<-readLines(con,n=1,warn=FALSE)
data <- c(data, as.numeric(strsplit(line,split=",")[[1]][1]))
}
close(con)
fft(data)
Let's assume your real question is: what happened to make apparently numeric data become non-numeric? Rather than slogging through the incredible number of type coercions in your code (csv to matrix to list to other list to as.matrix), I'm going to recommend you start by just plain reading one file into R and checking the typeof and class of each column. If anything turns out to be a factor rather than numeric , you may need to add the argument colClasses='character' .
If the data as read are numeric, then you're fouling it up in your subsequent conversions. Try simplifying the code as much as possible.
Related
I am trying to importa single column of a text file data set where each file is a single day of data. I want to take the mean of each day's wind speed. Here is the code I have written for that:
daily.wind.speed <- c()
file.names <- dir("C:\\Users\\User Name\\Desktop\\R project\\Sightings Data\\Weather Data", pattern =".txt")
for(i in 1:length(file.names))
{
##import data file into data frame
weather.data <-read.delim(file.names[i])
## extract wind speed column
wind.speed <- weather.data[3]
##Attempt to fix numeric error
##wind.speed.num <- as.numeric(wind.speed)
##Take the column mean of wind speed
daily.avg <- colMeans(wind.speed,na.rm=T)
##Add daily average to list
daily.wind.speed <- c(daily.wind.speed,daily.avg)
##Print for troubleshooting and progress
print(daily.wind.speed)
}
This code seems to work on some files in my data set, but others give me this error during this section of the code:
> daily.avg <- colMeans(wind.speed,na.rm=T)
Error in colMeans(wind.speed, na.rm = T) : 'x' must be numeric
I am also having trouble converting these values to numeric and am looking for options to either convert my data to numeric, or to possibly take the mean in a different way that dosen't encounter this issue.
> as.numeric(wind.speed.df)
Error: (list) object cannot be coerced to type 'double'
weather.data Example
Even though this is not a reproducible example the problem is that you are applying a matrix function to a vector so it won't work. Just change the colMeans for mean
I am trying to do Hartigan's diptest in R, however, I get the following error: 'x' must be numeric.
Apologies for such a basic question, but how do I ensure that the data that I load is numeric?
If I make up a set of values as follows (code below), the diptest works without problems:
library(diptest)
x = c(1,2,3,4,4,4,4,4,4,4,5,6,7,8,9,9,9,9,9,9,9,9,9)
hist(x)
dip.test(x)
But for example, when the same values are saved in an Excel file/tab delimited .txt file (saved as one column of values), and imported into R, when I run the diptest the 'x' must be numeric error occurs.
x <- read.csv("x.csv") # comes back in as a data frame
hist(x)
dip.test(x)
Is there a way to check what format the imported data from an Excel/.txt file is in R, and subsequently change it to be numeric? Thanks again.
Any help will be much appreciated thank you.
Here's what's happening. If you run the code that you know works, it's working because the data class is numeric as it should be. When you read it back in it's a data.frame, however. So you need to point to the numeric element of the data.frame:
library(diptest)
x = c(1,2,3,4,4,4,4,4,4,4,5,6,7,8,9,9,9,9,9,9,9,9,9)
write.csv(x, "x.csv", row.names=F)
x <- read.csv("x.csv") # comes back in as a data frame
hist(x$x)
dip.test(x$x)
Hartigans' dip test for unimodality / multimodality
data: x$x
D = 0.15217, p-value = 2.216e-05
alternative hypothesis: non-unimodal, i.e., at least bimodal
If you were to save the file to a .RDS instead of .csv then you could avoid this problem.
You could also check if your data frame contains any non-numeric characters as follows:
which(!grepl('^[0-9]',your_data_frame[[1]]))
I have a file called Schedule.csv, which is structured as follows:
URLs
http://www.basketball-reference.com/friv/dailyleaders.cgi?month=10&day=27&year=2015
http://www.basketball-reference.com/friv/dailyleaders.cgi?month=10&day=28&year=2015
I am trying to use the explanation provided in the following question to scrape the html tables but it isn't working: How to scrape HTML tables from a list of links
My current code is as follows:
library(XML)
schedule<-read.csv("Schedule.csv")
stats <- list()
for(i in seq_along(schedule))
{
print(i)
total <- readHTMLTable(schedule[i])
n.rows <- unlist(lapply(total, function(t) dim(t)[1]))
stats[[i]] <- as.data.frame(total[[which.max(n.rows)]])
}
I get an error when I run this code as follows:
Error in (function (classes, fdef, mtable) : unable to find an inherited method for function ‘readHTMLTable’ for signature ‘"data.frame"’
If I manually type the URL's in a vector as per below I get exactly what I want when I run the readHTMLTable code.
schedule<-c("http://www.basketball-reference.com/friv/dailyleaders.cgi?month=10&day=27&year=2015","http://www.basketball-reference.com/friv/dailyleaders.cgi?month=10&day=28&year=2015")
Can someone please explain to me why the read.csv is not giving me a usable vector of information to input into the readHTMLTable function?
read.csv creates a data.frame in your shcedule. Then you want to access it by rows (seq_along and schedule[i] work along the columns of the data frame)
In your case you can do:
for (i in 1:nrow (schedule)) {
total <- readHTMLTable(schedule[i, 1])
as I understand you want the first column of your data.frame, change the , 1] or use column names otherwise.
Also notice that read.csv will read your first column as a factor so you may prefer to read it as a character:
schedule<-read.csv("Schedule.csv", as.is = TRUE)
An other alternative if your file has a unique column is to use readLines an then you can keep your loop as it was...
schedule<-readLines("Schedule.csv")
stats <- list()
for(i in seq_along(schedule))
{
print(i)
total <- readHTMLTable(schedule[i])
...
but be careful with the column names because they will be in the first element of your schedule vector
I have used the search box for this and have found similar questions, but not identical ones. It seems that this is an easy problem though (I'm an R-newbee).
I am simply trying to create a new data frame and adding values to it. Not surprisingly, R throws an error saying that the rows don't match.
Here's the code
d <- data.frame()
files <- list.files(pattern="*.lst", full.names=T, recursive=FALSE)
d$fileName <- lapply(files, basename)
d$node <- gsub("([^.]+)\.[^\.lst]+\.lst", "$1", d$fileName, perl=TRUE)
And here's the error
Error in $<-.data.frame(*tmp*, "fileName", value =
list("A-bom.WR-P-E-A.lst", : replacement has 337 rows, data has 0
How would I go about this problem? I thought about filling d with the same amount of rows as there are files, but I don't think that that's the best way?
Simply create your data frame when it's used the fist time, so you don't "add" rows to a data frame with zero rows. And you may use sapply to return a (named) vector instead of a list.
files <- list.files(pattern="*.lst", full.names=T, recursive=FALSE)
d <- data.frame(fileName = unname(sapply(files, basename)))
d$node <- gsub("([^.]+)\\.[^\\.lst]+\\.lst", "$1", d$fileName, perl=TRUE)
Your regular expression caused an error, however, I'm not that familiar with regex, so you probably have to fix my fixes ;-)
I'm building a table from a CSV file. When the file is initially loaded I need to load as characters.
datset <- read.csv("outcome-of-care-measures.csv", colClasses = "character")
I have function to convert a factor containing number (from other stack q)
as.numeric.factor <- function(x) {as.numeric(levels(x))[x]}
I clean up the file with
i<-17
datset[datset=="Not Available"]<-NA
datset<-datset[complete.cases(datset[,i]),]
x<- as.numeric.factor(datset[, i])
The datset table contains lots of columns I don't need so I build a new table :
dat <- data.frame(cbind("HospitalName"= datset[,2], "State"= datset[,7],"Rating" = x))
My problem is that even though x is numeric, it gets turned into a factor when loaded to the dataframe. I can verify this from debug mode with :
class(x)
"Numeric"
class(dat[,3])
"Factor"
In later code I'm trying to sort the Rating column but it's failing due it being a factor - I guess.
I've even tried appending stringsAsFactors = FALSE to read.csv but this has no effect.
How can I prevent x from being converted into a factor when loading to a DF?
As Henrik explained in his comment, this:
dat <- data.frame(cbind("HospitalName"= datset[,2], "State"= datset[,7],"Rating" = x))
is a poor way to construct a data frame. cbind converts everything to a matrix, which can only hold a single data type. Hence the coercion.
It would be better to do:
dat <- data.frame(HospitalName = dataset[,2],state = dataset[,7],rating = x)
However, it is also true as Roland mentioned that you should be able to specify this one column to be numeric when reading the data in via:
colclasses <- rep("character", 40)
colclasses[7] <- "numeric"
and then passing that in read.csv.