Error Creating Multiclass NN with rxNeuralNet - r

I am developing Neural Networks in my SQLServer2017 with R.
I use the package MicrosoftML and the NYC TaxiData.
Goal: Neural Network to predict the "Ratecode" of a single TaxiRide
Here is the Code:
library(MicrosoftML)
library(dplyr)
dat_all <- InputData;
sizeAll <- length(InputData$tip_amount);
sample_train <- base::sample(nrow(dat_all),
size = (sizeAll*0.9))
sample_test <- base::sample((1:nrow(dat_all))[-sample_train],
size = (sizeAll*0.1))
dat_train <- dat_all %>%
slice(sample_train)
dat_test <- dat_all %>%
slice(sample_test);
form <- Rate ~ total_amount+trip_distance+duration_in_minutes+passenger_count+PULocationID+DOLocationID;
model <- rxNeuralNet(
formula = form,
data = dat_train,
type = "multiClass",
verbose = 1);
trained_model <- data.frame(payload = as.raw(serialize(model, connection=NULL)));
The Rate is successfully detected as a factor with size 5, representing different rates such as "Standard" or "JFK".
When running the Code, I get the following Error:
Error: All rows in data has missing values (N/A).  Please clean
missing data before training. Error in processing machine learning
request. Fehler in doTryCatch(return(expr), name, parentenv, handler)
: Error: All rows in data has missing values (N/A).  Please clean
missing data before training. Error in processing machine learning
request. Ruft auf: source ... tryCatch -> tryCatchList -> tryCatchOne
-> doTryCatch -> .Call
The very same error occurs when replacing the rate with a rateID.
I estimate that there is some form of Transformation to get this working, but somewhat the documentation of MS is lacking at this Point.
Here is the verbose of my NN before it wipes:
***** Net definition *****
input Data [6];
STDOUT message(s) from external script:
hidden H [100] sigmoid { // Depth 1
from Data all;
}
output Result [5] softmax { // Depth 0
from H all;
}
***** End net definition *****
Input count: 6
Output count: 5
Output Function: SoftMax
Loss Function: LogLoss
PreTrainer: NoPreTrainer
___________________________________________________________________
Starting training...
Learning rate: 0,001000
Momentum: 0,000000
InitWtsDiameter: 0,100000
___________________________________________________________________
Initializing 1 Hidden Layers, 1205 Weights...
Elapsed time: 00:00:00.7222942

I figured it out, here is the working Code:
library(MicrosoftML)
library(dplyr)
netDefinition <- ("
input Data auto;
hidden Mystery [100] sigmoid from Data all;
hidden Magic [100] sigmoid from Mystery all;
output Result auto softmax from Magic all;
")
dat_all <- InputData;
LocationLevels <- as.factor(c(1:265));
dat_all$PULocationID <- factor(dat_all$PULocationID, levels=LocationLevels);
dat_all$DOLocationID <- factor(dat_all$DOLocationID, levels=LocationLevels);
dat_all$RatecodeID <- factor(dat_all$RatecodeID, levels=as.factor(c(1:6)) );
form <- RatecodeID ~ trip_distance+total_amount+duration_in_minutes+passenger_count+PULocationID+DOLocationID;
model <- rxNeuralNet(
formula = form,
data = dat_all,
netDefinition=netDefinition,
type = "multiClass",
numIterations = 100,
verbose = 1);
trained_model <- data.frame(payload = as.raw(serialize(model, connection=NULL)));
Main Issue was Factorizing the Data correctly.

Related

External Cluster Validation - Categorical Data R

I've recently been attempting to evaluate output from k-modes (a cluster label), relative to a so-called True cluster label (labelled 'class' below).
In other words: I've been attempting to external validate the clustering output. However, when I tried external validation measures from the 'fpc' package, I was unsuccessful (error term posted below script).
I've attached my code for the mushroom dataset. I would appreciate if anyone could show me how to successful execute these external validation measures in the context of categorical data.
Any help appreciated.
# LIBRARIES
install.packages('klaR')
install.packages('fpc')
library(klaR)
library(fpc)
#MUSHROOM DATA
mushrooms <- read.csv(file = "https://raw.githubusercontent.com/miachen410/Mushrooms/master/mushrooms.csv", header = FALSE)
names(mushrooms) <- c("edibility", "cap-shape", "cap-surface", "cap-color",
"bruises", "odor", "gill-attachment", "gill-spacing",
"gill-size", "gill-color", "stalk-shape", "stalk-root",
"stalk-surface-above-ring", "stalk-surface-below-ring",
"stalk-color-above-ring", "stalk-color-below-ring", "veil-type",
"veil-color", "ring-number", "ring-type", "spore-print-color",
"population", "habitat")
names(mushrooms)[names(mushrooms)=="edibility"] <- "class"
indexes <- apply(mushrooms, 2, function(x) any(is.na(x) | is.infinite(x)))
colnames(mushrooms)[indexes]
table(mushrooms$class)
str(mushrooms)
#REMOVING CLASS VARIABLE
mushroom.df <- subset(mushrooms, select = -c(class))
#KMODES ANALYSIS
result.kmode <- kmodes(mushroom.df, 2, iter.max = 50, weighted = FALSE)
#EXTERNAL VALIDATION ATTEMPT
mushrooms$class <- as.factor(mushrooms$class)
class <- as.numeric(mushrooms$class))
clust_stats <- cluster.stats(d = dist(mushroom.df),
class, result.kmode$cluster)
#ERROR TERM
Error in silhouette.default(clustering, dmatrix = dmat) :
NA/NaN/Inf in foreign function call (arg 1)
In addition: Warning message:
In dist(mushroom.df) : NAs introduced by coercion

Error when using `refund::prf` without model intercept

I'm trying to run a functional data analysis model with scalar-on-function and on scalar regression in R. But by removing the intercept I'm getting the following error (the example is based on this discussion),
library(refund)
data(DTI)
DTI1 <- DTI[DTI$visit==1 & complete.cases(DTI),]
par(mfrow=c(1,2))
fit_af <- pfr(pasat ~ -1 +sex + case + af(cca, k=c(5, 8), bs="ps"), data = DTI1)
#Error in str2lang(x) : <text>:1:9: unexpected symbol
#1: pasat~0 sex
# ^
#Calls: pfr -> formula -> formula.character -> str2lang
How can I remove the intercept from a pfr model?
This is a bug. Running prf() in the debugging mode identifies the problem. When there is no intercept, it does:
if (!attr(tf, "intercept")) {
newfrml <- paste(newfrml, "0", sep = "")
}
which I think should be fixed to
if (!attr(tf, "intercept")) {
newfrml <- paste(newfrml, "0 +", sep = "")
}
Consider reporting this Stack Overflow thread, i.e., https://stackoverflow.com/q/72856108 to package maintainer Julia Wrobel: julia.wrobel#cuanschutz.edu
Note: This was tested using the latest refund_0.1-26 to date (released to CRAN on 2022-04-16).

Error in eval(expr, p): object 'X' not found; predict (BayesARIMAX)

I am trying to use BayesARIMAX to model and predict us gdp (you can find the data here: https://fred.stlouisfed.org/series/GDP).I followed the example (https://cran.r-project.org/web/packages/BayesARIMAX/BayesARIMAX.pdf) to build my model. I didnt have any major issue to build the model(used error handling to overcome Getting chol.default error when using BayesARIMAX in R issue). However could not get the prediction of the model. I tried to look for solution and there is no example of predicting the model that is build using BayesARIMAX. Every time that I run the "predict" I get the following error:
"Error in eval(expr, p) : object 'X' not found"
Here is my code.
library(xts)
library(zoo)
library(tseries)
library(tidyverse)
library(fpp2)
gdp <- read.csv("GDP.csv", head = T)
date.q <- as.Date(gdp[, 1], "%Y-%m-%d")
gdp <- xts(gdp[,2],date.q)
train.row <- 248
number.row <- dim(merge.data)[1]
gdp.train <- gdp[1:train.row]
gdp.test <- gdp[(train.row+1):number.row]
date.test <- date.q[(train.row+1):number.row]
library(BayesARIMAX)
#wrote this function to handle randomly procuded error due to MCMC simulation
test_function <- function(a,b,P=1,Q=1,D=1,error_count = 0)
{
tryCatch(
{
model = BayesARIMAX(Y=a,X = b,p=P,q=Q,d=D)
return(model)
},
error = function(cond)
{
error_count=error_count+1
if (error_count <40)
{
test_function(a,b,P,Q,D,error_count = error_count)
}
else
{
print(paste("Model doesnt converge for ARIMA(",P,D,Q,")"))
print(cond)
}
}
)
}
set.seed(1)
x = rnorm(length(gdp.train),4,1)
bayes_arima_model <- test_function(a = gdp.train,b=x,P = 3,D = 2,Q = 2)
bayes_arima_pred <- xts(predict(bayes_arima_model[[1]],newxreg = x[1:3])$pred,date.test)
and here is the error code
Error in eval(expr, p) : object 'X' not found
Here is how I resolve the issue after reading through the BayesARIMAX code (https://rdrr.io/cran/BayesARIMAX/src/R/BayesianARIMAX.R) . I basically created the variable "X" and passed it to predict function to get the result. You just have to set the length of X variable equal to number of prediction.
here is the solution code for prediction.
X <- c(1:3)
bayes_arima_pred <- xts(predict(bayes_arima_model[[1]],newxreg = X[1:3])$pred,date.test)
which gave me the following results.
bayes_arima_pred
[,1]
2009-01-01 14462.24
2009-04-01 14459.73
2009-07-01 14457.23

R Model returning the error: Too many open devices

I've been working on the creation of a training model in R for MS Azure. When I initially set up the model it all worked fine. Now it's continuously returning the below:
{"error":{"code":"LibraryExecutionError","message":"Module execution encountered an internal library error.","details":[{"code":"FailedToEvaluateRScript","target":"Score Model (RPackage)","message":"The following error occurred during evaluation of R script: R_tryEval: return error: Error in png(file = \"3e25ea05d5bc49d683f4471ff40780bcrViz%03d.png\", bg = \"transparent\") : \n too many open devices\n"}]}}
I haven't changed anything, and have looked around online only to find references to other issues. My code is as follows:
Trainer R Script
# Modify Datatype, factor Level, Replace NA to 0
x <- dataset
for (i in seq_along(x)) {
if (class(x[[i]]) == "character") {
#Convert Type
x[[i]] <- type.convert(x[[i]])
#Apply Levels
# levels(x[[i]]) <- levels(cols_modeled[, names(x)[i]]) # linked with levels in model
}
if (is.numeric(x[[i]]) && is.na(x[[i]]) ){
#print("*** Updating NA to 0")
x[[i]] <- 0
}
}
df1 <- x
rm(x)
set.seed(1234)
model <- svm(Paid ~ ., data= df1, type= "C")
Scorer R Script
library(e1071)
scores <- data.frame( predicted_result = predict(model, dataset))
Has anyone come across this before?

replacement has length zero in list() in r

I'm trying to run this code, and I'm using mhadaptive package, but the problem is that when I run these code without writing metropolis_hastings (that is one part of mhadaptive package) error does not occur, but when I add mhadaptive package the error occur. What should I do?
li_F1<-function(pars,data) #defining first function
{
a01<-pars[1] #defining parameters
a11<-pars[2]
epsilon<<-pars[3]
b11<-pars[4]
a02<-pars[5]
a12<-pars[6]
b12<-pars[7]
h<-pars[8]
h[[i]]<-list() #I want my output is be listed in the h
h[[1]]<-0.32082184 #My first value of h is known and other values should calculate by formula
for(i in 2:nrow(F_2_))
{
h[[i]]<- ((a01+a11*(h[[i-1]])*(epsilon^2)*(h[[i-1]])*b11)+(F1[,2])*((a02+a12*(h[[i-1]])*(epsilon^2)+(h[[i-1]])*b12)))
pred<- h[[i]]
}
log_likelihood<-sum(dnorm(prod(h[i]),pred,sd = 1 ,log = TRUE))
return(h[i])
prior<- prior_reg(pars)
return(log_likelihood + prior)
options(digits = 22)
}
prior_reg<-function(pars) #defining another function
{
epsilon<<-pars[3] #error
prior_epsilon<-pt(0.95,5,lower.tail = TRUE,log.p = FALSE)
return(prior_epsilon)
}
F1<-as.matrix(F_2_) #defining my importing data and simulatunig data with them
x<-F1[,1]
y<-F1[,2]
d<-cbind(x,y)
#using mhadaptive package
mcmc_r<-Metro_Hastings(li_func = li_F1,pars=c(10,15,10,10,10,15),par_names=c('a01','a02','a11','a12','b11','b12'),data=d)
By running this code this error occur.
Error in h[[i]] <- list() : replacement has length zero
I'll so much appreciate who help me.

Resources