I'm using the following R code to save into database. However, the date field is not properly saved.
storedata$update = as.character(excelSerialNumToDate(tape$date),"%Y-%m-%d")
sqlSave(channel = dbhandle,dat = storedata,tablename = 'dbo.storeinfo',append = T,rownames = F,colnames = F,verbose = T,safer = F,fast = F, nastring = NULL)
What is the best way to save into a date field using SqlSave in R?
Wrote my own SqlSave to eliminate these issues.
Related
My goal is to create a decision tree model in Rattle for a school project. I've been able to determine the variables that I would need for my research question and created a new dataset from the original .csv file. After saving the new dataset as not only an .xls file and a .rdata file, I received an error message after loading the file into Rattle. This is my first time creating a decision tree model so I'm struggling a bit. Thanks in advance for your help!
Here's what I have so far:
install.packages(readxl)
library(readxl)
library(rattle)
setwd("C:/Users/river/OneDrive/Documents/Random Data")
edu <- read_excel('pfi_pu.xlsx')
eduu <- data.frame(c("P1HRSWK" = c(edu$P1HRSWK),
"P1EMPL" = c(edu$P1EMPL),
"P2HRSWK" = c(edu$P2HRSWK),
"P2EMPL" = c(edu$P2EMPL),
"P1ENRL" = c(edu$P1ENRL),
"P2ENRL" = c(edu$P2ENRL),
"P1EDUC" = c(edu$P1EDUC),
"P2EDUC" = c(edu$P2EDUC),
"P1HISPRM" = c(edu$P1HISPRM),
"P2HISPRM" = c(edu$P2HISPRM),
"P1PACI" = c(edu$P1PACI),
"P2PACI" = c(edu$P2PACI),
"P1BLACK" = c(edu$P1BLACK),
"P2BLACK" = c(edu$P2BLACK),
"P1ASIAN" = c(edu$P1ASIAN),
"P2ASIAN" = c(edu$P2ASIAN),
"P1AMIND" = c(edu$P1AMIND),
"P2AMIND" = c(edu$P2AMIND),
"P1HISPAN" = c(edu$P1HISPAN),
"P2HISPAN" = c(edu$P2HISPAN),
"P1LKWRK" = c(edu$P1LKWRK),
"P2LKWRK" = c(edu$P2LKWRK),
"P1MTHSWRK" = c(edu$P1MTHSWRK),
"P1REL" = c(edu$P1REL),
"P2REL" = c(edu$P2REL),
"P1SEX" = c(edu$P1SEX),
"P2SEX" = c(edu$P2SEX),
"P1MRSTA" = c(edu$P1MRSTA),
"SEFUTUREX" = c(edu$SEFUTUREX),
"HSFUTUREX" = c(edu$HSFUTUREX),
"PARGRADEX" = c(edu$PARGRADEX),
"TTLHHINC" = c(edu$TTLHHINC),
"PAR1EMPL" = c(edu$PAR1EMPL),
"PAR2EMPL" = c(edu$PAR2EMPL),
"SEEXPEL" = c(edu$SEEXPEL),
"SESUSPIN" = c(edu$SESUSPIN),
"SESUSOUT" = c(edu$SESUSOUT),
"SEGRADEQ" = c(edu$SEGRADEQ)
,dim = c(14075,38,1))
save(eduu,file="eduu.xls")
error message
Seems your problem is about writing a file. The command save must be used to save .RData files, not Excel files. According to this post, you may try:
openxlsx::write.xlsx(eduu, 'eduu.xlsx')
xlsx::write.xlsx(eduu, 'eduu.xlsx')
writexl::write_xlsx(eduu, 'eduu.xlsx')
I need to update a wide table on an SQL SERVER from R. So the package DBI seems to be very useful for that.
The problem is that the R data.frame contains strings of more than 3000 characters and when I use the DBI dbSendQuery function, all strings are truncated to 256 characters.
Here could be a code example :
con <- odbc::dbConnect(drv = odbc::odbc(),
dsn = '***',
UID = '***',
PWD = '***')
df = data.frame(TEST = paste(rep("A", 300), collapse=""),
TEST_ID = 1068858)
df$TEST = df$TEST %>% as.character
query = paste0('UPDATE MY_TABLE SET "TEST"=? WHERE TEST_ID=?')
update <- DBI::dbSendQuery(con, query)
DBI::dbBind(update, df)
DBI::dbClearResult(update)
odbc::dbDisconnect(con)
Then the following request return 256 instead of 300 :
SELECT LEN(TEST) FROM MY_TABLE WHERE TEST_ID = 1068858
NB : TEST is of type (varchar(max), NULL) and already contains strings of more than 256 chars.
Thanks in advance for any advice
In the end, I choose to get rid of sophisticated functions. A solution was to write the table in .csv file and bulk insert it into the database. Here is an example using RODBC package :
write.table(x = df,
file = "/path/DBI_error_test.csv",
sep = ";",
row.names = FALSE, col.names = FALSE,
na = "NULL",
quote = FALSE)
Query = paste("CREATE TABLE #MY_TABLE_TMP (
TEST varchar(max),
TEST_ID int
);
BULK INSERT #MY_TABLE_TMP
FROM 'C:\\DBI_error_test.csv'
WITH
(
FIELDTERMINATOR = ';',
ROWTERMINATOR = '\n',
BATCHSIZE = 500000,
CHECK_CONSTRAINTS
)
UPDATE R
SET R.TEST = #MY_TABLE_TMP.TEST
FROM MY_TABLE AS R
INNER JOIN #MY_TABLE_TMP ON #MY_TABLE_TMP.TEST_ID = R.TEST_ID;
DROP TABLE #MY_TABLE_TMP;
")
channel <- RODBC::odbcConnect(dsn = .DB_DSN_NAME,
uid = .DB_UID,
pwd = .DB_PWD)
RODBC::sqlQuery(channel = channel, query = query, believeNRows = FALSE)
RODBC::odbcClose(channel = channel)
I'm currently working on an ArangoDB POC. I find that the time taken for document creation is very high in ArangoDB with PyArango. It takes about 5 minutes to insert 300 documents. I've pasted the rough code below, please let me know if there are better ways to speed this up :
with open('abc.csv') as fp:
for line in fp:
dataList = line.split(",")
aaa = dbObj['aaa'].createDocument()
bbb = dbObj['bbb'].createDocument()
ccc = dbObj['ccc'].createEdge()
bbb['bbb'] = dataList[1]
aaa['aaa'] = dataList[0]
aaa._key = dataList[0]
aaa.save()
bbb.save()
ccc.links(aaa,bbb)
ccc['related_to'] = "gfdgf"
ccc['weight'] = 0
ccc.save()
The different collections are created by the below code :
dbObj.createCollection(className='aaa', waitForSync=False)
for your problem with the batch mode in the arango java driver. if you know the key attributes of the vertices you can build the document handle by "collectionName" + "/" + "documentKey".
Example:
arangoDriver.startBatchMode();
for(String line : lines)
{
String[] data = line.split(",");
BaseDocument device = new BaseDocument();
BaseDocument phyAddress = new BaseDocument();
BaseDocument conn = new BaseDocument();
String keyDevice = data[0];
String handleDevice = "DeviceId/" + keyDevice;
device.setDocumentKey(keyDevice);
device.addAttribute("device_id",data[0]);
String keyPhyAddress = data[1];
String handlePhyAddress = "PhysicalLocation/" + keyPhyAddress;
phyAddress.setDocumentKey(keyPhyAddress);
phyAddress.addAttribute("address",data[1]);
final DocumentEntity<BaseDocument> from = arangoDriver.graphCreateVertex("testGraph", "DeviceId", device, null);
final DocumentEntity<BaseDocument> to = arangoDriver.graphCreateVertex("testGraph", "PhysicalLocation", phyAddress, null);
arangoDriver.graphCreateEdge("testGraph", "DeviceId_PhysicalLocation", null, handleDevice, handlePhyAddress, null, null);
}
arangoDriver.executeBatch();
I would build all of the data to be inserted into a json formatted string and use createDocumentRaw to create them all at once with one save.
I have written a code in ASP.NET to fetch data from Oracle stored procedure. But I get an error when trying to fetch the data:
ORA-06550: line 1, column 7: PLS-00306: wrong number or types of arguments in call to 'USER_FEEDBACK' ORA-06550: line 1, column 7: PL/SQL: Statement ignored
Procedure code is this:
create or replace PROCEDURE user_feedback(cv_results out sys_refcursor,start_date IN VARCHAR2)
IS
BEGIN
open cv_results for
select pi.first_name || ' ' || pi.last_name initiator
......
from request_workflow w inner join request_workflow_attribute waRating
on waRating.request_workflow_id = w.row_id
and waRating.attr_name = 'UserRating'
.............
where w.date_stamp_utc between start_date and '31-dec-2015'
order by waRating.attr_value desc, eform_name;
END ;
This is my ASP.NET code:
Oracle.DataAccess.Client.OracleCommand objCmd = new Oracle.DataAccess.Client.OracleCommand();
objCmd.Connection = objConn;
objCmd.CommandText = "user_feedback";
objCmd.CommandType = CommandType.StoredProcedure;
Oracle.DataAccess.Client.OracleParameter oraP = new Oracle.DataAccess.Client.OracleParameter();
// System.Data.OracleClient.OracleParameter op = null;
oraP.OracleDbType = OracleDbType.RefCursor;
oraP.ParameterName = "cv_results";
oraP.Direction = System.Data.ParameterDirection.Output;
oraP.ParameterName = "start_date";
oraP.OracleDbType = OracleDbType.Varchar2;
oraP.Value = "01-Dec-2015";
oraP.Direction = System.Data.ParameterDirection.Input;
objCmd.Parameters.Add(oraP);
objConn.Open();
objCmd.ExecuteReader();
objCmd.ExecuteNonQuery();
Please suggest how to make it work.
You're only creating/adding a single parameter but assigning it twice with different values resulting in just one parameter to the call.
Instead, create and add two separate parameters, something like;
var oraP1 = new Oracle.DataAccess.Client.OracleParameter();
oraP1.OracleDbType = OracleDbType.RefCursor;
oraP1.ParameterName = "cv_results";
oraP1.Direction = System.Data.ParameterDirection.Output;
objCmd.Parameters.Add(oraP1);
var oraP2 = new Oracle.DataAccess.Client.OracleParameter();
oraP2.ParameterName = "start_date";
oraP2.OracleDbType = OracleDbType.Varchar2;
oraP2.Value = "01-Dec-2015";
oraP2.Direction = System.Data.ParameterDirection.Input;
objCmd.Parameters.Add(oraP2);
It's probably because you don't Add the first parameter. Try something like this?
...
oraP.ParameterName = "cv_results";
oraP.Direction = System.Data.ParameterDirection.Output;
objCmd.Parameters.Add(oraP);
oraP = new Oracle.DataAccess.Client.OracleParameter();
oraP.ParameterName = "start_date";
...
Though, it'd be better to declare another variable for the first param if you'll be reading from that refcursor later.
I'm currently trying to open an MsRdpClient connection in a web browser by opening a new tab and passing it dynamic server names. However, the following code doesn't seem to work and I can't dynimically populate any of the required values, namely servername and available screen width and height.
resWidth = request.querystring("width")
resHeight = request.querystring("height")
MsRdpClient.DesktopWidth = resWidth
MsRdpClient.DesktopHeight = resHeight
MsRdpClient.Width = resWidth
MsRdpClient.Height = resHeight
MsRdpClient.server = request.querystring("fqdn")
MsRdpClient.username = "username"
MsRdpClient.AdvancedSettings.ClearTextPassword = "password"
MsRdpClient.AdvancedSettings2.RDPPort = "3389"
MsRdpClient.Connect
I'm not really sure where to go from here. I see it's been asked on a few boards but no one has seem to come up with an answer. Any help would be greatly appreciated.
Below is the script I ended up using. putting the required variables in via scriptlets is what did the trick. i.e. the "<%=fqdn%>"
resWidth = (screen.AvailWidth - 45)
resHeight = (screen.AvailHeight - 150)
MsRdpClient.DesktopWidth = resWidth
MsRdpClient.DesktopHeight = resHeight
MsRdpClient.Width = resWidth
MsRdpClient.Height = resHeight
MsRdpClient.server = "<%=fqdn%>"
MsRdpClient.AdvancedSettings2.RDPPort = "3389"
MsRdpClient.Connect
sub MsRdpClient_OnDisconnected(disconnectCode)
history.go(-1)
end sub