Import data in to R from MongoDB in JSON Format - r

I want to import data in JSON format from MongoDB in to R. I am using Mongolite package to connect MongoDB to R, but when i use mongo$find('{}') data is getting stored as dataframe. Please check my Rcode,
library(mongolite)
mongo <- mongolite::mongo(collection = "Attributes", db = "Test", url =
"mongodb://IP:PORT",verbose = TRUE)
df1 <- mongo$find('{}')
df1 is getting stored as dataframe, but I want the data in JSON format only. Please give your suggestions for the same.
Edit -
Actual json structure converted in to list
But when i load data from MongoDB to R using mongolite package, data is getting stored as dataframe and then if i convert to list, the structure is getting changed and few extra columns are inserted in to list.
Please let me know on how to solve this issue.
Thanks
SJB

Related

How to import a dynamodb table attribute into a dataframe in R and plot the dataframe using ggplot

I am trying to import dynamodb in R using paws.database library. I am successful in retrieving the required attribute into R(using scan operation). However the imported data is in the form of a nested list i.e. in [[]] form. My intention is to format the imported dynamodb attribute into a dataframe and later be able to plot it using ggpplot. I have tried using options such as
df <- ldply (list_a, data.frame), ldply (list_a, data.frame),
data.frame(matrix(unlist(list_a), nrow=length(list_a),
byrow=TRUE),stringsAsFactors=FALSE),
as.data.frame(do.call(cbind,list_a))
so far and was unable to convert the data in a proper dataframe format. The final error I get in ggplot is "
Error: data must be a data frame, or other object coercible by fortify(), not a list "
Could anyone please help ?
See this similar issue.
I'm also using paws. Here's what I did to work with a small DynamoDB table:
dyna <- paws::dynamodb()
Table<-dyna$scan("table_name")
newtable<-rbindlist(Table$Items,fill = TRUE)
Then I create a new dataframe by using unlist() on each column of newtable.

How to save an empty data to adlsgen2 using R?

I've explored but didn't find any suggestions.
I want to know how to save an empty file to adlsgen2 using R and then read it back to the same code.
Thanks for the help.
Since you want to write an empty file to ADLS gen2 using R and read it from that location, first create a dataframe.
The dataframe can either be completely empty, or it can be a dataframe with column names but no rows. You can use the following code to create one.
df <- data.frame() #OR
df <- data.frame(<col_name>=<type>)
Once you have created a dataframe, you must establish connection to ADLS gen2 account. You can use the method specified in the following link to do that.
https://saketbi.wordpress.com/2019/05/11/how-to-connect-to-adls-gen2-using-sparkr-from-databricks-rstudio-while-integrating-securely-with-azure-key-vault/
After making the connection you can use read and write functions in R language using the path to ADLS gen2 storage. The following link provides numerous functions that can be used according to your requirement.
https://www.geeksforgeeks.org/reading-files-in-r-programming/
Sharing the solution to my question:
I've used SparkR to create an empty data frame and save the same in the adls after setting up the spark context.
Solution is below:
Step 1: Create a schema
fcstSchema <- structType(structField("ABC", "string",TRUE))
new_df<- data.frame(ABC=NULL,stringsAsFactors=FALSE)
n<-createDataFrame(new_df,fcstSchema)
SparkR::saveDF(n,path = "abfss://<account_name>#<datalake>.dfs.core.windows.net/<path>/",source = "delta",mode = "overwrite")

Create database in Azure Databricks using R dataframe

I have my final output in R dataframe. I need to write this output to a database in Azure Databricks. Can someone help me with the syntax? I used this code:
require(SparkR)
data1 <- createDataFrame(output)
write.df(data1, path="dbfs:/datainput/sample_dataset.parquet",
source="parquet", mode="overwrite")
This code runs without error, but i dont see the database in the datainput folder (mentioned in the path). Is there some other way to do it?
I believe you are looking for saveAsTable function. write.df is particularly to save the data in a file system only, not to tag the data as table.
require(SparkR)
data1 <- createDataFrame(output)
saveAsTable(data1, tableName = "default.sample_table", source="parquet", mode="overwrite")
In the above code, default is some existing database name, under which a new table will get created having name as sample_table. If you mention sample_table instead of default.sample_table then it will be saved in the default database.

Oracle bulk copy importing wrong data

I am trying to import data from an excel sheet to an Oracle table. I am able to extract the correct data, but when I run the following code:
$bulkCopy = new-object ("Oracle.DataAccess.Client.OracleBulkCopy") $oraClientConnString
$bulkCopy.DestinationTableName = $entity
$bulkCopy.BatchSize = 5000
$bulkCopy.BulkCopyTimeout = 10000
$bulkCopy.WriteToServer($dt)
$bulkcopy.close()
$bulkcopy.Dispose()
The data inserted in the table is some garbage values, consisting of 0's and 10's.
Value received from excel is stored in a data table ($dt).
Any help will be highly appreciated.
Please check the data type of the values in your data table. I have experienced this issue in .Net with the data type Double. When I change my data type to Decimal, everything was fine.

Writing financial data to database using Rblpapi and RMySQL

I used Rblpapi to retrieve historic data, format is a list of data.frames. I now want to write that list of data into a MySQL database, I tried
dbWriteTable(con,name="db_all",data,header=T), which gives me
"unable to find an inherited method for function ‘dbWriteTable’ for
signature ‘"MySQLConnection", "character", "list"’"
I have the feeling the problem is the list of data.frames, is there any easy way to insert this data from R into MySQL?
Did you try:
df <- do.call("rbind", data)
dbWriteTable(con, name="db_all", df, header=T)

Resources