Snowflake|Airflow: How to unload data from Snowflake into xls format - airflow

Is there any way to achieve unloading data from Snowflake to XLS?
We are using Airflow to load data from Snowflake to XLS file or Converting from CSV to XLS

If you are leveraging Airflow, then you could potentially use Snowflake's Python Connector to load data from Snowflake to a Panda dataframe and then use Panda's to_excel() function to write that data out to Excel.
https://docs.snowflake.com/en/user-guide/python-connector.html
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_excel.html

Related

How to load a CSV (or TSV) into BigQuery directly with R---without creating a dataframe

I'm working with CSV files in an R notebook. To upload the data I'm using a bash chunk and bq command. I'd like to be able to do it in an R chunk.
I use bigrquery and dbplyr for my BigQuery work in R. But bigrquery and DBI don't seem to have methods to allow a disk file to pass through to BigQuery without first creating a dataframe.
Is there a way to do this other than a bash chunk or system command?

Named pipe concept in unix

I used unix script to convert oracle data to csv file and export csv file data to teradata database. It takes long time to load. How to use piping concept for this script in unix

How to store r results in tableau extract? We want to share the tableau workbook with users who do not have R or Rserver installed

There are users who do not have R installed on their machine. We want to share the tableau workbook with them. Is it possible to include the results from R script execution in the extract and use that data instead of live execution every time you open the workbook?
Tableau's R integration requires LIVE connection to R Server, therefore it cannot store any data that you calculate using R.
SOLUTION: Run R code and store the result to Rdata/CSV file that you can then directly connect to in Tableau.
Let's say you have a Tableau workbook that uses as a connection excel files (one or many). If the excel files are generated with R, or with another automatic process, I embed those files with the Tableau workbook with extension .twbx, so I can share the report.
In order to embed the excel file/s with the Tableau Workbook I do the following:
Make sure you have in the same folder a Tableau workbook and the connection files. The Tableau workbook has extension .twb, and the connection files are, for instance, 'data_a.xlsx' and 'data_b.xlsx'.
Then I just zip the Tableau workbook with the data files with extension .twbx:
workbook_name <- 'charges'
workbook_data <- c('data_a.xlsx', 'data_b.xlsx')
zip(paste0(workbook_name, '.twbx'),
files=c(workbook_data,
paste0(workbook_name, '.twb'))
)
Then you have a Tableau workbook with the data updated using R.

How to access data stored in hbase from spark in R

I need to get data stored in hbase to analyse in R but I need to do it through Spark because the data does not fit in memory.
Does anybody know how to access data in hbase through Spark in R?
I've searched both the web and SO but no joy. I've found pages that explain how to access data in hbase from R but they don't do it through Spark. And all the pages I've seen explaining how to access data in R and Spark (with sparklyr) provide examples with the iris dataset :(
Any help is much appreciated!
One option seems to be to install rhbase and get the data from hbase and save it to csv, first, then use sparkr to read the data from the csv file and proceed to analyse etc. blogs.wandisco.com/2014/08/19/experiences-r-big-data/
Is there a better way? One that does not require saving the data to a csv file?

Can we write BSON files in R without MongoDB

I am trying to explore alternate serialization/un-serialization in R.
Is it possible to read /write data in BSON file format in R without actually creating or using a MongoDB database?
I browsed the rmongodb package description, and it appeared that the package might require one to use a MongoDB database for reading/writing BSON files.

Resources