I want to build a Jenkins cron job, which fetches data from firebase into json format and then convert that data into csv or xlsx file and put it into one drive.
I have achieved fetching data from firebase, but stuck on converting it to csv and then save to one drive.
There are multiple ways to convert a json file to csv but considering the scenario that you are working with jenkins and might have a shell as build step, so in that case you can use jq utility. there are multiple answers which can help you.
Use jq to Convert json File to csv
How to convert json into csv file using jq?
if you are open to use any other option for build then you can use python also. which makes it much easier.
you can use pandas module from python to convert a json file to csv using below code.
import pandas as pd
with open('data.json', encoding='utf-8') as stream:
df = pd.read_json(stream)
df.to_csv('data.csv', encoding='utf-8', index=False)
Related
I'm working with h2o (latest version 3.26.0.10) on a Hadoop cluster. I've read in a parquet file from HDFS and have performed some manipulation on it, built a model, etc.
I've stored some important results in an H2OFrame that I wish to export to local storage, instead of HDFS. Is there a way to export this file as a parquet?
I tried using h2o.exportFile, documentation here: http://docs.h2o.ai/h2o/latest-stable/h2o-r/docs/reference/h2o.exportFile.html but all the examples are for writing .csv. I tried using the a file path with .parquet as an extension and that didn't work. It wrote a file but I think it was basically a .csv as it was identical file size to the .csv.
example: h2o.exportFile(iris_hf, path = "/path/on/h2o/server/filesystem/iris.parquet")
On a related note, if I were to export my H2OFrame to HDFS instead of local storage, would it be possible to write that in parquet format? I could at least then move that to local storage.
I am looking for ways to process VSAM files with R and export as a csv.
I have been searching the web and have not been able to find any methods of using R to read VSAM files.
A little more information would be of use. How are you going to get the data from the VSAM files? Are you reading directly from an IBM system? What access method will you be using? What is the structure of the file you are reading since since if you want it to be put in a data.frame, is it something like a CSV file already?. So any other particulars would be helpful.
I used unix script to convert oracle data to csv file and export csv file data to teradata database. It takes long time to load. How to use piping concept for this script in unix
I have realm database (.realm) from an existing application, which has more than 400k records.
I dug my head in to this for more than 3-4 hours, but I couldn't find any fruitful results towards converting a .realm file in to a sqlite file.
Any data conversion so far has been done manually. People have converted from Core Data to Realm by grabbing the objects from Core Data and then saving them to a Realm.
I imagine your best bet will be similar here. Grab each object from the Realm and then convert it to the tables/rows/and columns in sqlite. Then you can take that file anywhere.
You can open the realm file in Realm Browser & there you have the option to convert the file in to CSV ( File => Export => CSV ). Then you can import that CSV file to mysql database.
If you want sqlite more specifically, You can import tables separately to your sqlite db through SQLiteBrowser . Create or open your database file and select the specific table you want , then you can import table from CSV.
I'm not at home with this, but I found this link hopefully it can help.
I want the results to be in a excel sheet.
The results should be made as a report.
I am testing a android device and want the result.
monkeyrunner
If you want to automate the whole process you can try xlwt podule to write Excel files from python.
If monkeyrunner gives you some problem importing python modules give AndroidViewClient/culebra a try.
Additionally, ask yourself whether you need Excel at all when you can generate nice reports from python using Sphinx or pod.