I tried doing it that way but that only gives me the path of those folders but not the images in side.
### A Pluto.jl notebook ###
# v0.14.3
using Markdown
using InteractiveUtils
using Pkg
Pkg.activate("Project.toml")
Pkg.add("Flux")
#load data
trainData = "../src/dataset/chest_xray/train/"
testData = "../src/dataset/chest_xray/test/"
validationData = "../src/dataset/chest_xray/val/"
trainNormal = "../src/dataset/chest_xray/train/NORMAL/"
TrainPneumonia = "../src/dataset/chest_xray/train/PNEUMONIA/"
length(trainNormal)
#logankilpatrick what I am simply asking is how to import images kaggle.com/paultimothymooney/chest-xray-pneumonia in julia using the flux package. and the sample code that I posted its me just trying to do so but not working
Related
I have written some scripts in R which I have to run in azure ml notebook but I have not found much documentation how to create a dataset by running code in notebook with R kernel. I have written the following python code which works with python kernel as:
from azureml.core import Dataset, Datastore,Workspace
subscription_id = 'abc'
resource_group = 'pqr'
workspace_name = 'xyz'
workspace = Workspace(subscription_id, resource_group, workspace_name)
datastore = Datastore.get(workspace, 'workspaceblobstore')
# create tabular dataset from all parquet files in the directory
tabular_dataset_3 = Dataset.Tabular.from_parquet_files(path=(datastore,'/UI/09-17-2022_125003_UTC/userdata1.parquet'))
df=tabular_dataset_3.to_pandas_dataframe()
It works fine with python kernel but I want to execute the equivalent R code in notebook with R kernel.
Can anyone please help me what is the equivalent R code ? Any help would be appreciated.
To create an R script and use the dataset, first we need to register the dataset to the portal. Once the dataset is added to the portal, we need to get the dataset URL and open the notebook and use the R kernel.
Upload the dataset and get the data source URL
Go to Machine Learning studio and create a new notebook.
Use the below R script to get the dataset and convert that to dataframe.
azureml_main <- function(dataframe1, dataframe2){
print("R script run.")
run = get_current_run()
ws = workspacename
dataset = azureml$core$dataset$Dataset$get_by_name(ws, “./path/insurance.csv")
dataframe2 <- dataset$to_pandas_dataframe()
# Return datasets as a Named List
return(list(dataset1=dataframe1, dataset2=dataframe2))
}
(optional read) Greater Objective: PowerBI Web doesn't support a few R packages when published on the internet. It throws the below error ("Missing R Package"). Hence, I am working towards saving the output from R as an image (.jpeg) to a remote location (such as FTP) or cloud storage (secure and open source) and then import it to PowerBI. This workaround might resolve the package conflict (hoping).
Specific Objective*: The below code illustrates a trivial way of saving an R output(.jpeg) image locally. However, is there a way to save the image directly to the FTP server, provided I have the username/password etc? (unfortunately, I cannot share the server details)
library(outbreaks)
library(incidence)
cases = subset(nipah_malaysia, select = c("perak", "negeri_sembilan", "selangor",
"singapore"))
i = as.incidence(cases, dates = nipah_malaysia$date, interval = 7L)
jpeg(file = "plot.jpeg")
plot(i)
dev.off()
I did come across this post on employing ftpUpload function from the "rcurl" package. However, to upload it to FTP, I might still need to save it locally which defeats my purpose in this use-case.
Any suggestions would be helpful.
If saving a temporary file (as you suggested in a comment) is an option, then you can do that with the following code:
library(outbreaks)
library(incidence)
library(RCurl)
cases = subset(nipah_malaysia, select = c("perak", "negeri_sembilan", "selangor",
"singapore"))
i = as.incidence(cases, dates = nipah_malaysia$date, interval = 7L)
jpeg(file = filename <- tempfile())
plot(i)
dev.off()
ftpUpload(filename, "ftp://User:Password#FTPServer/destfile.jpeg")
If you're ok with having the output in PNG format (EDIT: I updated the code to show output to JPEG format) try the code below, with chunks borrowed from this answer that discusses how to save an image in memory:
EDIT: Updated to output to jpeg format
library(outbreaks)
library(incidence)
cases = subset(nipah_malaysia, select = c("perak", "negeri_sembilan", "selangor",
"singapore"))
orig_i = as.incidence(cases, dates = nipah_malaysia$date, interval = 7L)
plot(orig_i)
#### This section adapted from
#### https://stackoverflow.com/questions/7171523/in-r-how-to-plot-into-a-memory-buffer-instead-of-a-fileinstead-of-a-file
#### loads image data to memory rather than a file
library(Cairo)
library(png)
library(ggplot2)
Cairo(file='/dev/null')
plot(orig_i) #your plot
# hidden stuff in Cairo
i = Cairo:::.image(dev.cur())
r = Cairo:::.ptr.to.raw(i$ref, 0, i$width * i$height * 4)
dev.off()
dim(r) = c(4, i$width, i$height) # RGBA planes
# have to swap the red & blue components for some reason
r[c(1,3),,] = r[c(3,1),,]
# now use the jpeg library to write the raw vector
library(jpeg)
p = writeJPEG(r, raw()) # raw JPEG bytes
#DEBUGGING - check that this actually works
#Note: Windows 10 has an error that might report this as a file system error
#In windows, drag and drop the file into an open chrome window to see the image
writeBin(p, con= "yourpathhere/check_output.jpg")
#adapted code from #tfehring's example for the updload
library(RCurl)
ftpUpload(p, "ftp://User:Password#FTPServer/destfile.jpg")
So far I created several pngs in R and used ani.options and im.convert from the animation package to create a gif animation (ImageMagick is installed on Windows). It works without any problems:
`ani.options(nmax = 100, loop = 1, interval = 0.1)
for(i in 1:100){
name = rename(i)
png(name)
plot(...)
dev.off()
}
im.convert("*.png", output ="animation.gif", convert = c("convert"),
cmd.fun = if (.Platform$OS.type == "windows") shell else system,
extra.opts = "",clean = TRUE)`
Insead of png-files I would like to convert pdfs to a gif animation. Again I generated and saved several pdfs in a for-loop without any problems. The challenge is now to convert these pdfs to a gif-animation. I tried different approaches but I can't figure out how to modify the im.convert command and how to set ani.options parameters to combine the pdfs in a gif-animation.
So far, I tried setting ani.type and ani.dev to pdf and changed ".png" in im.convert to ".pdf".
I am gratefull for any suggestions.
I found the answer here:
Error using magick R to import PDF
After installing the 64-bit version of GhostScript it worked.
I have built out a Jupyter notebook for my previous analysis.
And I want to start a different branch of analysis, using the some of the same dataframes from previous analysis.
How do I use the previous dataframes without coping all my code to rebuild my previous analysis, in my new notebook?
You can share data across notebooks with Jupyter magics. For example:
Given
# Notebook 1
import pandas as pd
d = {"one" : pd.Series([1., 2., 3.], index=list("abc"))}
df = pd.DataFrame(d)
Code
%store df
Recall the DataFrame in a separate notebook:
# Notebook 2
%store -r df
df
Output
More on this in the older IPython docs. See also Jupyter's %bookmark magic for sharing directories.
You can pickle the dataframe then load the dataframe in your new notebook. This is fine for short term data reuse. For long term data storage, writing then reading a text csv file may be more reliable.
pickle_save.py
import os
import pandas as pd
pickle_location = r'd:\temp\pickle_file'
df = pd.DataFrame({'A':1,'B':2}, index=[0])
df.to_pickle(pickle_location)
if os.path.exists(pickle_location):
print('pickle created')
pickle_load.py
import os
import pandas as pd
pickle_location = r'd:\temp\pickle_file'
df_load = pd.read_pickle(pickle_location)
print(df_load)
I use LIWC2015 as student.
I would like to use it with R.
I found the package LIWCalike with which it is possible to use LIWC dictonary.
I have installed the dictionary to my computer.
However I can't find with file I should include into my path in order to use it with. There is the executable version, also a jar file and I extracted dictonaries however they are only available into pdf format.
What file should I use from LIWC2015 dictonary in order to use it in R?
This example code is from package but I don't have a cat file
liwc2007dict <- dictionary(file = "~/Dropbox/QUANTESS/dictionaries/LIWC/LIWC2007.cat",
format = "wordstat")
tail(liwc2007dict, 1)
You need to change the format of the file in the code in R.
Try this:
liwc2015dict <- dictionary(file = "~/Dropbox/QUANTESS/dictionaries/LIWC/LIWC2015_English_Flat.dic",
format = "LIWC")
It's documented here.