I am trying to import some raw data files (unknown file type) from an MIT facial recognition project site. The files each contain a 16384x1 array of pixel values for a 128*128 image of a face.
Here is the site with the data:
http://courses.media.mit.edu/2004fall/mas622j/04.projects/faces/
Raw data is provided in an unknown file format (some sort of .bin or .dat, I'm really not sure), and should encode for a 16284X1 array. I have tried reading in the files with readBin(), which results in a 4096x1 array (code shown below, "1227" is the file name). In addition, the values of this array have a huge range (min=-2139063177, max= 2139781749), which does not allow them to be pixel values with image().
data<-readBin("1227", "integer",n=128*128, endian = "big")
Changing endian to "little" results in a array of length zero. I had a friend try to read the file in matlab, and they were successful with the following code:
fid=fopen('rawdata/1223');
I = fread(fid);
imagesc(reshape(I, 128, 128)'); colormap(gray(256));
The files are available at the link I posted above, but let me know if I should post the contents of one the files to be read.
The data appear to be an array of bytes which code for a greyscale value. You can read the data into R and plot it with
xx <- matrix(as.numeric(readBin("1223", "raw", 128*128)), ncol=128, byrow = TRUE)
plot(as.raster(xx, max=255))
which will return
Related
I am following a tutorial on remote sensing using R. The tutorial is available from here. pg 44.
I would like to read the metadata for a Landsat 5 image, specifically for 1984-06-22 path/row 170/072. The Landsat Product ID is LT05_L2SP_170072_19840622_20200918_02_T1. Here is the L5 metadata.
I am using the readMeta function from the RSToolbox package. The work should be pretty straightforward in that I put in the path to my metadata file and specify raw = F so that the metadata can be put in a format for further analyses.
mtl <- readMeta(file = ".", raw = F)
After doing this (reading the L5 using readMeta) I get this error.
Error in `.rowNamesDF<-`(x, value = value) : invalid 'row.names' length
Now of course there are many ways of killing a rat, so I used this method here - whereby read.delim function is used to read the metadata file. This brings in a dataframe with all the metadata values alright. However, when this dataframe is put into the radCor function in order to convert the L5 DNs to Top-of-Atmosphere radiance, the following error appears:
Error in radCor(june_landsat2, metaData = metadata_june, method = "rad") :
metaData must be a path to the MTL file or an ImageMetaData object (see readMeta)
Seems like radCor will accept nothing else apart from what is read by readMeta or the path to the MTL file itself. Not even the result from read.delim will do. Because the first error readMeta mentioned row.names length issue, I thought deleting the last row without a value in the metadata file would solve the issue, but this brings more complicated errors.
In short, I would like to find a way to make readMeta read my L5 metadata file, since the result from readMeta is being used in other places of the tutorial. Thanks
I want to import json data from some kind of fitness tracker in order to run some analysis on them. The single json files are quite large, while I am only interested in specific numbers per training session (each json file is a training session).
I managed to read in the names of the files & to grab the interesting content out of the files. Unfortunately my code does obviously not work correctly if one or more information are missing in some of the json files (e.g. distance is not availaible as it was an indoor training).
I stored all json files with training sessions in a folder (=path in the code) and asked R to get a list of the files in that folder:
json_files<- list.files(path,pattern = ".json",full.names = TRUE) #this is the list of files
jlist<-as.list(json_files)
Then I wrote this function to get the data im interested in from each single file (as reading in all the content for each file exceeded my available RAM capacity...)
importPFData <- function(x)
{
testimport<-fromJSON(x)
sport<-testimport$exercises$sport
starttimesession<-testimport$exercises$startTime
endtimesession<-testimport$exercises$stopTime
distance<-testimport$exercises$distance
durationsport<-testimport$exercises$duration
maxHRsession <- testimport$exercises$heartRate$max
minHRsession <- testimport$exercises$heartRate$min
avgHRsession <- testimport$exercises$heartRate$avg
calories <- testimport$exercises$kiloCalories
VO2max_overall <- testimport$physicalInformationSnapshot$vo2Max
return(c(starttimesession,endtimesession,sport,distance,durationsport,
maxHRsession,minHRsession,avgHRsession,calories,VO2max_overall))
}
Next I applied this function to all elements of my list of files:
dataTest<-sapply(jlist, importPFData)
I receive a list with one entry per file, as expected. Unfortunately not all of the data was available per file, which results in some entries having 7, other having 8,9 or 10 entries.
I struggle with getting this into a proper dataframe as the infomation is not shown as NA or 0, its just left out.
Is there an easy way to include NA in the function above if no information is found in the individual json file for that specific detail (e.g. distance not available --> "NA" for distance for this single entry)?
Example (csv) of the content of a file with 10 entries:
"","c..2013.01.06T08.52.38.000....2013.01.06T09.52.46.600....RUNNING..."
"1","2013-01-06T08:52:38.000"
"2","2013-01-06T09:52:46.600"
"3","RUNNING"
"4","6890"
"5","PT3608.600S"
"6","234"
"7","94"
"8","139"
"9","700"
"10","48"
Example (csv) for a file with only 7 entries (columns won´t macht to Example 1):
"","c..2014.01.22T18.38.30.000....2014.01.22T18.38.32.000....RUNNING..."
"1","2014-01-22T18:38:30.000"
"2","2014-01-22T18:38:32.000"
"3","RUNNING"
"4","0"
"5","PT2S"
"6","0"
"7","46"
I have an habitat classification map from Iceland (https://vistgerdakort.ni.is/) with 72 classes in a tif file of 5m*5m pixel size. I want to simplify it, so that there is only 14 classes. I open the files (a tif file and a text file containing the reclassification rules) and use the function classify in the terra package as follow on a subset of the map.
raster <- rast("habitat_subset.tif")
reclass_table<-read.table("reclass_habitat.txt")
habitat_simple<-classify(raster, reclass_table, othersNA=TRUE)
It does exactly what I need it to do and I am able to save the file back to tif using
writeRaster(habitat_simple, "reclass_hab.tif")
The problem is that my initial tif file was 105MB and my new reclassify tif file is 420MB. Since my goal is to reclassify the whole extent of the country, I can't afford to have the file become so big. Any insights on how to make it smaller? I could not find any comments online in relation to this issue.
You can specify the datatype, in your case you should be able to use "INT1U" (i.e., byte values between 0 and 254 --- 255 is used for NA, at least that is the default). That should give a file that is 4 times smaller than when you write it with the default "FLT4S". Based on your question, the original data come with that datatype. In addition you could use compression; I am not sure how well they work with "INT1U". You could have found out about this in the documentation, see ?writeRaster
writeRaster(habitat_simple, "reclass_hab.tif",
wopt=list(datatype="INT1U", gdal="COMPRESS=LZW"))
You could also skip the writeRaster step and do (with terra >= 1.1-4) you can just do
habitat_simple <- classify(raster, reclass_table, othersNA=TRUE,
datatype="INT1U", gdal="COMPRESS=LZW")
I would like to load a raw image data (such as the .raw ones from http://eeweb.poly.edu/~yao/EL5123/SampleData.html) into Julia and display them.
Basically, I am looking for a way to load file into Julia as Array{xxx} type.
Any ideas?
Here is the code and along with the resulting plot:
using Plots, Images, HTTP
r = HTTP.request("GET", "http://eeweb.poly.edu/%7Eyao/EL5123/image/lena_gray.raw")
img = reshape(r.body, 512, 512)
v = rotr90(colorview(Gray, img./256));
Plots.plot(v)
savefig("lena.png")
I'm rather new to R and image processing. I have a series of images stored as csv files that I would like to read into EBImage. I have been using ImageJ in the past. I've been using the imageData(y) command but keep getting the error that my "object must be an array" when I try to read in the file.
I think the issue I'm having is that I've imported the csv as a spatial indexed matrix (x,y) and I need to convert to an array (x,y,value).
test=read.csv('Fe_Kα1.csv', sep=",", header=FALSE)
test1 <- as.matrix(test)
writeImage('test1')
Error in validImage(x) : object must be an array
Thank you