I am having troubles with the encoding of the osm data.
Here is a reproducible example using the osmar package:
osmData <- osmar::get_osm(osmar::center_bbox(23.334360, 42.693180, 100, 100))
osmData$nodes$tags[80:100, ] #the output is not UTF-8
I have also downloaded a planet file from https://download.geofabrik.de/europe/
After unzipping it and using it with osmar::get_osm I still have the same issue. The cyrilics letters are not readable.
Any ideas how can I counter this?
Ok, Answering my own question:
I ran the above code in linux and understood that the issue was with the windows locale. The workaround I found was to use iconv with from and to parameters set to "UTF-8".
iconv(osmData$nodes$tags[80:100,3][11], from="UTF-8", to="UTF-8")
This works and could be applied to all columns.
Related
I have a 5.1 GB json file that I would like to read in R using rjson. I want afterwards to construct a dataframe from it, however it won't load because the size is too large.
Is there any way to work around it?
Thank you for your help =)
Nina, I would recommend you using jsonlite package instead of rjson.
library(jsonlite)
your_json <- "your_path.json"
unpacked_json <- jsonlite::stream_in(textConnection(readLines(your_json, n=100000)),verbose=F)
Here you limit the page size to let IDE correctly read your JSON file. For more information I would also recommend you to make some research on this topic:
https://community.rstudio.com/t/how-to-read-large-json-file-in-r/13486
Reading a huge json file in R , issues
I know for sure that it is sometimes really hard to cope with documentation (and as all other human beings we are lazy); and I don't like to read doc-n myself, but I highly recommend you to make yourself familiar with jsonlite documentation and vignettes. Here's the CRAN link: https://cran.r-project.org/web/packages/jsonlite/index.html
When I use non-standard letters, the Rstudio view function tends to use a different encoding than print. If you run the following code you will see the difference. I've tried setting all possible encodings, but View keeps displaying them wrongly. Any solutions?
x <- data.frame(test=c('a','b','c','d','é'))
View(x)
print(x)
I will upload an image as soon as I have more reputation.
This issue is solved in the newest version of Rstudio (1.0.136).
I am new to R and was following the following tutorial on the ggplot2 package found here. However the readShapePoly() function throws an error whenever I try to load the basic shapefile. I have used the following code:
library("ggplot2")
library("gpclib")
library("maptools")
setwd("~/Documents/R Projects/Intro to ggplot2")
#Intro to ggplot 2 contains the .shp file and associated data
sport <- readShapePoly("london_sport.shp")
which gets me:
Error in getinfo.shape(filen) : Error opening SHP file
I have tried omitting the file extension. I have also tried downloading other .shp files which throw the same error too. I have also tried calling readShapePoly using the full file path, which doesn't work either. I am using R studio (mac OSX), but I get the same error using the standard R window. I have tried the suggestions on the previous closed threat "Error opening SHP file in Rstudio", but to no avail.
Edit: the error was with a missing .dbf file. Thanks to #Spacedman for the fix.
I had a similar issue, and it was because there were several other files along with the '*.shp' shape file in the zip package that I downloaded. Then I only moved the shape file to another folder and it didn't work. When including all files together, it was fine and I could readShapeSpatial() function okay.
Forget ggplot and gpcclib. Stick to maptools and rgdal that actually provide tools for reading a shapefile.
Don't just say you've tried "this and that", outline the details. For example, does `file.exists("london_sport.shp") return TRUE?
Also, what makes you think readShapePoly() is the right function? It only knows how to read polygon shapefiles, try readShapePoints() and readShapeLines() as well.
If you can, try readOGR which can read a shapefile despite many caveats (including the geometry type).
library(rgdal)
readOGR("~/Documents/R Projects/Intro to ggplot2", "london_sport")
If you can report on all of those things it's likely someone could help.
There is one more, but not much automatic solution which helped me:
file<- readShapePoly(file.choose())
Then just find your *.shp file and run it.
Include three of those files (extensions: dbf, shp, shx) in the same folder.
In case this helps anyone:
I had the same problem but none of the solutions worked. Worst, I the same was going on with an script that I'm 100% sure was working previously.
Turns out that it could also be that the shapefile gets damaged. Apparently this can happen while the file is being manipulated (or so that say my geographer friends), so next time you try to open it won't work for no apparent reason. Downloading it again worked fine, but makes me think to have a copy of the more precious ones just in case.
I just managed to fix this problem with a shapefile I was trying to read by typing:
file<- readShapeSpatial("filename.shp")
instead of typing in the full file path.
Before then, I tried all the suggestions, including making sure that the .dbf and .shx files were also present. Don't know why this should be the case.
I had the same problem. I found out that basically you need three of those files: .shp, .shx and .dbf
Does anyone know why savePlot can't save to pdf in linux by default?
> savePlot("rv-3.pdf", type="pdf")
Error in match.arg(type) :
'arg' should be one of “png”, “jpeg”, “tiff”, “bmp”
lizard:~images$ R --version
R version 2.14.1 (2011-12-22)
...
?savePlot is pretty clear about this:
This works by copying the image surface to a file.
Hence you start with a raster representation and therefore can only go to a raster representation. It would be somewhat perverse to pipe a raster version of the plot in a PDF, which is a vector format (yes I know you can have rasters inside PDFs).
The functionality is limited to cario-based X11 devices and the documentation refers to copying the "on screen" representation hence the restrictions.
I suppose the other Answer to your question is: that functionality has not been implemented yet.
dev.copy2pdf does what you want:
plot(1:10)
dev.copy2pdf(file="~/test.pdf")
From reading the help files, I take it this will effectively replot your figure as a vector image in the file, which will usually be preferable to exporting your vector image into a raster format, as savePlot appears to do.
Try this:
pdf(file="rv-3.pdf")
plot(x,y)
dev.off()
you can also change the size by by adding height= or width= to the pdf function.
I am trying to read a matlab file into R using R.matlab but am encountering this error:
require(R.matlab)
r <- readMat("file.mat", verbose=T)
Trying to read MAT v5 file stream...
Error in readTag(this) : Unknown data type. Not in range [1,19]: 18569
In addition: Warning message:
In readMat5Header(this, firstFourBytes = firstFourBytes) :
Unknown MAT version tag: 512. Will assume version 5.
How can this issue be solved or is there an alternative way to load matlab files? I can use hdf5load but have heard this can mess with the data. Thanks!
This is a bit late on the response, but I've recently been running into the same issues. For me, the issue was that I was saving matlab files by default using the '-v7.3' option. After extensive searching, the R.matlab source documentation (http://cran.r-project.org/web/packages/R.matlab/R.matlab.pdf) indicates the following:
Reading compressed MAT files
From MATLAB v7, compressed MAT version 5 files are used by default
[3,4]. This function supports reading such
files, if running R v2.10.0 or newer. For older versions of R, the
Rcompression package is used. To install that package, please see
instructions at http://www.omegahat.org/ cranRepository.html. As a
last resort, use save -V6 in MATLAB to write MAT files that are
compatible with MATLAB v6, that is, to write non-compressed MAT
version 5 files.
About MAT files saved in MATLAB using ’-v7.3’
This function does not
support MAT files saved in MATLAB as save('foo.mat',
'-v7.3'). Such MAT files are of a completely different file format
[5,6] compared to those saved with, say, '-v7'."
adding the '-v7' option at the end of my save command fixed this issue.
i.e.: save('filename', 'variable', '-v7')
i had a very similar problem until i pointed the function to an actual .mat file that existed. before that i'd been specifying two files of the same name, but one was .mat and the other was .txt, so it may have been trying to open the other.
i realize this may not directly solve your issue (the only difference i saw in my error message was the absence of that first line "Trying ..." and the specific numbers thereafter as well as the presence of another couple similar warnings with odd numbers), but it might point to some simple filename problem as the issue.
i use the latest matlab on 64 bit vista and the latest R on 32 bit xp.