Related
I am very sorry for asking a question that is probably very easy if you know how to solve it, and where many versions of the same question has been asked before. However, I am creating a new post since I have not found an answer to this specific question.
Basically, I have a 200cm x 200cm square that I am recording with a camera above it. However, the camera distorts the square slightly, see example here.. I am wondering how I go from transforming the x,y coordinates in the camera to real-life x,y coordinates (e.g., between 0-200 cm for each side). I understand that I probably need to apply some kind of transformation matrix, but I do not know which one, nor how to determine the transformation matrix. I haven't done any serious linear-algebra in a long time, so I appreciate any pointers for what to read up on, or how to get it done. I am working in python, so if there is some ready code for doing the transformation that would also be useful to know.
Thanks a lot!
I will show this using python and numpy.
import numpy as np
First, you have to understand the projection model
def apply_homography(H, p1):
p = H # p1.T
return (p[:2] / p[2]).T
With some algebraic manipulation you can determine the points at the plane z=1 that produced the given points.
def revert_homography(H, p2):
Hb = np.linalg.inv(H)
# 1 figure out which z coordinate should be added to p2
# order to get z=1 for p1
z = 1/(Hb[2,2] + (Hb[2,0] * p2[:,0] + Hb[2,1]*p2[:,1]))
p2 = np.hstack([p2[:,:2] * z[:,None], z[:, None]])
return p2 # Hb.T
The projection is not invertible, but under the complanarity assumption it may be inverted successfully.
Now, let's see how to determine the H matrix from the given points (assuming they are coplanar).
If you have the four corners in order in order you can simply specify the (x,y) coordinates of the cornder, then you can use the projection equations to determine the homography matrix like here, or here.
This requires at least 5 points to be determined as there is 9 coefficients, but we can fix one element of the matrix and make it an inhomogeneous equation.
def find_homography(p1, p2):
A = np.zeros((8, 2*len(p1)))
# x2'*(H[2,0]*x1+H[2,1]*x2)
A[6,0::2] = p1[:,0] * p2[:,0]
A[7,0::2] = p1[:,1] * p2[:,0]
# - (H[0,0]*x1+H[0,1]*y1+H[0,2])
A[0,0::2] = -p1[:,0]
A[1,0::2] = -p1[:,1]
A[2,0::2] = -1
# y2'*(H[2,0]*x1+H[2,1]*x2)
A[6,1::2] = p1[:,0] * p2[:,1]
A[7,1::2] = p1[:,1] * p2[:,1]
# - (H[1,0]*x1+H[1,1]*y1+H[1,2])
A[3,1::2] = -p1[:,0]
A[4,1::2] = -p1[:,1]
A[5,1::2] = -1
# assuming H[2,2] = 1 we can pass its coefficient
# to the independent term making an inhomogeneous
# equation
b = np.zeros(2*len(p2))
b[0::2] = -p2[:,0]
b[1::2] = -p2[:,1]
h = np.ones(9)
h[:8] = np.linalg.lstsq(A.T, b, rcond=None)[0]
return h.reshape(3,3)
Here a complete usage example. I pick a random H and transform four random points, this is what you have, I show how to find the transformation matrix H_. Next I create a test set of points, and I show how to find the world coordinates from the image coordinates.
# Pick a random Homography
H = np.random.rand(3,3)
H[2,2] = 1
# Pick a set of random points
p1 = np.random.randn(4, 3);
p1[:,2] = 1;
# The coordinates of the points in the image
p2 = apply_homography(H, p1)
# testing
# Create a set of random points
p_test = np.random.randn(20, 3)
p_test[:,2] = 1;
p_test2 = apply_homography(H, p_test)
# Now using only the corners find the homography
# Find a homography transform
H_ = find_homography(p1, p2)
assert np.allclose(H, H_)
# Predict the plane points for the test points
p_test_predicted = revert_homography(H_, p_test2)
assert np.allclose(p_test_predicted, p_test)
I read in colored jpg images using readJPEG() from the jpeg package. Now I have my images as three-dimensional arrays (width, height, channels) in R.
I want to convert these image arrays into the HSL or HSV color space, mutate the images and save them as JPGs in the RGB format again. However, as the images are quite large (5000 x 8000), it would be too time consuming to loop through every single cell. I found the package OpenImageRto convert the image to the HSV color space quickly, however, I am confused by large negative values in the "saturation" channel. Also, the package contains no functions to convert the image back.
Is there any package to perform fast conversions from RGB to HSL or HSV (and back)? Or is there any other way to perform the converison quickly?
These are my current attempts for converting into one direction, element-wise:
# load packages
library(jpeg)
library(plotwidgets)
# load image
img <- readJPEG(img_path)
img <- img * 255
# new empty image
img_new <- array(NA, dim = dim(img))
# this takes way too long
for (img_row in 1:dim(img)[1]) {
for (img_col in 1:dim(img)[2]) {
img_new[img_row,img_col,] <- round(rgb2hsl(as.matrix(img[img_row,img_col,])))
}
}
# this takes also way too long
for (img_row in 1:dim(img)[1]) {
img_new[img_row,,] <- t(round(rgb2hsl(t(matrix(img[img_row,,], ncol = 3)))))
}
# this takes also ages
rgb_hsl_fun <- function(x) {
as.numeric(rgb2hsl(matrix(x)))
}
img_hsl <- apply(X = img, MARGIN = c(1,2), FUN = rgb_hsl_fun)
The whole thing is quite simple to do.
Use the colorspace library for this.
Here is my original img.jpg file.
Here is the code.
library(jpeg)
library(colorspace)
#Reading a jpg file
img = readJPEG("img.jpg") * 255
#Row-by-row conversion
for(i in 1:dim(img)[1]){
#Convert to HSV format
hsv = RGB(img[i,,1], img[i,,2], img[i,,3]) |> as("HSV")
#Mutation of H, S, V components
attributes(hsv)$coords[,"H"] = attributes(hsv)$coords[,"H"]/2
attributes(hsv)$coords[,"S"] = attributes(hsv)$coords[,"S"]*.998
attributes(hsv)$coords[,"V"] = attributes(hsv)$coords[,"V"]-1
#Convert to RGB format and save to the current line.
rgb = as(hsv, "RGB")
img[i,,1] = attributes(rgb)$coords[,"R"]
img[i,,2] = attributes(rgb)$coords[,"G"]
img[i,,3] = attributes(rgb)$coords[,"B"]
}
#Save to JPG file
writeJPEG(img / 255, "img_hsv.jpg")
Just note that to get to the individual H, S, V (or R, G, B) components you have to use the coords attribute.
As you can see, my mutation of the components H, S, V was as follows:
H = H / 2
S = S * 0.998
V = V-1
After this mutation, the original file looks like this.
However, if you prefer to carry out the mutation on the HLS palette, it is possible.
#Reading a jpg file
img = readJPEG("img.jpg") * 255
#Row-by-row conversion
for(i in 1:dim(img)[1]){
#Convert to HLS format
hls = RGB(img[i,,1], img[i,,2], img[i,,3]) |> as("HLS")
#Mutation of H, S, V components
attributes(hls)$coords[,"H"] = attributes(hls)$coords[,"H"]/2
attributes(hls)$coords[,"L"] = attributes(hls)$coords[,"L"]/2
attributes(hls)$coords[,"S"] = attributes(hls)$coords[,"S"]/2
#Convert to RGB format and save to the current line.
rgb = as(hls, "RGB")
img[i,,1] = attributes(rgb)$coords[,"R"]
img[i,,2] = attributes(rgb)$coords[,"G"]
img[i,,3] = attributes(rgb)$coords[,"B"]
}
#Save to JPG file
writeJPEG(img / 255, "img_hls.jpg")
Here is the image with H/2, L/2 and S/2 conversion.
Hope this is what you were looking for.
It would be wise to open an issue to the Github repository (in case that there is a quick fix to the error case for the HSV transformation). For the record I'm the author and maintainer of the OpenImageR package.
I took a look once again to the code of the RGB_to_HSV function and as I mention at the top of the function of the Rcpp code the implementation is based on the paper
Analytical Study of Colour Spaces for Plant Pixel Detection, Pankaj Kumar and Stanley J. Miklavcic, 2018, Journal of Imaging (page 3 of 12) or section 2.1.3,
The negative values of the saturation channel were highly probable related to a mistake of the following line,
S(i) = 1.0 - (3.0 * s_val) * (R(i) + G(i) + B(i));
which actually (based on the paper) should have been:
S(i) = 1.0 - (3.0 * s_val) / (R(i) + G(i) + B(i));
(division rather than multiplication of the last term)
I uploaded the updated version to Github and you can install it using
remotes::install_github('mlampros/OpenImageR')
and please report back if it works so that I can upload the new version to CRAN.
The package does not include a transformation from HSV to RGB (from what I understand you want to modify the pixel values and then convert to RGB).
I am trying to create a 3D mesh of a specific building from points that I extracted from a lidar point cloud. I then created a matrix from the x, y and z values to feed into the as.mesh3d function from the rlg package and since its from a lidar survey, I have 27,000+ points for this one building. I run into an error when I try to create the mesh. I've copied in a sample of 20 points from the point cloud:
X <- c(1566328,1566328,1566328,1566328,1566328,1566327,1566327,1566327,
1566327,1566327,1566327,1566327,1566327,1566327,1566327,1566327,
1566326,1566326,1566326,1566326)
Y <- c(5180937,5180937,5180936,5180935,5180936,5180937,5180937,5180936,
5180936,5180935,5180935,5180935,5180936,5180936,5180937,5180938,
5180938,5180937,5180936,5180936)
Z <- c(19.92300028,19.98300046,19.93700046,19.88099962,19.93500046,19.99500046,
20.00400046,20.00600046,19.97199962,19.92499962,19.95400046,
19.99099991,20.01199991,19.97600020,19.95800008,19.93200008,
19.95300008,19.94800008,19.94300020,19.98399991)
#created a matrix
xyz <- matrix(c(X, Y, Z), byrow = TRUE, ncol = 3)
The problem arises when I try to create the mesh using as.mesh3d():
mesh <- as.mesh3d(xyz, y = NULL, Z = NULL, type = "triangle", col = "red")
This is what I get: Error in as.mesh3d.default(xyz, y = NULL, Z = NULL, type = "triangle", : Wrong number of vertices
The same error happens for the original dataset of 27000+ points despite all being of the same length.
I'm really not advanced in R and was hoping I could get some advice or solutions on how to get past this.
Thankyou
The as.mesh3d function assumes the points are already organized as triangles. Since you're giving it 20 points, that's not possible: it needs a multiple of 3 points.
There's a problem with your calculation of xyz: you say byrow = TRUE, but you're specifying values by column. Using
xyz <- cbind(X, Y, Z)
would work.
If I plot all of your points using text3d(xyz, text=1:20), it looks as though there are a lot of repeats.
There are several ways to triangulate those points, but they depend on assumptions about the surface. For example, if you know there is only one Z value for each (X, Y) pair, you could use as.mesh3d.deldir (see the help page) to triangulate. Here's the code and output for your sample:
dxyz <- deldir::deldir(X - mean(X), Y - mean(Y), z = Z)
# Warning message:
# In deldir::deldir(X - mean(X), Y - mean(Y), z = Z) :
# There were different z "weights" corresponding to
# duplicated points.
persp3d(dxyz, col = "red")
I had to subtract the means from X and Y because rounding errors caused it to look very bad without that: rgl does a lot of things in single precision, which only gives 7 or 8 decimal place accuracy.
I was working with 3 vectors x , y , z , each of length N (Assume N to be some large natural number, say 20000). In order to visualize this, I was able to plot this easily using the following R code :
library("plot3D")
lines3D(x, y, z, type = "l")
Now, I was thinking if we can make a little 3D animation (i.e. a 3D GIF) from the vectors x , y , z. Is it possible in R ?
NOTE : I've previously done 2D GIFs in R, with the help of packages like ggplot2 , gganimate , magick etc. However, I'm curious whether the same thing can be done for 3D data. Thanks in advance.
It is very simple to create .gif with the animation package (note it requires ‘ImageMagick’ or ‘GraphicsMagick’ to run). After the installation, you could do something like this (I assume that you want to display your plot with different point of view):
library(plot3D)
library(animation)
x <- runif(1000)
y <- x*2+runif(1000)
z <- sample(1:10,length(x),replace = T)*x/y
ani.options(interval=0.5,nmax=35)
saveGIF(for(t in seq(0,360,10)){
lines3D(x, y, z, type = "l",theta=t)
}, movie.name = "animation.gif")
I am not an R expert, but i use it for all kinds of image processing. Now I am trying to apply Gaussian blur smoothing (spatstat package) on my satellite S-2 image. Original type of my image is Raster (Raster layer) tiff, actually a subtract image from two Sentinel-2 bands (green and blue). To apply blur on this kind of image I have to first convert it to a pixel image. I've tried doing this following few other questions (like this one Converting a raster object to an im object in R), but i did not succed. I have tried few possibilities, like converting raster image into matrix and than to pixel image, but this does not work, because the image is than too large (although I use small, croped area of the whole Sentinel-2 image).
So, my function in brief looks something like that:
blue <- raster("S2A_OPER_MSI_T33TWH_B02.tif")
green <- raster("S2A_OPER_MSI_T33TWH_B03.tif")
subt <- function(r1, r2) {
return(r2-r1)
}
out_sub1 <- (blue, green, fun = subt)
I tried to apply blur directly on a Raster image, but i soon realized its not working on raster data:
gauss_sub1 <- blur(out_sub1, sigma = 5)
#Error: is.im(x) is not TRUE
So, I try to convert my image into a pixel one
out_sub11 <- as.im(X = "out_sub1")
Error in as.im.function(X, W, ..., dimyx = dimyx, na.replace = na.replace): A window W is required
Therefore I try to define a window following my raster extent
e <- out_sub1#extent
sp_w <- as(e, "SpatialPolygons")
W <- as(sp_w, "owin")
Error in as(SP.win, "owin") : no method or default for coercing “SpatialPolygons” to “owin”
Can anyone tell me what am I doing wrong or how can I convert spatial polygon into owin object class, so I can further process blur command?
And can please someone explain me what difference there is between raster image and a pixel image in R?
You can apply a filter using raster library:
library(raster)
r <- blue - green
# 3 by 3 mean filter
r_mf <- focal(r, w=matrix(1/9,nrow=3,ncol=3))
# gaussian filter
gf <- focalWeight(r, 2, "Gauss")
r_gf <- focal(r, w=gf)