Hi I am an undergraduate student working on a graduate project, and a beginner to computer vision.
After I went through the tutorial "Camera Calibration and 3D Reconstruction" provided by OpenCV (link) : https://docs.opencv.org/master/d9/db7...
I failed to see the connections between the second part to the final part. What I understand here is :
The intrinsic and extrinsic parameters of a camera is required to
estimate the position of the camera and the captured object
To reconstruct a 3D model multiple point clouds are needed, and to
generate a point cloud a disparity map is required.
What I do not understand is :
The importance of estimating the position of the camera or the object to compute the epiline or epipole in either image planes.
The importance of epipolar geometry, and finding out the location of epiline and epipole to compute the disparity map.
As far as I am aware, the code below generate a disparity map
stereo = cv2.createStereoBM(numDisparities=16, blockSize=15)
disparity = stereo.compute(imgL,imgR)
and the input includes a pair of stereo images, minDisparities, numDisparities and blockSize, but not the position of the camera nor the epiline/epipole.
Any help would be greatly appreciated.
Related
I'm making a procedurally generated minecraft-like voxel terrain in Unity. Mesh generation and albedo channel texturing is flawless; however I need to apply different normal map textures for different cube faces regarding whether they're neighboring to another cube or not. Materials accepts only single normal map file and doesn't provide a sprite-sheet-editor kind of functionality for normal maps. So I have no idea about how to use selected slices out of normal map file as if they were albedo textures. I couldn't find any related resources about the problem. Any help will be appreciated. Thanks...
First of all, I'm not an expert in this area, though I am going to try to help you based on my limited and incomplete understanding of parts of Unity.
If there are a finite number of "normal face maps" that can exist, I suggest something like you indicated ("sprite sheet") and create a single texture (also sometimes called a texture atlas) that contains all these normal maps.
The next step, which I'm not sure whether the Standard material shader will be to handle for your situation is to generate UV/texture coordinates for the normal map and pass those along with your vertex xyz positions to the shader. The UV coordinates need to be for each vertex of each face; they are specified as a 2-D (U, V) offset into your atlas of normal maps and are floating point values with a range of [0.0, 1.0], that map to the full X and Y coordinates of the actual normal texture. For instance, if you had an atlas with a grid of textures in 4 rows and 4 columns, a face that should use the top-left texture would have UV coords of [(0,0), (0.25,0), (0.25,0.25), (0, 0.25)].
The difficulty here may depend if you are you using UV coordinates for doing other texture mapping (e.g. in the Albedo or wherever else). If this is the case, I think the Unity Standard Shader permits two sets of texture coordinates, and if you need more, you might have to roll your own shader or find a Shader asset elsewhere that allows for more UV sets. This is where my understanding of gets shaky, as I'm not exactly sure how the shader uses these two UV coordinate sets, and whether there is some existing convention for how these UV coordinate are used, as the standard shader supports secondary/detail maps, which may mean you have to share the UV0 set with all non-detail maps, so albedo, normal, height, occlusion, etc.
I want to reconstruct a surface from a point cloud of xyz points only, but the example provided requires a normal for each point. I have 2 questions: 1) what does the normal represent. 2) what do I do if I don't have the normal.
If you have a point cloud that represent a surface, the associated normal is the normal direction on the surface at the point. If you don't have such normals, CGAL provides some methods to estimate and orient those normals:
pca_estimate_normals() or jet_estimate_normals() for the estimation and mst_orient_normals() for the orientation.
Note that the links I'm giving are for the upcoming 5.1 release but the functions exist in previous releases. You can also read the new reconstruction tutorial here.
I'm running a model-scene match between a set of point clouds in order to test the matching results.
The match is based on 3D features such as normals and point feature histogram.
I'm using the normal estimation of point cloud library (pcl) to compute the histogram after I'd resampled the point cloud of both model and scene.
My question is, how can I test the accuracy of selecting different radius values in the nearest-neighbor estimation step.
I need to use that values for normal estimation, resampling and histogram in objects such as cup/knife/hummer etc.
I tried to visualize those objects using the pcl visulizer with different radius values and choosing which one that gives correct normals (In terms of how perpendicular were the normals orientation to the surfaces).
But I think that this visual testing is not enough and I would like to know if there are some empiric ways to estimate the optimal radius value.
I would appreciate any suggestion or help ,share your thoughts :)
Thank you.
I think you should start from a ground test: create a point cloud from a mesh using the mesh normals (using CloudCompare for example), then load it twice: once with full data (including normals) and once without normals.
Rebuild normals using the search radius to be tested then you can directly compare de obtained normals with the one extracted from the mesh...
I am working with disparity maps (1024 x 768) obtained via stereo and I am able to get point clouds with XYZRGB pcl::Points. However not all pixels from the disparity map are valid depth hence there will never be 1024x768 = 786432 XYZRGB points. Fortunately I am able to save the point clouds unorganized (i.e. height=1). Unfortunately, some normal estimation methods etc, are tailored for organized pointclouds. How can I create organised pointclouds from this ?
I believe that this is not possible.
First of all unorganized point cloud (PC) is just list of points in random order written in file
On the other hand organized PC carries information of in which order orginal points were obtained by depth camera and some other information. This information is stored in lets call it grid.
Once you destroy this grid omiting some points theres no algorithm that can put it back together as it originally was
You can use other methods which provides PCL that doesnt take OPC as an argument. Result will be same as if you would use organized point cloud only little bit slower (depends on size of your input cloud)
I assume that you do have the calibration parameters that are necessary to transform the image points and their depth into 3D points, right?
In this case, you simply create a 2D point cloud and do the following for each pixel of the disparity map:
If the point is valid:
set the corresponding point in the point cloud to the 3D point
else:
set the corresponding point in the cloud to NaN (i.e. a 3D point with NaN as coordinates)
I'm working on camera calibration and pose estimation.Do you know any software for producing synthetic data? I want to put my camera and my calibration object in the scene and move the calibration object. I want the simulator to give me the positions of calibration object in world's coordinate system and camera's coordinate system and some info like this...
Do you know such a thing?
You can have a look at Blender: you specify the positions of camera and objects in the scene by means of 3D coordinates; then with the render command you get the image for the camera.