I get an organized point cloud (using pcl and an ASUS Xtion Pro Live), which of course contains NANs and the like. I also get an RGB image of the same scene.
The first step for processing is removing those NANs, which converts the point cloud to unorganized. I then perform a few other steps, but that's not relevant to the question (I think, see P.S.1). What COULD (I'm not sure) be relevant is that I run extract multiple times, and so have quite a few intermediate point clouds. I believe this means I can no longer assume that the points are in the same order they were at the start.
For clarification, I do understand what an unorganized point cloud it and how it differs from unorganized, both theoretically and in terms of how the data is actually stored.
After chopping off various points, I now have a much smaller point cloud which consists only of points in the original point cloud (but much less of them). How do I map these points back to the matching points in the original point cloud? I probably could iterate through the entire cloud to find matches, but this seems hacked together. Is there a better way to do this?
My main aim is to be able to say that 'point A in my final point cloud is of interest to me' and furthermore to map that to pixel K in the RGB image I first obtained. It seems to me that matching the final point cloud with the initial one is the best way to do this, but alternatives are also welcome.
P.S.1 - One of the last few steps in my process is finding a convex hull and then extracting a polygonal prism from the original point cloud. If all else fails, I will just interrogate the (20-50) points on the convex hull to match them with my initial point cloud (minimizing computation) and hence to match them with the original RGB images.
P.S.2 - Random musing - since I know the original size of the RGB image, the origin of the camera relative to the point cloud (or, rather, the position of the points relative to the camera used to take them), and can trivially obtain the camera parameters, could I simply use ray-tracing through each point in my final point cloud to produce an RGB image? The image may need registration with the 'real' RGB image, or it probably won't since nothing will have actually moved except for rounding error.
Related
Okay, so I have one OBJ file which I read into PCLpointcloud2. Now I want to feed it into a K-dTree. Which is not taking PCLPointCloud2 as input. I want to query any general point if it lies on the surface of my OBJ file.
I am finding it hard to understand their documentation. So how can it be done?
Plus, kindly point me to a good reference easily interpretable. And what is "PointT" BTW? Is it custom build type defined by us? please elaborate.
Look at the code in the provided tool pcl_mesh_sampling (in the PCL code directory under tools/mesh_sampling.cpp). It is relatively simple. It loads a model from PLY or OBJ then for each triangle it samples random points from the triangle. The final point cloud then undergoes a voxel-grid sample to make the points relatively uniform. Alternatively, you can just run the pcl_mesh_sampling program on your obj file to get an output PCD which you can then visualise with pcl_viewer before loading the PCD file into your own code.
Once you have the final point cloud, you can build and use a KD-Tree as per http://pointclouds.org/documentation/tutorials/kdtree_search.php
PointT is the template argument. The point cloud library can handle a variety of point types, from simple PointXYZ (having just x,y,z) to more complicated points like PointXYZRGBNormal (having x,y,z,normal_x,normal_y,normal_z, curvature, r, g, and b channels). Each algorithm is templated on the point type that you want to use. It would probably be easier if you used PointXYZ with your OBJ file, so use pcl::PointXYZ for all your template arguments. For more on templates see http://www.tutorialspoint.com/cplusplus/cpp_templates.htm and http://pointclouds.org/documentation/tutorials/adding_custom_ptype.php.
Update (reply to latest comment)
Added here because this reply is too long for a comment.
I think I see what you are getting at. So when you sample points from the point cloud & build a KD-tree of the object surface, and for each point you keep track which faces are nearby that point (probably all the faces adjacent to the face from which the point was sampled should be sufficient? Just one face is definitely insufficient). Then when the query point is given, you find the nearest point in the KD-tree and check whether the query point is on the "outside" or inside of the full list of nearby faces associated with that point in the KD-tree. If it's on the "inside" of all of them perhaps it is an interior point. But I cannot guarantee that this is true. That is my thinking on that question at the moment. But I do wonder if you want a mesh-based approach really. By the way, if you break your mesh up into convex parts then you can have nice guarantees when processing each convex part.
I'm maintaining software which uses PCL. I'm myself not much experienced in PCL, I've only tried some examples and tried to understand the official PCL-Ducumentation (which is unfortunately mainly sparse, doxygen-generated text). My impression is, only a PCL contributors have real change to use the library efficiently.
One feature I have to fix in the software is aligning two clouds. The clouds are two objects, which should be stacked together with a layer in-between (The actual task is to calculate the volume of the layer ).
I hope the picture explains the task well. The objects are scanned both from the sides to be stacked (one from above and the other from below). On both clouds the user selects manually two points. Then, as I hope there should be a mean in PCL to align two clouds providing the two clouds and the coordinates of the points. The alignment is required only in X-Y Plane.
Unfortunately I can't find out which function should I use for this, partly because the PCL documentation is IHMO really humble, partly because of lack of experience.
My desperate idea was to stack the clouds using P1 as the origin of both and then rotate the second cloud manually using the calculated angle between P11,P21 and P12,P22. This works, but since the task appears to me very common, I'd expect PCL to provide a dedicated function for that.
Could you point me to a proper API-function, code-snippet, example, similar project or a good book helping to understand PCL API and usage?
Many thanks!
I think this problem does not need PCL. It is simple enough to form the correct linear equation and solve it.
If you want to use PCL without worrying about the maths too much (though, if the above is a mystery to you, then studying some computational geometry would be very useful), here is my suggestion.
Most PCL operations work on 3D point clouds. I understand from your question that you only have 2D point clouds OR you don't care about the 3rd dimension. In this case if I were you I would represent the points as a 3D point cloud and set the z dimension to zero.
You will only need two point clouds with 3 points as that is how many points you are feeding to the transformation estimation algorithm. The first 2 points in the clouds will be the points chosen by the user. The third one will be any point that you have chosen that you know is the same in both clouds. You need this third one otherwise the transform is still ambiguous if it is a general transform that is being computed. You can calculate however such a point as you know 2 points already and you know that all the points are on a common plane (as you have projected them by losing the z values). Just don't choose it co-linear with the other two points. For example, halfway between the two points and 2cm in the perpendicular direction (ensuring to go in the correct direction).
Then you can use the estimateRigidTransformation functions to find the transform.
http://docs.pointclouds.org/1.7.0/classpcl_1_1registration_1_1_transformation_estimation_s_v_d.html
This function is also good for over-determined problems (it is the workhorse of the ICP algorithm in PCL) but as long as you have enough points to determine the transform it should work.
I am trying to enlarge a point cloud data set. Suppose I have a point cloud data set consisting of 100 points & I want to enlarge it to say 5 times. Actually I am studying some specific structure which is very small, so I want to zoom in & do some computations. I want something like imresize() in Matlab.
Is there any function to do this? What does resize() function do in PCL? Any idea about how can I do it?
Why would you need this? Points are just numbers, regardless whether they are 1 or 100, until all of them are on the same scale and in the same coordinate system. Their size on the screen is just a visual representation, you can zoom in and out as you wish.
You want them to be a thousandth of their original value (eg. millimeters -> meters change)? Divide them by 1000.
You want them spread out in a 5 times larger space in that particular coordinate system? Multiply their coordinates with 5. But even so, their visual representations will look exactly the same on the screen. The data remains basically the same, they will not be resized per se, they numeric representation will change a bit. It is the simplest affine transform, just a single multiplication.
You want to have finer or coarser resolution of your numeric representation? Or have different range? Change your data type accordingly.
That is, if you deal with a single set.
If you deal with different sets, say, recorded with different kinds of sensors and the numeric representations differ a bit (there are angles between the coordinate systems, mm vs cm scale, etc.) you just have to find the transformation from one coordinate system to the other one and apply it to the first one.
Since you want to increase the number of points while preserving shape/structure of the cloud, I think you want to do something like 'upsampling'.
Here is another SO question on this.
The PCL offers a class for bilateral upsampling.
And as always google gives you a lot of hints on this topic.
Beside (what Ziker mentioned) increasing allocated memory (that's not what you want, right?) or zooming in in visualization you could just rescale your point cloud.
This can be done by multiplying each points dimensions with a constant factor or using an affine transformation. So you can e.g switch from mm to m.
If i understand your question correctly
If you have defined your cloud like this
pcl::PointCloud<pcl::PointXYZ>::Ptr cloud (new pcl::PointCloud<pcl::PointXYZ>);
in fact you can do resize
cloud->points.resize (cloud->width * cloud->height);
Note that doing resize does nothing more than allocate more memory for variable thus after resizing original data remain in cloud. So if you want to have empty resized cloud dont forget to add cloud->clear();
If you just want zoom some pcd for visual puposes(i.e you cant see what is shape of cloud because its too small) why dont you use PCL Visualization and zoom by scrolling up/down
I am working with disparity maps (1024 x 768) obtained via stereo and I am able to get point clouds with XYZRGB pcl::Points. However not all pixels from the disparity map are valid depth hence there will never be 1024x768 = 786432 XYZRGB points. Fortunately I am able to save the point clouds unorganized (i.e. height=1). Unfortunately, some normal estimation methods etc, are tailored for organized pointclouds. How can I create organised pointclouds from this ?
I believe that this is not possible.
First of all unorganized point cloud (PC) is just list of points in random order written in file
On the other hand organized PC carries information of in which order orginal points were obtained by depth camera and some other information. This information is stored in lets call it grid.
Once you destroy this grid omiting some points theres no algorithm that can put it back together as it originally was
You can use other methods which provides PCL that doesnt take OPC as an argument. Result will be same as if you would use organized point cloud only little bit slower (depends on size of your input cloud)
I assume that you do have the calibration parameters that are necessary to transform the image points and their depth into 3D points, right?
In this case, you simply create a 2D point cloud and do the following for each pixel of the disparity map:
If the point is valid:
set the corresponding point in the point cloud to the 3D point
else:
set the corresponding point in the cloud to NaN (i.e. a 3D point with NaN as coordinates)
I'm very new to PCL.
I try to detect the floor under an object for checking if the object topples or is it positioned horizontally.
I've checked API and found the method: pcl::PointCloud< T >::at.
Seems like I could detect Z-value of a point using at. Is it correct?
If yes, I'm confused, how it should work. Mathematically a point is infinite small. On my scans I see the point-density the smaller the more distinct they are in Z-direction.
Will at always return a point? Is the value the mean of nearest physical points?
As referenced in the documentation, pcl::PointCloud< T >::at returns the information of a single point (the coordinates plus other data depending on the point format) given column and row information (roughly the X,Y in the depth image). For this reason, this method just works on organized clouds.
Unfortunately, not every point is a valid point. Unless you filter the point cloud, you could find invalid measurements (points which have NaN components). This is pretty normal, just discard those points using a filter. Your intuition is right, the point density is smaller the further away you go from the sensor.
As for what you're trying to achieve, you should take a look at the planar segmentation tutorial on the PCL website and at the Table Object Detector software by Nicolas Burrus. The latter extracts a plane, and the clusters of objects on top of it.