I'm very new to PCL.
I try to detect the floor under an object for checking if the object topples or is it positioned horizontally.
I've checked API and found the method: pcl::PointCloud< T >::at.
Seems like I could detect Z-value of a point using at. Is it correct?
If yes, I'm confused, how it should work. Mathematically a point is infinite small. On my scans I see the point-density the smaller the more distinct they are in Z-direction.
Will at always return a point? Is the value the mean of nearest physical points?
As referenced in the documentation, pcl::PointCloud< T >::at returns the information of a single point (the coordinates plus other data depending on the point format) given column and row information (roughly the X,Y in the depth image). For this reason, this method just works on organized clouds.
Unfortunately, not every point is a valid point. Unless you filter the point cloud, you could find invalid measurements (points which have NaN components). This is pretty normal, just discard those points using a filter. Your intuition is right, the point density is smaller the further away you go from the sensor.
As for what you're trying to achieve, you should take a look at the planar segmentation tutorial on the PCL website and at the Table Object Detector software by Nicolas Burrus. The latter extracts a plane, and the clusters of objects on top of it.
Related
I am trying to understand how to manually generate objects.
I have a mesh, part of which I delete and create a new geometry in its place. I have information about the normals of deleted vertices. On the basis of which I have to build new faces (in a different size and quantity) looking in the same direction.
But I don’t understand how to choose the correct winding. It sounds easy when the lessons talk about CCW winding in screen space. But what if I have a bunch of almost chaotic points in the model space? How then to determine this CCW, which axis is used for this? I suggest that the nearest old normals might help. But what is the cheapest method to determine the correct order?
It turned out to be easier than I thought. It is necessary to find the cross product of the first two vectors from the vertices of a triangle, then find the dot of the resulting vector and the normal vector, if the result is negative, then during generation it is necessary to change the order of vertices.
I get an organized point cloud (using pcl and an ASUS Xtion Pro Live), which of course contains NANs and the like. I also get an RGB image of the same scene.
The first step for processing is removing those NANs, which converts the point cloud to unorganized. I then perform a few other steps, but that's not relevant to the question (I think, see P.S.1). What COULD (I'm not sure) be relevant is that I run extract multiple times, and so have quite a few intermediate point clouds. I believe this means I can no longer assume that the points are in the same order they were at the start.
For clarification, I do understand what an unorganized point cloud it and how it differs from unorganized, both theoretically and in terms of how the data is actually stored.
After chopping off various points, I now have a much smaller point cloud which consists only of points in the original point cloud (but much less of them). How do I map these points back to the matching points in the original point cloud? I probably could iterate through the entire cloud to find matches, but this seems hacked together. Is there a better way to do this?
My main aim is to be able to say that 'point A in my final point cloud is of interest to me' and furthermore to map that to pixel K in the RGB image I first obtained. It seems to me that matching the final point cloud with the initial one is the best way to do this, but alternatives are also welcome.
P.S.1 - One of the last few steps in my process is finding a convex hull and then extracting a polygonal prism from the original point cloud. If all else fails, I will just interrogate the (20-50) points on the convex hull to match them with my initial point cloud (minimizing computation) and hence to match them with the original RGB images.
P.S.2 - Random musing - since I know the original size of the RGB image, the origin of the camera relative to the point cloud (or, rather, the position of the points relative to the camera used to take them), and can trivially obtain the camera parameters, could I simply use ray-tracing through each point in my final point cloud to produce an RGB image? The image may need registration with the 'real' RGB image, or it probably won't since nothing will have actually moved except for rounding error.
I am working with disparity maps (1024 x 768) obtained via stereo and I am able to get point clouds with XYZRGB pcl::Points. However not all pixels from the disparity map are valid depth hence there will never be 1024x768 = 786432 XYZRGB points. Fortunately I am able to save the point clouds unorganized (i.e. height=1). Unfortunately, some normal estimation methods etc, are tailored for organized pointclouds. How can I create organised pointclouds from this ?
I believe that this is not possible.
First of all unorganized point cloud (PC) is just list of points in random order written in file
On the other hand organized PC carries information of in which order orginal points were obtained by depth camera and some other information. This information is stored in lets call it grid.
Once you destroy this grid omiting some points theres no algorithm that can put it back together as it originally was
You can use other methods which provides PCL that doesnt take OPC as an argument. Result will be same as if you would use organized point cloud only little bit slower (depends on size of your input cloud)
I assume that you do have the calibration parameters that are necessary to transform the image points and their depth into 3D points, right?
In this case, you simply create a 2D point cloud and do the following for each pixel of the disparity map:
If the point is valid:
set the corresponding point in the point cloud to the 3D point
else:
set the corresponding point in the cloud to NaN (i.e. a 3D point with NaN as coordinates)
I have a map of a mountainous landscape, http://skimap.org/data/989/60/1218033025.jpg. It contains a number of known points, the lat-longs of which can be easily found out using Google maps. I wish to be able to pin any latitude longitude coordinate on the map, of course within the bounds of the landscape.
For this, I tried an approach that seems to be largely failing. I assumed the map to be equivalent to an aerial photograph of the Swiss landscape, without any info about the altitude or other coordinates of the camera. So, I assumed the plane perpendicular to the camera lens normal to be Ax+By+Cz-d=0.
I attempt to find the plane constants, using the known points. I fix my origin at a point, with z=0 at the sea level. I take two known points in the landscape, and using the equation for a line in 3D, I find the length of the projection of this line segment joining the two known points, on the plane. I multiply it by another constant K to account for the resizing of this length on a static 2d representation of this 3D image. The length between the two points on a 2d static representation of this image on this screen can be easily found in pixels, and the actual length of the line joining the two points, can be easily found, since I can calculate the distance between the two points with their lat-longs, and their heights above sea level.
So, I end up with an equation directly relating the distance between the two points on the screen 2d representation, lets call it Ls, and the actual length in the landscape, L. I have many other known points, so plugging them into the equation should give me values of the 4 constants. For this, I needed 8 known points (known parameters being their name, lat-long, and heights above sea level), one being my orogin, and the second being a fixed reference point. The rest 6 points generate a system of 6 linear equations in A^2, B^2, C^2, AB, BC and CA. Solving the system using a online tool, I get the result that the system has a unique solution with all 6 constants being 0.
So, it seems that the assumption that the map is equivalent to an aerial photograph taken from an aircraft, is faulty. Can someone please give me some pointers or any other ideas to get this to work? Do open street maps have a Mercator projection?
I would say that this impossible to do in an automatic way. The skimap should be considered as an image rather than a map, a map is an projection of the real world into one plane, since this doesn't fit skimaps very well they are drawn instead.
The best way is probably to manually define a lot of points in the skimap with known or estimated coordinates and use them to estimate the points betwween. To get an acceptable result you probably have to assign coordinates to each pixel in the skimap.
You could do something like the following: http://magazin.unic.com/en/2012/02/16/making-of-interactive-mobile-piste-map-by-laax/
I am solving the exact same issue. It is pretty hard and lots of maths. Taking me a few weeks to solve it. Interpolation is the key as well with lots of manual mapping. I would say that for a ski mountain it will take at least 1000/1500 points to be able to get the very basic. So, not a trivial task unless you can automate the collection of these points (what I am doing!) ;)
For use in a rigid body simulation, I want to compute the mass and inertia tensor (moment of inertia), given a triangle mesh representing the boundary of the (not necessarily convex) object, and assuming constant density in the interior.
Assuming your trimesh is closed (whether convex or not) there is a way!
As dmckee points out, the general approach is building tetrahedrons from each surface triangle, then applying the obvious math to total up the mass and moment contributions from each tet. The trick comes in when the surface of the body has concavities that make internal pockets when viewed from whatever your reference point is.
So, to get started, pick some reference point (the origin in model coordinates will work fine), it doesn't even need to be inside of the body. For every triangle, connect the three points of that triangle to the reference point to form a tetrahedron. Here's the trick: use the triangle's surface normal to figure out if the triangle is facing towards or away from the reference point (which you can find by looking at the sign of the dot product of the normal and a vector pointing at the centroid of the triangle). If the triangle is facing away from the reference point, treat its mass and moment normally, but if it is facing towards the reference point (suggesting that there is open space between the reference point and the solid body), negate your results for that tet.
Effectively what this does is over-count chunks of volume and then correct once those areas are shown to be not part of the solid body. If a body has lots of blubbery flanges and grotesque folds (got that image?), a particular piece of volume may be over-counted by a hefty factor, but it will be subtracted off just enough times to cancel it out if your mesh is closed. Working this way you can even handle internal bubbles of space in your objects (assuming the normals are set correctly). On top of that, each triangle can be handled independently so you can parallelize at will. Enjoy!
Afterthought: You might wonder what happens when that dot product gives you a value at or near zero. This only happens when the triangle face is parallel (its normal is perpendicular) do the direction to the reference point -- which only happens for degenerate tets with small or zero area anyway. That is to say, the decision to add or subtract a tet's contribution is only questionable when the tet wasn't going to contribute anything anyway.
Decompose your object into a set of tetrahedrons around the selected interior point. (That is solids using each triangular face element and the chosen center.)
You should be able to look up the volume of each element. The moment of inertia should also be available.
It gets to be rather more trouble if the surface is non-convex.
I seem to have miss-remembered by nomenclature and skew is not the adjective I wanted. I mean non-regular.
This is covered in the book "Game Physics, Second Edition" by D. Eberly. The chapter 2.5.5 and sample code is available online. (Just found it, haven't tried it out yet.)
Also note that the polyhedron doesn't have to be convex for the formulas to work, it just has to be simple.
I'd take a look at vtkMassProperties. This is a fairly robust algorithm for computing this, given a surface enclosing a volume.
If your polydedron is complicated, consider using Monte Carlo integration, which is often used for multidimensional integrals. You will need an enclosing hypercube, and you will need to be able to test whether a given point is inside or outside the polyhedron. And you will need to be patient, as Monte Carlo integration is slow.
Start as usual at Wikipedia, and then follow the external links pages for further reading.
(For those unfamiliar with Monte Carlo integration, here's how to compute a mass. Pick a point in the containing hypercube. Add to the point_total counter. Is it in the polyhedron? If yes, add to the point_internal counter. Do this lots (see the convergence and error bound estimates.) Then
mass_polyhedron/mass_hypercube \approx points_internal/points_total.
For a moment of inertia, you weight each count by the square of the distance of the point to the reference axis.
The tricky part is testing whether a point is inside or outside your polyhedron. I'm sure that there are computational geometry algorithms for that.