How to visualize descriptors and keypoints from raw pointcloud? - point-cloud-library

Question

I am trying to extract NARF keypoints and descriptors for raw pointcloud data using pcl::NarfKeypoint , pcl::NarfDescriptor.
In visualization process, I can simply plot my keypoints along with the range image generated from the original ponitcloud.
The problem, however, deals with visualizing descriptors along with keypoints.
As far as I have understood, Narf computes all indices for keypoints and using getVector3fMap(), one can simply visualize them with pcl::visualization.
When it comes to descriptors, the output would be x, y, z, roll, pitch, yaw and more importantly descriptors[36].
Does anyone know how to visualize the descriptors with keypoints in PCL?
Do we really need to utilize those 36 points in descriptors[36] to address this problem?
My sample code:
// --------------------------------
// -----Extract NARF keypoints-----
// --------------------------------
clock_t begin = clock();
pcl::RangeImageBorderExtractor range_image_border_extractor;
pcl::NarfKeypoint narfKp (&range_image_border_extractor);
narfKp.setRangeImage (&range_image);
narfKp.getParameters().support_size = support_size;
narfKp.getParameters().calculate_sparse_interest_image = true;
narfKp.getParameters().use_recursive_scale_reduction = true;
pcl::PointCloud<int> keyPoIdx;
narfKp.compute (keyPoIdx);
cout << "range image = " << range_image << "\n \n";
cout << "keypoint = "<< keyPoIdx <<"\n";
cout << "time to compute NARF keyPoints = " << (float)(clock() - begin) / CLOCKS_PER_SEC << " [sec] \n";
// --------------------------------
// ----Extract NARF descriptors----
// --------------------------------
vector<int> desIdx;
desIdx.resize(keyPoIdx.points.size());
for (unsigned int i = 0; i < desIdx.size(); i++)
{
desIdx[i] = keyPoIdx.points[i];
}
pcl::NarfDescriptor narfDes (&range_image, &desIdx);
narfDes.getParameters().support_size = support_size;
narfDes.getParameters().rotation_invariant = true; // cause more descriptors than keypoints
pcl::PointCloud<pcl::Narf36> outputNarfDes;
narfDes.compute(outputNarfDes);
cout << "Extracted "<< outputNarfDes.size() <<" descriptors for " << keyPoIdx.points.size() << " keypoints.\n";
//------------------------------------------------------------------ //
//-----------------------Visualization-------------------------------//
// ----------------------------------------------------------------- //
// ----------------------------------------------
// -----Show keypoints in range image widget-----
// ----------------------------------------------
//for (size_t i=0; i<keyPoIdx.points.size (); ++i)
//range_image_widget.markPoint (keyPoIdx.points[i]%range_image.width,
//keyPoIdx.points[i]/range_image.width);
// ---------------------------------------
// -----Show Descriptors in 3D viewer-----
// ---------------------------------------
pcl::PointCloud<pcl::PointXYZ>::Ptr descriptors_ptr (new pcl::PointCloud<pcl::PointXYZ>);
pcl::PointCloud<pcl::PointXYZ>& desVIZ = *descriptors_ptr;
desVIZ.points.resize(outputNarfDes.size());
cout << "descriptor index size = " << desVIZ.points.size() << "\n";
for (size_t i=0; i < desVIZ.points.size(); ++i)
//for (size_t i=0; i<desIdx.size(); ++i)
{
// ??????????????? MY PROBLEM ???????????????????
desVIZ.points[i].getVector3fMap () = range_image.points[outputNarfDes.points[i]].getVector3fMap ();
// ??????????????? MY PROBLEM ???????????????????
}
pcl::visualization::PointCloudColorHandlerCustom<pcl::PointXYZ> des_color_handler (descriptors_ptr, 200, 0, 50);
viewer.addPointCloud<pcl::PointXYZ> (descriptors_ptr, des_color_handler, "descriptors");
viewer.setPointCloudRenderingProperties (pcl::visualization::PCL_VISUALIZER_POINT_SIZE, 10, "descriptors");
// -------------------------------------
// -----Show keypoints in 3D viewer-----
// -------------------------------------
pcl::PointCloud<pcl::PointXYZ>::Ptr keypoints_ptr (new pcl::PointCloud<pcl::PointXYZ>);
pcl::PointCloud<pcl::PointXYZ>& keyPo = *keypoints_ptr;
keyPo.points.resize(keyPoIdx.points.size());
for (size_t i=0; i<keyPoIdx.points.size(); ++i)
{
keyPo.points[i].getVector3fMap () = range_image.points[keyPoIdx.points[i]].getVector3fMap ();
}
pcl::visualization::PointCloudColorHandlerCustom<pcl::PointXYZ> keypoints_color_handler (keypoints_ptr, 0, 200, 0);
viewer.addPointCloud<pcl::PointXYZ> (keypoints_ptr, keypoints_color_handler, "keypoints");
viewer.setPointCloudRenderingProperties (pcl::visualization::PCL_VISUALIZER_POINT_SIZE, 5, "keypoints");
//--------------------
// -----Main loop-----
//--------------------
while (!viewer.wasStopped ())
{
range_image_widget.spinOnce (); // process GUI events
viewer.spinOnce ();
pcl_sleep(0.01);
}

Answer 1

You can visualize the roll/pitch/yaw, by converting it to a vector. Refer to this answer for details. This vector can either be used as the normal of each points - that way, for each view, only keypoints sharing the same orientation will have a color. Alternatively you may try to draw arrows in the position of the keypoints.
To visualize the descriptor, you can project it to a three dimensional space using PCA. Than, it can be used to set the color of your keypoints.