In pytorch, nn.CrossEntropy has a parameter: ignore_index.
How is it implemented?
Maybe like this? "compute CE loss with torch, but not use the function nn.CrossEntropy"
The loss is simply set to 0 for the values with the ignored index.
In PyTorch, nn.CrossEntropy uses negative log likelihood.
It has several implementations, for example this one.
Perhaps these lines are the most important for you:
if (cur_target == ignore_index) {
output[index] = static_cast<scalar_t>(0);
continue;
}
Related
I have this function for fitting a plane to a pointcloud using PCL's sac model fitting. I want the best result I can get, so I want to run with seg.setOptimizeCoefficients(true).
The problem is that a lot of the time, the pointcloud passed in, will not have enough points to optimise coefficients, so I get a continuous stream of:
[pcl::SampleConsensusModelPlane::optimizeModelCoefficients] Not enough inliers found to optimize model coefficients (0)! Returning the same coefficients.
I would like to have coefficient optimisation to run when it can, and when it can't to just carry on without polluting the CLI output with many red warning messages.
according to this issue this message just means that there are fewer than 3 inlier points for the SAC model fitting. I do extract the inlier points, so I could manually check if there are 3 or more. But I can't see how to do this first, and THEN find the optimized model coefficients. Is there a way?
inline void fit_plane_to_points(
const pcl::PointCloud<pcl::PointXYZI>::ConstPtr& det_points,
const pcl::ModelCoefficients::Ptr& coefficients,
const Eigen::Vector3f& vec,
const pcl::PointCloud<pcl::PointXYZI>::Ptr& inlier_pts) {
// if no det points to work with, don't try and segment
if (det_points->size() < 3) {
return;
}
// fit surface point samples to a plane
pcl::PointIndices::Ptr inlier_indices(new pcl::PointIndices);
pcl::SACSegmentation<pcl::PointXYZI> seg;
seg.setModelType(pcl::SACMODEL_PERPENDICULAR_PLANE);
// max allowed difference between the plane normal and the given axis
seg.setEpsAngle(sac_angle_threshold_);
seg.setAxis(vec);
seg.setMethodType(pcl::SAC_RANSAC);
seg.setDistanceThreshold(sac_distance_threshold_);
seg.setMaxIterations(1000);
seg.setInputCloud(det_points);
seg.setOptimizeCoefficients(sac_optimise_coefficients_);
seg.segment(*inlier_indices, *coefficients);
if (inlier_indices->indices.empty()) {
// if no inlier points don't try and extract
return;
}
// extract the planar points
pcl::ExtractIndices<pcl::PointXYZI> extract;
extract.setInputCloud(det_points);
extract.setIndices(inlier_indices);
extract.setNegative(false);
extract.filter(*inlier_pts);
return;
}
I would say the best way to do this is to disable setOptimizeCoefficients and then do that manually after seg.segment. You basically have to recreate these lines: https://github.com/PointCloudLibrary/pcl/blob/10235c9c1ad47989bdcfebe47f4a369871357e2a/segmentation/include/pcl/segmentation/impl/sac_segmentation.hpp#L115-L123 .
You can access the model via getModel() (https://pointclouds.org/documentation/classpcl_1_1_s_a_c_segmentation.html#ac7b9564ceba35754837b4848cf448d78).
Ultimately got it working, with advice from IBitMyBytes by setting seg.setOptimizeCoefficients(false); and then manually optimising after doing my own check:
// if we can, optimise model coefficients
if (sac_optimise_coefficients_ && inlier_indices->indices.size() > 4) {
pcl::SampleConsensusModel<pcl::PointXYZI>::Ptr model = seg.getModel();
Eigen::VectorXf coeff_refined;
Eigen::Vector4f coeff_raw(coefficients->values.data());
model->optimizeModelCoefficients(inlier_indices->indices,
coeff_raw, coeff_refined);
coefficients->values.resize(coeff_refined.size());
memcpy(&coefficients->values[0], &coeff_refined[0],
coeff_refined.size() * sizeof (float));
// Refine inliers
model->selectWithinDistance(coeff_refined, sac_distance_threshold_,
inlier_indices->indices);
}
I want to have a computed property that tracks that historical max of another property. Both these attributes are a member of an ndb class.
My naive approach was the following:
number = ndb.IntegerProperty(default=0)
highest_ever_number = ndb.ComputedProperty(lambda self: self.highest_ever_number if self.highest_ever_number >= self.number else self.number)
The intent was to set the highest_ever_number to a new number if number ever surpassed highest_ever_number.
This doesn't work because highest_ever_number is initially unset. I cannot set it to a default value using "default=0". Is there a workaround to this?
Thanks
I don't think you can use such conditions inside a Computed Property. This whole functionality can be done within a function, so it doesn't need to be a ComputedProperty. A better alternative to this problem could be to create a class method and call it whenever necessary.
Instead of using a computed property, you should use a normal IntegerProperty but use a _pre_put hook (https://cloud.google.com/appengine/docs/standard/python/ndb/modelclass#hooks) to verify/update highest_ever_number.
I can't test it right now, but this might do the trick:
number = ndb.IntegerProperty(default=0)
highest_ever_number = ndb.ComputedProperty(lambda self: self.highest_ever_number if
self.highest_ever_number and self.highest_ever_number >= self.number else self.number)
I hope everyone is well; I have a question it is may be looked as a dumb one but I really need someone to explain it for me. I also though it will be useful for some, since it has been asked before with no satisfactory answer.
Since , I have mixed data type matrix, I was looking for K-nearst neighbors algorithem that works with gower distance in R. I found the function Knngow under the package dprep that claims to perform this.
http://finzi.psych.upenn.edu/library/dprep/html/knngow.html
The function take three argument knngow( Training_Set, Testing_set, K_number) and return the predicted class.
I was playing around with it and was wondering how the function can recognize what is my target vector? Put differently, how does it return the predicted class, without me acknowledging it in advance with my target column.
please find the source code below ( I retrieved it using the function edit)
function (train, test, k)
{
p = dim(train)[2]
ntest = dim(test)[1]
ntrain = dim(train)[1]
classes = rep(0, ntest)
if (ntest == ntrain) {
for (i in 1:ntest) {
tempo = order(gower.dist(test[i, -p], train[-i,
-p]))[1:k]
classes[i] = moda(train[tempo, p])[1]
}
}
else {
for (i in 1:ntest) {
tempo = order(StatMatch::gower.dist(test[i, -p],
train[, -p]))[1:k]
classes[i] = moda(train[tempo, p])[1]
}
}
classes
}
please can someone explain for me the code?
I hope I have post the question in the correct form, please let me know if I have to move it to somewhere else.
Thank you very much for your time.
knngow function takes the last column of the train as the target attribute. Also p = dim(train)[2]) indicates your column number.
Column p (the last column of your training data) is not used for calculating Gower dist. It is only taken into account when it comes to predict the class label of test samples.
I wrote the codes in Modelica as below:
model TestIniitial
extends Modelica.Icons.Example;
parameter Integer nWri= 2;
Real u[nWri](each start= 10, fixed=false);
Real uPre[nWri];
parameter Real _uStart[nWri] = fill(10, nWri);
parameter Modelica.SIunits.Time startTime = 0;
parameter Modelica.SIunits.Time samplePeriod = 1;
Boolean sampleTrigger "True, if sample time instant";
initial equation
u[1] = 1;
u[2] = 2;
equation
sampleTrigger = sample(startTime, samplePeriod);
when sampleTrigger then
for i in 1: nWri loop
uPre[i] = pre(u[i]);
end for;
end when;
for i in 1:nWri loop
u[i] = (i+1)*time;
end for;
end TestIniitial;
Basically I want to initialize the u before simulation. However, I got below complaints(the initialization of u is over-specified) from translation:
The Modelica Language Specification 3.2.1 specifies that if a real variable, v,
is appearing in an expression as pre(v), but not assigned by a when equation,
then the equation v = pre(v) should be added to the initialization problem.
For this problem the following equations were added:
u[1] = pre(u[1]);
u[2] = pre(u[2]);
I can't understand the complaints since pre(v) was assigned in when equation already. What can I do if I want to initialize the u in above codes?
Thanks.
Looking at this, my guess is that the error message is trying to provide you some diagnostics but it is incorrect about the source. I suspect (again, I do not know for sure) that it sees the fact that pre(u) appears in the model and that there is an initialization problem and assumes a specific issue.
My guess is that the issue stems from the fact that you have fixed=true set on u. I see no reason to do that and my guess is that it will lead to too many constraints on the initialization problem as well. Get rid of the fixed=true and see what happens. Report back if that doesn't address the problem.
Good luck.
I have been working on MLM (multi level marketing) application.
Below is the code snippet (not entire code) of recursive function which I had written in initial phase and was working properly. But now the MLM tree is too deep and recursive function stops. It says maximum nesting level exceeded. I increased nesting function call levels few times but now I dont want to increase it further as I know that's not right solution.
Can anyone suggest a alternative code (may be iterative) to me for this?
<?php
function findallpairs($username, $totalusers= 0)
{
$sql = "select username,package_id from tbl_user where
parent_id = '".$username."' order by username";
$result = mysql_query($sql);
if(mysql_num_rows($result) > 0)
{
while($row = mysql_fetch_array($result))
{
$username = $row["username"];
$totalusers++;
$arrtmp = findallpairs($username, $totalusers);
$totalusers = $arrtmp["totalusers"];
}
}
$arrpoints["totalusers"] = $totalusers;
return $arrpoints;
}
?>
Note : Please remember my original code is too big but I have been pasting just the important aspect of the logic here.
It would be a great help for me if I find the alternative solution to this.
Thanks in advance!
How deep are you going?
The day makes a mutliway tree within your sql database. Trees are recursive structures, and recursive code is what naturally fits.
You may be able use use what i'm calling quasi-memiozation.
This should be easy if you have the children listed in the DB structure. Take a result for all users with no childrin, memioize their value into a hash or tree with the key being the user ID and the value 1. Then just mass iterate over each user (or just the parents of memiozed entries) and if it has values memiozed for all its children, add them together and memoioze that value. Repeat the iteration until you find the root (a user with no parent)
If you don't have a record of children it's likely terribly inefficient.