Let's assume that I have data with 1000 features. I want to apply SVM-RFE on this data where each time 10% for the features are removed.
How one can get the accuracy overall the levels of the elimination stages. For example, I want to get performance over 1000 features, 900 features, 800 features,....,2 features, 1 feature.
Also, I want to keep track of the features in each level.
Current framework doesn't score the model / store the feature set at each iteration for RFE.
May be you can get the scoring using the private function, which is intended to used for RFECV class.
>>> from sklearn.datasets import make_friedman1
>>> from sklearn.feature_selection import RFE
>>> from sklearn.svm import SVR
>>> from sklearn.model_selection._validation import _score
>>> X, y = make_friedman1(n_samples=50, n_features=10, random_state=0)
>>> estimator = SVR(kernel="linear")
>>> selector = RFE(estimator, 5, step=1)
>>> from sklearn.metrics import check_scoring
>>> scorer = check_scoring(estimator, 'r2')
>>> selector._fit(
... X, y, lambda estimator, features:
... _score(estimator, X[:, features], y, scorer))
RFE(estimator=SVR(C=1.0, cache_size=200, coef0=0.0, degree=3, epsilon=0.1,
gamma='scale', kernel='linear', max_iter=-1, shrinking=True,
tol=0.001, verbose=False),
n_features_to_select=5, step=1, verbose=0)
>>> selector.scores_
[0.6752027280057595, 0.6750531506827873, 0.6722333425078437, 0.6684835939207456, 0.6669024507875724, 0.6751247326304468]
>>> selector.ranking_
array([1, 1, 1, 1, 1, 6, 4, 3, 2, 5])
If you want to retrieve the feature set at each level/iteration, you need to edit the fit method.
Another option, you can iterator on top of rfe and then store the feature set and performance.
Related
I'd like to check the total derivatives of an output with respect to a large array of inputs, but I don't want to check the derivative with respect to every member of the array, since the array is too large, and the complex steps (or finite differences) across each member of the array would take too long. Is there a way to check_totals wrt just a single member of an array?
Alternatively, is there a way to perform a directional derivative across the entire array for check_totals? This feature seems to exist for check_partials only?
As of Version 3.1.1 of OpenMDAO we don't have directional checking for totals, but it is a good idea and we are probably going to implement it when we figure out the best way.
As a workaround for now, I think the easiest way to take a directional derivative of your model is to temporarily modify your model by creating a component that takes a "step" in some random direction, and then inserting it in front of your component with wide inputs. I've put together a simple example here:
import numpy as np
import openmdao.api as om
n = 50
class DirectionalComp(om.ExplicitComponent):
def setup(self):
self.add_input('x', 1.0)
self.add_output('y', np.ones(n))
self.A = -1.0 + 2.0 * np.random.random(n)
self.declare_partials('y', 'x', rows=np.arange(n), cols=np.repeat(0, n), val=self.A)
def compute(self, inputs, outputs, discrete_inputs=None, discrete_outputs=None):
x = inputs['x']
outputs['y'] = x * self.A
prob = om.Problem()
model = prob.model
# Add something like this
model.add_subsystem('p', om.IndepVarComp('x', 1.0))
model.add_subsystem('direction', DirectionalComp())
model.connect('p.x', 'direction.x')
model.connect('direction.y', 'comp.x')
model.add_design_var('p.x')
# Old Model
model.add_subsystem('comp', om.ExecComp('y = 2.0*x', x=np.ones((n, )), y=np.ones((n, ))))
model.add_constraint('comp.y', lower=0.0)
prob.setup()
prob.run_model()
totals = prob.check_totals()
I wanna create a Neural Network in PyTorch, that will have 2 inputs and 3 outputs with 1 hidden layer. The two inputs will be float numbers that represents features of an image and 3 outputs will be real numbers between 0 and 1. For example output (1, 0, 0) would mean that it is square and (0,1,0) would mean it is rectangle. Any idea how to do it in pytorch?
The network can be defined like this:
import torch
import torch.nn as nn
import torch.optim as optim
import torch.autograd as autograd
import torch.nn.functional as F
from torch.autograd import Variable
class Net(nn.Module):
def __init__(self, num_inputs=2, num_outputs=3,hidden_dim=5):
# define your network here
super(Net, self).__init__()
self.layer1 = nn.Linear(num_inputs,hidden_dim)
self.layer2 = nn.Linear(hidden_dim,num_outputs)
def forward(self, x):
# implement the forward pass
x = F.relu(self.layer1(x))
x = F.sigmoid(self.layer2(x))
return x
Although I have defined the network here, you should maybe look at some examples on the official pytorch website for example on how to train your model.
I am trying to solve an optimization problem where I need to specify the problem and the constraints using a 2D matrix. I have been using SCIPY, where the 1D arrays are the requirements. I want to check if GEKKO allows one to specify the objective function, bounds and constraints using a 2D matrix.
I have provided details and a reproducible version of the problem in the post here:
SCIPY - building constraints without listing each variable separately
Thanks
C
You can use the m.Array function in gekko. I don't recommend that you use the np.triu() with the Gekko array because the eliminated variables will still solve but potentially be hidden from the results. Here is a solution:
import numpy as np
import scipy.optimize as opt
from gekko import GEKKO
p= np.array([4, 5, 6.65, 12]) #p = prices
pmx = np.triu(p - p[:, np.newaxis]) #pmx = price matrix, upper triangular
m = GEKKO(remote=False)
q = m.Array(m.Var,(4,4),lb=0,ub=10)
# only upper triangular can change
for i in range(4):
for j in range(4):
if j<=i:
q[i,j].upper=0 # set upper bound = 0
def profit(q):
profit = np.sum(q.flatten() * pmx.flatten())
return profit
for i in range(4):
m.Equation(np.sum(q[i,:])<=10)
m.Equation(np.sum(q[:,i])<=8)
m.Maximize(profit(q))
m.solve()
print(q)
This gives the solution:
[[[0.0] [2.5432017412] [3.7228765674] [3.7339217013]]
[[0.0] [0.0] [4.2771234426] [4.2660783187]]
[[0.0] [0.0] [0.0] [0.0]]
[[0.0] [0.0] [0.0] [0.0]]]
In my data, there are about 70 classes and I am using lightGBM to predict the correct class label.
In R, would like to have a customised "metric" function where I can evaluate whether top 3 predictions by lightgbm cover the true label.
The link here is inspiring to see
def lgb_f1_score(y_hat, data):
y_true = data.get_label()
y_hat = np.round(y_hat) # scikits f1 doesn't like probabilities
return 'f1', f1_score(y_true, y_hat), True
however I don't know the dimensionality of the arguments going to function. seems data are shuffled for some reason.
Scikit-learn implementation
from sklearn.metrics import f1_score
def lgb_f1_score(y_true, y_pred):
preds = y_pred.reshape(len(np.unique(y_true)), -1)
preds = preds.argmax(axis = 0)
print(preds.shape)
print(y_true.shape)
return 'f1', f1_score(y_true, preds,average='weighted'), True
After reading through the docs for lgb.train and lgb.cv, I had to make a separate function get_ith_pred and then call that repeatedly within lgb_f1_score.
The function's docstring explains how it works. I have used the same argument names as in the LightGBM docs. This can work for any number of classes but does not work for binary classification. In the binary case, preds is a 1D array containing the probability of the positive class.
from sklearn.metrics import f1_score
def get_ith_pred(preds, i, num_data, num_class):
"""
preds: 1D NumPY array
A 1D numpy array containing predicted probabilities. Has shape
(num_data * num_class,). So, For binary classification with
100 rows of data in your training set, preds is shape (200,),
i.e. (100 * 2,).
i: int
The row/sample in your training data you wish to calculate
the prediction for.
num_data: int
The number of rows/samples in your training data
num_class: int
The number of classes in your classification task.
Must be greater than 2.
LightGBM docs tell us that to get the probability of class 0 for
the 5th row of the dataset we do preds[0 * num_data + 5].
For class 1 prediction of 7th row, do preds[1 * num_data + 7].
sklearn's f1_score(y_true, y_pred) expects y_pred to be of the form
[0, 1, 1, 1, 1, 0...] and not probabilities.
This function translates preds into the form sklearn's f1_score
understands.
"""
# Only works for multiclass classification
assert num_class > 2
preds_for_ith_row = [preds[class_label * num_data + i]
for class_label in range(num_class)]
# The element with the highest probability is predicted
return np.argmax(preds_for_ith_row)
def lgb_f1_score(preds, train_data):
y_true = train_data.get_label()
num_data = len(y_true)
num_class = 70
y_pred = []
for i in range(num_data):
ith_pred = get_ith_pred(preds, i, num_data, num_class)
y_pred.append(ith_pred)
return 'f1', f1_score(y_true, y_pred, average='weighted'), True
I am using Conv2D model of Keras 2.0. However, I cannot fully understand what the function is doing mathematically. I try to understand the math using randomly generated data and a very simple network:
import numpy as np
import keras
from keras.layers import Input, Conv2D
from keras.models import Model
from keras import backend as K
# create the model
inputs = Input(shape=(10,10,1)) # 1 channel, 10x10 image
outputs = Conv2D(32, (3, 3), activation='relu', name='block1_conv1')(inputs)
model = Model(outputs=outputs, inputs=inputs)
# input
x = np.random.random(100).reshape((10,10))
# predicted output for x
y_pred = model.predict(x.reshape((1,10,10,1))) # y_pred.shape = (1,8,8,32)
I tried to calculate, for example, the value of the first row, the first column in the first feature map, following the demo in here.
w = model.layers[1].get_weights()[0] # w.shape = (3,3,1,32)
w0 = w[:,:,0,0]
b = model.layers[1].get_weights()[1] # b.shape = (32,)
b0 = b[0] # b0 = 0
y_pred_000 = np.sum(x[0:3,0:3] * w0) + b0
But relu(y_pred_000) is not equal to y_pred[0][0][0][0].
Could anyone point out what's wrong with my understanding? Thank you.
It's easy and it comes from Theano dim ordering. The result of applying filter in stored in a so called channel dimension. In case of TensorFlow this is the last dimension and that's why results are good. In case of Theano it's second dimension (convolution result has shape (cases, channels, width, height) so in order to solve your problem you need to change prediction line to:
y_pred = model.predict(x.reshape((1,1,10,10)))
Also you need to change the way you get the weights as weights in Theano has shape (output_channels, input_channels, width, height) you need to change the weight getter to:
w = model.layers[1].get_weights()[0] # w.shape = (32,1,3,3)
w0 = w[0,0,:,:]