Understanding the equations behind the 'logistic' learner when using fitclinear

I recently performed two-class predictions using the "fitclinear" function in MATLAB, implementing the "logistic" learner.

In the description of the "learner" input argument, the documentation for fitclinear lists only a single equation for regression:

f(x) = Bx + b

This equation only applies to linear regression, and not the logistic learner option. Would it be possible to list/describe the equations that yield the predictions when using the logistic learner? The description of the loss function in this section is informative, but I would appreciate seeing an explicit description of the logistic function.

Answer 1

[Ilya]

predict help:

If the linear classification model consists of logistic regression learners, then the software applies the 'logit' score transformation to the raw classification scores (see ScoreTransform).

Then click on ScoreTransform.

[Brian Odegaard]

Thank you!

One quick follow-up: Is there a standard way to rank features when implementing the logistic learner in this function? I am aware of several options for feature ranking in MATLAB but am not sure which method is best to incorporate with fitclinear.

[Ilya]

Use lasso regularization with a small step in the regularization parameter and record what features survive at what lambda. As you increase regularization, you will get less features. The order in which feature coefficients drop to zero determines the feature ranks. Something like this would do:

lambda = logspace(-8,2,100);
mdl = fitclinear(X,y,'Learner','logistic','Lambda','lambda','Regularization','lasso','Solver','sparsa');

You may need to adjust the range of lambda for your problem.