MLPACK
1.0.4
|
An implementation of LARS, a stage-wise homotopy-based algorithm for l1-regularized linear regression (LASSO) and l1+l2 regularized linear regression (Elastic Net). More...
Public Member Functions | |
LARS (const bool useCholesky, const double lambda1=0.0, const double lambda2=0.0, const double tolerance=1e-16) | |
Set the parameters to LARS. | |
LARS (const bool useCholesky, const arma::mat &gramMatrix, const double lambda1=0.0, const double lambda2=0.0, const double tolerance=1e-16) | |
Set the parameters to LARS, and pass in a precalculated Gram matrix. | |
const std::vector< size_t > & | ActiveSet () const |
Access the set of active dimensions. | |
const std::vector< arma::vec > & | BetaPath () const |
Access the set of coefficients after each iteration; the solution is the last element. | |
const std::vector< double > & | LambdaPath () const |
Access the set of values for lambda1 after each iteration; the solution is the last element. | |
const arma::mat & | MatUtriCholFactor () const |
Access the upper triangular cholesky factor. | |
void | Regress (const arma::mat &data, const arma::vec &responses, arma::vec &beta, const bool rowMajor=false) |
Run LARS. | |
Private Member Functions | |
void | Activate (const size_t varInd) |
Add dimension varInd to active set. | |
void | CholeskyDelete (const size_t colToKill) |
void | CholeskyInsert (const arma::vec &newX, const arma::mat &X) |
void | CholeskyInsert (double sqNormNewX, const arma::vec &newGramCol) |
void | ComputeYHatDirection (const arma::mat &matX, const arma::vec &betaDirection, arma::vec &yHatDirection) |
void | Deactivate (const size_t activeVarInd) |
Remove activeVarInd'th element from active set. | |
void | GivensRotate (const arma::vec::fixed< 2 > &x, arma::vec::fixed< 2 > &rotatedX, arma::mat &G) |
void | InterpolateBeta () |
Private Attributes | |
std::vector< size_t > | activeSet |
Active set of dimensions. | |
std::vector< arma::vec > | betaPath |
Solution path. | |
bool | elasticNet |
True if this is the elastic net problem. | |
std::vector< bool > | isActive |
Active set membership indicator (for each dimension). | |
double | lambda1 |
Regularization parameter for l1 penalty. | |
double | lambda2 |
Regularization parameter for l2 penalty. | |
std::vector< double > | lambdaPath |
Value of lambda_1 for each solution in solution path. | |
bool | lasso |
True if this is the LASSO problem. | |
const arma::mat & | matGram |
Reference to the Gram matrix we will use. | |
arma::mat | matGramInternal |
Gram matrix. | |
arma::mat | matUtriCholFactor |
Upper triangular cholesky factor; initially 0x0 matrix. | |
double | tolerance |
Tolerance for main loop. | |
bool | useCholesky |
Whether or not to use Cholesky decomposition when solving linear system. |
An implementation of LARS, a stage-wise homotopy-based algorithm for l1-regularized linear regression (LASSO) and l1+l2 regularized linear regression (Elastic Net).
Let be a matrix where each row is a point and each column is a dimension and let
be a vector of responses.
The Elastic Net problem is to solve
where is the vector of regression coefficients.
If and
, the problem is the LASSO. If
and
, the problem is the elastic net. If
and
, the problem is ridge regression. If
and
, the problem is unregularized linear regression.
Note: This algorithm is not recommended for use (in terms of efficiency) when = 0.
For more details, see the following papers:
@article{efron2004least, title={Least angle regression}, author={Efron, B. and Hastie, T. and Johnstone, I. and Tibshirani, R.}, journal={The Annals of statistics}, volume={32}, number={2}, pages={407--499}, year={2004}, publisher={Institute of Mathematical Statistics} }
@article{zou2005regularization, title={Regularization and variable selection via the elastic net}, author={Zou, H. and Hastie, T.}, journal={Journal of the Royal Statistical Society Series B}, volume={67}, number={2}, pages={301--320}, year={2005}, publisher={Royal Statistical Society} }
mlpack::regression::LARS::LARS | ( | const bool | useCholesky, |
const double | lambda1 = 0.0 , |
||
const double | lambda2 = 0.0 , |
||
const double | tolerance = 1e-16 |
||
) |
Set the parameters to LARS.
Both lambda1 and lambda2 default to 0.
useCholesky | Whether or not to use Cholesky decomposition when solving linear system (as opposed to using the full Gram matrix). |
lambda1 | Regularization parameter for l1-norm penalty. |
lambda2 | Regularization parameter for l2-norm penalty. |
tolerance | Run until the maximum correlation of elements in (X^T y) is less than this. |
mlpack::regression::LARS::LARS | ( | const bool | useCholesky, |
const arma::mat & | gramMatrix, | ||
const double | lambda1 = 0.0 , |
||
const double | lambda2 = 0.0 , |
||
const double | tolerance = 1e-16 |
||
) |
Set the parameters to LARS, and pass in a precalculated Gram matrix.
Both lambda1 and lambda2 default to 0.
useCholesky | Whether or not to use Cholesky decomposition when solving linear system (as opposed to using the full Gram matrix). |
gramMatrix | Gram matrix. |
lambda1 | Regularization parameter for l1-norm penalty. |
lambda2 | Regularization parameter for l2-norm penalty. |
tolerance | Run until the maximum correlation of elements in (X^T y) is less than this. |
void mlpack::regression::LARS::Activate | ( | const size_t | varInd | ) | [private] |
Add dimension varInd to active set.
varInd | Dimension to add to active set. |
const std::vector<size_t>& mlpack::regression::LARS::ActiveSet | ( | ) | const [inline] |
const std::vector<arma::vec>& mlpack::regression::LARS::BetaPath | ( | ) | const [inline] |
void mlpack::regression::LARS::CholeskyDelete | ( | const size_t | colToKill | ) | [private] |
void mlpack::regression::LARS::CholeskyInsert | ( | const arma::vec & | newX, |
const arma::mat & | X | ||
) | [private] |
void mlpack::regression::LARS::CholeskyInsert | ( | double | sqNormNewX, |
const arma::vec & | newGramCol | ||
) | [private] |
void mlpack::regression::LARS::ComputeYHatDirection | ( | const arma::mat & | matX, |
const arma::vec & | betaDirection, | ||
arma::vec & | yHatDirection | ||
) | [private] |
void mlpack::regression::LARS::Deactivate | ( | const size_t | activeVarInd | ) | [private] |
Remove activeVarInd'th element from active set.
activeVarInd | Index of element to remove from active set. |
void mlpack::regression::LARS::GivensRotate | ( | const arma::vec::fixed< 2 > & | x, |
arma::vec::fixed< 2 > & | rotatedX, | ||
arma::mat & | G | ||
) | [private] |
void mlpack::regression::LARS::InterpolateBeta | ( | ) | [private] |
const std::vector<double>& mlpack::regression::LARS::LambdaPath | ( | ) | const [inline] |
Access the set of values for lambda1 after each iteration; the solution is the last element.
Definition at line 164 of file lars.hpp.
References lambdaPath.
const arma::mat& mlpack::regression::LARS::MatUtriCholFactor | ( | ) | const [inline] |
Access the upper triangular cholesky factor.
Definition at line 167 of file lars.hpp.
References matUtriCholFactor.
void mlpack::regression::LARS::Regress | ( | const arma::mat & | data, |
const arma::vec & | responses, | ||
arma::vec & | beta, | ||
const bool | rowMajor = false |
||
) |
Run LARS.
The input matrix (like all MLPACK matrices) should be column-major -- each column is an observation and each row is a dimension. However, because LARS is more efficient on a row-major matrix, this method will (internally) transpose the matrix. If this transposition is not necessary (i.e., you want to pass in a row-major matrix), pass 'true' for the rowMajor parameter.
data | Column-major input data (or row-major input data if rowMajor = true). |
responses | A vector of targets. |
beta | Vector to store the solution (the coefficients) in. |
rowMajor | Set to true if matX is row-major. |
std::vector<size_t> mlpack::regression::LARS::activeSet [private] |
std::vector<arma::vec> mlpack::regression::LARS::betaPath [private] |
bool mlpack::regression::LARS::elasticNet [private] |
std::vector<bool> mlpack::regression::LARS::isActive [private] |
double mlpack::regression::LARS::lambda1 [private] |
double mlpack::regression::LARS::lambda2 [private] |
std::vector<double> mlpack::regression::LARS::lambdaPath [private] |
Value of lambda_1 for each solution in solution path.
Definition at line 199 of file lars.hpp.
Referenced by LambdaPath().
bool mlpack::regression::LARS::lasso [private] |
const arma::mat& mlpack::regression::LARS::matGram [private] |
arma::mat mlpack::regression::LARS::matGramInternal [private] |
arma::mat mlpack::regression::LARS::matUtriCholFactor [private] |
Upper triangular cholesky factor; initially 0x0 matrix.
Definition at line 177 of file lars.hpp.
Referenced by MatUtriCholFactor().
double mlpack::regression::LARS::tolerance [private] |
bool mlpack::regression::LARS::useCholesky [private] |