Class LassoCoordinateDescent

source code

            object --+        
                     |        
   bases.BaseAlgorithm --+    
                         |    
   bases.ImplicitAlgorithm --+
                             |
                object --+   |
                         |   |
  bases.IterativeAlgorithm --+
                             |
                object --+   |
                         |   |
bases.InformationAlgorithm --+
                             |
                            LassoCoordinateDescent

A coordinate descent algorithm for the lasso.

This algorithm is very similar to the ShootingAlgorithm, but uses soft
thresholding instead of splitting the cases of the subgradient of L1.
It seems ShootingAlgorithm is slightly faster.

Parameters
----------
l : Non-negative float. The Lagrange multiplier, or regularisation
        constant, of the Lasso.

mean : Boolean. Whether to compute the squared loss or the mean squared
        loss. Default is True, the mean squared loss.

penalty_start : Non-negative integer. The number of columns, variables
        etc., to be exempt from penalisation. Equivalently, the first index
        to be penalised. Default is 0, all columns are included.

eps : Positive float. Tolerance for the stopping criterion.

info : List or tuple of utils.Info. What, if any, extra run information
        should be stored. Default is an empty list, which means that no
        run information is computed nor returned.

max_iter : Non-negative integer. Maximum allowed number of iterations.
        Default is 10000.

min_iter : Non-negative integer less than or equal to max_iter. Minimum
        number of iterations that must be performed. Default is 1.

Examples
--------
>>> from parsimony.algorithms.coordinate import LassoCoordinateDescent
>>> import parsimony.functions as functions
>>> import parsimony.functions.penalties as penalties
>>> import numpy as np
>>> np.random.seed(42)
>>> X = np.random.rand(100, 50)
>>> beta_star = np.random.rand(50, 1)
>>> beta_star[beta_star < 0.5] = 0.0
>>> y = np.dot(X, beta_star) + 0.001 * np.random.randn(100, 1)
>>> l = 0.0618
>>> alg = LassoCoordinateDescent(l)
>>> function = functions.CombinedFunction()
>>> function.add_function(functions.losses.LinearRegression(X, y,
...                                                         mean=False))
>>> function.add_prox(penalties.L1(l=l))
>>> beta = alg.run(X, y)
>>> round(np.linalg.norm(beta_star - beta), 14)
0.34655181469595

Find the minimiser of the associated function, starting at beta.

Parameters
----------
X : Numpy array, shape n-by-p. The matrix X with independent
        variables.

y : Numpy array, shape n-by-1. The response variable y.

beta : Numpy array. Optional starting point.