Class LinearRegressionL1L2GL

source code

     object --+        
              |        
  BaseEstimator --+    
                  |    
RegressionEstimator --+
                      |
                     LinearRegressionL1L2GL

Linear regression with L1, L2 and Group lasso penalties:

    f(beta, X, y) = (1 / (2 * n)) * ||Xbeta - y||²_2
                    + l1 * ||beta||_1
                    + (l2 / 2) * ||beta||²_2
                    + gl * GL(beta)

Parameters
----------
l1 : Non-negative float. The L1 regularization parameter.

l2 : Non-negative float. The L2 regularization parameter.

tv : Non-negative float. The group lasso regularization parameter.

A : Numpy or (usually) scipy.sparse array. The linear operator for the
        smoothed group lasso Nesterov function. A must be given.

mu : Non-negative float. The regularisation constant for the smoothing.

algorithm : ExplicitAlgorithm. The algorithm that should be applied.
        Should be one of:
            1. FISTA(...)
            2. ISTA(...)
            3. StaticCONESTA(...)

        Default is FISTA(...).

algorithm_params : A dict. The dictionary algorithm_params contains
        parameters that should be set in the algorithm. Passing
        algorithm=FISTA(**params) is equivalent to passing
        algorithm=FISTA() and algorithm_params=params. Default is an empty
        dictionary.

penalty_start : Non-negative integer. The number of columns, variables
        etc., to be exempt from penalisation. Equivalently, the first
        index to be penalised. Default is 0, all columns are included.

mean : Boolean. Whether to compute the squared loss or the mean squared
        loss. Default is True, the mean squared loss.

Examples
--------
>>> import numpy as np
>>> import parsimony.estimators as estimators
>>> import parsimony.algorithms.proximal as proximal
>>> import parsimony.functions.nesterov.gl as group_lasso
>>> n = 10
>>> p = 15
>>>
>>> np.random.seed(42)
>>> X = np.random.rand(n, p)
>>> y = np.random.rand(n, 1)
>>> l1 = 0.1  # L1 coefficient
>>> l2 = 0.9  # Ridge coefficient
>>> gl = 1.0  # GL coefficient
>>> groups = [range(0, 10), range(5, 15)]
>>> A = group_lasso.linear_operator_from_groups(p, groups, weights=None,
...                                             penalty_start=0)
>>> lr = estimators.LinearRegressionL1L2GL(l1, l2, gl, A,
...                                   algorithm=proximal.StaticCONESTA(),
...                                   algorithm_params=dict(max_iter=1000),
...                                   mean=False)
>>> res = lr.fit(X, y)
>>> round(lr.score(X, y), 13)
0.6101838224235
>>>
>>> lr = estimators.LinearRegressionL1L2GL(l1, l2, gl, A,
...                                  algorithm=proximal.CONESTA(),
...                                  algorithm_params=dict(max_iter=1000),
...                                  mean=False)
>>> res = lr.fit(X, y)
>>> round(lr.score(X, y), 11)
0.61139525439
>>>
>>> lr = estimators.LinearRegressionL1L2GL(l1, l2, gl, A,
...                                   algorithm=proximal.FISTA(),
...                                   algorithm_params=dict(max_iter=1000),
...                                   mean=False)
>>> lr = lr.fit(X, y)
>>> round(lr.score(X, y), 12)
10.7465249393
>>>
>>> lr = estimators.LinearRegressionL1L2GL(l1, l2, gl, A,
...                                   algorithm=proximal.ISTA(),
...                                   algorithm_params=dict(max_iter=1000),
...                                   mean=False)
>>> lr = lr.fit(X, y)
>>> round(lr.score(X, y), 14)
11.02462114246791