Class LogisticRegressionL1L2GL

source code

             object --+        
                      |        
          BaseEstimator --+    
                          |    
LogisticRegressionEstimator --+
                              |
                             LogisticRegressionL1L2GL

Logistic regression (re-weighted log-likelihood aka. cross-entropy)
with L1, L2 and Group Lasso penalties:

    f(beta) = -loglik / n_samples
              + l1 * ||beta||_1
              + (l2 / (2 * n)) * ||beta||²_2
              + gl * GL(beta)
where
    loglik = Sum wi * (yi * log(pi) + (1 − yi) * log(1 − pi)),

    pi = p(y=1|xi, beta) = 1 / (1 + exp(-xi'*beta)),

    wi = weight of sample i.

Parameters
----------
l1 : Non-negative float. The Lagrange multiplier, or regularisation
        constant, for the L1 penalty.

l2 : Non-negative float. The Lagrange multiplier, or regularisation
        constant, for the ridge (L2) penalty.

gl : Non-negative float. The Lagrange multiplier, or regularisation
        constant, of the group lasso function.

A : Numpy or (usually) scipy.sparse array. The linear operator for the
        smoothed total variation Nesterov function. A must be given.

mu : Non-negative float. The regularisation constant for the Nesterov
        smoothing.

algorithm : ExplicitAlgorithm. The algorithm that should be applied.
        Should be one of:
            1. FISTA(...)
            2. ISTA(...)
            3. StaticCONESTA(...)

        Default is FISTA(...).

algorithm_params : A dict. The dictionary algorithm_params contains
        parameters that should be set in the algorithm. Passing
        algorithm=FISTA(**params) is equivalent to passing
        algorithm=FISTA() and algorithm_params=params. Default is an empty
        dictionary.

class_weight : Dict, 'auto' or None. If 'auto', class weights will be
        given inverse proportional to the frequency of the class in
        the data. If a dictionary is given, keys are classes and values
        are corresponding class weights. If None is given, the class
        weights will be uniform.

penalty_start : Non-negative integer. The number of columns, variables
        etc., to be exempt from penalisation. Equivalently, the first
        index to be penalised. Default is 0, all columns are included.

mean : Boolean. Whether to compute the squared loss or the mean squared
        loss. Default is True, the mean squared loss.

Examples
--------
>>> import numpy as np
>>> import parsimony.estimators as estimators
>>> import parsimony.algorithms.proximal as proximal
>>> import parsimony.functions.nesterov.gl as group_lasso
>>>
>>> np.random.seed(42)
>>>
>>> n, p = 10, 16
>>> groups = [range(0, p / 2), range(p / 2, p)]
>>> weights = [1.5, 0.5]
>>> A = group_lasso.linear_operator_from_groups(p, groups=groups,
...                                             weights=weights)
>>>
>>> X = np.random.rand(n, p)
>>> y = np.random.randint(0, 2, (n, 1))
>>> l1 = 0.1  # L1 coefficient
>>> l2 = 0.9  # Ridge coefficient
>>> gl = 1.0  # TV coefficient
>>> lr = estimators.LogisticRegressionL1L2GL(l1, l2, gl, A=A,
...                      algorithm=proximal.StaticCONESTA(max_iter=1000),
...                      mean=False)
>>> res = lr.fit(X, y)
>>> error = lr.score(X, y)
>>> print "error = ", error
error =  0.7
>>> lr = estimators.LogisticRegressionL1L2GL(l1, l2, gl, A=A,
...                                algorithm=proximal.FISTA(max_iter=1000),
...                                mean=False)
>>> lr = lr.fit(X, y)
>>> error = lr.score(X, y)
>>> print "error = ", error
error =  0.7
>>> lr = estimators.LogisticRegressionL1L2GL(l1, l2, gl, A,
...                                 algorithm=proximal.ISTA(max_iter=1000),
...                                 mean=False)
>>> lr = lr.fit(X, y)
>>> error = lr.score(X, y)
>>> print "error = ", error
error =  0.7