RegressionLinear - Linear regression model for high-dimensional data - MATLAB (original) (raw)
Linear regression model for high-dimensional data
Description
RegressionLinear is a trained linear model object for regression; the linear model is a support vector machine regression (SVM) or linear regression model. fitrlinear fits a RegressionLinear model by minimizing the objective function using techniques that reduce computation time for high-dimensional data sets (e.g., stochastic gradient descent). The regression loss plus the regularization term compose the objective function.
Unlike other regression models, and for economical memory usage, RegressionLinear model objects do not store the training data. However, they do store, for example, the estimated linear model coefficients, estimated coefficients, and the regularization strength.
You can use trained RegressionLinear models to predict responses for new data. For details, see predict.
Construction
Create a RegressionLinear object by using fitrlinear.
Properties
Linear Regression Properties
Half of the width of the epsilon insensitive band, specified as a nonnegative scalar.
If Learner is not 'svm', then Epsilon is an empty array ([]).
Data Types: single | double
Regularization term strength, specified as a nonnegative scalar or vector of nonnegative values.
Data Types: double | single
Linear regression model type, specified as 'leastsquares' or 'svm'.
In this table, f(x)=xβ+b.
- β is a vector of p coefficients.
- x is an observation from p predictor variables.
- b is the scalar bias.
| Value | Algorithm | Loss Function | FittedLoss Value |
|---|---|---|---|
| 'svm' | Support vector machine regression | Epsilon insensitive: ℓ[y,f(x)]=max[0,|y−f(x) | −ε] |
| 'leastsquares' | Linear regression through ordinary least squares | Mean squared error (MSE): ℓ[y,f(x)]=12[y−f(x)]2 | 'mse' |
Linear coefficient estimates, specified as a numeric vector with length equal to the number of expanded predictors (see ExpandedPredictorNames).
Data Types: double
Estimated bias term or model intercept, specified as a numeric scalar.
Data Types: double
Loss function used to fit the model, specified as 'epsiloninsensitive' or 'mse'.
| Value | Algorithm | Loss Function | Learner Value |
|---|---|---|---|
| 'epsiloninsensitive' | Support vector machine regression | Epsilon insensitive: ℓ[y,f(x)]=max[0,|y−f(x) | −ε] |
| 'mse' | Linear regression through ordinary least squares | Mean squared error (MSE): ℓ[y,f(x)]=12[y−f(x)]2 | 'leastsquares' |
Other Regression Properties
Categorical predictor indices, specified as a vector of positive integers. CategoricalPredictors contains index values indicating that the corresponding predictors are categorical. The index values are between 1 and p, where p is the number of predictors used to train the model. If none of the predictors are categorical, then this property is empty ([]).
Data Types: single | double
Parameters used for training the RegressionLinear model, specified as a structure.
Access fields of ModelParameters using dot notation. For example, access the relative tolerance on the linear coefficients and the bias term by usingMdl.ModelParameters.BetaTolerance.
Data Types: struct
Predictor names in order of their appearance in the predictor data, specified as a cell array of character vectors. The length of PredictorNames is equal to the number of variables in the training data X orTbl used as predictor variables.
Data Types: cell
Response variable name, specified as a character vector.
Data Types: char
Data Types: char | string | function_handle
Object Functions
| incrementalLearner | Convert linear regression model to incremental learner |
|---|---|
| lime | Local interpretable model-agnostic explanations (LIME) |
| loss | Regression loss for linear regression models |
| partialDependence | Compute partial dependence |
| plotPartialDependence | Create partial dependence plot (PDP) and individual conditional expectation (ICE) plots |
| predict | Predict response of linear regression model |
| selectModels | Select fitted regularized linear regression models |
| shapley | Shapley values |
| update | Update model parameters for code generation |
Copy Semantics
Value. To learn how value classes affect copy operations, see Copying Objects.
Examples
Train a linear regression model using SVM, dual SGD, and ridge regularization.
Simulate 10000 observations from this model
y=x100+2x200+e.
- X=x1,...,x1000 is a 10000-by-1000 sparse matrix with 10% nonzero standard normal elements.
- e is random normal error with mean 0 and standard deviation 0.3.
rng(1) % For reproducibility n = 1e4; d = 1e3; nz = 0.1; X = sprandn(n,d,nz); Y = X(:,100) + 2X(:,200) + 0.3randn(n,1);
Train a linear regression model. By default, fitrlinear uses support vector machines with a ridge penalty, and optimizes using dual SGD for SVM. Determine how well the optimization algorithm fit the model to the data by extracting a fit summary.
[Mdl,FitInfo] = fitrlinear(X,Y)
Mdl = RegressionLinear ResponseName: 'Y' ResponseTransform: 'none' Beta: [1000×1 double] Bias: -0.0056 Lambda: 1.0000e-04 Learner: 'svm'
Properties, Methods
FitInfo = struct with fields: Lambda: 1.0000e-04 Objective: 0.2725 PassLimit: 10 NumPasses: 10 BatchLimit: [] NumIterations: 100000 GradientNorm: NaN GradientTolerance: 0 RelativeChangeInBeta: 0.4907 BetaTolerance: 1.0000e-04 DeltaGradient: 1.5816 DeltaGradientTolerance: 0.1000 TerminationCode: 0 TerminationStatus: {'Iteration limit exceeded.'} Alpha: [10000×1 double] History: [] FitTime: 0.0720 Solver: {'dual'}
Mdl is a RegressionLinear model. You can pass Mdl and the training or new data to loss to inspect the in-sample mean-squared error. Or, you can pass Mdl and new predictor data to predict to predict responses for new observations.
FitInfo is a structure array containing, among other things, the termination status (TerminationStatus) and how long the solver took to fit the model to the data (FitTime). It is good practice to use FitInfo to determine whether optimization-termination measurements are satisfactory. In this case, fitrlinear reached the maximum number of iterations. Because training time is fast, you can retrain the model, but increase the number of passes through the data. Or, try another solver, such as LBFGS.
Simulate 10000 observations from this model
y=x100+2x200+e.
- X={x1,...,x1000} is a 10000-by-1000 sparse matrix with 10% nonzero standard normal elements.
- e is random normal error with mean 0 and standard deviation 0.3.
rng(1) % For reproducibility n = 1e4; d = 1e3; nz = 0.1; X = sprandn(n,d,nz); Y = X(:,100) + 2X(:,200) + 0.3randn(n,1);
Hold out 5% of the data.
rng(1); % For reproducibility cvp = cvpartition(n,'Holdout',0.05)
cvp = Hold-out cross validation partition NumObservations: 10000 NumTestSets: 1 TrainSize: 9500 TestSize: 500 IsCustom: 0 IsGrouped: 0 IsStratified: 0
Properties, Methods
cvp is a CVPartition object that defines the random partition of n data into training and test sets.
Train a linear regression model using the training set. For faster training time, orient the predictor data matrix so that the observations are in columns.
idxTrain = training(cvp); % Extract training set indices X = X'; Mdl = fitrlinear(X(:,idxTrain),Y(idxTrain),'ObservationsIn','columns');
Predict observations and the mean squared error (MSE) for the hold out sample.
idxTest = test(cvp); % Extract test set indices yHat = predict(Mdl,X(:,idxTest),'ObservationsIn','columns'); L = loss(Mdl,X(:,idxTest),Y(idxTest),'ObservationsIn','columns')
The hold-out sample MSE is 0.1852.
Extended Capabilities
Usage notes and limitations:
- The predict and update functions support code generation.
- When you train a linear regression model by using fitrlinear, the following restrictions apply.
- If the predictor data input argument value is a matrix, it must be a full, numeric matrix. Code generation does not support sparse data.
- You can specify only one regularization strength, either
'auto'or a nonnegative scalar for the 'Lambda' name-value pair argument. - The value of the 'ResponseTransform' name-value pair argument cannot be an anonymous function.
- Code generation with a coder configurer does not support categorical predictors (
logical,categorical,char,string, orcell). You cannot use the'CategoricalPredictors'name-value argument. To include categorical predictors in a model, preprocess them by using dummyvar before fitting the model.
For more information, see Introduction to Code Generation.
- The following object functions fully support GPU arrays:
- The object functions execute on a GPU if at least one of the following applies:
- The model was fitted with GPU arrays.
- The predictor data that you pass to the object function is a GPU array.
- The response data that you pass to the object function is a GPU array.
Version History
Introduced in R2016a
You can fit a RegressionLinear object with GPU arrays by usingfitrlinear. MostRegressionLinear object functions now support GPU array input arguments so that they can execute on a GPU. The object functions that do not support GPU array inputs are incrementalLearner, lime,shapley, and update.