Structural risk minimization (original) (raw)
Structural risk minimization (SRM) is an inductive principle of use in machine learning. Commonly in machine learning, a generalized model must be selected from a finite data set, with the consequent problem of overfitting – the model becoming too strongly tailored to the particularities of the training set and generalizing poorly to new data. The SRM principle addresses this problem by balancing the model's complexity against its success at fitting the training data. This principle was first set out in a 1974 paper by Vladimir Vapnik and Alexey Chervonenkis and uses the VC dimension.
| Property | Value |
|---|---|
| dbo:abstract | Structural risk minimization (SRM) is an inductive principle of use in machine learning. Commonly in machine learning, a generalized model must be selected from a finite data set, with the consequent problem of overfitting – the model becoming too strongly tailored to the particularities of the training set and generalizing poorly to new data. The SRM principle addresses this problem by balancing the model's complexity against its success at fitting the training data. This principle was first set out in a 1974 paper by Vladimir Vapnik and Alexey Chervonenkis and uses the VC dimension. In practical terms, Structural Risk Minimization is implemented by minimizing , where is the train error, the function is called a regularization function, and is a constant. is chosen such that it takes large values on parameters that belong to high-capacity subsets of the parameter space. Minimizing in effect limits the capacity of the accessible subsets of the parameter space, thereby controlling the trade-off between minimizing the training error and minimizing the expected gap between the training error and test error. The SRM problem can be formulated in terms of data. Given n data points consisting of data x and labels y, the objective is often expressed in the following manner: The first term is the mean squared error (MSE) term between the value of the learned model, , and the given labels . This term is the training error, , that was discussed earlier. The second term, places a prior over the weights, to favor sparsity and penalize larger weights. The trade-off coefficient, , is a hyperparameter that places more or less importance on the regularization term. Larger encourages sparser weights at the expense of a more optimal MSE, and smaller relaxes regularization allowing the model to fit to data. Note that as the weights become zero, and as , the model typically suffers from overfitting. (en) |
| dbo:wikiPageExternalLink | http://www.svms.org/srm/ |
| dbo:wikiPageID | 10704974 (xsd:integer) |
| dbo:wikiPageLength | 3081 (xsd:nonNegativeInteger) |
| dbo:wikiPageRevisionID | 977568847 (xsd:integer) |
| dbo:wikiPageWikiLink | dbr:Vapnik–Chervonenkis_theory dbr:Vladimir_Vapnik dbr:Alexey_Chervonenkis dbr:Model_selection dbr:Occam_Learning dbr:Machine_learning dbr:Empirical_risk_minimization dbc:Machine_learning dbr:Support_vector_machines dbr:VC_dimension dbr:Overfitting |
| dbp:wikiPageUsesTemplate | dbt:Compu-sci-stub dbt:! dbt:Otheruses dbt:Reflist |
| dct:subject | dbc:Machine_learning |
| gold:hypernym | dbr:Principle |
| rdf:type | dbo:Agent |
| rdfs:comment | Structural risk minimization (SRM) is an inductive principle of use in machine learning. Commonly in machine learning, a generalized model must be selected from a finite data set, with the consequent problem of overfitting – the model becoming too strongly tailored to the particularities of the training set and generalizing poorly to new data. The SRM principle addresses this problem by balancing the model's complexity against its success at fitting the training data. This principle was first set out in a 1974 paper by Vladimir Vapnik and Alexey Chervonenkis and uses the VC dimension. (en) |
| rdfs:label | Structural risk minimization (en) |
| owl:sameAs | freebase:Structural risk minimization wikidata:Structural risk minimization https://global.dbpedia.org/id/4vmmN |
| prov:wasDerivedFrom | wikipedia-en:Structural_risk_minimization?oldid=977568847&ns=0 |
| foaf:isPrimaryTopicOf | wikipedia-en:Structural_risk_minimization |
| is dbo:knownFor of | dbr:Vladimir_Vapnik |
| is dbo:wikiPageDisambiguates of | dbr:SRM |
| is dbo:wikiPageRedirects of | dbr:Structural_Risk_Minimization |
| is dbo:wikiPageWikiLink of | dbr:Minimisation dbr:Bregman_method dbr:Vladimir_Vapnik dbr:Model_selection dbr:Kernel_adaptive_filter dbr:Least-squares_support_vector_machine dbr:Multiple_kernel_learning dbr:SRM dbr:Outline_of_machine_learning dbr:Supervised_learning dbr:Structural_Risk_Minimization |
| is dbp:knownFor of | dbr:Vladimir_Vapnik |
| is foaf:primaryTopic of | wikipedia-en:Structural_risk_minimization |