Neural tangent kernel (original) (raw)
In the study of artificial neural networks (ANNs), the neural tangent kernel (NTK) is a kernel that describes the evolution of deep artificial neural networks during their training by gradient descent. It allows ANNs to be studied using theoretical tools from kernel methods. The NTK was introduced in 2018 by Arthur Jacot, Franck Gabriel and Clément Hongler. It was implicit in contemporaneous work on overparameterization.
| Property | Value |
|---|---|
| dbo:abstract | En l'estudi de les xarxes neuronals artificials (ANN), el kernel de tangent neural (amb acrònim anglès NTK) és un que descriu l'evolució de les xarxes neuronals artificials profundes durant el seu entrenament per descens de gradient. Permet estudiar les ANN utilitzant eines teòriques dels mètodes nucli. Per a les arquitectures de xarxes neuronals més comunes, en el límit de l'amplada de capa gran, l'NTK es torna constant. Això permet fer declaracions senzilles de forma tancada sobre prediccions de xarxes neuronals, dinàmiques d'entrenament, generalització i superfícies de pèrdua. Per exemple, garanteix que les ANN prou amples convergeixen a un mínim global quan s'entrenen per minimitzar una pèrdua empírica. El NTK de les xarxes d'amplada gran també està relacionat amb diversos . El NTK va ser presentat el 2018 per Arthur Jacot, Franck Gabriel i Clément Hongler. Estava implícit en treballs contemporanis sobre sobreparametrització. (ca) In the study of artificial neural networks (ANNs), the neural tangent kernel (NTK) is a kernel that describes the evolution of deep artificial neural networks during their training by gradient descent. It allows ANNs to be studied using theoretical tools from kernel methods. For most common neural network architectures, in the limit of large layer width the NTK becomes constant. This enables simple closed form statements to be made about neural network predictions, training dynamics, generalization, and loss surfaces. For example, it guarantees that wide enough ANNs converge to a global minimum when trained to minimize an empirical loss. The NTK of large width networks is also related to several other large width limits of neural networks. The NTK was introduced in 2018 by Arthur Jacot, Franck Gabriel and Clément Hongler. It was implicit in contemporaneous work on overparameterization. (en) No estudo de redes neurais artificiais (RNAs), o kernel de tangente neural (KTN) é um kernel que descreve a evolução de redes neurais artificiais profundas durante seu treinamento por gradiente descendente . Ele permite que RNAs sejam estudadas usando algoritmos do tipo Máquina de vetores de suporte. Para a maioria das arquiteturas de rede neural, no limite da largura da camada, o KTN se torna constante. Isso permite que declarações simples de forma fechada sejam feitas sobre previsões de rede neural, dinâmicas de treinamento, generalização e superfícies de perda. Por exemplo, ele garante que RNAs largas o suficiente convergem para um mínimo global quando treinados para minimizar uma perda empírica. O KTN de redes de grande largura também está relacionado a vários outros limites de largura de redes neurais. O KTN foi lançado em 2018 por Arthur Jacot, Franck Gabriel e Clément Hongler. Também estava implícito em alguns trabalhos contemporâneos. (pt) |
| dbo:wikiPageExternalLink | https://github.com/google/neural-tangents https://www.quantamagazine.org/a-new-link-to-an-old-model-could-crack-the-mystery-of-deep-learning-20211011/%7Cwebsite= |
| dbo:wikiPageID | 62443864 (xsd:integer) |
| dbo:wikiPageLength | 19653 (xsd:nonNegativeInteger) |
| dbo:wikiPageRevisionID | 1103547779 (xsd:integer) |
| dbo:wikiPageWikiLink | dbr:Python_(programming_language) dbc:Kernel_methods_for_machine_learning dbr:Kernel_methods dbr:Deep_learning dbr:Quadratic_loss_function dbr:Maxima_and_minima dbr:Global_minimum dbr:Gradient_descent dbr:Convex_function dbr:Convolutional_neural_network dbr:Recurrent_neural_networks dbr:Closed-form_expression dbr:Feature_(machine_learning) dbr:Kernel_method dbr:Kernel_methods_for_vector_output dbr:Kernel_regression dbr:Activation_function dbr:Large_width_limits_of_neural_networks dbr:Affine_transformation dbr:Gaussian_distribution dbr:Gaussian_random_variable dbr:Quanta_Magazine dbr:Artificial_neural_network dbr:Transformer_(machine_learning_model) dbr:Positive-definite_kernel dbr:Convolutional_neural_networks dbr:Dataset_(machine_learning) dbr:Independent_and_identically_distributed_random_variables dbr:Ordinary_differential_equation dbr:Loss_function dbr:Neural_network_Gaussian_process dbr:Free_and_open-source |
| dbp:wikiPageUsesTemplate | dbt:Cite_web dbt:Reflist dbt:Toclimit |
| dct:subject | dbc:Kernel_methods_for_machine_learning |
| rdfs:comment | In the study of artificial neural networks (ANNs), the neural tangent kernel (NTK) is a kernel that describes the evolution of deep artificial neural networks during their training by gradient descent. It allows ANNs to be studied using theoretical tools from kernel methods. The NTK was introduced in 2018 by Arthur Jacot, Franck Gabriel and Clément Hongler. It was implicit in contemporaneous work on overparameterization. (en) No estudo de redes neurais artificiais (RNAs), o kernel de tangente neural (KTN) é um kernel que descreve a evolução de redes neurais artificiais profundas durante seu treinamento por gradiente descendente . Ele permite que RNAs sejam estudadas usando algoritmos do tipo Máquina de vetores de suporte. O KTN foi lançado em 2018 por Arthur Jacot, Franck Gabriel e Clément Hongler. Também estava implícito em alguns trabalhos contemporâneos. (pt) |
| rdfs:label | Kernel de tangent neural (ca) Neural tangent kernel (en) Kernel de tangente neural (pt) |
| owl:sameAs | wikidata:Neural tangent kernel dbpedia-ca:Neural tangent kernel dbpedia-pt:Neural tangent kernel https://global.dbpedia.org/id/BzW1v |
| prov:wasDerivedFrom | wikipedia-en:Neural_tangent_kernel?oldid=1103547779&ns=0 |
| foaf:isPrimaryTopicOf | wikipedia-en:Neural_tangent_kernel |
| is dbo:wikiPageDisambiguates of | dbr:NTK |
| is dbo:wikiPageWikiLink of | dbr:NTK dbr:Kernel_method dbr:Large_width_limits_of_neural_networks dbr:Neural_network_Gaussian_process |
| is foaf:primaryTopic of | wikipedia-en:Neural_tangent_kernel |