O. Kinouchi | Universidade de São Paulo (original) (raw)

Papers by O. Kinouchi

PLoS Computational Biology, 2009

Since the first experimental evidences of active conductances in dendrites, most neurons have bee... more Since the first experimental evidences of active conductances in dendrites, most neurons have been shown to exhibit dendritic excitability through the expression of a variety of voltage-gated ion channels. However, despite experimental and theoretical efforts undertaken in the past decades, the role of this excitability for some kind of dendritic computation has remained elusive. Here we show that, owing to very general properties of excitable media, the average output of a model of an active dendritic tree is a highly non-linear function of its afferent rate, attaining extremely large dynamic ranges (above 50 dB). Moreover, the model yields double-sigmoid response functions as experimentally observed in retinal ganglion cells. We claim that enhancement of dynamic range is the primary functional role of active dendritic conductances. We predict that neurons with larger dendritic trees should have larger dynamic range and that blocking of active conductances should lead to a decrease in dynamic range.

Recently Hirsch [1,2] has proposed a new scalar index h to quantify individual's scientific resea... more Recently Hirsch [1,2] has proposed a new scalar index h to quantify individual's scientific research impact. A researcher with index h has h papers with at least h citations. This index has several advantages: (i) it combines productivity with impact, (ii) the necessary data is easy to access at the Thomson ISI Web of Science database, (iii) it is not sensitive to extreme values, (iv) it is hard to inflate. However, this index remains sensitive to the research field. It is even difficult to compare researchers from different areas within a given discipline. Further, since h is an integer number, many researchers may have the same index h, so that discriminating or listing them demands further indexes.

The ideas of optimization of learning algorithms in Artificial Neural Networks are reviewed empha... more The ideas of optimization of learning algorithms in Artificial Neural Networks are reviewed emphasizing generic properties and the online implementations are interpreted from a biological perspective. A simple model of the relevant subsidiary variables needed to improve learning in artificial feedforward networks and the 'time ordering' of the appearance of the respective information processing systems is proposed. We discuss the possibility that these results might be relevant in other contexts, not being restricted to the simple models from which they stem. The analysis of a few examples, which range from the lowest cellular scale to the macroscopic level, suggests that similar ideas could be applied to biological systems.

arXiv: Adaptation and Self-Organizing Systems, 2018

In the last decade, several models with network adaptive mechanisms (link deletion-creation, dyna... more In the last decade, several models with network adaptive mechanisms (link deletion-creation, dynamic synapses, dynamic gains) have been proposed as examples of self-organized criticality (SOC) to explain neuronal avalanches. However, all these systems present stochastic oscillations hovering around the critical region that are incompatible with standard SOC. This phenomenology has been called self-organized quasi-criticality (SOqC). Here we make a linear stability analysis of the mean field fixed points of two SOqC systems: a fully connected network of discrete time stochastic spiking neurons with firing rate adaptation produced by dynamic neuronal gains and an excitable cellular automata with depressing synapses. We find that the fixed point corresponds to a stable focus that loses stability at criticality. We argue that when this focus is close to become indifferent, demographic noise can elicit stochastic oscillations that frequently fall into the absorbing state. This mechanism ...

International Journal of Bifurcation and Chaos, 1996

We introduce a simple generalization of graded response formal neurons which presents very comple... more We introduce a simple generalization of graded response formal neurons which presents very complex behavior. Phase diagrams in full parameter space are given, showing regions with fixed points, periodic, quasiperiodic and chaotic behavior. These diagrams also represent the possible time series learnable by the simplest feed-forward network, a two input single-layer perceptron. This simple formal neuron (‘dynamical perceptron’) behaves as an excitable ele ment with characteristics very similar to those appearing in more complicated neuron models like FitzHugh-Nagumo and Hodgkin-Huxley systems: natural threshold for action potentials, dampened subthreshold oscillations, rebound response, repetitive firing under constant input, nerve blocking effect etc. We also introduce an ‘adaptive dynamical perceptron’ as a simple model of a bursting neuron of Rose-Hindmarsh type. We show that networks of such elements are interesting models which lie at the interface of neural networks, coupled ma...

International Journal of Astrobiology, 2018

Non-homogeneous fractal-like colonization processes, where the cluster of visited sites has large... more Non-homogeneous fractal-like colonization processes, where the cluster of visited sites has large voids and grows slowly, could explain the negative results of Search for Extraterrestrial Intelligence (SETI) preserving the possibility of a galactic spanning civilization. Here we present a generalized invasion percolation model to illustrate a minimal colonization process with large voids and delayed colonization. Spatial correlation between unvisited sites, in the form of large empty regions, suggests that to search civilizations in the Sun neighbourhood may be a misdirected SETI strategy. A weaker form of the Fermi Paradox also suggests this last conclusion.

We study a simple map as a minimal model of excitable cells. The map has two fast variables which... more We study a simple map as a minimal model of excitable cells. The map has two fast variables which mimic the behavior of class I neurons, undergoing a sub-critical Hopf bifurcation. Adding a third slow variable allows the system to present bursts and other interesting biological behaviors. Bifurcation lines which locate the excitability region are obtained for different planes in parameter space.

We study the problem of determining the Hamiltonian of a fully connected Ising Spin Glass of N un... more We study the problem of determining the Hamiltonian of a fully connected Ising Spin Glass of N units from a set of measurements, whose sizes needs to be O(N 2 ) bits. The student-teacher scenario, used to study learning in feed-forward neural networks, is here extended to spin systems with arbitrary couplings. The set of measurements consists of data about the local minima of the rugged energy landscape. We compare simulations and analytical approximations for the resulting learning curves obtained by using different algorithms. Key words: neural networks, generalization, spin glasses, inverse problems, on-line learning. PACS number : 07.05.Mh, 84.35.+i, 87.10+e, 02.50-r, 05.90+m. 1 Introduction The study of the dynamics or statistical properties of a system usually consists in making predictions of its behavior based on assumed microscopic laws such as, for example, using knowledge about its Hamiltonian. However, ill posed and inverse problems can be found in a vast array of areas....

In a landscape composed of N randomly distributed sites in Euclidean space, a walker (“tourist”) ... more In a landscape composed of N randomly distributed sites in Euclidean space, a walker (“tourist”) goes to the nearest one that has not been visited in the last τ steps. This procedure leads to trajectories composed of a transient part and a final cyclic attractor of period p. The tourist walk presents a simple scaling with respect to τ and can be performed in a wide range of networks that can be viewed as ordinal neighborhood graphs. As an example, we show that graphs defined by thesaurus dictionaries share some of the statistical properties of low dimensional (d = 2) Euclidean graphs and are easily distinguished from random link networks which correspond to the d → ∞ limit. This approach furnishes complementary information to the usual clustering coefficient and mean minimum separation length. 1

The training of multilayered neural networks in the presence of different types of noise is studi... more The training of multilayered neural networks in the presence of different types of noise is studied. We consider the learning of realizable rules in nonoverlapping architectures. Achieving optimal generalization depends on the knowledge of the noise level, however its misestimation may lead to partial or complete loss of the generalization ability. We demonstrate this effect in the framework of online learning and present the results in terms of noise robustness phase diagrams. While for additive (weight) noise the robustness properties depend on the architecture and size of the networks, this is not so for multiplicative (output) noise. In this case we nd a universal behaviour independent of the machine size for both the tree parity and committee machines.

Physical Review E, 1997

We consider the synchronous updating of a fully connected Ising neural network with separable but... more We consider the synchronous updating of a fully connected Ising neural network with separable but asymmetric couplings. In the thermodynamic limit, and away from saturation, it is possible to write a nonlinear mapping for the time evolution of the macroscopic order parameters. A detailed analysis of this mapping is performed for a simple case, with p=2 stored patterns. The dynamical phase diagram, in terms of the degree of noise and the parameters of the embedding matrix, displays a rich structure of locked regions into different cycles, in association with nonstandard Farey trees. In some regions of the dynamical phase diagram, we show the coexistence of two different Farey sequences, giving rise to the overlapping of several locked regions.

A random-neighbor extremal stick-slip model is introduced. In the thermodynamic limit, the distri... more A random-neighbor extremal stick-slip model is introduced. In the thermodynamic limit, the distribution of states has a simple analytical form and the mean avalanche size, as a function of the coupling parameter, is exactly calculable. The system is critical only at a special point Jc in coupling parameter space. However, the critical region around this point, where approximate scale invariance holds, is very large, suggesting a mechanism for explaining the ubiquity of power laws in Nature.

Neurocomputing, 2001

We propose a simple map (a dynamical system with discrete time) as a minimal formal model of exci... more We propose a simple map (a dynamical system with discrete time) as a minimal formal model of excitable and bursting cells. The map has two fast variables and a single slow one and presents all the usual behavior of excitable cells like fast spiking, regular spiking, bursting, plateau action potentials and adaptation phenomena. The simplicity of the map enables us

We review the application of Statistical Mechanics methods to the study of online learning of a d... more We review the application of Statistical Mechanics methods to the study of online learning of a drifting concept in the limit of large systems. The model where a feed-forward network learns from examples generated by a time dependent teacher of the same architecture is analyzed. The best possible generalization ability is determined exactly, through the use of a variational method. The constructive variational method also suggests a learning algorithm. It depends, however, on some unavailable quantities, such as the present performance of the student. The construction of estimators for these quantities permits the implementation of a very effective, highly adaptive algorithm. Several other algorithms are also studied for comparison with the optimal bound and the adaptive algorithm, for different types of time evolution of the rule.

PLoS Computational Biology, 2009

Since the first experimental evidences of active conductances in dendrites, most neurons have bee... more Since the first experimental evidences of active conductances in dendrites, most neurons have been shown to exhibit dendritic excitability through the expression of a variety of voltage-gated ion channels. However, despite experimental and theoretical efforts undertaken in the past decades, the role of this excitability for some kind of dendritic computation has remained elusive. Here we show that, owing to very general properties of excitable media, the average output of a model of an active dendritic tree is a highly non-linear function of its afferent rate, attaining extremely large dynamic ranges (above 50 dB). Moreover, the model yields double-sigmoid response functions as experimentally observed in retinal ganglion cells. We claim that enhancement of dynamic range is the primary functional role of active dendritic conductances. We predict that neurons with larger dendritic trees should have larger dynamic range and that blocking of active conductances should lead to a decrease in dynamic range.

Recently Hirsch [1,2] has proposed a new scalar index h to quantify individual's scientific resea... more Recently Hirsch [1,2] has proposed a new scalar index h to quantify individual's scientific research impact. A researcher with index h has h papers with at least h citations. This index has several advantages: (i) it combines productivity with impact, (ii) the necessary data is easy to access at the Thomson ISI Web of Science database, (iii) it is not sensitive to extreme values, (iv) it is hard to inflate. However, this index remains sensitive to the research field. It is even difficult to compare researchers from different areas within a given discipline. Further, since h is an integer number, many researchers may have the same index h, so that discriminating or listing them demands further indexes.

The ideas of optimization of learning algorithms in Artificial Neural Networks are reviewed empha... more The ideas of optimization of learning algorithms in Artificial Neural Networks are reviewed emphasizing generic properties and the online implementations are interpreted from a biological perspective. A simple model of the relevant subsidiary variables needed to improve learning in artificial feedforward networks and the 'time ordering' of the appearance of the respective information processing systems is proposed. We discuss the possibility that these results might be relevant in other contexts, not being restricted to the simple models from which they stem. The analysis of a few examples, which range from the lowest cellular scale to the macroscopic level, suggests that similar ideas could be applied to biological systems.

arXiv: Adaptation and Self-Organizing Systems, 2018

In the last decade, several models with network adaptive mechanisms (link deletion-creation, dyna... more In the last decade, several models with network adaptive mechanisms (link deletion-creation, dynamic synapses, dynamic gains) have been proposed as examples of self-organized criticality (SOC) to explain neuronal avalanches. However, all these systems present stochastic oscillations hovering around the critical region that are incompatible with standard SOC. This phenomenology has been called self-organized quasi-criticality (SOqC). Here we make a linear stability analysis of the mean field fixed points of two SOqC systems: a fully connected network of discrete time stochastic spiking neurons with firing rate adaptation produced by dynamic neuronal gains and an excitable cellular automata with depressing synapses. We find that the fixed point corresponds to a stable focus that loses stability at criticality. We argue that when this focus is close to become indifferent, demographic noise can elicit stochastic oscillations that frequently fall into the absorbing state. This mechanism ...

International Journal of Bifurcation and Chaos, 1996

We introduce a simple generalization of graded response formal neurons which presents very comple... more We introduce a simple generalization of graded response formal neurons which presents very complex behavior. Phase diagrams in full parameter space are given, showing regions with fixed points, periodic, quasiperiodic and chaotic behavior. These diagrams also represent the possible time series learnable by the simplest feed-forward network, a two input single-layer perceptron. This simple formal neuron (‘dynamical perceptron’) behaves as an excitable ele ment with characteristics very similar to those appearing in more complicated neuron models like FitzHugh-Nagumo and Hodgkin-Huxley systems: natural threshold for action potentials, dampened subthreshold oscillations, rebound response, repetitive firing under constant input, nerve blocking effect etc. We also introduce an ‘adaptive dynamical perceptron’ as a simple model of a bursting neuron of Rose-Hindmarsh type. We show that networks of such elements are interesting models which lie at the interface of neural networks, coupled ma...

International Journal of Astrobiology, 2018

Non-homogeneous fractal-like colonization processes, where the cluster of visited sites has large... more Non-homogeneous fractal-like colonization processes, where the cluster of visited sites has large voids and grows slowly, could explain the negative results of Search for Extraterrestrial Intelligence (SETI) preserving the possibility of a galactic spanning civilization. Here we present a generalized invasion percolation model to illustrate a minimal colonization process with large voids and delayed colonization. Spatial correlation between unvisited sites, in the form of large empty regions, suggests that to search civilizations in the Sun neighbourhood may be a misdirected SETI strategy. A weaker form of the Fermi Paradox also suggests this last conclusion.

We study a simple map as a minimal model of excitable cells. The map has two fast variables which... more We study a simple map as a minimal model of excitable cells. The map has two fast variables which mimic the behavior of class I neurons, undergoing a sub-critical Hopf bifurcation. Adding a third slow variable allows the system to present bursts and other interesting biological behaviors. Bifurcation lines which locate the excitability region are obtained for different planes in parameter space.

We study the problem of determining the Hamiltonian of a fully connected Ising Spin Glass of N un... more We study the problem of determining the Hamiltonian of a fully connected Ising Spin Glass of N units from a set of measurements, whose sizes needs to be O(N 2 ) bits. The student-teacher scenario, used to study learning in feed-forward neural networks, is here extended to spin systems with arbitrary couplings. The set of measurements consists of data about the local minima of the rugged energy landscape. We compare simulations and analytical approximations for the resulting learning curves obtained by using different algorithms. Key words: neural networks, generalization, spin glasses, inverse problems, on-line learning. PACS number : 07.05.Mh, 84.35.+i, 87.10+e, 02.50-r, 05.90+m. 1 Introduction The study of the dynamics or statistical properties of a system usually consists in making predictions of its behavior based on assumed microscopic laws such as, for example, using knowledge about its Hamiltonian. However, ill posed and inverse problems can be found in a vast array of areas....

In a landscape composed of N randomly distributed sites in Euclidean space, a walker (“tourist”) ... more In a landscape composed of N randomly distributed sites in Euclidean space, a walker (“tourist”) goes to the nearest one that has not been visited in the last τ steps. This procedure leads to trajectories composed of a transient part and a final cyclic attractor of period p. The tourist walk presents a simple scaling with respect to τ and can be performed in a wide range of networks that can be viewed as ordinal neighborhood graphs. As an example, we show that graphs defined by thesaurus dictionaries share some of the statistical properties of low dimensional (d = 2) Euclidean graphs and are easily distinguished from random link networks which correspond to the d → ∞ limit. This approach furnishes complementary information to the usual clustering coefficient and mean minimum separation length. 1

The training of multilayered neural networks in the presence of different types of noise is studi... more The training of multilayered neural networks in the presence of different types of noise is studied. We consider the learning of realizable rules in nonoverlapping architectures. Achieving optimal generalization depends on the knowledge of the noise level, however its misestimation may lead to partial or complete loss of the generalization ability. We demonstrate this effect in the framework of online learning and present the results in terms of noise robustness phase diagrams. While for additive (weight) noise the robustness properties depend on the architecture and size of the networks, this is not so for multiplicative (output) noise. In this case we nd a universal behaviour independent of the machine size for both the tree parity and committee machines.

Physical Review E, 1997

We consider the synchronous updating of a fully connected Ising neural network with separable but... more We consider the synchronous updating of a fully connected Ising neural network with separable but asymmetric couplings. In the thermodynamic limit, and away from saturation, it is possible to write a nonlinear mapping for the time evolution of the macroscopic order parameters. A detailed analysis of this mapping is performed for a simple case, with p=2 stored patterns. The dynamical phase diagram, in terms of the degree of noise and the parameters of the embedding matrix, displays a rich structure of locked regions into different cycles, in association with nonstandard Farey trees. In some regions of the dynamical phase diagram, we show the coexistence of two different Farey sequences, giving rise to the overlapping of several locked regions.

A random-neighbor extremal stick-slip model is introduced. In the thermodynamic limit, the distri... more A random-neighbor extremal stick-slip model is introduced. In the thermodynamic limit, the distribution of states has a simple analytical form and the mean avalanche size, as a function of the coupling parameter, is exactly calculable. The system is critical only at a special point Jc in coupling parameter space. However, the critical region around this point, where approximate scale invariance holds, is very large, suggesting a mechanism for explaining the ubiquity of power laws in Nature.

Neurocomputing, 2001

We propose a simple map (a dynamical system with discrete time) as a minimal formal model of exci... more We propose a simple map (a dynamical system with discrete time) as a minimal formal model of excitable and bursting cells. The map has two fast variables and a single slow one and presents all the usual behavior of excitable cells like fast spiking, regular spiking, bursting, plateau action potentials and adaptation phenomena. The simplicity of the map enables us

We review the application of Statistical Mechanics methods to the study of online learning of a d... more We review the application of Statistical Mechanics methods to the study of online learning of a drifting concept in the limit of large systems. The model where a feed-forward network learns from examples generated by a time dependent teacher of the same architecture is analyzed. The best possible generalization ability is determined exactly, through the use of a variational method. The constructive variational method also suggests a learning algorithm. It depends, however, on some unavailable quantities, such as the present performance of the student. The construction of estimators for these quantities permits the implementation of a very effective, highly adaptive algorithm. Several other algorithms are also studied for comparison with the optimal bound and the adaptive algorithm, for different types of time evolution of the rule.