José Antonio Martín H. | Universidad Complutense de Madrid (original) (raw)

José Antonio Martín H.

I obtained my PhD in Computer Science and Artificial Intelligence (March 2009) from the Technical University of Madrid (UPM – Universidad Politécnica de Madrid). My PhD thesis work, “Studies on adaptive systems with applications to autonomous robots and intelligent agents”, was oriented to create a model about different phenomena observed in natural and artificial systems: adaptation, anticipation, rationality and intelligence, from a system oriented point of view, see for instance Wikipedia: Adaptive System, “The Law of Adaptation”. I have also done research on basic psychology (under a second PhD program) and presented a master thesis about the psychological phenomenon of equivalence class formation.

As a result of my research activities, I have gained a deep understanding and expertise in many different subfields such as:

* Autonomous robot's controllers.
* Development and application of new algorithms for sequential decision making, e.g., k-armed bandits, contextual bandits, Reinforcement Learning algorithms.
* Algorithmic randomness and model-learning applied to sequence/graph prediction tasks.
* Bayesian classification and prediction methods.
* Different forms of Neuro-Evolution, Neuro-Controllers and Incremental learning in Artificial Neural Networks.
* Evolutionary and other forms of stochastic global optimization methods.
* Machine Perception, e.g. Computer Vision, Clustering, Classification and Filtering.
* Graph theory and combinatorial optimization algorithms.

I have obtained two first places and two second places (JAMH Team) at the Second and Third Reinforcement Learning Competitions (ICML2008 in Helsinki and ICML2009 in Montreal)

I have also participated in the Exploration and Exploitation 3 Challenge (ICML2012 in Edinburgh) on Yahoo! dataset (to serve-recommend news articles on a web site). I obtained the 2th place (very little gap between the winner and the second place) from 38 teams from around the world.
Especialidades

Computer science: Reinforcement Learning, Evolutionary Computation, Stochastic Optimization, Neural Networks, Combinatorial Optimization, Graph Theory

less

Uploads

Papers by José Antonio Martín H.

Research paper thumbnail of Reinforcement Learning in System Identification

arXiv (Cornell University), Dec 14, 2022

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Search and retrieval of plasma wave forms: Structural pattern recognition approach

Abstract Databases for fusion experiments are designed to store several million wave forms. Tempo... more Abstract Databases for fusion experiments are designed to store several million wave forms. Temporal evolution signals show the same patterns under the same plasma conditions and, therefore, pattern recognition techniques can allow identification of similar plasma behaviors. Further developments in this area must be focused on four aspects: large databases, feature extraction, similarity function, and search/retrieval efficiency. This article describes an approach for pattern searching within wave forms.

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Solving Hard Computational Problems Efficiently: Asymptotic Parametric Complexity 3-Coloring Algorithm

PLoS ONE, Jan 14, 2013

Many practical problems in almost all scientific and technological disciplines have been classifi... more Many practical problems in almost all scientific and technological disciplines have been classified as computationally hard (NP-hard or even NP-complete). In life sciences, combinatorial optimization problems frequently arise in molecular biology, e.g., genome sequencing; global alignment of multiple genomes; identifying siblings or discovery of dysregulated pathways. In almost all of these problems, there is the need for proving a hypothesis about certain property of an object that can be present if and only if it adopts some particular admissible structure (an NP-certificate) or be absent (no admissible structure), however, none of the standard approaches can discard the hypothesis when no solution can be found, since none can provide a proof that there is no admissible structure. This article presents an algorithm that introduces a novel type of solution method to “efficiently” solve the graph 3-coloring problem; an NP-complete problem. The proposed method provides certificates (proofs) in both cases: present or absent, so it is possible to accept or reject the hypothesis on the basis of a rigorous proof. It provides exact solutions and is polynomial-time (i.e., efficient) however parametric. The only requirement is sufficient computational power, which is controlled by the parameter . Nevertheless, here it is proved that the probability of requiring a value of to obtain a solution for a random graph decreases exponentially: , making tractable almost all problem instances. Thorough experimental analyses were performed. The algorithm was tested on random graphs, planar graphs and 4-regular planar graphs. The obtained experimental results are in accordance with the theoretical expected results.

Bookmarks Related papers MentionsView impact

Research paper thumbnail of A computational model of the equivalence class formation psychological phenomenon

Innovations in Hybrid …, Jan 1, 2007

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Evolution of Neuro-controllers for Multi-link Robots

Innovations in Hybrid Intelligent Systems, Jan 1, 2007

A general method to learn the inverse kinematics of multi-link robots by means of neuro-controlle... more A general method to learn the inverse kinematics of multi-link robots by means of neuro-controllers is presented. We can find analytical solutions for the most used and known robots in the bibliography. However, these solutions are specific to a particular robot configuration and are not generally applicable to other robot morphologies. The proposed method is general in the sense that it is not dependant on the robot morphology. We base our method in the Evolutionary Computation paradigm for obtaining incrementally better neuro- ...

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Applying Reinforcement Learning to Multi-robot Team Coordination

Hybrid Artificial Intelligence Systems, Jan 1, 2008

Multi-robot systems are one of the most challenging problems in autonomous robots. Teams of homog... more Multi-robot systems are one of the most challenging problems in autonomous robots. Teams of homogeneous or heterogeneous robots must be able to solve complex tasks. Sometimes the tasks have a cooperative basis in which the global objective is shared by all the robots. In other situations, the robots can be different and even contradictory goals, defining a kind of competitive problems. The multi-robot systems domain is a perfect example in which the uncertainty and vagueness in sensor readings and robot odometry must be handled by ...

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Formalización de maniobras en robots con múltiples grados de libertad como sistemas multiagente

car.upm-csic.es

Bookmarks Related papers MentionsView impact

Research paper thumbnail of José Antonio Martín H., Hybridizing evolutionary computation and reinforcement learning for the design of almost universal controllers for autonomous …

Neurocomputing, Jan 1, 2009

Bookmarks Related papers MentionsView impact

Research paper thumbnail of A computational model of the equivalence class formation psychological phenomenon

Innovations in Hybrid …, Jan 1, 2007

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Evolution of Neuro-controllers for Multi-link Robots

Innovations in Hybrid Intelligent Systems, Jan 1, 2007

A general method to learn the inverse kinematics of multi-link robots by means of neuro-controlle... more A general method to learn the inverse kinematics of multi-link robots by means of neuro-controllers is presented. We can find analytical solutions for the most used and known robots in the bibliography. However, these solutions are specific to a particular robot configuration and are not generally applicable to other robot morphologies. The proposed method is general in the sense that it is not dependant on the robot morphology. We base our method in the Evolutionary Computation paradigm for obtaining incrementally better neuro- ...

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Applying Reinforcement Learning to Multi-robot Team Coordination

Hybrid Artificial Intelligence Systems, Jan 1, 2008

Multi-robot systems are one of the most challenging problems in autonomous robots. Teams of homog... more Multi-robot systems are one of the most challenging problems in autonomous robots. Teams of homogeneous or heterogeneous robots must be able to solve complex tasks. Sometimes the tasks have a cooperative basis in which the global objective is shared by all the robots. In other situations, the robots can be different and even contradictory goals, defining a kind of competitive problems. The multi-robot systems domain is a perfect example in which the uncertainty and vagueness in sensor readings and robot odometry must be handled by ...

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Formalización de maniobras en robots con múltiples grados de libertad como sistemas multiagente

car.upm-csic.es

Bookmarks Related papers MentionsView impact

Research paper thumbnail of José Antonio Martín H., Hybridizing evolutionary computation and reinforcement learning for the design of almost universal controllers for autonomous …

Neurocomputing, Jan 1, 2009

Bookmarks Related papers MentionsView impact

Research paper thumbnail of APPLICATION OF ORTHOGONAL VARIANT MOMENTS TO COMPUTER VISION

Computational Intelligence in …, Jan 1, 2008

ABSTRACT The Orthogonal Variant Moments (OVM) are proposed in this paper as a way characterizing ... more ABSTRACT The Orthogonal Variant Moments (OVM) are proposed in this paper as a way characterizing any function or signal in general. Our approach to the theory of visual perception is based on the study of the low level vision system by Orthogonal Variant Moments while most of the works on this field use invariant-moments. An application of this method to computer-vision proves the efficiency of this approach.

Bookmarks Related papers MentionsView impact

Research paper thumbnail of DYNAMIC GOAL COORDINATION IN PHYSICAL AGENTS

Bookmarks Related papers MentionsView impact

Research paper thumbnail of A fuzzy expressions processor with modifiers and hedges, conceptual design and testing

dia.fi.upm.es

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Hidden Edges in Graph Coloring

Citeseer

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Learning autonomous helicopter flight with evolutionary reinforcement learning

Computer Aided Systems Theory-EUROCAST …, Jan 1, 2009

Bookmarks Related papers MentionsView impact

Research paper thumbnail of A distributed reinforcement learning architecture for multi-link robots

4th International Conference on Informatics in Control, …, Jan 1, 2007

Bookmarks Related papers MentionsView impact

Research paper thumbnail of A model for the dynamic coordination of multiple competing goals

Journal of Experimental & …, Jan 1, 2009

A general framework for the problem of coordination of multiple competing goals in dynamic enviro... more A general framework for the problem of coordination of multiple competing goals in dynamic environments for physical agents is presented. This approach to goal coordination is a novel tool to incorporate a deep coordination ability to pure reactive agents. The framework presented is based on the notion of multi-objective optimisation. In this article we propose a kind of ‘aggregating functions’

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Reinforcement Learning in System Identification

arXiv (Cornell University), Dec 14, 2022

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Search and retrieval of plasma wave forms: Structural pattern recognition approach

Abstract Databases for fusion experiments are designed to store several million wave forms. Tempo... more Abstract Databases for fusion experiments are designed to store several million wave forms. Temporal evolution signals show the same patterns under the same plasma conditions and, therefore, pattern recognition techniques can allow identification of similar plasma behaviors. Further developments in this area must be focused on four aspects: large databases, feature extraction, similarity function, and search/retrieval efficiency. This article describes an approach for pattern searching within wave forms.

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Solving Hard Computational Problems Efficiently: Asymptotic Parametric Complexity 3-Coloring Algorithm

PLoS ONE, Jan 14, 2013

Many practical problems in almost all scientific and technological disciplines have been classifi... more Many practical problems in almost all scientific and technological disciplines have been classified as computationally hard (NP-hard or even NP-complete). In life sciences, combinatorial optimization problems frequently arise in molecular biology, e.g., genome sequencing; global alignment of multiple genomes; identifying siblings or discovery of dysregulated pathways. In almost all of these problems, there is the need for proving a hypothesis about certain property of an object that can be present if and only if it adopts some particular admissible structure (an NP-certificate) or be absent (no admissible structure), however, none of the standard approaches can discard the hypothesis when no solution can be found, since none can provide a proof that there is no admissible structure. This article presents an algorithm that introduces a novel type of solution method to “efficiently” solve the graph 3-coloring problem; an NP-complete problem. The proposed method provides certificates (proofs) in both cases: present or absent, so it is possible to accept or reject the hypothesis on the basis of a rigorous proof. It provides exact solutions and is polynomial-time (i.e., efficient) however parametric. The only requirement is sufficient computational power, which is controlled by the parameter . Nevertheless, here it is proved that the probability of requiring a value of to obtain a solution for a random graph decreases exponentially: , making tractable almost all problem instances. Thorough experimental analyses were performed. The algorithm was tested on random graphs, planar graphs and 4-regular planar graphs. The obtained experimental results are in accordance with the theoretical expected results.

Bookmarks Related papers MentionsView impact

Research paper thumbnail of A computational model of the equivalence class formation psychological phenomenon

Innovations in Hybrid …, Jan 1, 2007

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Evolution of Neuro-controllers for Multi-link Robots

Innovations in Hybrid Intelligent Systems, Jan 1, 2007

A general method to learn the inverse kinematics of multi-link robots by means of neuro-controlle... more A general method to learn the inverse kinematics of multi-link robots by means of neuro-controllers is presented. We can find analytical solutions for the most used and known robots in the bibliography. However, these solutions are specific to a particular robot configuration and are not generally applicable to other robot morphologies. The proposed method is general in the sense that it is not dependant on the robot morphology. We base our method in the Evolutionary Computation paradigm for obtaining incrementally better neuro- ...

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Applying Reinforcement Learning to Multi-robot Team Coordination

Hybrid Artificial Intelligence Systems, Jan 1, 2008

Multi-robot systems are one of the most challenging problems in autonomous robots. Teams of homog... more Multi-robot systems are one of the most challenging problems in autonomous robots. Teams of homogeneous or heterogeneous robots must be able to solve complex tasks. Sometimes the tasks have a cooperative basis in which the global objective is shared by all the robots. In other situations, the robots can be different and even contradictory goals, defining a kind of competitive problems. The multi-robot systems domain is a perfect example in which the uncertainty and vagueness in sensor readings and robot odometry must be handled by ...

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Formalización de maniobras en robots con múltiples grados de libertad como sistemas multiagente

car.upm-csic.es

Bookmarks Related papers MentionsView impact

Research paper thumbnail of José Antonio Martín H., Hybridizing evolutionary computation and reinforcement learning for the design of almost universal controllers for autonomous …

Neurocomputing, Jan 1, 2009

Bookmarks Related papers MentionsView impact

Research paper thumbnail of A computational model of the equivalence class formation psychological phenomenon

Innovations in Hybrid …, Jan 1, 2007

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Evolution of Neuro-controllers for Multi-link Robots

Innovations in Hybrid Intelligent Systems, Jan 1, 2007

A general method to learn the inverse kinematics of multi-link robots by means of neuro-controlle... more A general method to learn the inverse kinematics of multi-link robots by means of neuro-controllers is presented. We can find analytical solutions for the most used and known robots in the bibliography. However, these solutions are specific to a particular robot configuration and are not generally applicable to other robot morphologies. The proposed method is general in the sense that it is not dependant on the robot morphology. We base our method in the Evolutionary Computation paradigm for obtaining incrementally better neuro- ...

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Applying Reinforcement Learning to Multi-robot Team Coordination

Hybrid Artificial Intelligence Systems, Jan 1, 2008

Multi-robot systems are one of the most challenging problems in autonomous robots. Teams of homog... more Multi-robot systems are one of the most challenging problems in autonomous robots. Teams of homogeneous or heterogeneous robots must be able to solve complex tasks. Sometimes the tasks have a cooperative basis in which the global objective is shared by all the robots. In other situations, the robots can be different and even contradictory goals, defining a kind of competitive problems. The multi-robot systems domain is a perfect example in which the uncertainty and vagueness in sensor readings and robot odometry must be handled by ...

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Formalización de maniobras en robots con múltiples grados de libertad como sistemas multiagente

car.upm-csic.es

Bookmarks Related papers MentionsView impact

Research paper thumbnail of José Antonio Martín H., Hybridizing evolutionary computation and reinforcement learning for the design of almost universal controllers for autonomous …

Neurocomputing, Jan 1, 2009

Bookmarks Related papers MentionsView impact

Research paper thumbnail of APPLICATION OF ORTHOGONAL VARIANT MOMENTS TO COMPUTER VISION

Computational Intelligence in …, Jan 1, 2008

ABSTRACT The Orthogonal Variant Moments (OVM) are proposed in this paper as a way characterizing ... more ABSTRACT The Orthogonal Variant Moments (OVM) are proposed in this paper as a way characterizing any function or signal in general. Our approach to the theory of visual perception is based on the study of the low level vision system by Orthogonal Variant Moments while most of the works on this field use invariant-moments. An application of this method to computer-vision proves the efficiency of this approach.

Bookmarks Related papers MentionsView impact

Research paper thumbnail of DYNAMIC GOAL COORDINATION IN PHYSICAL AGENTS

Bookmarks Related papers MentionsView impact

Research paper thumbnail of A fuzzy expressions processor with modifiers and hedges, conceptual design and testing

dia.fi.upm.es

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Hidden Edges in Graph Coloring

Citeseer

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Learning autonomous helicopter flight with evolutionary reinforcement learning

Computer Aided Systems Theory-EUROCAST …, Jan 1, 2009

Bookmarks Related papers MentionsView impact

Research paper thumbnail of A distributed reinforcement learning architecture for multi-link robots

4th International Conference on Informatics in Control, …, Jan 1, 2007

Bookmarks Related papers MentionsView impact

Research paper thumbnail of A model for the dynamic coordination of multiple competing goals

Journal of Experimental & …, Jan 1, 2009

A general framework for the problem of coordination of multiple competing goals in dynamic enviro... more A general framework for the problem of coordination of multiple competing goals in dynamic environments for physical agents is presented. This approach to goal coordination is a novel tool to incorporate a deep coordination ability to pure reactive agents. The framework presented is based on the notion of multi-objective optimisation. In this article we propose a kind of ‘aggregating functions’

Bookmarks Related papers MentionsView impact

Research paper thumbnail of Contextual Bandits: Approximate Linear Bayes for Large Contexts

Contextual bandits, and in general informed decision making, can be studied in the general stocha... more Contextual bandits, and in general informed decision making, can be studied in the general stochastic/statistical setting by means of the conditional probability paradigm where Bayes' theorem plays a central role. However, when informed decisions have to be made considering very large contextual information or the information is contained in too many variables with large history of observations and the time to take decisions is critical, the exact calculation of the Bayes' rule, to produce the best decision given the available information, is unaffordable.

In this increasingly common setting some derivations and approximations to conditional probability and the Bayes' rule will progressively gain greater applicability. In this article, an algorithm able to handle large contextual information in the form of binary features for optimal decision making in contextual bandits is presented. The algorithm is analyzed with respect to its scalability in terms of the time required to select the best choice and the time required to update its policy.

Last but not least, we address the exploration and exploitation issue explaining, despite the incomputability of an optimal tradeoff, the way in which the proposed algorithm "naturally"' balances exploration and exploitation by using common sense.

Bookmarks Related papers MentionsView impact