João F. Henriques (original) (raw)
Hi, my name is João F. Henriques. (Sounds a bit like “joo-au” in English.) I like to work in the convex hull of machine learning, deep learning and computer vision. Perhaps my most well-known works are on visual tracking, but I have many favourite topics: robotics, AI safety, 3D (NeRFs/splats), and optimisation.
My talented DPhil students:
Marian Longa · Tim Franzmeyer · Dominik Kloepfer · Yash Bhalgat · Shivani Mall · Lorenza Prospero · Mark Eid
(Graduated: Xu Ji · Mandela Patrick · Shu Ishida · Andreea Oncescu)
Preprints
Sneak peek at upcoming research
Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Image
S. Szymanowicz, E. Insafutdinov, C. Zheng, D. Campbell, J. F. Henriques, C. Rupprecht, A. Vedaldi
arXiv, 2024
3D-Aware Instance Segmentation and Tracking in Egocentric Videos
Y. Bhalgat, V. Tschernezki, I. Laina, J. F. Henriques, A. Vedaldi, A. Zisserman
arXiv, 2024
SOAP-RL: Sequential Option Advantage Propagation for Reinforcement Learning in POMDP Environments
S. Ishida, J. F. Henriques
arXiv, 2024
RapidVol: Rapid Reconstruction of 3D Ultrasound Volumes from Sensorless 2D Scans
M. C. Eid, P. Yeung, M. K. Wyburd, J. F. Henriques, A. I. L. Namburete
arXiv, 2024
GST: Precise 3D Human Body from a Single Image with Gaussian Splatting Transformers
L. Prospero, A. Hamdi, J. F. Henriques, C. Rupprecht
arXiv, 2024
Research
Publications, talks and source-code
Filter by topic
All topics Computer vision Reinforcement learning Robotics 3D vision Multimodal and video ML optimization Meta-learning Object tracking Language modeling Audio Transformation invariance
Unsupervised Object Detection with Theoretical Guarantees
M. Longa, J. F. Henriques
NeurIPS, 2024
Interpretable Representation Learning from Videos using Nonlinear Priors
M. Longa, J. F. Henriques
BMVC, 2024
Rapid Motor Adaptation for Robotic Manipulator Arms
Y. Liang, K. Ellis, J. F. Henriques
CVPR, 2024
N2F2: Hierarchical Scene Understanding with Nested Neural Feature Fields
Y. Bhalgat, I. Laina, J. F. Henriques, A. Zisserman, A. Vedaldi
ECCV, 2024
Select to Perfect: Imitating desired behavior from large multi-agent data
T. Franzmeyer, E. Elkind, P. Torr, J. Foerster, J. F. Henriques
ICLR, 2024
Illusory Attacks: Information-theoretic detectability matters in adversarial attacks
T. Franzmeyer, S. McAleer, J. F. Henriques, J. Foerster, P. Torr, A. Bibi, C. Schroeder de Witt
ICLR, 2024
T. Franzmeyer, A. Shtedritski, S. Albanie, P. Torr, J. F. Henriques, J. N. Foerster
ACL, 2024
SCENES: Subpixel Correspondence Estimation with Epipolar Supervision
D. Kloepfer, D. Campbell, J. F. Henriques
3DV, 2024
A Sound Approach: Using Large Language Models to generate audio descriptions for egocentric text-audio retrieval
A. Oncescu, J. F. Henriques, A. Zisserman, S. Albanie, A. S. Koepke
ICASSP, 2024
Dissecting Temporal Understanding in Text-to-Audio Retrieval
A. Oncescu, J. F. Henriques, A. S. Koepke
ACM International Conference on Multimedia, 2024
Text2Loc: 3D Point Cloud Localization from Natural Language
Y. Xia, L. Shi, Z. Ding, J. F. Henriques, D. Cremers
CVPR, 2024
LangProp: A code optimization framework using Large Language Models applied to driving
S. Ishida, G. Corrado, G. Fedoseev, H. Yeo, L. Russell, J. Shotton, J. F. Henriques, A. Hu
ICLR Workshop on LLM Agents, 2024
Neural Fields for Co-Reconstructing 3D Objects from Incidental 2D Data
D. Campbell, E. Insafutdinov, J. F. Henriques, A. Vedaldi
CVPR Workshop on Neural Rendering Intelligence, 2024
Contrastive Lift: 3D Object Instance Segmentation by Slow-Fast Contrastive Fusion
Y. Bhalgat, I. Laina, J. F. Henriques, A. Zisserman, A. Vedaldi
NeurIPS, 2023
LoCUS: Learning Multiscale 3D-consistent Features from Posed Images
D. Kloepfer, D. Campbell, J. F. Henriques
ICCV, 2023
RbA: Segmenting Unknown Regions Rejected by All
N. Nayal, M. Yavuz, J. F. Henriques, F. Güney
ICCV, 2023
CASSPR: Cross Attention Single Scan Place Recognition
Y. Xia, M. Gladkova, R. Wang, Q. Li, U. Stilla, J. F. Henriques, D. Cremers
ICCV, 2023
A Light Touch Approach to Teaching Transformers Multi-view Geometry
Y. Bhalgat, J. F. Henriques, A. Zisserman
CVPR, 2023
Generalised Lookahead Optimiser
C. Oncescu, J. Valmadre, J. F. Henriques
Tiny Papers at ICLR, 2023
Learn what matters: cross-domain imitation learning with task-relevant embeddings
T. Franzmeyer, P. Torr, J. F. Henriques
NeurIPS, 2022
SNeS: Learning Probably Symmetric Neural Surfaces from Incomplete Data
E. Insafutdinov, D. Campbell, J. F. Henriques, A. Vedaldi
ECCV, 2022
We augment neural radiance fields to render views of partially-symmetric objects that are not seen in the data, such as when seeing a car from just one side. Since shadows and reflections break object symmetry, in the process we decompose scenes into geometry, light and material properties.
Towards real-world navigation with deep differentiable planners
S. Ishida, J. F. Henriques
CVPR, 2022
We train robot agents to explore and seek semantic goals, without hazardous trial-and-error, by using only safe demonstrations. We achieve this by extending and improving on Value Iteration Networks, enabling robots to cope even with mazes with a high branching factor.
Audio retrieval with natural language queries: A benchmark study
A. S. Koepke, A. Oncescu, J. Henriques, Z. Akata, S. Albanie
IEEE Transactions on Multimedia, 2022
Illusionary Attacks on Sequential Decision Makers and Countermeasures
T. Franzmeyer, J. F. Henriques, J. N. Foerster, P. H. Torr, A. Bibi, C. S. de Witt
arXiv, 2022
Keeping your eye on the ball: Trajectory attention in video transformers
M. Patrick, D. Campbell, Y. M. Asano, I. M. F. Metze, C. Feichtenhofer, A. Vedaldi, J. F. Henriques
NeurIPS, 2021 (oral presentation)
We improve video transformers (e.g. for action recognition) by encouraging attention pooling over motion paths. We also reduce the quadratic computational complexity of attention to linear, with a rigorous probabilistic approximation based on orthogonal prototypes.
Multi-modal self-supervision from generalized data transformations
M. Patrick, Y. M. Asano, P. Kuznetsova, R. Fong, J. F. Henriques, G. Zweig, A. Vedaldi
ICCV, 2021
Most contrastive self-supervised methods learn representations that are distinctive to individual examples, and invariant to several other factors. We propose a framework to systematically evaluate valid combinations of distinctive and invariant factors, yielding superior performance in many multi-modal learning tasks.
Space-time crop & Attend: Improving cross-modal video representation learning
M. Patrick, Y. M. Asano, P. Huang, I. Misra, F. Metze, J. F. Henriques, A. Vedaldi
ICCV, 2021
J. Jiao, J. F. Henriques
BMVC, 2021
Support-set bottlenecks for video-text representation learning
M. Patrick, P. Huang, Y. Asano, F. Metze, A. G. Hauptmann, J. F. Henriques, A. Vedaldi
ICLR, 2021
We investigate noise-contrastive learning of video-text neural networks. We find that learning to reconstruct video captions with video retrieval as a representational bottleneck yields better semantic representations.
Audio retrieval with natural language queries
A. Oncescu, A. S. Koepke, J. F. Henriques, Z. Akata, S. Albanie
Interspeech, 2021 (nominated for best student paper award)
Creating a content-based audio search engine. Similar to Google Images, but for audio instead.
Automatic Recall Machines: Internal replay, continual learning and the brain
X. Ji, J. Henriques, T. Tuytelaars, A. Vedaldi
NeurIPS Workshops, 2020
Avoiding catastrophic forgetting with context-sensitive generative recall, inspired by biological memory.
360º camera alignment via segmentation
B. Davidson, M. S. Alvi, J. F. Henriques
ECCV, 2020
P. Martins, J. F. Henriques, J. Batista
IJCV, 2020
Bayesian constrained local models revisited
P. Martins, J. F. Henriques, R. Caseiro, J. Batista
TPAMI, 2016
Learning feed-forward one-shot learners
L. Bertinetto, J. F. Henriques, J. Valmadre, P. H. S. Torr, A. Vedaldi
NeurIPS, 2016
Early work on meta-learning for one-shot learning, where a deep network predicts the parameters of another network, given a few examples of a classification task.
Beyond the shortest path: Unsupervised domain adaptation by sampling subspaces along the spline flow
R. Caseiro, P. Martins, J. F. Henriques, J. Batista
CVPR, 2015
Likelihood-enhanced bayesian constrained local models
P. Martins, R. Caseiro, J. F. Henriques, J. Batista
ICIP, 2014 (top 10% of accepted papers)
Rolling riemannian manifolds to solve the multi-class classification problem
R. Caseiro, P. Martins, J. F. Henriques, J. Carreira, J. Batista
CVPR, 2013 (oral presentation)
Semi-intrinsic mean shift on riemannian manifolds
R. Caseiro, J. F. Henriques, P. Martins, J. Batista
ECCV, 2012
Discriminative bayesian active shape models
P. Martins, R. Caseiro, J. F. Henriques, J. Batista
ECCV, 2012
Let the shape speak: Face alignment using conjugate priors
P. Martins, R. Caseiro, J. F. Henriques, J. Batista
BMVC, 2012 (oral presentation)
A nonparametric riemannian framework on tensor field with application to foreground segmentation
R. Caseiro, P. Martins, J. F. Henriques, J. Batista
Pattern Recognition, 2012
A nonparametric riemannian framework on tensor field with application to foreground segmentation
R. Caseiro, J. F. Henriques, P. Martins, J. Batista
ICCV, 2011
Tracking in streamed video by updating globally optimal matchings
J. F. Henriques, R. Caseiro, J. Batista
ICIP, 2010
R. Caseiro, J. F. Henriques, J. Batista
ICIP, 2010
More
Research-related
Workshops on Preregistration
An alternative publication model for machine learning research
Preregistration separates the generation and confirmation of hypotheses:
Come up with an exciting research question
Write a paper proposal without confirmatory experiments
After the paper is accepted, run the experiments and report your results
There are several advantages in this model: 1) A healthy mix of positive and negative results; 2) Reasonable ideas that don’t work still get published, avoiding wasteful replications; 3) Papers are evaluated on the basis of scientific interest, not whether they achieve the best results; 4) It is easier to plan research; and 5) results are statistically stronger. Check the pages below for more information, including talks and preregistered machine learning papers.
OverBoard
A pure Python dashboard for monitoring deep learning experiments
OverBoard is a lightweight yet powerful dashboard to monitor your experiments. It includes:
A table of hyper-parameters with Python-syntax filtering
Multiple views of the same data (i.e. custom X/Y axes)
Hyper-parameter visualisation (i.e. bubble plots)
Percentile intervals for multiple runs (i.e. shaded plots)
Custom visualisations (tensors and any custom plot with familiar MatPlotLib syntax)
Fast client-side rendering (the training code is kept lightweight)
You can install it with: pip install overboard
Its only dependences are PyQtGraph (conda install pyqt pyqtgraph -c anaconda) and Python 3.
Fun
Not mutually-exclusive with research
S. Albanie, J. Thewlis, S. Ehrhardt, J. F. Henriques
Narrowly missed SIGBOVIK, 2019
The most end-to-end network ever proposed, and a sunnier alternative to cloud computing. Narrowly missing the deadline for SIGBOVIK 2019, received the Most timely paper award at SIGBOVIK 2020.
Stopping GAN violence: Generative unadversarial networks
S. Albanie, S. Ehrhardt, J. F. Henriques
SIGBOVIK, 2017
An attempt to end the madness of pitting network-against-network (GAN training). This paper achieved moderate success on social media, which meant that all subsequent papers were doomed to obscurity (but that didn't stop us).
Surprisingly, there is an entirely serious paper that experiments with generative unadversarial training and credits our joke paper as the inspiration! (With full knowledge that it is not to be taken seriously of course.) Mission accomplished.
