Prof. dr. Michael S. Lew (original) (raw)

PhD, University of Illinois Urbana-Champaign (UIUC)

Research - short version

My research lies in the field of artificial intelligence with a primary focus on the area of deep learning. I develop new deep learning algorithms toward understanding large databases and libraries of information by learning synergistic methods using multiple modalities within multimedia including but not limited to vision, audio and text. The goal is to bring to light all of the information in the world and at the same time automatically be able to detect misinformation (e.g. fake news).

Research - longer version

My research is toward developing artificial intelligence (AI) and currently focuses on the paradigm of deep learning.

Artificial Intelligence is the most promising paradigm to help society. Major breakthroughs are expected in the near future in a numerous important areas of society (e.g. medicine, automatic self-driving vehicles, fake news detection, robotics, etc.) but only with access to very large datasets. In many areas, the datasets exist but scientists do not have access. An example I am often asked by medical researchers: what it would take to have expert level classifiers for well known diseases and cancers. My reply is usually that one needs access to a very large dataset of examples. These datasets exist already at many research hospitals around the world. However, the datasets can not be used for training the deep classifiers due to privacy reasons. In my opinion, while it is important to develop AI to detect diseases and cancers, it is even more important to protect human privacy rights. Currently, there is no easy answer for the dataset access issue and each country is wrestling with the tradeoff of privacy vs scientific advances in their own way.

My current research is exploring how to both find information and also how to detect false information. Both are necessary to a modern information retrieval system. Because humans want to express their search queries using high level human concepts, the grand challenge in multimedia retrieval has been "bridging" the semantic gap which is the gap between the high level concepts of humans and the low level features from multimedia. For a specific example, the problem of image classification is especially important because it can bridge the semantic gap by taking an input image and outputting human level concepts. If we can solve the semantic gap, then it would make all of the collected art/heritage, scientific, WWW and personal images searchable and accessible. Furthermore, there is major interest in explanation based modeling which seeks to give better understanding of the underlying reasons and causes involved in deep network decisions.

Misinformation is one of the key 21st century problems and manifests in many different ways. "Fake News" is perhaps the most well known type and has affected areas from medicine (dangerous treatments for COVID) to politics (unfounded claims of election fraud). Misinformation also occurs in more subtle ways such as intentionally misrepresenting someone's viewpoints such as taking a small part of a discussion and presenting it as the main viewpoint. The context is important. Other modern ways in which misinformation is happening is with misrepresentation of authorship (see ChatGPT) and plagiarism. Part of my current research examines methods of detecting fake information using deep learning.

Teaching/Courses

Conference Organization

Scientific Conference Program Committees

Representative Publications (over 180 peer-reviewed in ACM, IEEE, and LNCS)
Browse LIACS publications

Content-Based Multimedia Information Retrieval: State of the Art and Challenges, Citations: 2210 in Google Scholar - Cache Oct.2013

Michael S. Lew, Nicu Sebe, Chabane Djeraba, Ramesh Jain, ACM Transactions on Multimedia Computing, Communications, and Applications, vol. 2, issue 1, 2006, pp. 1-19.
-> New ideas and paradigms in multimedia information retrieval

The MIR FLICKR Retrieval Evaluation, Citations: 1655 in Google Scholar

M.S. Lew, et al., ACM International Conference on Multimedia Information Retrieval (MIR'08), Vancouver, Canada, 2008.
-> Discusses the important directions for future multimedia and computer vision test sets and introduces an open and consistently annotated image retrieval test set and evaluation approach

Deep learning for visual understanding, Citations: 2120 in Google Scholar

Y Guo, Y Liu, A Oerlemans, S Lao, S Wu, MS Lew, Neurocomputing, vol. 187, pp 27-48, 2016.
-> describes the new paradigms in deep learning. It is currently the most cited article in the journal Neurocomputing

Image Retrieval using Wavelet-based Salient Points,Citations: 209 in Google Scholar - Cache 1/2010

M.S. Lew, et al., Journal of Electronic Imaging, Vol. 10, No. 4, October, 2001, pp. 835-849.
-> Earlier version: "amongst the best submitted to the SPIE Conf. on Storage and Retrieval of Media Databases" - SPIE Program Committee

A review of semantic segmentation using deep neural networks, Citations: 372 in Google Scholar

M.S. Lew, et al., International Journal of Multimedia Information Retrieval, Springer, London, 2018.
-> New ideas and research directions in semantic segmentation using deep neural networks

Principles of Visual Information Retrieval, Citations: 273 in Google Scholar

Michael S. Lew, Springer-Verlag, London, ISBN 1-852333-381-2, January 2001.
-> Covers the state of the art in visual information retrieval. It has been used in more than 7 universities worldwide including Columbia University (USA), Tsinghua University (China), etc.

Next Generation Web Searches for Visual Content, Citations: 201 in Google Scholar - Cache 4/2010

Michael S. Lew, IEEE Computer, November, 2000, pp. 46-53.
-> Describes the ImageScape System which performs semantic multimedia searching on an index of over 25 million images covering the Web

Learning and Feature Selection in Stereo Matching, Citations: 172 in Google Scholar

Michael S. Lew, Thomas S. Huang, Kam W. Wong, IEEE Transactions on Pattern Analysis and Machine Intelligence on Learning in Computer Vision, September, 1994, pp. 869-881.
-> This paper was about learning which features would be best for matching. It also touches on a core interest of mine - to integrate methods from artificial intelligence into computer vision and image understanding.

The Distributed ASCI Supercomputer Project, Citations: 153 in Google Scholar - Cache

M. Lew, et al., Operating Systems Review, 2000,
-> Describes the Netherlands distributed supercomputer

Authentic Facial Expression Analysis, Citations: 470 in Google Scholar

M.S. Lew, et al., Image and Vision Computing, Vol. 25, No. 12, 2007, pp. 1856-1863.
-> Arguably the first evaluation on machine detection of authentic human emotions

Toward Improved Ranking Metrics, Citations: 155 in Google Scholar

M.S. Lew, et al., IEEE Transactions on Pattern Analysis and Machine Intelligence October, 2000, pp. 1132-1143.
-> Despite most papers assuming that noise is Gaussian, this paper shows that it is not!

Comparing salient point detectors, Citations: 157 in Google Scholar

M.S. Lew, et al., Pattern Recognition Letters, 24 (2003) 89\9696.
-> A nice evaluation of salient point techniques

The State of the Art in Image and Video Retrieval, Citations: 132 in Google Scholar

M. Lew, et al., Proceedings of the International Conference on Image and Video Retrieval (CIVR), Urbana, IL, 2003.
-> A VIR overview

Information Theory and Face Detection, Citations: 76 in Google Scholar

Michael S. Lew, Nies Huijsmans, Proceedings of the International Conference on Pattern Recognition, Vienna, Austria, August 25-30, 1996, pp. 601-605.
-> This paper contains one of the most surprising results I've seen

New trends and ideas in visual concept detection, Citations: 411 in Google Scholar

Mark Huiskes, Bart Thomee, Michael S. Lew, Proceedings of the ACM International Conference on Multimedia Information Retrieval (MIR), Philadelphia, USA, 2010.
-> Some insights into large scale semantic image retrieval

Emotion Recognition Using a Cauchy Naive Bayes Classifier, Citations: 194 in Google Scholar

M.S. Lew, et al., Proceedings of the International Conference on Pattern Recognition, Quebec, Canada, 2002, pp. 17-20.
-> This paper is an early work on automatic emotion recognition

Publications in 2004

Older Publications

Selected Past Research Milestones

- My first IEEE Trans PAMI article (in the special issue on Learning in Computer Vision), 1994

- My first position as Chair Full Professor at Tsinghua (first contract), 2003

- My first General co-Chair service for the International Conference on Image and Video Retrieval (CIVR), 2003

Recent Graduate Students (2016-)

Interesting Projects

Other Activities: Student Advising and Research Grants (Click Here)

Contact information

Main Contact:

**Telephone:**31-71-527-7034

**Fax:**31-71-527-6985

Postal Address: 

Leiden Institute of Advanced Computer Science
Leiden University
Niels Bohrweg 1
2333 CA Leiden
The Netherlands

Deep learning and multimedia retrieval

Deep Learning Classification demo and publications

Deep Learning Caption demo and publications

Publications (technical reports, preliminary work)

Content-based tag recommendation algorithms for unstructured data

Improving SIFT accuracy with use of perspective transforms

Improving the LSDh-tree for fast approximate nearest neighbor search

Information-Synthesis Network for Facial Landmarks Estimation

Preliminary Evaluation of CNN Classification by Objective Testers

Sub-Image Search Engine

Image Similarity Using Color Histograms

Rating Inference

PhD at Leiden University

PhD Ceremony

PhD Protocol During Defense

PhD at Leiden Guidelines

PhD Training and Compulsory Courses

ASCI Research School

CSC and Leiden

PhD Defense Rules and Guidelines - Cache

Bus

Bus - Corpus

Bus/Train Info

Bus - Niels Bohrweg

Bus - Universiteitsterrein

Bus - De Kempenaerstraat

Bus - Leidsebuurt

Bus - Van Assendelftstraat

Bus - Endegeest

Links - University

LIACS Thesis Repository (with students and supervisors index)

Leiden Teaching Tools

Committees - Masters

Lucris/Converis

Remote Campus (Citrix)

ServicePlein

Course Edit

Leiden Account Info

SNAPP

Brightspace

MyTimeTable

Leiden Account Services

Zoom for Leiden

Edit Courses

DS Lab details

Links - General/Random

MS Edge New Tab Extension

Media Bias Check

BuienRadar Pollen

RIVM Vaccination Overview

EU Corona Certificate

Booster Shot

Corona Vaccinations by Age

Covid Vaccinations

Google.com - A good WWW search engine for general information and guides

Google Search Operators

Bing Search Operators (e.g. inbody:)

Thesis Check - Turnitin

IBM Watson Speech to Text

Bus Corpus

Bus Niels Bohrweg

Dutch Basisverzekering

Doctor Boender

DBLP - Michael S. Lew

DBLP - International Journal of Multimedia Information Retrieval

MarBo Young on Writing

Log out Amazon.com

Log out Amazon.co.uk

Archive.org - Way Back Machine (historical WWW)

SAP Leiden

LIM

Waist to Height - Penn Health

Sunblock Ingredients

Random links

Lily

Campaign Promises - BBC

Campaign Promises - USA Today

Campaign Promises - Github

Six Party Talks and Signed Declaration 2005

GOPDND

Covid live

Dilbert

Perldoc.com - Perl Documentation

Multimedia Conferences 2011

AMSR links

Infinity Web Server

ACM International Conference on Multimedia Retrieval

ACM International Conference on Multimedia Retrieval, Glasgow, 2014 Web Archive

ACM TOMCCAP MIR Survey (vol. 2, issue 1, pp. 1-19, 2006)