Augmented Reality on Handheld Computers to Flight Displays (original) (raw)

Computer-vision-based registration techniques for augmented reality

1996

Augmented reality is a term used to describe systems in which computer-generated information is superimposed on top of the real world; for example, through the use of a see-through head-mounted display. A human user of such a system could still see and interact with the real world, but have valuable additional information, such as descriptions of important features or instructions for performing physical tasks, superimposed on the world. For example, the computer could identify objects and overlay them with graphic outlines, labels, and schematics. The graphics are registered to the real-world objects and appear to be "painted" onto those objects. Augmented reality systems can be used to make productivity aids for tasks such as inspection, manufacturing, and navigation.

Vision-based Registration for Augmented Reality - A Short Survey

IEEE International Conference on Signal and Image Processing Applications (ICSIPA), 2015

The purpose of this paper is to explore some existing techniques in vision-based registration for Augmented Reality (AR) and present them collectively. AR is a branch of computer vision which generally overlays Virtual Objects (VOs) on actual images of real-world scenes in order to provide additional information about the scene to the user. Due to its wide range of applications in the fields of medical, robotics and automotive, geographic and remote sensing, military and aerospace, it has gained high demand. In any AR system, registration is the key to make the augmented scene appearing natural. Registration process must avoid occlusion of VOs and objects in the real world and align the VOs precisely. Optics-based and video-based are two well-known industrial AR systems. Researchers show that even with a single camera model registration for an AR is plausible but, VOs may be registered in front of real-world objects. It is because the registration process lacks depth information of the scene. However, employing stereo vision system and utilizing available natural features in a real-world scene and set of arbitrary multiple planes one can improve accuracy of VO registration. Thus, an AR system becomes robust if it is devised with algorithms to extract and track natural features in real-time.

IJERT-A Report on Registration Problems in Augmented Reality

International Journal of Engineering Research and Technology (IJERT), 2014

https://www.ijert.org/a-report-on-registration-problems-in-augmented-reality https://www.ijert.org/research/a-report-on-registration-problems-in-augmented-reality-IJERTV3IS040730.pdf Registration is the accurate alignment of real and virtual objects. Without accurate registration, the illusion that the virtual objects exist in the real environment is severely compromised. Registration is a difficult problem and a topic of continuing research. The goal of this paper is to cover the recent advances in Augmented Reality registration problem (error).Registration as the base technology of augmented reality should be capable to reflect the location and orientation change quickly during the virtual information loaded in the real target scene. The Registration error has many region such as calibration error, tracker error ,system delay, misalignment of the model and optical distortion. Augmented reality (AR) is a technology which supplements the real world with the virtual image, text and other information aligned with real scenes to augment perception and experiences for the real environment.

A system for synthetic vision and augmented reality in future flight decks

2000

ABSTRACT Rockwell Science Center is investigating novel human-computer interface techniques for enhancing the situational awareness in future flight decks. One aspect is to provide intuitive displays which provide the vital information and the spatial awareness by augmenting the real world with an overlay of relevant information registered to the real world.

Registration Using Natural Features for Augmented Reality Systems

IEEE Transactions on Visualization and Computer Graphics, 2006

Registration is one of the most difficult problems in augmented reality (AR) systems. In this paper, a simple registration method using natural features based on the projective reconstruction technique is proposed. This method consists of two steps: embedding and rendering. Embedding involves specifying four points to build the world coordinate system on which a virtual object will be superimposed. In rendering, the Kanade-Lucas-Tomasi (KLT) feature tracker is used to track the natural feature correspondences in the live video. The natural features that have been tracked are used to estimate the corresponding projective matrix in the image sequence. Next, the projective reconstruction technique is used to transfer the four specified points to compute the registration matrix for augmentation. This paper also proposes a robust method for estimating the projective matrix, where the natural features that have been tracked are normalized (translation and scaling) and used as the input data. The estimated projective matrix will be used as an initial estimate for a nonlinear optimization method that minimizes the actual residual errors based on the Levenberg-Marquardt (LM) minimization method, thus making the results more robust and stable. The proposed registration method has three major advantages: 1) It is simple, as no predefined fiducials or markers are used for registration for either indoor and outdoor AR applications. 2) It is robust, because it remains effective as long as at least six natural features are tracked during the entire augmentation, and the existence of the corresponding projective matrices in the live video is guaranteed. Meanwhile, the robust method to estimate the projective matrix can obtain stable results even when there are some outliers during the tracking process.

Visual registration for unprepared augmented reality environments

Personal and Ubiquitous …, 2003

Despite the increasing sophistication of augmented reality (AR) tracking technology, tracking in unprepared environments still remains an enormous challenge according to a recent survey. Most current systems are based on a calculation of the optical flow between the current and previous frames to adjust the label position. Here we present two alternative algorithms based on geometrical image constraints. The first is based on epipolar geometry and provides a general description of the constraints on image flow between two static scenes. The second is based on the calculation of a homography relationship between the current frame and a stored representation of the scene. A homography can exactly describe the image motion when the scene is planar, or when the camera movement is a pure rotation, and provides a good approximation when these conditions are nearly met. We assess all three styles of algorithms with a number of criteria including robustness, speed and accuracy. We demonstrate two real-time AR systems here, which are based on the estimation of homography. One is an outdoor geographical labelling/ overlaying system, and the other is an AR Pacman game application.

Temporal Registration using a Kalman Filter for Augmented Reality Applications

Augmented Reality uses see-through head-mounted displays to superimpose synthetically generated information on a three-dimensional scene. Information is rendered in alignment with physical objects to enhance the user's ability to perceive and interact with the world. A significant technical challenge related to Augmented Reality is determining the transformation that will correctly align the synthetic data with corresponding physical objects. This is made more difficult by the fact that the user and the scene may be in motion. Alignment must be both stable and accurate in order to produce the "illusion" that synthetic objects are an integral part of the environment. This paper presents a tracking algorithm capable of computing the three-dimensional pose of objects with respect to a headmounted camera as they are moved in the scene. Using the fixed transform from the headmounted camera and the see-through device, information can then be displayed in alignment with the current view of the object being tracked. In contrast to approaches that solve for the absolute position of the viewer within a fixed geometry , this object-centered approach to Augmented Reality models motion of both the object and the camerasdisplay system mounted on the user's head using a single transformation involving six parameters.

System for synthetic vision and augmented reality in future flight decks

SPIE Proceedings, 2000

Rockwell Science Center is investigating novel human-computer interaction techniques for enhancing the situational awareness in future flight decks. One aspect is to provide intuitive displays that provide the vital information and the spatial awareness by augmenting the real world with an overlay of relevant information registered to the real world. Such Augmented Reality (AR) techniques can be employed during bad weather scenarios to permit flying in Visual Flight Rules (VFR) in conditions which would normally require Instrumental Flight Rules (IFR). These systems could easily be implemented on heads-up displays (HUD). The advantage of AR systems vs. purely synthetic vision (SV) systems is that the pilot can relate the information overlay to real objects in the world, whereas SV systems provide a constant virtual view, where inconsistencies can hardly be detected. The development of components for such a system led to a demonstrator implemented on a PC. A camera grabs video images which are overlaid with registered information. Orientation of the camera is obtained from an inclinometer and a magnetometer; position is acquired from GPS. In a possible implementation in an airplane, the on-board attitude information can be used for obtaining correct registration. If visibility is sufficient, computer vision modules can be used to fine-tune the registration by matching visual cues with database features. This technology would be especially useful for landing approaches. The current demonstrator provides a frame-rate of 15 fps, using a live video feed as background with an overlay of avionics symbology in the foreground. In addition, terrain rendering from a 1 arc sec. digital elevation model database can be overlaid to provide synthetic vision in case of limited visibility. For true outdoor testing (on ground level), the system has been implemented on a wearable computer.

Linear Augmented Reality Registration

Lecture Notes in Computer Science, 2001

Augmented reality requires the geometric registration of virtual or remote worlds with the visual stimulus of the user. This registration can be achieved by tracking the head pose of the user with respect to the reference coordinate system of the virtual objects. If tracking is achieved with headmounted cameras, registration becomes pose estimation as it is known in computer vision. Augmented reality is by definition a real-time problem, so we are interested only in bounded and short computational time. We propose a new linear algorithm for pose estimation. The algorithm shows a better performance than the linear algorithm by Quan and Lan and is comparable to the non-predicted time iterative algorithm by Kumar and Hanson.