A visual medium for programmatic control of interactive applications (original) (raw)

The VisMap system provides for "visual manipulation" of arbitrary off-the-shelf applications, through an application's graphical user interface. VisMap's API-independent control has advantages for tasks that can benefit from direct access to the functions of the user interface. We describe the design goals and architecture of the system, and we discuss two applications, a user-controlled visual scripting program and an autonomous solitaire-playing program, which together demonstrate some of the capabilities and limitations of the approach.