Jun Kato | The University of Tokyo (original) (raw)

Papers by Jun Kato

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Lyric apps as websites Mass Distribution of Lyric Apps (4.3) Lyric app users (always unique exper... more Lyric apps as websites Mass Distribution of Lyric Apps (4.3) Lyric app users (always unique experience) Lyric app: Musician Programmer Musicians can upload new musical pieces Programmers and musicians can choose from existing online music or Web-based Development Workflow (4.1) APIs for Building Lyric App Interactions (4.2) Lyric App Framework (https://developer.textalive.jp) A lyric app is a new form of lyric-driven visual art that can render different lyrical content depending on user interaction (2) Design space exploration collecting 52 examples of lyric apps revealed 8 representative categories (6) Lyric video audiences (always same content) Lyric video (existing static medium): Musician Motion graphic designer Video-sharing services TextAlive App API (npm: textalive-app-api) Figure 1: The Lyric App Framework opens up a novel design space for programmers and musicians to develop lyric apps. These apps enable users to interact with lyric-driven visual art synchronized with music, addressing the limitations of lyric videos.

Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

Research on creativity support tools (CSTs) has a long history in Human-Computer Interaction (HCI... more Research on creativity support tools (CSTs) has a long history in Human-Computer Interaction (HCI); however, researchers often focus on developing novel CSTs and verifying them in a controlled lab setting, rather than on capturing the creative process in the wild. In reality, creative activity is exploratory, laborious, and involves multiple CSTs; which together form a creativity support environment or ecology. Creative activity is also social, cultural, and collaborative with people distributing, modifying, and reacting to the creations of others. This process can inspire subsequent iterations. To understand and support open-ended, culturally embedded, collaborative creativity, HCI researchers are seeking new methods to study the sociocultural aspects of creativity support. This Special Interest Group on Creativity and Cultures in Computing (SIGCCC) invites diverse researchers to provide a forum for CST discussions from a wide sociocultural lens. The participants will identify and discuss the state-of-the-art and conceptualize future directions for creativity support research.

27th International Conference on Intelligent User Interfaces, 2022

An interactive touch surface is getting more and more popular as the primary input source for com... more An interactive touch surface is getting more and more popular as the primary input source for computers, including smartphones and tablet devices. Such devices rarely come with a set of a mouse and keyboard, making it difficult for the programmer to write code in a traditional text-based development environment. Meanwhile, there is an increasing demand on the use of complex data types e.g. images, reflecting the increase in computational power. Current programming languages and development environments usually do not have built-in support for such complex data types, resulting in poor programming experience. In this paper, we propose Visionsketch, a domain-specific language for computer vision programming along with its gesture-based development environment optimized for a touchscreen.

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2013

Current programming environments use textual or symbolic representations. While these representat... more Current programming environments use textual or symbolic representations. While these representations are appropriate for describing logical processes, they are not appropriate for representing raw values such as human and robot posture data, which are necessary for handling gesture input and controlling robots. To address this issue, we propose Picode, a text-based development environment augmented with inline visual representations: photos of human and robots. With Picode, the user first takes a photo to bind it to posture data. She then drag-and-drops the photo into the code editor, where it is displayed as an inline image. A preliminary user study revealed positive effects of taking photos on the programming experience.

ACM SIGGRAPH 2009 Emerging Technologies on - SIGGRAPH '09, 2009

ACM SIGPLAN Notices, 2013

Live programming allows programmers to edit the code of a running program and immediately see the... more Live programming allows programmers to edit the code of a running program and immediately see the effect of the code changes. This tightening of the traditional edit-compile-run cycle reduces the cognitive gap between program code and execution, improving the learning experience of beginning programmers while boosting the productivity of seasoned ones. Unfortunately, live programming is difficult to realize in practice as imperative languages lack well-defined abstraction boundaries that make live programming responsive or its feedback comprehensible. This paper enables live programming for user interface programming by cleanly separating the rendering and non-rendering aspects of a UI program, allowing the display to be refreshed on a code change without restarting the program. A type and effect system formalizes this separation and provides an evaluation model that incorporates the code update step. By putting live programming on a more formal footing, we hope to enable critical a...

Artificial intelligence (AI) such as deep learning has achieved significant success in a variety ... more Artificial intelligence (AI) such as deep learning has achieved significant success in a variety of application domains. Several visualization techniques have been proposed for understanding the overall behavior of the neural network defined by deep learning code. However, they show visualization only after the code or network definition is written and it remains complicated and unfriendly for newbies to build deep neural network models on a code editor. In this paper, to help user better understand the behavior of networks, we augment a code editor with instant and editable visualization of network model, inspired by live programming which provides continuous feedback to the programmer.

Companion to the first International Conference on the Art, Science and Engineering of Programming, 2017

Programmers write source code that compiles to programs, and users execute the programs to benefi... more Programmers write source code that compiles to programs, and users execute the programs to benefit from their features. While issue-tracking systems help communication between these two groups of people, feature requests have usually been written in text with optional figures that follows community guidelines and needs human interpretation to understand what to implement in which part of the source code. To make this process more direct, intuitive, and efficient, a streamlined interaction design called "User-Generated Variables (UGV)" is proposed. First, the users can declare parameters that they want to tweak in existing programs without reading or understanding the source code. Then, the system turns the proposal into variable declarations in the relevant part of the source code. Finally, the programmers are notified of the proposal and can implement the actual features to reflect changes in the variable value. The proposed interaction is implemented in two existing Web-...

Live Programming allows programmers to gain information about the program continuously during its... more Live Programming allows programmers to gain information about the program continuously during its development. While it has been implemented in various integrated development environments (IDEs) for programmers, its interaction techniques such as slider widgets for continuous parameter tuning are comprehensible for people without any prior knowledge of programming and have been widely used for a long time. In this paper, we aim to introduce prior work on Live Programming research from the interaction point of view and relate it to Human-Computer Interaction research. We then name the subset of Live Programming interaction that only involves changes in constant values “Live Tuning.” Our example IDEs that implement both Live Programming and Live Tuning interactions are showcased, followed by the discussion on the possible future direction of programming experience (PX) research.

Computers have become ubiquitous in our life and work, and the way that they are built and used n... more Computers have become ubiquitous in our life and work, and the way that they are built and used needs to be fundamentally improved. Most of the prior effort has been aimed at improving the programming experience for people with specific technical backgrounds (e.g., programmers, end-users, data scientists). In contrast, throughout this paper, we discuss how to make programming activities more inclusive and collaborative, involving people with diverse technical backgrounds. We rethink the programming environment from both technical and social perspectives. First, we briefly introduce our previous technical effort in which the programming environment is shared between the developers and users of programs, eliminating the distinction between programming and runtime environments and fostering communication between them. Second, we introduce our social effort to support people who are visually impaired in implementing customized smart glasses that read words with a camera and speakers. We...

Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015

This paper presents TextAlive, a graphical tool that allows interactive editing of kinetic typogr... more This paper presents TextAlive, a graphical tool that allows interactive editing of kinetic typography videos in which lyrics or transcripts are animated in synchrony with the corresponding music or speech. While existing systems have allowed the designer and casual user to create animations, most of them do not take into account synchronization with audio signals. They allow predefined motions to be applied to objects and parameters to be tweaked, but it is usually impossible to extend the predefined set of motion algorithms within these systems. We therefore propose an integrated design environment featuring (1) GUIs that designers can use to create and edit animations synchronized with audio signals, (2) integrated tools that programmers can use to implement animation algorithms, and (3) a framework for bridging the interfaces for designers and programmers. A preliminary user study with designers, programmers, and casual users demonstrated its capability in authoring various kinetic typography videos.

2013 IEEE 2nd Global Conference on Consumer Electronics (GCCE), 2013

This paper describes our experience of prototyping OpenPool (http://openpool.cc/), a system that ... more This paper describes our experience of prototyping OpenPool (http://openpool.cc/), a system that digitally augments a billiard table with audio and visual effects. Our observation on its growth as an open-source project revealed that entertainment system can be a good platform for open collaboration among people with diverse backgrounds. In this paper, we explain the brief overview of OpenPool, investigate how it has engaged many people to achieve successful collaboration, and characterize its development process named "community-based prototyping."

As computers become more pervasive, more programs deal with real-world input and output (real-wor... more As computers become more pervasive, more programs deal with real-world input and output (real-world I/O) such as processing camera images and controlling robots. The realworld I/O usually contains complex data hardly represented by text or symbols, while most of the current integrated development environments (IDEs) are equipped with textbased editors and debuggers. My thesis investigates how visual representations of the real world can be integrated within the text-based development environment to enhance the programming experience. In particular, we have designed and implemented IDEs for three scenarios, all of which make use of photos and videos representing the real world. Based on these experiences, we discuss "programming with example data," a technique where the programmer demonstrates examples to the IDE and writes text-based code with support of the examples.