CineVision: An Interactive Pre-visualization Storyboard System for Director-Cinematographer Collaboration

– Proceedings of the ACM Symposium on User Interface Software and Technology (UIST) 2025, Busan, Republic of Korea

Keywords – storyboard, pre-visualization, AI-assisted filmmaking, director-cinematographer collaboration

Summary – This paper introduces CineVision, an AI-driven interactive platform aimed at enhancing collaboration between film directors and cinematographers during the pre-production phase. Traditional methods rely heavily on hand-drawn storyboards and verbal briefings, which often fall short in visual clarity and iteration speed. CineVision integrates scriptwriting input with real-time visual pre-visualization, offering functionalities such as dynamic lighting control, filmmaker-style visual emulation, and customizable character design. In a lab study involving 24 participants, the system demonstrated reduced task times and improved usability compared to two baseline methods. The findings suggest that by facilitating rapid iteration, clearer visual communication, and aligned creative intent, CineVision has the potential to streamline early film production workflows and foster deeper synergy between directors and cinematographers.

full paper

ppt

ReCollection: Creating synthetic memories with AI in an interactive art installation

–Proceedings of the ACM on Computer Graphics and Interactive Techniques, Volume 7, Issue 4

Keywords – Experimental Visualization, Intelligent System Design, Interactive Art

Summary – This paper describes the conceptual background, artificial intelligent (AI) system design, and visualization strategies of an interactive AI art experience: ReCollection. In the art installation, the artwork assembles col-lective visual memories based on participants’ language input, blurring the boundaries between remembrance and imagination through intelligent system design and experimental visualization. By providing a conceptual framework for non-linear narratives, constituting symbiotic imaginations for memories, this artwork aims to provide an inclusive art experience that inspires collective memory reproduction by providing an intimate recollection of symbiosis between beings and apparatus. Furthermore, this work provides a future prototype that cultivates empathy for the dementia community by investigating the tensions in the correlations between visual representations and narratives, as well as these between the past, present, and future.

Full Paper : 1029_科技藝術書報討論_李鍵

Presentation PPT

Authors – Bo Xu, Junzhe Zheng, Jiayuan He, Yuxuan Sun, Hongfei Lin, Liang Zhao, Feng Xia

Keywords – meme understanding, metaphor, multimodal

Summary – Understanding memes is challenging because they contain metaphorical information that requires deep interpretation. Previous studies have added human-annotated metaphors as textual features in machine learning models but often ignored the link between metaphors and corresponding visual elements. This paper proposes MMMC (Multimodal Metaphorical feature for Meme Classification), which jointly models both textual and visual features for better meme understanding. Using a text-conditioned generative adversarial network (GAN), MMMC generates visual features from linguistic cues of metaphorical concepts and integrates them for classification. Experiments on the MET-Meme dataset show that MMMC significantly outperforms existing methods in emotion classification and intention detection.

 ACM MM 2024, Multimodal Reasoning & Inference

→ Full Paper

→ Class Presentation

BleacherBot: AI Agent as a Sports Co-Viewing Partner

Keywords

Co-viewing, AI Agents, Large Language Models (LLMs), Sports

Communication, Human-AI Interaction

Abstract

Co-viewing, traditionally defined as watching content together in the same physical space, enhances emotional connections through shared experiences. With the rise of remote viewing during the COVID-19 pandemic, existing solutions, such as second-screen  platforms and rule-based AI companions, struggle to facilitate meaningful social interactions. This study explores the potential of Large Language Models, which offer human-like interactions and personalization. Our formative study with ten participants revealed the importance of managing arousal levels, highlighting the need to balance between high- and low-arousal levels across different viewing contexts. Based on these insights, we developed ‘BleacherBot’, a sports co-viewing agent with distinct interaction styles that vary in arousal levels. Our main study with 27 participants demonstrated that matching users’ preferred arousal levels with the agent’s interaction style significantly enhanced their engagement and overall enjoyment. We propose design guidelines for AI co-viewing agents that consider their role as complements to human social interactions.

論文

科技藝術書報討論1029_陳昱璋

How Artists Use AI as a Responsive Material for Art Creation

Sonja Rozental, Michel van Dartel, Alwin de Rooij

ISEA 2025 Art Long Papers

References:
“How Artists Use AI as a Responsive Material for Art Creation” presented by Rozental, van Dartel and de Rooij – ISEA Symposium Archives
Project Results

Presentation PPT

She Works, He Works: Critical Explorations of Gender in AI-Generated Representations

– ISEA 2025 Long Paper

Author: Amalia Foka

Abstract

She Works, He Works is an interdisciplinary project that combines feminist visual theory, AI critique, and artistic practice to examine gender representation in AI-generated imagery. By juxtaposing text-to-image outputs of male and female construction workers, it reveals how societal biases manifest in generative AI systems. The study employs critically crafted prompts that blend traditionally gendered traits with unconventional contexts, exposing disparities in representation emphasizing aesthetics and agility for women versus authority and professionalism for men. Grounded in Griselda Pollock’s feminist insights on visual culture, the project positions AI as both a technological and cultural actor, capable of perpetuating stereotypes and enabling critique. As both a study and an artwork, the project underscores the transformative potential of AI to reimagine gender norms and inspire equitable narratives. It highlights the ethical implications of AI in creative industries, calling for transparent practices, diverse datasets, and reflexive engagement with AI’s cultural impact. Through its integration of critical analysis and artistic exploration, She Works, He Works contributes to broader discussions on inclusive AI development and the role of art in fostering societal change. This dual approach showcases the power of AI as a medium for critique and a tool for envisioning more just and inclusive futures.

Keywords

Gender representation, text-to-image models, critical prompt engineering, AI-generated art, bias in AI, feminist visual theory, cultural representation, ethical AI development, societal stereotypes

Reference

“She Works, He Works: Critical Explorations of Gender in AI-Generated Representations” presented by Foka

Amalia Foka, Computer Science Applications for the Arts, University of Ioannina

Presentation

File

Being in Virtual Worlds: How Interaction, Environment, and Touch Shape Embodiment in Immersive Experiences

– CHI 2025, ACM SIGCHI Conference on Human Factors in Computing Systems 

Keywords – embodiment, presence, interaction, touch, virtual reality

Summary –

This paper reconceptualizes embodiment in virtual reality by arguing that traditional approaches (focusing mainly on body ownership via visual-tactile synchronization) are too narrow. Instead, the authors propose that VR embodiment emerges through the dynamic interplay of interaction, environment, and touch. They build an Interactional Framework of Virtual Embodiment, integrating perspectives from philosophy, psychology, HCI, and VR. Through case comparisons (such as pseudo-haptics, shared avatars, and creative VR installations) and reflective design practice, they derive 12 VR design principles aimed at enhancing presence and meaningful embodied experience in virtual worlds.

→ Class Presentation

→ Full Paper

Algorithmic Miner: Humanity in Service – An AI-Driven VR Journey into Machine Logic

–Proceedings of the ACM on Computer Graphics and Interactive Techniques, Volume 8, Issue 3

Keywords – AI-Driven Creativity, Algorithmic Control, Virtual Reality Interaction, Machine Logic Dominance, Immersive Environments, Technological Authority

Summary –

The installation aims to symbolically reconstruct the monotonous nature of data‑annotation tasks, inviting participants to engage in repetitive image and text annotation tasks, mirroring the tedious and dehumanizing aspects of real-world data labeling. The object recognition component introduces a “reverse scenario” where human agency is subordinated to AI logic, compelling participants to alter their virtual environment according to machine instructions. As participants complete these tasks, their efforts become golden jellyfish, ascending to a virtual utopia composed of gleaming steel and lush natural landscapes. This transformation symbolizes the abstraction and exploitation of human creativity within AI-driven systems.

Algorithmic Miner envisions a future where machine logic dominates discourse, dictating human creativity and redefining reality, resonating with Baudrillard’s notion of the “hyperreal simulacrum.” By exposing the hidden labor and ethical implications inherent in the tech industry, the installation urges audiences to reconsider the complex interplay between society, the environment, and technology. It invites reflection on the human costs associated with technological advancement, highlighting the need for a more equitable and conscious integration of human labor within AI systems.

Full Paper : https://dl.acm.org/doi/10.1145/3736784

Presentation PPT

Strategy and Skill Learning for Physics-based Table Tennis Animation

SIGGRAPH Conference Papers ’24

Keywords

Character Animation, Physics-based Characters, Deep Reinforcement Learning, Multicharacter Interaction, Virtual Reality, Table Tennis

ABSTRACT

Recent advancements in physics-based character animation leverage deep learning to generate agile and natural motion, enabling characters to execute movements such as backflips, boxing, and tennis. However, reproducing the selection and use of diverse motor skills in dynamic environments to solve complex tasks, as humans do, still remains a challenge. We present a strategy and skill learning approach for physics-based table tennis animation. Our method addresses the issue of mode collapse, where the characters do not fully utilize the motor skills they need to perform to execute complex tasks. More specifically, we demonstrate a hierarchical control system for diversified skill learning and a strategy learning framework for effective decision-making. We showcase the efficacy of our method through comparative analysis with state-of-the-art methods, demonstrating its capabilities in executing various skills for table tennis. Our strategy learning framework is validated through both agent-agent interaction and human-agent interaction in Virtual Reality, handling both competitive and cooperative tasks.

Paper

科技藝術書報討論1015_陳昱璋

AI Nüshu: An Exploration of Language Emergence in Sisterhood Through the Lens of Computational Linguistics

Yuqian Sun, Yuying Tang, Ze Gao, Zhijun Pan, Chuyan Xu, Yurou Chen, Kejiang Qian, Zhigang Wang, Tristan Braud, Chang Hee Lee, Ali Asadipour

SIGGRAPH Asia 2023 Art Long Papers

References:
AI Nüshu: An Exploration of Language Emergence in Sisterhood Through the Lens of Computational Linguistics | SIGGRAPH Asia 2023 Art Papers
AI Nüshu Video

Presentation PPT

From Virtual to Physical: Investigating the Carryover Effects of Avatar-Mediated Communication in Intergenerational Contexts

Kensuke Nomura(JP) , Yuji Hatada(JP),Takuji Narumi (JP), Hideaki Kuzuoka (JP)

 – ACM Symposium on Applied Perception 2025, Avatars & Entertainment

Keywords – avatar, interpersonal impression, intergenerational communication, virtual reality

Summary – Previous studies show that avatars hiding personal traits can reduce psychological barriers and increase closeness in virtual settings, but it is unclear if this effect lasts offline. This study examined how avatar-based communication influences impressions and later face-to-face interactions between young adults (20–39) and older adults (40–59). Results showed that avatars initially reduced social distance online, but the effect did not continue offline. Participants judged partners mainly by voice and conversation, not avatar appearance. The findings suggest that avatars have limited short-term impact unless users strongly identify with them.

→ Class Presentation

→ Full Paper

DazzleVR: Enhancement of Brightness by Presenting Afterimage and Pseudo Dazzle Reflex Sensation in Virtual Reality
Juro Hosoi (JP), Takahiro Ito (JP), Yuki Ban (JP), Shin’ichi Warisawa (JP)
→ SIGGRAPH Asia 2023, Emerging Technologies

Key words – afterimage, dazzle reflex, visuo-haptic feedback, pseudo-haptics, VR perceptio

Summary –
DazzleVR is a VR system designed to induce the sensation of dazzle (intense brightness) by combining visual and tactile cues. It utilizes afterimages by rendering supplemental residual images at the user’s gaze point and implements a pseudo dazzle reflex by temporarily darkening the display (simulating eye closure) and applying subtle vibration to the eyelid area. Together, these mechanisms aim to evoke an authentic dazzling experience in VR without needing excessively bright displays. Users feel both a lingering brightness and a reflexive “blink-like” effect, enhancing immersion and perceived realism

Image Description
The visual shows a VR scene with a strong light source (e.g. a spotlight or flare), while a faint complementary afterimage overlays the user’s gaze direction. The vignette may include labels such as “DazzleVR” or “Afterimage + Reflex,” reinforcing that it’s a perceptual interaction demo.

→ Class Presentation
→ Full Paper
→ SIGGRAPH Asia Emerging Technologies Program Page

Spread Your Wings: Soft Floating Robotic Avatar with Flapping Wings for Physical Interactions

ACM SIGGRAPH 2025 Emerging Technologies

ABSTRACT

We demonstrate a new robotic avatar concept in which pilots can embody a fish-like form, “swimming” through indoor spaces to interact, play, and accompany others remotely. Our soft flapping-wing floating avatar offers three merits: (1) The soft body and blade-free design enable safe, close-proximity flight around people; (2) The wings not only provide propulsion but also serve as a medium for physical interaction, allowing affective gestures such as hugs, shoulder pats, and high-fives; and (3) The bioinspired design provides a more organic and lifelike movement compared to propeller-based avatars, potentially enhancing the sense of presence and evoking a feeling of interacting with a living creature. Furthermore, by mimicking the flapping flight of birds, pilots can control the avatar through body movements, while engaging in interactions with others through the avatar’s soft body and wings. This work opens new possibilities for telepresence and introduces a wing-based approach to remote physical interactions.

Keywords: 

Affectionate behavior, Positive affection, Close-proximity interaction, Soft floating robot, Flapping-wing robot

Reference :

https://dl.acm.org/doi/abs/10.1145/3721257.3734034

Presentation PDF

Parasitic Finger: Coexistence with Artificial Organism

– ACM SIGGRAPH 2025, Emerging Technologies

Authors: Akira Nakayasu, Saki Sakaguchi, Mina Shibasaki

Abstract

The Parasitic Finger project explores how humans coexist with uncontrollable finger augmentation with an SMA actuator. What would happen if our fingers, which we use freely in daily life, could move in unrealistic ways, independent of our will? The finger actuator unit weighs just 5 grams and is lightweight, making it comfortable to wear on the fingertips. By regulating the heat of the SMA wires, the SMA actuator can bend in four directions. Unlike human finger joints, they move like tentacles, performing actions such as waving, touching objects, or grasping items. We designed the finger movements so that participants would feel a sense of biological agency or animacy.

Keywords

Human Augmentation, Cyborgs, Digital Cyborgs, Human-machine Integration, Soft Actuator, Shape-Memory Alloy

Reference

https://dl.acm.org/doi/10.1145/3721257.3734023

https://nakayasu.com/parasitic-finger

https://www.toki.co.jp/biometal/products/WhtsBM.php

https://old2-lecture.nakayasu.com/index_p=2896.html

Presentation

file

Public Hand: Proximity-Aware Manipulation of Robotic Hand Shared with Multi-User in Robotic Room

Masahiro Ohta、Takatoshi Yoshida、Tomoya Sasaki、Kouta Minamizawa

ACM Siggraph 2025 Emerging Technologies

References:
Presentation – SIGGRAPH 2025 Conference Schedule
Public Hand: Proximity-Aware Manipulation of Robotic Hand Shared with Multi-User in Robotic Room | ACM SIGGRAPH 2025 Emerging Technologies

Presentation PPT