ACME

Minority Report-inspired interface allows us to explore time in new ways

Michael Kwolek — Fri, 26 Sep 2025 14:48:13 +0000

Minority Report-inspired interface allows us to explore time in new ways Michael Kwolek Fri, 09/26/2025 - 08:48

Michael Kwolek

Imagine a scene—a bird feeder on a summer afternoon, the dark of night descending over the Flatirons, a fall day on a university campus. Now imagine moving backwards and forwards through time on a single aspect of that setting while everything else remains. One ATLAS engineer is building technology that lets us experience multiple time scales all at once.

The Proteus Team

David Hunter developed Proteus with expertise from ACME Lab members Suibi Weng, Rishi Vanukuru, Annika Mahajan, Ada Zhao and Leo Ma, and advising from professor and lab director Ellen Do.

Brad Gallagher and Chris Petillo in the B2 Center for Media, Arts and Performance provided critical technical support to make the project come alive in the B2 Black Box Studio.

“Proteus: Spatiotemporal Manipulations” by David Hunter, ATLAS PhD student, in collaboration with his ACME Lab colleagues, allows people to simultaneously observe different moments in time through a full-scale interactive experience combining video projection, motion capture, audio and cooperative elements.

The project was shown through the creative residency program in the B2 Center for Media, Art and Performance at the Roser ATLAS Center. Housed in the B2 Black Box Studio, the project team used 270-degree video projections, a spatial audio array and motion capture technology, creating an often larger-than-life way to explore many different time scales at once. Simultaneous projections highlighted the lifecycle of a bacteria in a petri dish, a day in the life of a street corner on campus, and weather patterns at a global scale among other vignettes.

This tangible manipulation of time within a space can feel disorienting at first. It takes a while to adjust to what is happening, a unique sensation that has an expansive, almost psychedelic quality. But it may also have practical applications.

We spoke to Hunter about the inspiration behind Proteus, possible use cases and what comes next.

This Q&A has been lightly edited for length and clarity.

Several people can explore different moments in time simultaneously.

Tell us about the inspiration for Proteus.

There are scenes and settings where you want to see a place or situation at two distinct time frames, and for that comparison to not necessarily be hard-edged or side by side. You want it to be interpolated through time, so you can see patterns of change in space over that time.

Describe the early iterations of this project.

It was originally a tabletop setup with projection over small robots, and you could manipulate the robots to produce similar kinds of effects. There was a version running with a camera giving you a live video feed, but we switched it to curated videos as it was easier to understand what was happening with time manipulation. You're potentially making quite a confusing image for yourself, so the robots gave you something tangible to hold on to.

How did it evolve into the large-scale installation it is now?

We thought it would be interesting to see this at a really large scale in the B2, and the creative residencies made that possible. That's when we moved away from robots and it was like, “Well, how would you control it in this sort of space?”

I took a projection mapping course last semester and worked on a large-scale projection, but then we changed it to hand interaction—gesture-based in the air, kind of like “Minority Report.”

As you closer or farther from the screen, you change the time scale of part of the scene.

How do you describe what takes place when people interact with Proteus?

The key is interaction—people can actually control the time lapse. Usually, time lapses are linear, not two dimensional, and we don't have control over them. Here, you can focus on what interests you across different time periods, or hold two points in time side by side to see patterns and relationships as they change across space and time. This also enables multiple visitors to find the things they are interested in; there isn't one controller of the scene, it is collaborative.

What are some of the use cases you’ve thought about for this technology?

Anything with a geospatial component—a complex scene where many things are happening at once. You might want to keep track of something happening in the past while still tracking something else happening at another time.

You can use these portals to freeze multiple bits of action or set them up to visualize where things have gone at different points in time and space.

It's always been about collaboration—situation awareness where lots of people are trying to interrogate one image and see what everyone else is doing at the same time.

Then there’s film analysis: Can we put in a whole movie and perhaps you can find interesting relationships and compositions around that? This could be a fun way to spatially explore a narrative, too.

We're also looking at how it could be used for mockup design. Let's say you're prototyping an app and you have 50 different variations. You could collapse those all into one space, interrogate them through “time,” then mix and match different portions of your designs to come up with new combinations. It also works with volumetric images like body scans, where we swap time for depth.

What are the creative influences that drove the visual style of the piece?

I've long been interested in time lapses, like skateboard photography where multiple snapshots are overlaid on the same space as a single image.

There's all this work by Muybridge and Marey, who invented chronophotography. That's how they worked out that horses leave the ground while they run.

David Hockney did a ton of Polaroid work. There's a famous one of people playing Scrabble, shot from his perspective. All these different Polaroids are stuck down next to each other, not as a true representation of space but as a way of capturing time within that space—breaking the unity of the image.

Khronos Projector by Alvaro Cassinelli kicked off this research sprint and prompted me to look back at my interest in time and photography.

Several time lapse videos are projected simultaneously on the Black Box Studio's 270-degree screen.

Proteus is controlled with an app on a mobile phone modified to work with motion capture.

It does force your brain to work in a way that it is not used to, which is a really cool thing to happen in a creative sense but also in a technical sense.

We aren't used to seeing anything with non-uniform time. Whenever we're watching a video, we want to find a point in the video, then we see the whole image rewind or fast forward. Of course that makes sense in a lot of situations, but there could be interesting use-cases for interactive non-uniform time.

What makes ATLAS an ideal place for this type of research?

There's all the people, whether it's research faculty who are interested in asking questions like, “How can we make a novel system or improve research that's going on in this area?” Or it's super strong technical expertise, which is like, “Let's have a go at making this work in a projection environment.”

What’s next for the Proteus project?

At the moment, I can only compare by changing the time on a region of space and I can compare a region of space against another region at different times. But it might be interesting to be able to break the image and say, “Actually, I want to clone this region and see it from a different time period.” Can I reconstitute the image in some way?

Experience Proteus during Research & Innovation Week

Proteus will be running in the B2 Black Box Studio as part of:

ATLAS Research Open Labs
Roser ATLAS Center
October 10, 2025
3-5pm
FREE, no registration needed

ATLAS PhD student David Hunter researches novel ways to interact with different moments in time across a single video stream.

Off

Traditional

White

New research explores tinkering as a key classroom learning method

Michael Kwolek — Wed, 16 Jul 2025 16:21:34 +0000

New research explores tinkering as a key classroom learning method Michael Kwolek Wed, 07/16/2025 - 10:21

Michael Kwolek

When kids tinker in the classroom, they get to build many useful skills from computing to collaboration to creativity and more.

Krithik Ranjan, PhD student and member of the ACME Lab, studies low-cost forms of human-computer interaction that enable more people to explore their creativity through technology. And tinkering plays a big part in that.

Ranjan presented his work at Constructionism 2025 conference in Zurich, Switzerland, which explores how constructionist ideas can inspire advancements in learning technologies and methodologies. He filled us in on what he presented.

Tell us about your research focus.

I've been trying to build ways for people to create with technology in a more open-ended, tinkering-friendly way. Tinkering is a way to learn where you can explore, you can experiment, you can playfully interact with things to learn a concept, whether it is computer science, physics, astronomy or anything like that.

Can you describe the intention behind your paper on “The Design Space of Tangible Interfaces for Computational Tinkerability”?

There are two elements to it. One is the tangible side where the idea is that you're interacting with computational elements in the physical space, either stuff like paper or robots or different components that you can put together. And the other aspect is computational tinkerability, that playful open-ended aspect about creating something computationally.

I want to understand how people have previously developed design spaces, which is this concept of a framework to understand what people have done, what it means, and how we can analyze and categorize different types of projects in that space. I reviewed 33 different projects to figure out: What are kids tinkering with? What are children making? And how are they making it?

This project was from the perspective of a designer to inform future designers who are going to create such interfaces and projects.

What did you discover in conducting this research?

We figured out there is a range of how tinkerable or how expressive an interface can be, so we try to categorize that based on a “spectrum of tinkerability.”

The other important takeaway is the idea of expanding beyond code. There's so much work, both commercially and in research, around enabling students to code. But a lot of researchers also found that this line-by-line type of programming is also a bit discouraging to students from underrepresented groups. So there's a lot of work in expanding the ways you can create with computers to not just rewrite lines of code to program or make a game or make a 3D model. But more diverse ways that suit different interests.

Who might be the audience for this research and what might they do with it?

The goal was to categorize this space so people can refer to it and design based on that. The audience is other researchers and designers of interfaces for learning with computers. Based on the implications, they can better design ways that students learn by making [tools] more expressive, more open-ended, learner-driven, and catering to different interests instead of just code or just one type of way to interact.

A lot of your work is focused on using simple materials that are more accessible to students all over the world. How might this research help educators expand the tools they have access to for students?

In practical situations like classrooms and formal learning centers, there are always constraints with resources, with the number of people, with the kind of things you can get access to. And quite often we might find that projects, interfaces and tools in the market are usually one-off and they are a couple hundred dollars or more. So I was trying to look at these projects in terms of the kind of materials they use and how they enable people to interact with the material.

There are some projects like [ATLAS PhD] Ruhan Yang's Paper Robots and a couple other projects that we looked at where the focus was DIY-based interfaces that educators can fabricate themselves for their classrooms. These projects stressed on publishing the plans and instructables and stuff like that online so that anybody can use them to build these interactive interfaces themselves.

And part of this was also using platforms that are already available. Arduino, Microbit and Raspberry Pi are commonly used electronic platforms in education for many different purposes. There's a way to make these interfaces more accessible if you use those existing platforms instead of making custom electronics.

How does a student who uses your Cartoonimator tool, for example, learn in the process of figuring out how to use and then make animations?

This idea of tinkering and learning by making is based on these learning philosophies of constructionism. The idea is that you're learning by building artifacts, building mental models yourself instead of being instructed by somebody else.

A big part is the idea of being stuck and then trying to work through the problem, trying to figure out what's wrong, what needs to change and get something working. This way of learning is focused on the learner's motivation.

What's next for this research?

Cartoonimator is one example where I looked at these principles about expanding beyond code and working with more accessible materials to build an interface that's open ended and tinkering-friendly for learning something like animation.

I'm looking further at how we can engage students with physical computing using paper, because that is more accessible, easier to expand on and gives you this space of creative exploration that you may not usually have with devices.

If you're new with technology, you might be afraid of breaking something apart, but that's really a core part of tinkering, so I'm looking at how paper-based interfaces can foster the idea.

PhD student Krithik Ranjan analyzed 33 student learning tools and developed a “spectrum of tinkerability” that offers designers new ways to think about teaching computational skills.

Off

Traditional

White

ATLAS community presents new research on interactive systems at DIS 2025

Michael Kwolek — Thu, 26 Jun 2025 17:14:27 +0000

ATLAS community presents new research on interactive systems at DIS 2025 Michael Kwolek Thu, 06/26/2025 - 11:14

The 2025 ACM Designing Interactive Systems Conference (DIS) in Madeira, Portugal, features work from ten ATLAS community members representing three labs. This year’s event has five focus areas: Critical Computing and Design Theory, Design Methods and Processes, Artifacts and Systems, Research Through Design, and AI and Design with an overall theme around “design that transcends human-centered perspectives.”

ATLAS researchers study a broad range of topics, from human-computer interaction to biomaterials to woven forms.

Ellen Do, professor and ACME director, explains what connects the work our community is presenting at the conference: “I think all of the papers and presentations we have are on designing interactive systems. Some of the systems could be physical, some could be digital, some could be human-and-people, human-and-physical objects. So I think the theme about interactive systems and how you make systems interactive, what kind of user experience or human experience or immersive experience with the object or system or even the ecosystem, or the human communication system—I think that's all there.”

ATLAS research at DIS 2025

"Chaotic, Exciting, Impactful": Stories of Material-led Designers in Interdisciplinary Collaboration
Gabrielle Benabdallah, Eldy S. Lazaro Vasquez (ATLAS PhD student), Laura Devendorf (ATLAS Unstable Design Lab director, associate professor), Mirela Alistar (ATLAS Living Matter Lab director, assistant professor)

This paper explores the dynamics of interdisciplinary collaboration between designers, scientists, and engineers through ten stories as told from the perspective of material-led designers. These stories focus on material-led designers working in contexts like biodesign and smart textiles, where novel materials, fabrication methods, and technology often intersect, requiring cross-disciplinary collaboration. By including perspectives from designers within and adjacent to HCI, the study broadens the understanding of interdisciplinary teamwork that combines scientific, technical, and craft-based expertise. Our analysis highlights how designers navigate challenges like differing terminologies, epistemic hierarchies, and conflicting priorities. We discuss strategies such as material prototypes, attitudes of inquiry and openness, switching lexicons, and the value of interdisciplinary contexts. This research underscores designers as “translators” who mediate epistemological tensions, use tangible artifacts to communicate, and articulate possible applications. This research contributes ten stories as narrative resources for understanding strategies and fostering interdisciplinary spaces within HCI.

Towards Yarnier Interactive Textiles: Mapping a Design Journey through Hand Spun Conductive Yarns
Etta W. Sandry (ATLAS PhD student), Lily M. Gabriel (ATLAS undergraduate student), Eldy S. Lazaro Vasquez (ATLAS PhD student), Laura Devendorf (ATLAS Unstable Design Lab Director, associate professor)

The ability to create a wide and varied set of interactive textiles depends on the materials that one has available. Currently, the range of yarns that can be used to bring interactivity to textiles is greatly limited, especially considering the diversity available in non-conductive yarns. This pictorial traces a design journey into hand spinning that seeks to address this limitation and contributes samples of techniques and materials that could be used to create conductive yarns along with reflection on design methods that enabled us to explore a wider range of aesthetic expressions. We advocate for an approach that reconnects with the textiles in e-textiles, embraces divergence, and prioritizes the material as the driver of a design concept. We offer pathways for readers and researchers to continue this exploration within varied domains and practices.

Connect! A Circuit-Driven Card Game
Ruhan Yang (ATLAS PhD alum), Ellen Yi-Luen Do (ATLAS ACME Lab director, professor)

Hybrid physical-digital games often rely on screen-based interactions, which can detract from their tactile nature. We introduce Connect!, a card game that integrates paper circuits and real-time LED feedback, enabling players to construct functional circuits as part of gameplay. Unlike traditional hybrid games, Connect! embeds feedback directly into physical components while preserving material interaction. We conducted a user study comparing gameplay with and without electronic feedback. Our findings suggest that real-time feedback not only increased engagement but also altered players' behavior, encouraging rule exploration and emergent play. Our work contributes to tangible interaction and game-based learning, demonstrating the potential of low-cost electronics in enhancing interactive experiences.

Connect! game cards

From Data to Discussion: Interfaces for Collective Inquiry and Open-Ended Data Creation
David Hunter (ATLAS PhD student)

Data can enrich our understanding of the world and improve our society. However the datafication of our society comes with challenges for empowering communities. In designing systems for recording and representing data, a theme has emerged of these interfaces as the site of conversations and sense-making, and the participatory nature is valuable beyond the data itself. This insight has led me to investigate tools and experiences that enable open-ended data creation and exploration as a grounding for discussion and prompting action. The goal is to design interfaces and systems for exploring places and futures through data, to empower communities and supporting civic participation, learning and making, situational awareness, and scenario planning. In this pictorial I present five ongoing research projects investigating these ideas.

How to Data Walk

Knitting with unknown trees: assembling a more-than-human practice
Doenja Oogjes, Ege Kökel, Netta Ofer (ATLAS PhD alum), Hsiang-Lin Kuo, Jasmijn Vugts, Troy Nachtigall, Torin Hopkins (ATLAS PhD alum)

In this pictorial, we explore alternative ways of knowing urban trees through a more-than-human lens. Using a municipal tree dataset, we focus on “unknown” trees—entries unclassified due to error, decay, or absence—highlighting the limits of quantification and fixed knowledge systems. Urban trees, while critical for ecosystems, are often shaped by technological interventions (e.g., GIS, IoT sensors, AI diagnostics) that prioritize their utility over other expressions. We engage in knitting as a material inquiry to foreground nonhuman agencies and relational entanglements. Through reflective shifts and compromises, this project questions normative design practices, seeking to amplify nonhuman participation. We make two contributions. Firstly, we offer insights into fostering alternative, relational engagements with urban ecologies. Secondly, we reflect on our process of surfacing and working with agentic capacities, articulating guidance for other design researchers. Through this, we advocate for fragmented approaches that embrace complicity and complexity in more-than-human design.

Designing Interfaces that Support Temporal Work Across Meetings with Generative AI
Rishi Vanukuru (ATLAS PhD student), Payod Panda, Xinyue Chen, Ava Elizabeth Scott, Lev Tankelevitch, Sean Rintel

Temporal work is an essential part of the modern knowledge workplace, where multiple threads of meetings and projects are connected across time by the acts of looking back (retrospection) and ahead (prospection). As we develop Generative AI interfaces to support knowledge work, this lens of temporality can help ground design in real workplace needs. Building upon research in routine dynamics and cognitive science, and an exploratory analysis of real recurring meetings, we develop a framework and a tool for the synergistic exploration of temporal work and the capabilities of Generative AI. We then use these to design a series of interface concepts and prototypes to better support work that spans multiple scales of time. Through this approach, we demonstrate how the design of new Generative AI tools can be guided by our understanding of how work really happens across meetings and projects.

Members of three ATLAS labs show how interactive technology can create possibilities for new means of productivity, data analysis, creativity and play.

Off

Traditional

White

Colorado-based Computer Graphics Professionals Make Their Mark at SIGGRAPH 2024

Anonymous — Fri, 02 Aug 2024 16:30:29 +0000

Colorado-based Computer Graphics Professionals Make Their Mark at SIGGRAPH 2024 Anonymous (not verified) Fri, 08/02/2024 - 10:30

ATLAS community members, including professor Ellen Do and PhD student Ruhan Yang, presented at this year's conference in Denver.

Off

Traditional

White

ATLAS PhD student deploys papercraft to make engineering tangible and fun

Anonymous — Tue, 30 Jul 2024 19:50:41 +0000

ATLAS PhD student deploys papercraft to make engineering tangible and fun Anonymous (not verified) Tue, 07/30/2024 - 13:50

ATLAS PhD student Ruhan Yang blends papercraft and circuit design to make engineering more tangible, accessible and fun for tinkerers of all ages.

Off

Traditional

White

Public-private partnership drives attention for ATLAS research in augmented and mixed reality

Anonymous — Thu, 18 Jul 2024 16:41:59 +0000

Public-private partnership drives attention for ATLAS research in augmented and mixed reality Anonymous (not verified) Thu, 07/18/2024 - 10:41

Michael Kwolek

Partnerships between universities and industry can yield important research and commercial breakthroughs. ATLAS professor Ellen Do has worked to cultivate relationships between CU Boulder and industry players, including as a member of the Pervasive Personalized Intelligence (PPI) Center, to support graduate students and enhance opportunities for commercialization of ATLAS research.

The PPI Center, which recently concluded its tenure, was founded “with a mission of bringing industry and university talent together to solve the intelligence challenges faced by software and computer engineers in Internet of Things systems." It operated under the supervision of the National Science Foundation and included members from NEC, Intel and Trimble.

“It’s been such a good experience. We’ve learned a lot. Ellen Do and her team have helped to expand our thinking and encouraged us to explore new areas.” - Dr. Haifeng Chen, Head of Data Science Department at NEC Laboratories, and his colleague Kai Ishikawa, Principal Researcher (PPI Center event recap)

The PPI Center’s Spring 2024 Industry Advisory Board Meeting in Portland, OR, included a research poster session, and ATLAS students were honored with three of the four awards industry attendees voted on at the event.

Suibi Che-Chuan Weng, PhD student, won "Most Industry Ready" for Editing Reality: Empowering Users to Manipulate Reality through Addition, Erasing, and Modification with Speech to Prompt in Mixed Reality.
Rishi Vanukuru, PhD student, won "Most Impactful" for Asynchronous spatial guidance using mobile devices and Augmented Reality.
Ada Zhao, MS student, won "Most Impactful" for The WizARd and Apprentice: Augmented Reality Expert Capture for Training Novices.

2 more ATLAS PhD students participated: Krithik Ranjan presented PuppetGuide: Tangible Personalized Museum Tour Guides using LLMs and David Hunter presented Tangible Interaction with Object Detection and Large Language Models.

As for the experience participating in the PPI Center, Do says, “it is good to know that the industry is interested in supporting research and considers our research relevant.” She sees ways ATLAS could form partnerships within several industry sectors on a range of themes due to the multidisciplinary nature of the research conducted here.

Since their involvement in PPI started, Do and her team have had a series of meetings with mentors from global technology firms, discussing collaborative research opportunities.

Vanukuru is currently doing an internship at Microsoft Research Cambridge focused on spatial computing in its VR/AR group. Weng and Zhao are working on research in the ACME Lab this summer, extending the Editing Reality (and PuppetGuide), and WizARd and Apprentice projects with interns from the CU SPUR program. Zhao is also conducting a pilot study, interviewing laser cutter operating experts about how they would demonstrate operations and how they can annotate their demonstration using the WizARd prototype for novice learners. Hunter has embarked on an internship with Trimble this summer, while he and Ranjan are also working in the ACME Lab.

ACME Lab members built relationships with industry players through the Pervasive Personalized Intelligence (PPI) Center by collaborating on solutions to challenges in building Internet of Things systems. Three ATLAS PhD students took home awards from the PPI Center's Spring 2024 Advisory Board Meeting.

Off

Traditional

White

ATLAS in Ireland: 12 community members present at TEI’24

Anonymous — Fri, 09 Feb 2024 19:05:23 +0000

ATLAS in Ireland: 12 community members present at TEI’24 Anonymous (not verified) Fri, 02/09/2024 - 12:05

Michael Kwolek

ATLAS is well-represented at #TEI2024 - the 18th ACM International Conference on Tangible, Embedded and Embodied Interaction. This year’s conference, in Cork, Ireland, celebrates “cutting-edge scientific research and art that is on the edge of disciplines and on the edge of new unique developments and possibilities.”

Research from 12 members of the ATLAS community including faculty, alumni and students is featured at the conference. The work spans a range of disciplines, including weaving, biomaterials, mixed reality and robotics. In addition, ACME Lab director, Ellen Do, acted as Co-Chair of Graduate Student Consortium; PhD student, Sandra Bae, was an Associate Chair for Pictorials; and ATLAS PhD alum, Fiona Bell, was an Associate Chair for Papers.

Research ATLAS PhD students presented at TEI’24

Loom Pedals: Retooling Jacquard Weaving for Improvisational Design Workflows
Shanel Wu, Xavier A Corr, Xi Gao, Sasha De Koninck, Robin Bowers, and Laura Devendorf

Abstract: We present the Loom Pedals, an open-source hardware/software interface for enhancing a weaver’s ability to create on-the-fly, improvised designs in Jacquard weaving. Learning from traditional handweaving and our own weaving experiences, we describe our process of designing, implementing, and using the prototype Loom Pedals system with a TC2 Digital Jacquard loom. The Loom Pedals include a set of modular, reconfigurable foot pedals which can be mapped to parametric Operations that generate and transform digital woven designs. Our novel interface integrates design and loom control, providing a customizable workflow for playful, improvisational Jacquard weaving. We conducted a formative evaluation of the prototype through autobiographical methods and collaboratively developed future Loom Pedals features. We contribute our prototype, design process, and conceptual reflections on weaving as a human-machine dialog between a weaver, the loom, and many other agents.

Bio-Digital Calendar: Attuning to Nonhuman Temporalities for Multispecies Understanding
Fiona Bell, Joshua Coffie, and Mirela Alistar

Abstract: We explore how actively engaging with the temporalities of a nonhuman organism can lead to multispecies understanding. To do so, we design a bio-digital calendar that brings attention to the growth and health of kombucha SCOBY, a symbiotic culture of bacteria and yeast that lives in a tea medium. The non-invasive bio-digital calendar surrounds the kombucha SCOBY to track (via sensors) and enhance (via sound) its growth. As we looked at and listened to our kombucha SCOBY calendar on a daily basis, we became attuned to the slowness of kombucha SCOBY. This multisensory noticing practice with the calendar, in turn, destabilized our preconceived human-centered positionality, leading to a more humble, decentered relationship between us and the organism. Through our experiences with the bio-digital calendar, we gained a better relational multispecies understanding of temporalities based on care, which, in the long term, might be a solution to a more sustainable future.

Wizard of Props: Mixed Reality Prototyping with Physical Props to Design Responsive Environments
Yuzhen Zhang, Ruixiang Han, Ran Zhou, Peter Gyory, Clement Zheng, Patrick C. Shih, Ellen Yi-Luen Do, Malte F Jung, Wendy Ju, and Daniel Leithinger

Abstract: Driven by the vision of future responsive environments, where everyday surroundings can perceive human behaviors and respond through intelligent robotic actuation, we propose Wizard of Props (WoP): a human-centered design workflow for creating expressive, implicit, and meaningful interactions. This collaborative experience prototyping approach integrates full-scale physical props with Mixed Reality (MR) to support ideation, prototyping, and rapid testing of responsive environments. We present two design explorations that showcase our investigations of diverse design solutions based on varying technology resources, contextual considerations, and target audiences. Design Exploration One focuses on mixed environment building, where we observe fluid prototyping methods. In Design Exploration Two, we explore how novice designers approach WoP, and illustrate their design ideas and behaviors. Our findings reveal that WoP complements conventional design methods, enabling intuitive body-storming, supporting flexible prototyping fidelity, and fostering expressive environment-human interactions through in-situ improvisational performance.

Making Biomaterials for Sustainable Tangible Interfaces
Fiona Bell, Shanel Wu, Nadia Campo Woytuk, Eldy S. Lazaro Vasquez, Mirela Alistar, and Leah Buechley

Abstract: In this studio, we will explore sustainable tangible interfaces by making a range of biomaterials that are bio-based and readily biodegradable. Building off of previous TEI studios that were centered around one specific biomaterial (i.e., bioplastics at TEI’22 and microbial cellulose at TEI’23), this studio will provide participants the ability to experience a wide variety of biomaterials from algae-based bioplastics, to food-waste-based bioclays, to gelatin-based biofoams. We will teach participants how to identify types of biomaterials that are applicable to their own research and how to make them. Through hands-on activities, we will demonstrate how to implement biomaterials in the design of sustainable tangible interfaces and discuss topics sensitized by biological media such as more-than-human temporalities, bioethics, care, and unmaking. Ultimately, our goal is to facilitate a space in which HCI researchers and designers can collaborate, create, and discuss the opportunities and challenges of working with sustainable biomaterials.

Paper Modular Robot: Circuit, Sensation Feedback, and 3D Geometry
Ruhan Yang

Abstract: Modular robots have proven valuable for STEM education. However, modular robot kits are often expensive, which makes them limited in accessibility. My research focuses on using paper and approachable techniques to create modular robots. The kit’s design encompasses three core technologies: paper circuits, sensation feedback mechanisms, and 3D geometry. I have developed proof-of-concept demonstrations of technologies for each aspect. I will integrate these technologies to design and build a paper modular robot kit. This kit includes various types of modules for input, output, and other functions. My dissertation will discuss the development of these technologies and how they are integrated. This research will address the considerations and techniques for paper as an interactive material, providing a guideline for future research and development of paper-based interaction.

Research from 12 members of the ATLAS community including faculty, alumni and students is featured at the 18th ACM International Conference on Tangible, Embedded and Embodied Interaction.

Off

Traditional

White

ATLAS PhD Students Present at ISMAR 2023

Anonymous — Wed, 25 Oct 2023 22:44:26 +0000

ATLAS PhD Students Present at ISMAR 2023 Anonymous (not verified) Wed, 10/25/2023 - 16:44

Michael Kwolek

Billed as the premier conference for Augmented Reality (AR), Mixed Reality (MR) and Virtual Reality (VR), IEEE ISMAR was the perfect location for ATLAS community members to showcase their work this month.

ATLAS PhD students Rishi Vanukuru, Torin Hopkins and Suibi Che-Chuan Weng attended ISMAR 2023 in Sydney, Australia, from October 16-20, along with leading researchers in academia and industry.

Vanukuru presented his work on DualStream, a system for mobile phone-based spatial communication employing AR to give people more immersive tools to “share spaces and places.” He also participated in the “1st Joint Workshop on Cross Reality” with his research on using mobile devices to support collaboration.

Meanwhile, Hopkins and Weng displayed their respective research on improving ways for musicians to collaborate remotely.

Research ATLAS PhD students presented at ISMAR 2023

DualStream: Spatially Sharing Selves and Surroundings using Mobile Devices and Augmented Reality
Rishi Vanukuru, Suibi Che-Chuan Weng, Krithik Ranjan, Torin Hopkins, Amy Banić, Mark D. Gross, Ellen Yi-Luen Do

Abstract: In-person human interaction relies on our spatial perception of each other and our surroundings. Current remote communication tools partially address each of these aspects. Video calls convey real user representations but without spatial interactions. Augmented and Virtual Reality (AR/VR) experiences are immersive and spatial but often use virtual environments and characters instead of real-life representations. Bridging these gaps, we introduce DualStream, a system for synchronous mobile AR remote communication that captures, streams, and displays spatial representations of users and their surroundings. DualStream supports transitions between user and environment representations with different levels of visuospatial fidelity, as well as the creation of persistent shared spaces using environment snapshots. We demonstrate how DualStream can enable spatial communication in real-world contexts, and support the creation of blended spaces for collaboration. A formative evaluation of DualStream revealed that users valued the ability to interact spatially and move between representations, and could see DualStream fitting into their own remote communication practices in the near future. Drawing from these findings, we discuss new opportunities for designing more widely accessible spatial communication tools, centered around the mobile phone.

Exploring the use of Mobile Devices as a Bridge for Cross-Reality Collaboration [Workshop Paper]
Rishi Vanukuru, Ellen Yi-Luen Do

Abstract: Augmented and Virtual Reality technologies enable powerful forms of spatial interaction with a wide range of digital information. While AR and VR headsets are more affordable today than they have ever been, their interfaces are relatively unfamiliar, and a large majority of people around the world do not yet have access to such devices. Inspired by contemporary research towards cross-reality systems that support interactions between mobile and head-mounted devices, we have been exploring the potential of mobile devices to bridge the gap between spatial collaboration and wider availability. In this paper, we outline the development of a cross-reality collaborative experience centered around mobile phones. Nearly fifty users interacted with the experience over a series of research demo days in our lab. We use the initial insights gained from these demonstrations to discuss potential research directions for bringing spatial computing and cross-reality collaboration to wider audiences in the near future.

Investigating the Effects of Limited Field of View on Jamming Experience in Extended Reality [Poster Paper]
Suibi Che-Chuan Weng, Torin Hopkins, Shih-Yu Ma, Chad Tobin, Amy Banić, Ellen Yi-Luen Do

Abstract: During musical collaboration, extra-musical visual cues are vital for communication between musicians. Extended Reality (XR) applications that support musical collaboration are often used with headmounted displays such as Augmented Reality (AR) glasses, which limit the field of view (FOV) of the players. We conducted a three part study to investigate the effects of limited FOV on co-presence. To investigate this issue further, we conducted a within-subjects user study (n=19) comparing an unrestricted FOV holographic setup to Nreal AR glasses with a 52◦ limited FOV. In the AR setup, we tested two conditions: 1) standard AR experience with 52◦-limited FOV, and 2) a modified AR experience, inspired by player feedback. Results showed that the holographic setup offered higher co-presence with avatars.

Networking AI-Driven Virtual Musicians in Extended Reality [Poster]
Torin Hopkins, Rishi Vanukuru, Suibi Che-Chuan Weng, Chad Tobin, Amy Banić, Mark D. Gross, Ellen Yi-Luen Do

Abstract: Music technology has embraced Artificial Intelligence as part of its evolution. This work investigates a new facet of this relationship, examining AI-driven virtual musicians in networked music experiences. Responding to an increased popularity due to the COVID-19 pandemic, networked music enables musicians to meet virtually, unhindered by many geographical restrictions. This work begins to extend existing research that has focused on networked human-human interaction by exploring AI-driven virtual musicians’ integration into online jam sessions. Preliminary feedback from a public demonstration of the system suggests that despite varied understanding levels and potential distractions, participants generally felt their partner’s presence, were task-oriented, and enjoyed the experience. This pilot aims to open opportunities for improving networked musical experiences with virtual AI-driven musicians and informs directions for future studies with the system.

ATLAS PhD students Rishi Vanukuru, Torin Hopkins and Suibi Che-Chuan Weng attended ISMAR 2023 in Sydney in October to present research on AR, VR and MR.

Off

Traditional

White

Ellen Yi-Luen Do Presents Keynote on Fun with Creative Technology & Design at TaiCHI 2023

Anonymous — Wed, 13 Sep 2023 18:51:39 +0000

Ellen Yi-Luen Do Presents Keynote on Fun with Creative Technology & Design at TaiCHI 2023 Anonymous (not verified) Wed, 09/13/2023 - 12:51

Michael Kwolek

ATLAS Professor Ellen Yi-Luen Do had the opportunity to be a keynote speaker at TaiCHI 2023, a symposium hosted by the Taiwan Human-Computer Interaction Society at Taiwan University in Taipei. The event gathered researchers and practitioners across a range of backgrounds in technology, design and human factors to deepen community connections and explore new ideas.

Sessions included presentations on fabrication, perception, interactions and other timely topics, with a surprising range in mediums from humble materials like felt and puppets to advanced VR technologies and metaverse interactivity.

As director of the ACME Lab at ATLAS, Do and her team conduct research on using everyday items as interfaces, creating objects to think with, new ways of working, and methods and tools to help others make things. Do delivered her presentation, entitled “Fun with Creative Technology & Design”, advocating for playful computing with easily accessible materials like paper and cardboard, while highlighting ways to make toolkits for others to create for themselves.

The audience, which included experts in computer science, psychology, media, art, design and business responded enthusiastically, finding common ground in this relatable, inclusive approach to otherwise complex technologies. Do received a particularly warm reception from students in the field. She noted, “Several students came to thank me for my talk, stating that they learned so much from me, and that they never thought research could be this fun and interesting.”

Do expressed excitement for a few standout presentations from the conference including Ed Chi, Distinguished Scientist at Google DeepMind, who delivered a keynote on the large language model revolution. She said, “I was happy to learn that Bard will be a tool-use application applying to many of the Google apps and services people already use, including Maps, Sheets, Gmail, Docs, and more.”

She also called out FeltingReel: Density Varying Soft Fabrication with Reeling and Felting by Ping-Yi Wang and Lung-Pan Cheng as particularly intriguing.

Back in 2015, Do wrote the article “A flourishing field: a guide to HCI in China, Taiwan, and Singapore”, and saw the founding of Taiwan HCI. Looking back, she reflects, “I’m happy to see TaiCHI 2023 have 300 people registered with vibrant discussions, demos and posters. It's definitely growing!”

ATLAS Professor Ellen Yi-Luen Do presented on Fun with Creative Technology & Design as keynote speaker at TaiCHI 2023.

Off

Traditional

White

Sandra Bae, ATLAS PhD Student, Awarded at VIS 2023

Anonymous — Wed, 30 Aug 2023 15:47:51 +0000

Sandra Bae, ATLAS PhD Student, Awarded at VIS 2023 Anonymous (not verified) Wed, 08/30/2023 - 09:47

Michael Kwolek

Sandra Bae, PhD student and member of the Utility Research Lab and ACME Lab at ATLAS, has been honored with a Best Paper Honorable Mention at VIS 2023 for her research on network physicalizations.

Billed as “the premier forum for advances in theory, methods and applications of visualization and visual analytics”, VIS 2023 will be held in Melbourne, Australia, from October 22-27, and is sponsored by IEEE. The Best Papers Committee bestows honorable mentions on the top 5% of publications submitted.

The paper introduces a computational design pipeline to 3D print physical representations of networks enabling touch interactivity via capacitive sensing and computational inference.

[video:https://youtu.be/uv0Yu0WUeSQ]

A Computational Design Process to Fabricate Sensing Network Physicalizations
S. Sandra Bae, Takanori Fujiwara, Anders Ynnerman, Ellen Yi-Luen Do, Michael L. Rivera, Danielle Albers Szafir

Abstract
Interaction is critical for data analysis and sensemaking. However, designing interactive physicalizations is challenging as it requires cross-disciplinary knowledge in visualization, fabrication, and electronics. Interactive physicalizations are typically produced in an unstructured manner, resulting in unique solutions for a specific dataset, problem, or interaction that cannot be easily extended or adapted to new scenarios or future physicalizations. To mitigate these challenges, we introduce a computational design pipeline to 3D print network physicalizations with integrated sensing capabilities. Networks are ubiquitous, yet their complex geometry also requires significant engineering considerations to provide intuitive, effective interactions for exploration. Using our pipeline, designers can readily produce network physicalizations supporting selection-the most critical atomic operation for interaction-by touch through capacitive sensing and computational inference. Our computational design pipeline introduces a new design paradigm by concurrently considering the form and interactivity of a physicalization into one cohesive fabrication workflow. We evaluate our approach using (i) computational evaluations, (ii) three usage scenarios focusing on general visualization tasks, and (iii) expert interviews. The design paradigm introduced by our pipeline can lower barriers to physicalization research, creation, and adoption.

Bae describes potential use cases for sensing network physicalizations:

Accessibility visualization - Accessible visualizations (e.g., tactile visualizations) focus on making data visualization more inclusive, particularly for those with low vision or blindness. However, most tactile visualizations are static and non-interactive, which reduces data expressiveness and inhibits data exploration. This technique can create more interactive tactile visualizations.
AR/VR - Most AR/VR devices use computer vision (CV), but most devices using CV cannot reproduce the haptic benefits that we naturally leverage (holding, rotating, tracing) with our sense of touch. Past studies confirm the importance of tangible inputs when virtually exploring data. But creating tangible devices for AR/VR requires too much instrumentation to make them interactive. Our technique would enable developers to more easily produce fully functional, responsive controllers right from the printer within a single pass.

The work continues as Bae plans to pursue more complex designs and richer interactivity including:

Fabricating bigger networks - The biggest network Bae has 3D printed so far is 20 nodes and 40 links, but this is rather small for most network datasets. She will scale this technique to support bigger networks.

Supporting output - Interactive objects receive input (e.g., from touch) and produce output (e.g., light, sound, color change) in a controlled manner. The sensing network currently addresses the first part of the interaction loop by responding to touch inputs, but she next wants to explore how to support output.

Bae showcased this research along with fellow ATLAS community members at the Rocky Mountain RepRap Festival earlier this year. We’re excited to see where her innovative research leads next.

Off

Traditional

White