Responsive Mobile Environments

A reflection on the invisible and otherworldly qualities in everyday technologies

© 2022. All rights reserved.

Investigation II - Device Hauntologies / Haunted Devices

Investigation II: Device Hauntologies / Haunted Devices

tldr: Engage a line of spooky inquiry. Unsettle the present by making a creative technology.
Part I Individual: To begin, identify a precedent project (see Case Study - Technical above). Review its own past and experiment with it by remaking/resurrecting it or an aspect of it or its experience. Use this as a resource for design in Part II.
Part II Collaborative (Pairs): Prepare an intentionally haunted- or hauntologically-informed research product that can be encounters in the conditions of everyday life.

About this Investigation

Investigations are a series of small exercises designed to explore a conceptual space and culminates with a made artefact. The format is rapid explorations of a theme, idea or theory. This investigation will be organized in two parts: first, a short four week exercise to become familiar with developing discursive concepts and prototypes; and second, a four week collaborative exploration that generates a refined discursive design.

In the last module, we introduced the frame of spooky technology and how it might be a productive means to attend to the frictions and breakdowns we encounter with everyday intelligences. This mdoule will pick up this thread. We will consider how mental models break down around our current technology, what gives rise to spooky or haunting outcomes, and how we might reflect some of those concerns by producing critical, speculative, and discursive technologies?

_“Devices whose computational capabilities exceed human comprehension have become our trusted companions, and yet we hardly know how they work.” - Betti Marenko

In this module, we’ll continue this exploration, explore this prompt, and ask questions about ubiquitous computing, by creating new and alternative smart home products that embody superstitions, rituals and belief systems.

Learning goals

This investigation will ask you to engage a spooky line of design inquiry and develop a discursive prototype in response. We’ll also develop a body of knowledge that explores how we can leverage the supernatural and the superstitious as a resource for design inquiry. Finally, we’ll explore the development of skills with the production of new forms of ubiquitous, physical and tangible computing and introduce strategies for prototyping everyday intelligences. As part of this exercise, you will:

  • Develop an understanding of concepts like critical making, counterfactuals, and material speculation and how they relate to the design of alternative devices;

  • Investigate the concerns and considerations that currently surround everyday technology and smart home devices;

  • Examine existing technologies and precedents to develop a critical position around issues such as predictability, privacy, or pervasive data;

  • Explore spookiness as a resource for critically informed design inquiry and identify strategies for engaging it in hands-on inquiry;

  • Build skills with prototyping hardware, electronics, and intelligent processes using tools like the Particle Platform, Alexa Skills Kit and Teachable Machine.

  • Speculate on how hybrid objects (blended physical digital objects) can be produced to present alternatives to contemporary interactions with devices;

  • Work collaboratively in an applied investigation to tease-out the broader considerations, issues and requirements in building alternative belief-based or superstitious smart home products.

Context and Precedents

Believe it Yourself Automata Farm, 2018

BIY: Believe it Yourself

BIYTM is a recent project by Automato.farm, presents a set of three belief-based computing kits. Each is comprised of a Raspberry Pi, sensors, and outputs for machine learning processes. A critical but playful exploration is grounded in close work with experts in alternative belief systems like divination and fortune telling from different cultures to translate their knowledge authentically into digital forms. By digitizing divination practices, BIY empowers non-objective interpretation of the world through AI-powered microcontrollers; it offers a nuanced interchange between belief, explainability and systems. Not only does this suggest a process for critical production, but it highlights the broader potential for divination, superstition, and ritualistic practices to be embedded into new digital devices and critical prototypes.

BIY, taken with a lineage of work in rituals suggests the value of new material enactments that examine and support alternative, occult, otherworldly rituals and practices.

See:

Ghost Bug and Wave Detectors. James Pierce and Carl DiSalvo. (2017)

Ghost Bug and Wave Detectors. James Pierce and Carl DiSalvo. (2017)

Ghost Bug and Wave Detectors James Pierce and Carl DiSalvo. (2017)

These are “a set of design packets that probe relationships between the paranormal, surveillance, and electro-pollution.” These prototypes and enactments suggest provocative ways to detect electronic ‘spirits’, reveal and perform Hertizan forces, or help to make visible the hidden systems and data we are entangled with avaiable for dialog.

See:

  • Pierce, J. and Carl DiSalvo. (2017) Projecting Network Anxieties with Alternative Design Metaphors. In Proceedings of DIS Conference on Designing Interactive Systems, Edinburgh, UK. DIS ’17. ACM Press, New York, NY.

eGregor Christine Geeng & Anon, 2020

eGregor is based on ‘an eldritch privacy mental model’ and recasts voice assistants like Alexa and Google Home as malevolent actors in our homes. Geeng explains: “Using science fiction concepts in conjunction with visceral notice, a concept that eschews purely text-based privacy indicators, eGregor more clearly represents the data prac- tices of a hypothetical parent company. It has various aes- thetic and auditory indicators and an intuitive and terrifying persona.” This project suggests how re-interpretations of existing smart home products can be a valuable strategies to present issues of privacy and explainability

Ghosts in the Smart Home. Joseph Lindley, Adrian Gradinar, and Paul Coulton (2020)

Ghosts in the Smart Home

This experimental animistic performance casts smart home devices as the leads in a short film. This suggests other ways to prototype and perform critical encounters with alternative IoT devices and “to establish new ways to see, to be, and to know, which respond to the 21st century’s complex socio-technical systems.”

See:

Content and Methods

To learn about the wide range of topics in deploying spooky lines of design inquiry, on Tuesday we will introduce conceptual frames, design approaches, and speculate together. On Thursday, we’ll build new skills with discursive prototyping by progressively introducing a suite of tools that you can use within this project to materalize strange, unsettling or alternative approaches to everyday devices.

Importantly, we’re going to learn from each other and explore this space together. Each of you will research a topic of interest to you as well as share relevant case study that can be resources for our making. This will help you build familiarity and give us as a group a catalog we can draw from in our explorations. The review of this catalog will provide groundwork to building an informed response to the creative project. Using your research, you’ll collaboratively develop a conceptual design and realize a ‘prototype’ of a haunted or hauntologically informed device.

Specifically, this module will introduce themes surrounding the electromagnetic imaginary, the history of hauntings, and creative responses. We’ll introduce a series of approaches to making haunted or hauntologically informed devices – through spooky lines of inquiry – and how they might intersect with ubiquitous computing. To do this, we’ll start by conducting weekly creative experiments that examine and practice these approaches for “open-ended interpretation and orientation” around systems and processes.” Case studies of research and practice will examine and suggests the value of new material enactments that examine and support alternative, occult, otherworldly encouners. We’ll speculate on alternative design metaphors and embodiments that attend to the aesthetics, systems and cultures of these supernatural traditions. In support this, we’ll continue to explore methods for designing provocative objects through making and introduce technical aporoaches to doing so. Specifically, we’ll look at how methods of material speculation can to craft experiential and critical objects.

Schedule

Date Type Description
Tues, Feb 1 Intro Review of Investigation I.
Intro to the Module
Thurs, Feb 3 Tech Intro/Refresher to the Particle Platform
Tues, Feb 8 Concepts Review of Cases.
Mental Models and Spooky User Research.
Thurs, Feb 10 Tech Introduction to Teachable Machine
Tues, Feb 15 Concepts Review of Spooky User Research.
Critical Design, Engineering and Making.
Thurs, Feb 17 Tech Building Alexa Skills (Part I)
Discussion on Tech Cases.
Tues, Feb 22 Concepts Review of Thinkpieces.
Electromagnetic Imaginaries, Counterfactuals and Material Speculations
Thurs, Feb 24 Tech Building Alexa Skills (Part II): Integrating with Particle
Discussion on proposal.
Tues, Mar 1 Concepts Proposal Reviews
Hauntologies and Haunted Research Products.
Thurs, Mar 3 Tech Intro to Webhooks
Exploring OpenAI’s GPT-3
Tues, Mar 8 No Class Spring Break
Thurs, Mar 10 No Class Spring Break
Tues, Mar 15 Concepts Desk Crits
Networked Anxieties and Beliefs.
Thurs, Mar 17 No Class Working with I2C.
Working with Accelerometers.
Tues, Mar 22 Desk Crits Feedback on creative project development;
15 mins per group.
Thurs, Mar 24 No Class Accelerometers continued.
Working with TensorFlowLite.
Tues, Mar 29 Concepts Crit of creative project.
Prepare a lightning Demo - 15 mins per group; 5 mins demo/presentation. 10 mins discussion.

Deliverables and Deadlines

Due Date Deliverable Details
Tues, Feb 8 Case Identify and describe a case study that explores a concept related to this module. Share on Slack in #cases
Tues, Feb 15 Exploration Conduct spooky user research with someone from beyond the course. Report your findings.
Thur, Feb 17 Case Identify and describe a case study of a hybrid object to represence. Share on Slack in #cases.
Thur, Feb 17 Project Log Post a short update on your experiments with critical making
Tues, Feb 22 Think Piece Research a think piece on haunted or hauntologically informed devices on Slack in #thinkpieces.
Thur, Feb 24 Project Log Post a short update on your experiments with critical making
Tues, Mar 1 Proposal Create a proposal for your creative project (200 words + illustrations) and share on the Slack
Tues, Mar 1 Exploration As a group, investigate and report on a historical analog for your project.
Thur, Mar 3 Project Log Post a short update on your experiments with critical making
Tues, Mar 15 Exploration Refine a short ghost story about your proposal
Tues, Mar 15 Project Bring a rough cut artifact to class for review and discussion.
Thur, Mar 17 Project Log Post a short update on your experiments with critical making
Tues, Mar 22 Project Bring a refined artifact to class to present and discuss during desk crits
Thur, Mar 24 Project Log Post a short update on your experiments with critical making
Tues, Mar 29 Project Present your prototype in class.
Wed, Mar 30, midnight Documentation Deliver documentation of your creative project

Readings

Review For Class

Thurs, Feb 23 Excerpts from Spooky Technology (forthcoming). Section 1: ‘Glitches’
Tues, Feb 8 Graham Dove and Anne-Laure Fayard. 2020. Monsters, Metaphors, and Machine Learning. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–17. DOI:https://doi.org/10.1145/3313831.3376275. Watch talk at: https://www.youtube.com/watch?v=fds8eZ7uyzY&list=PLqhXYFYmZ-VctgnS59-jZt13-yC4DXvGm
Thur, Feb 10 Precedent: Objectifier - Bjørn Karmann (2016) https://bjoernkarmann.dk/objectifier
Tues, Feb 15 Bleecker, Julian. “Design fiction: A short essay on design, science, fact and fiction. 2009.” Retrieved June 16 (2015).;
The Critical Engineering Manifesto https://criticalengineering.org
Thur, Feb 17 Precedent: Chthonic Rites - Wesley Goatley (2109) https://www.wesleygoatley.com/chthonic-rites/
Tues, Feb 22 Wakkary, Ron, William Odom, Sabrina Hauser, Garnet Hertz, and Henry Lin. “A short guide to material speculation: Actual artifacts for critical inquiry.” Interactions 23, no. 2 (2016): 44-48. DOI: https://dl.acm.org/doi/fullHtml/10.1145/2889278
Recommended Chapter 5. A Methodological Playground: Fictional Worlds and Thought Experiments. Dunne, Anthony, and Fiona Raby. Speculative everything: design, fiction, and social dreaming. MIT press, 2013.
Thur, Feb 24 Precedent: Chthonic Rites - Wesley Goatley (2109) https://www.wesleygoatley.com/chthonic-rites/
Tuesday, Mar 1 Fisher, M. (2012). What is hauntology?. Film Quarterly, 66(1), 16-24.

Odom, William, Ron Wakkary, Youn-kyung Lim, Audrey Desjardins, Bart Hengeveld, and Richard Banks. “From research prototype to research product.” In Proceedings of the 2016 CHI conference on human factors in computing systems, pp. 2549-2561. 2016. https://dl.acm.org/doi/abs/10.1145/2858036.2858447
Tuesday, Mar 15 Chapter 1: Mediums and Media from Haunted media: Electronic presence from telegraphy to television. Sconce, J. (2000). Duke University Press.

Pierce, James, and Carl DiSalvo. “Addressing network anxieties with alternative design metaphors.” In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1-13. 2018. DOI: https://doi.org/10.1145/3173574.3174123

Case Study: Conceptual

Identify and critically review a case study on designing technology around alternative beliefs, rituals and practices. The focus here is on a product or project that presents an interesting approach, method or strategy that can be leveraged in your own work. Report your discoveries. Read the full description.

Case Study: Technical

Identify and critically review a case study on designing technology around alternative beliefs, rituals and practices. The focus here is on a product or project that presents an interesting approach, method or strategy that can be leveraged in your own work. Report your discoveries. Read the full description.

Think Piece

On Haunted Devices and Hauntologies: Research and write a topical essay that raises an interesting question or perspective related to the themes of the module: relationship between designed technologies (ubiquitous computing, tangible devices, designed technologies) and the supernatural (myths, magic, monsters, folklore, superstitions, alternative belief systems, rituals and practices). Document and report your findings to the class and reflect on their implication for what and how we’ll make. Read the full description.

Creative Project - Brief and Approach

Engage a line of spooky inquiry. Unsettle the present by making a creative technology.

Part I: Re;presencing (individual) - 4 weeks

To begin, identify a precedent project (see Case Study - Technical above). Review its own past and experiment with it by remaking/resurrecting it or an aspect of it or its experience. Use this as a resource for design in Part II.

Within the first half of our investigation, we’ll focus on building resources, skills and familiarity with approaches through hands-on exercises. This will provide you with low-stakes ways to quickly inquire and engage with our exploration space.

Creative Experiments

During investigations, we’ll use creative experiments to try out ideas in low-stakes practices. These will range from quick design mock ups through to conducting home tours and interviews or engaging in out-of-class experiences that build first-hand knowledge of our topics.

Project Logs

On a weekly basis, spend 1-3 hours experimenting with electronics, components, experiences and making hybrid objects. Document your experiments. Read the full description of this activity.

Part II: Haunted Devices / Hauntologies (Pairs) - 4 weeks

With our quick experiments complete, we’ll use the second four weeks to dig in and develop a well-realized discursive design.

Part II Collaborative (Pairs): Prepare an intentionally haunted- or hauntologically-informed research product that can be encounters in the conditions of everyday life.

Read the full brief.

Resources

Below is a list of additional online material that relates to the module and provides a starting point for your explorations. This is by no means exhaustive i.e. you should read/research beyond it.

Projects

Prototyping Technologies & Tools

Collectives

Papers

  • Addressing network anxieties with alternative design metaphors - Carl DiSalvo, James Pierce (2018)

    Pierce, J., & DiSalvo, C. (2018, April). Addressing network anxieties with alternative design metaphors. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1-13).

  • A short guide to material speculation: Actual artifacts for critical inquiry - Garnet Hertz, Henry Lin, Ron Wakkary, Sabrina Hauser, Will Odom (2016)

    Wakkary, R., Odom, W., Hauser, S., Hertz, G., & Lin, H. (2016). A short guide to material speculation: Actual artifacts for critical inquiry. Interactions, 23(2), 44-48.

  • Animist User Expectations in a Ubicomp World: A Position paper for ‘Lost in Ambient Intelligence - Mike Kuniavsky (2007)

    Kuniavsky, M. (2007). Animist User Expectations in a Ubicomp World: A position paper for ‘Lost in Ambient Intelligence.’. San Francisco.

  • Witchcraft and HCI: Morality, modernity, and postcolonial computing in rural Bangladesh - Sharifa Sultana, Syed Ishtiaque Ahmed (2019)

    Sultana, S., & Ahmed, S. I. (2019, May). Witchcraft and hci: Morality, modernity, and postcolonial computing in rural bangladesh. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1-15).

  • Ghosts in the Smart Home - Adrian Gradinar, Joseph Lindley, Paul Coulton (2020)

    Lindley, J., Gradinar, A., & Coulton, P. (2020, July). Ghosts in the Smart Home. In Companion Publication of the 2020 ACM Designing Interactive Systems Conference (pp. 465-468).

  • IoT Data in the Home: Observing Entanglements and Drawing New Encounters - Audrey Desjardins, Cayla Key, Heidi R. Biggs, Jeremy E. Viny

    Desjardins, A., Biggs, H. R., Key, C., & Viny, J. E. (2020, April). IoT data in the home: Observing entanglements and drawing new encounters. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-13).

  • A hauntology of participatory speculation - Cally Gatehouse (2020)

    Gatehouse, C. (2020, June). A hauntology of participatory speculation. In Proceedings of the 16th Participatory Design Conference 2020-Participation (s) Otherwise-Volume 1 (pp. 116-125).

  • Design Fiction: A short essay on design, science, fact and fiction - Julian Bleecker (2009)

People

Articles

Prototyping Technologies, Tools and Examples

Collectives