Perugina. Airport. Sepia. Soldier. Rome. Yet, another precious family photo lost in Google Photos among mundane recipe screenshots. And, yet, another reminder that it’s all about the prompt. The prompt I never set up and lazily relegated to the wisdom of a Google algorithm.

When we’re using prompts in search, we use them like fishing lures slowly trolling cyberspace. The more technically proficient we are, the less we are bothered by combing through junk to reach what we need. But the emerging GPT-2 and GPT-3 language models has turned that expectation upside down, revolutionizing what we can do via prompt engineering. 

Now, we can not only retrieve, we can create.

And the breadth is relatively wide from NLP to AI synthesis, emails can be generated and images can be rendered through iterations of text prompts. The trick to getting it right is through effective prompt engineering. And that is a cultural issue. (No different than ethnographies are only as good as their recruitment.)

Lisa Li Xiang and Percy Liang write about prefix tuning, “a lightweight alternative to fine-tuning for natural language generation tasks” (Li, 2021). They propose that “Fine-tuning is the de facto way of leveraging large pretrained language models for downstream tasks. However, fine-tuning modifies all the language model parameters and therefore necessitates storing a full copy for each task…good generation performance can also be attained by updating a very small prefix.”

It’s within these small spaces that cultural context can be applied. We began this discussion in March 2020 at the Intelligent Human Systems Integration Conference in Modena, Italy. The challenge is to have the design remain open to human potential, rather than structural control.

“It’s no surprise that most people engaged in ‘tech-mediated intimacy’ prefer sexting with ‘adult’ chatbots over sex robots. They can imagine a human at the other end. As designers, we need to understand human traits and be ahead of that moment” (Stock, 2020).

Let’s not forget that according to Encyclopedia Brittanica, “the words engine and ingenious are derived from the same Latin root, ingenerare, which means “to create.” This is one thing when we are haphazardly throwing prompts at Midjourney. There, it’s okay to have others run with your prompt and see where it goes.

Jason M. Allen, a maker of tabletop games, won the 2022 Colorado State Fair’s annual art competition for his AI-generated work using Midjourney. “Théâtre D’opéra Spatial,” took first place in the digital category. “Some artists defended Mr. Allen, saying that using A.I. to create a piece was no different from using Photoshop or other digital image-manipulation tools, and that human creativity is still required to come up with the right prompts to generate an award-winning piece” (Roos, 2022).

But, what about when we need to land effectively and quickly get to that promising work? That’s when we need to start tuning upfront.

Already, virtual prototyping is becoming essential for automotive research and design (Girling, 2022). Think about how far we could go without heavy investments in building upfront.

Imagine if we could simply focus on the thinking? Implications reach far beyond the tangible.

Consider occupational identities. Julia Yates & Sharon Cahill state “Evidence suggests that occupational prototypes have an impact on career decisions. Psychology undergrads were asked to evoke a typical member of each of four occupational groups and describe their prototype in detail. A classic grounded theory analysis identified the characteristics which were symbolised by the features of the prototypes and resulted in eight dimensions: warm, energetic, fun, intelligent, conventional, highbrow, successful and cool” (Yates, 2019). What if we could leverage prompt engineering to bring better context to education and career design too? But, then again, what if I could simply find that missing Google photo?

– by Tim Stock and Marie Lena Tupot

Works cited

Girling, W. (2022, July 29). Virtual prototyping is becoming essential for automotive R&D. Automotive World. Retrieved September 2, 2022, from https://www.automotiveworld.com/articles/virtual-prototyping-is-becoming-essential-for-automotive-rd/ 

Li, X. L., & Liang, P. (2021, January 1). Prefix-tuning: Optimizing continuous prompts for generation. arXiv.org. Retrieved September 2, 2022, from https://arxiv.org/abs/2101.00190

Roose, K. (2022, September 2). An a.i.-generated picture won an art prize. artists aren’t happy. The New York Times. Retrieved September 2, 2022, from https://www.nytimes.com/2022/09/02/technology/ai-artificial-intelligence-artists.html 

Stock, T. J., & Tupot, M. L. (2020, January 22). Pre-emptive culture mapping: Exploring a system of language to better understand the abstract traits of human interaction.  Intelligent Human Systems Integration 2020. IHSI 2020. Advances in Intelligent Systems and Computing, vol 1131. Springer, Cham. https://doi.org/10.1007/978-3-030-39512-4_88

Yates, J., & Cahill, S. (2019). The characteristics of prototypical occupational identities: A grounded theory of four occupations. British Journal of Guidance & Counselling, 49(1), 115–131. https://doi.org/10.1080/03069885.2019.1706154