#159,094 | AsPredicted (original) (raw)

'Human-AI Collaboration & Creative Engagement (Replication Study)'

AsPredicted #: 159,094

Author(s)
Suqing Wu (Zhejiang University) - s.wu@zju.edu.cn
Yukun Liu (Zhejiang University) - liuyk@shanghaitech.edu.cn
Mengqi Ruan (Zhejiang University) - ruanmengqi@zju.edu.cn
Siyu Chen (Zhejiang University) - siyu_chen@zju.edu.cn
Xiao-Yun Xie (School of Management, Zhejiang University) - xiexy@zju.edu.cn

Pre-registered on
2024/01/22 20:32 (PT)

1) Have any data been collected for this study already?
No, no data have been collected for this study yet.

2) What's the main question being asked or hypothesis being tested in this study?
The primary focus of this study is to examine the persisting effects of human-Generative AI (GenAI) collaboration on individual human workers' psychological experiences and performance in both the collaborative task and the subsequent human solo task.

We have conducted a study (Study 1) that supported our ideas. The goal of the current study is to replicate those effects following the same design in Study 1. In this study, however, we will adopt two different task scenarios: 'Email Composition Task' for the human-GenAI collaborative task and a 'Product Promotion Task' for the human solo task.

We aim to demonstrate that the performance augmentation effect observed during initial collaboration with GenAI would continue to influence human performance in a subsequent human solo task. Additionally, we also expect that transitioning from a human-GenAI collaborative task to a human solo task would lead to changes in psychological experiences, including regained sense of control, reduced intrinsic motivation, and increased boredom.

3) Describe the key dependent variable(s) specifying how they will be measured.
Task Engagement: Measured with a three-item scale (Rich, LePine & Crawford, 2010).
Task Performance: Operationalized through subjective and objective evaluations in both tasks. Independent raters will rate the quality of participants' outputs, and the outputs will be objectively evaluated using appropriate measures.
Perceived Sense of Control: Measured with a three-item scale (Greenaway et al., 2015).
Boredom: Measured with a four-item scale (van Hooft & van Hooff, 2018).
Intrinsic Motivation: Measured with a four-item scale (Shin & Grant, 2019).

4) How many and which conditions will participants be assigned to?
In this replication study, participants will be assigned to one of two conditions:

Collaboration-Solo Condition:
• Task 1 (Human-GPT collaboration Task): Participants will collaborate with ChatGPT to compose an email. This task is designed to assess how ChatGPT assistance impacts human workers' task performance and their psychological experiences.
• Task 2 (Human-Solo Task): Following the collaborative task, participants will individually engage in a 'Product Promotion Task'. This task aims to evaluate the persisting effects of prior collaboration with ChatGPT on human workers' task performance and their psychological experiences.

Solo-Solo Condition:
• Task 1 (Human-Solo Task): Participants will independently compose an email on their own, without the assistance of ChatGPT. This serves as a control condition to assess baseline performance and psychological experiences in a human solo work setting.
• Task 2 (Human-Solo Task): Participants will then proceed to the 'Product Promotion Task', performed on their own. This will allow for comparisons of the effects of prior conditions on human workers' task performance and their psychological experiences in the subsequent task.

5) Specify exactly which analyses you will conduct to examine the main question/hypothesis.
We will conduct independent samples t-tests, multiple regressions, moderation, and mediation analyses, among others.

6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations.
We will use an attention check to identify participants whose data are less reliable and thus should not be included in analyses.

7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined.
We have determined that "d = 0.20" represents the smallest effect size of theoretical interest for our study. Therefore, we conducted an a priori power analysis using G*Power, factoring in this effect size. To detect this effect with an α error probability of 0.05 and a power (1-β error probability) of 0.80, the estimated sample size required for our replication study is 788 participants.

8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?)
We will measure and control task load (Reid & Nygren, 1988) after each task, respectively. For explorative purpose, we will measure individual self-construal, creative role identity, and cognitive styles, investigating their moderating effects.

Version of AsPredicted Questions: 2.00