Chaehan So | Yonsei University (original) (raw)

Papers by Chaehan So

arXiv (Cornell University), Mar 3, 2020

People often rely on online reviews to make purchase decisions. The present work aimed to gain an... more People often rely on online reviews to make purchase decisions. The present work aimed to gain an understanding of a machine learning model's prediction mechanism by visualizing the effect of sentiments extracted from online hotel reviews with explainable AI (XAI) methodology. Study 1 used the extracted sentiments as features to predict the review ratings by five machine learning algorithms (knn, CART decision trees, support vector machines, random forests, gradient boosting machines) and identified random forests as best algorithm. Study 2 analyzed the random forests model by feature importance and revealed the sentiments joy, disgust, positive and negative as the most predictive features. Furthermore, the visualization of additive variable attributions and their prediction distribution showed correct prediction in direction and effect size for the 5-star rating but partially wrong direction and insufficient effect size for the 1-star rating. These prediction details were corroborated by a what-if analysis for the four top features. In conclusion, the prediction mechanism of a machine learning model can be uncovered by visualization of particular observations. Comparing instances of contrasting ground truth values can draw a differential picture of the prediction mechanism and inform decisions for model improvement.

International Journal of Human–Computer Interaction

Computer Supported Cooperative Work, Jul 15, 2023

Computer Animation and Virtual Worlds

Making Software Teams Effective

Making Software Teams Effective

Human Factors: The Journal of the Human Factors and Ergonomics Society

Objective The present work explores how the horizontal viewing angle of a virtual character’s fac... more Objective The present work explores how the horizontal viewing angle of a virtual character’s face influences perceptions of credibility and approachability. Background When encountering virtual characters, people rely both on credibility and approachability judgments to form a first impression of the depicted virtual character. Research shows that certain perceptions are preferred either on frontal or tilted faces, but not how approachability or credibility judgments relate to horizontal viewing angles in finer granularity between 0° and 45°. Method 52 participants performed a two-alternative forced choice (2AFC) task rating 240 pairwise comparisons of 20 virtual character faces shown in four horizontal viewing angles (0°, 15°, 30°, and 45°) on approachability and credibility. They also rated scales on individual differences based on the BIS-BAS framework (behavioral inhibition system, drive, and reward responsiveness), self-esteem, and personality traits (neuroticism, loneliness)....

Artificial Intelligence in HCI, 2020

When people buy products online, they primarily base their decisions on the recommendations of ot... more When people buy products online, they primarily base their decisions on the recommendations of others given in online reviews. The current work analyzed these online reviews by sentiment analysis and used the extracted sentiments as features to predict the product ratings by several machine learning algorithms. These predictions were disentangled by various methods of explainable AI (XAI) to understand whether the model showed any bias during prediction. Study 1 benchmarked these algorithms (knn, support vector machines, random forests, gradient boosting machines, XGBoost) and identified random forests and XGBoost as best algorithms for predicting the product ratings. In Study 2, the analysis of global feature importance identified the sentiment joy and the emotional valence negative as most predictive features. Two XAI visualization methods, local feature attributions and partial dependency plots, revealed several incorrect prediction mechanisms on the instance-level. Performing the benchmarking as classification, Study 3 identified a high no-information rate of 64.4% that indicated high class imbalance as underlying reason for the identified problems. In conclusion, good performance by machine learning algorithms must be taken with caution because the dataset, as encountered in this work, could be biased towards certain predictions. This work demonstrates how XAI methods reveal such prediction bias.

Research on data science has largely viewed data as an abstract input in service of algorithms de... more Research on data science has largely viewed data as an abstract input in service of algorithms developed by data scientists. In this view, data science activities are made sustainable by the efficient flow of data to improve the algorithms. Recent studies in CSCW and HCI, however, point to how the effectiveness of algorithms critically depends on sustainably collecting reliable, complete data situated in domain experts’ practices and settings. Drawing on ethnographic fieldwork and a pilot machine learning project at a craft brewery, we describe three types of situations where brewers’ data practices led to unreliable, incomplete data, and how such data practices limited the effectiveness of data science activities. We analyze sources of misalignment between their data practices and data science activities, which we use to offer design implications for sustainability. Extending research on end-user software development that views sustainability as driven by domain experts as ‘owners ...

J. of Design Research, 2019

The present study investigated the specific life circumstances that determine designers' happ... more The present study investigated the specific life circumstances that determine designers' happiness measured in terms of life satisfaction and subjective well-being. To this aim, self-reports of 252 participants in an online survey were analysed using psychological measurement instruments for pressure, self-aspects, happiness and mindfulness. The findings highlight that social pressure and time pressure are negatively related to designers' happiness and to positive self-aspects (self-esteem, creative self-efficacy) by a small to medium effect size (r = .21). Judging partially mediates this detrimental effect by a small effect size (r = .09), is highly related to social comparison frequency (r = .50) and global happiness (r = –.40). Present focus and present value relate more positively to global happiness than future focus and future value (r = .17). A positive implication for designers derives from the result that creative self-efficacy closely relates to positive emotions (r = .43) and life satisfaction (r = .40) but is unaffected by social and time pressure.

International Journal of Design Creativity and Innovation, 2019

This paper investigates how the immersion into a real-world user context influences design studen... more This paper investigates how the immersion into a real-world user context influences design students' design inspiration and mood improvement. To this aim, the present study analyzed the perception of 16 design students who immersed themselves into a dance studio and compared it with the external evaluation of 74 professional designers. Results indicate that (a) design students can attribute the immersion experience to multiple stimulus sources; (b) the mood improvement and design inspiration induced by the immersion experience predict their satisfaction level; and (c) the design inspiration induced by the immersion experience can be recognized by external observers. These findings can be harnessed for leveraging designers' user research effectiveness by embedding it in an immersion experience.

Journal of Integrated Design Research, 2018

The Design Journal, 2017

The purpose of this study was to test whether the priming of a brainstorming task by a persona in... more The purpose of this study was to test whether the priming of a brainstorming task by a persona increases ideational fluency and originality, i.e. the quantitative and qualitative dimensions of creative performance. We conducted a preliminary (n = 18) and final (n = 32) experiment with international students of business. These experiments revealed that priming of brainstorming by a persona increases originality of ideas by a large effect size (Cohen's d = .91, p = .02), and not significantly ideational fluency by a medium effect size (Cohen's d = .33, p = .39). As an alternative explanation to empathy, the found creativity effect may be attributed to priming that retrieves related memory items and thereby facilitates idea generation. As practical implications, design thinking practitioners can expect more original ideas and overcome design fixation if they brainstorm on a persona which is modelled in a concise and consistent way that caters to understanding the user need.

Journal of Engineering, Design and Technology, 2017

Purpose This paper aims to present a conceptual framework of how software teams can leverage the ... more Purpose This paper aims to present a conceptual framework of how software teams can leverage the implicit information of implemented acceptance tests to cater to the needs of decision makers. The research questions on this framework were how business stakeholders can receive project status information in an intuitive way and how this framework can guarantee the traceability of tests to requirements. Design/methodology/approach The conceptual framework delineates the design of an acceptance test framework in three aspects: how the requirements model reflects the evolving states of requirement maturity over a project, how the acceptance test model becomes synchronized with the requirements model without a traceability matrix and how the acceptance test model communicates business value to the decision makers. Findings In an industrial case study, the presented framework yielded the positive effects of intuitive understanding by business stakeholders, high test coverage of requirements...

The Design Journal, 2019

This paper investigates how nonexperts perceive digital design, and which psychological dimension... more This paper investigates how nonexperts perceive digital design, and which psychological dimensions are underlying this perception of design. It thus constructs a measurement instrument to analyse user response to online displayed design and to predict design preference. Study 1 let non

Lecture Notes in Business Information Processing, 2009

Summary. Rising interest on social-psychological effects of agile prac-tices necessitate the deve... more Summary. Rising interest on social-psychological effects of agile prac-tices necessitate the development of appropriate measurement instru-ments for future quantitative studies. This study has constructed such instruments for eight agile practices, namely iteration planning, itera-tive ...

Neural style transfer is a popular deep learning algorithm to generate images to mimic human arti... more Neural style transfer is a popular deep learning algorithm to generate images to mimic human artistry. This work applies the psychological method of the two-alternative forced choice (2afc) task to measure aesthetic preferences for neural style generated images. Portrait photos of three popular celebrities were generated by varying three parameters of neural style transfer in five configuration levels. Participants had to choose the image they preferred aesthetically from all pairwise combinations of configurations per style. The rate of being chosen was calculated for each neural style transfer configuration level. The findings show a differentiated picture of aesthetic preferences. On the one side, they indicate that people prefer images rendered with 500 iterations and a learning rate of 2e1, i.e. configurations that allow them to recognize the structure of the portrait image despite the stylization. On the other side, aesthetic preferences peak for two distinctly different conte...

arXiv (Cornell University), Mar 3, 2020

People often rely on online reviews to make purchase decisions. The present work aimed to gain an... more People often rely on online reviews to make purchase decisions. The present work aimed to gain an understanding of a machine learning model's prediction mechanism by visualizing the effect of sentiments extracted from online hotel reviews with explainable AI (XAI) methodology. Study 1 used the extracted sentiments as features to predict the review ratings by five machine learning algorithms (knn, CART decision trees, support vector machines, random forests, gradient boosting machines) and identified random forests as best algorithm. Study 2 analyzed the random forests model by feature importance and revealed the sentiments joy, disgust, positive and negative as the most predictive features. Furthermore, the visualization of additive variable attributions and their prediction distribution showed correct prediction in direction and effect size for the 5-star rating but partially wrong direction and insufficient effect size for the 1-star rating. These prediction details were corroborated by a what-if analysis for the four top features. In conclusion, the prediction mechanism of a machine learning model can be uncovered by visualization of particular observations. Comparing instances of contrasting ground truth values can draw a differential picture of the prediction mechanism and inform decisions for model improvement.

International Journal of Human–Computer Interaction

Computer Supported Cooperative Work, Jul 15, 2023

Computer Animation and Virtual Worlds

Making Software Teams Effective

Making Software Teams Effective

Human Factors: The Journal of the Human Factors and Ergonomics Society

Objective The present work explores how the horizontal viewing angle of a virtual character’s fac... more Objective The present work explores how the horizontal viewing angle of a virtual character’s face influences perceptions of credibility and approachability. Background When encountering virtual characters, people rely both on credibility and approachability judgments to form a first impression of the depicted virtual character. Research shows that certain perceptions are preferred either on frontal or tilted faces, but not how approachability or credibility judgments relate to horizontal viewing angles in finer granularity between 0° and 45°. Method 52 participants performed a two-alternative forced choice (2AFC) task rating 240 pairwise comparisons of 20 virtual character faces shown in four horizontal viewing angles (0°, 15°, 30°, and 45°) on approachability and credibility. They also rated scales on individual differences based on the BIS-BAS framework (behavioral inhibition system, drive, and reward responsiveness), self-esteem, and personality traits (neuroticism, loneliness)....

Artificial Intelligence in HCI, 2020

When people buy products online, they primarily base their decisions on the recommendations of ot... more When people buy products online, they primarily base their decisions on the recommendations of others given in online reviews. The current work analyzed these online reviews by sentiment analysis and used the extracted sentiments as features to predict the product ratings by several machine learning algorithms. These predictions were disentangled by various methods of explainable AI (XAI) to understand whether the model showed any bias during prediction. Study 1 benchmarked these algorithms (knn, support vector machines, random forests, gradient boosting machines, XGBoost) and identified random forests and XGBoost as best algorithms for predicting the product ratings. In Study 2, the analysis of global feature importance identified the sentiment joy and the emotional valence negative as most predictive features. Two XAI visualization methods, local feature attributions and partial dependency plots, revealed several incorrect prediction mechanisms on the instance-level. Performing the benchmarking as classification, Study 3 identified a high no-information rate of 64.4% that indicated high class imbalance as underlying reason for the identified problems. In conclusion, good performance by machine learning algorithms must be taken with caution because the dataset, as encountered in this work, could be biased towards certain predictions. This work demonstrates how XAI methods reveal such prediction bias.

Research on data science has largely viewed data as an abstract input in service of algorithms de... more Research on data science has largely viewed data as an abstract input in service of algorithms developed by data scientists. In this view, data science activities are made sustainable by the efficient flow of data to improve the algorithms. Recent studies in CSCW and HCI, however, point to how the effectiveness of algorithms critically depends on sustainably collecting reliable, complete data situated in domain experts’ practices and settings. Drawing on ethnographic fieldwork and a pilot machine learning project at a craft brewery, we describe three types of situations where brewers’ data practices led to unreliable, incomplete data, and how such data practices limited the effectiveness of data science activities. We analyze sources of misalignment between their data practices and data science activities, which we use to offer design implications for sustainability. Extending research on end-user software development that views sustainability as driven by domain experts as ‘owners ...

J. of Design Research, 2019

The present study investigated the specific life circumstances that determine designers' happ... more The present study investigated the specific life circumstances that determine designers' happiness measured in terms of life satisfaction and subjective well-being. To this aim, self-reports of 252 participants in an online survey were analysed using psychological measurement instruments for pressure, self-aspects, happiness and mindfulness. The findings highlight that social pressure and time pressure are negatively related to designers' happiness and to positive self-aspects (self-esteem, creative self-efficacy) by a small to medium effect size (r = .21). Judging partially mediates this detrimental effect by a small effect size (r = .09), is highly related to social comparison frequency (r = .50) and global happiness (r = –.40). Present focus and present value relate more positively to global happiness than future focus and future value (r = .17). A positive implication for designers derives from the result that creative self-efficacy closely relates to positive emotions (r = .43) and life satisfaction (r = .40) but is unaffected by social and time pressure.

International Journal of Design Creativity and Innovation, 2019

This paper investigates how the immersion into a real-world user context influences design studen... more This paper investigates how the immersion into a real-world user context influences design students' design inspiration and mood improvement. To this aim, the present study analyzed the perception of 16 design students who immersed themselves into a dance studio and compared it with the external evaluation of 74 professional designers. Results indicate that (a) design students can attribute the immersion experience to multiple stimulus sources; (b) the mood improvement and design inspiration induced by the immersion experience predict their satisfaction level; and (c) the design inspiration induced by the immersion experience can be recognized by external observers. These findings can be harnessed for leveraging designers' user research effectiveness by embedding it in an immersion experience.

Journal of Integrated Design Research, 2018

The Design Journal, 2017

The purpose of this study was to test whether the priming of a brainstorming task by a persona in... more The purpose of this study was to test whether the priming of a brainstorming task by a persona increases ideational fluency and originality, i.e. the quantitative and qualitative dimensions of creative performance. We conducted a preliminary (n = 18) and final (n = 32) experiment with international students of business. These experiments revealed that priming of brainstorming by a persona increases originality of ideas by a large effect size (Cohen's d = .91, p = .02), and not significantly ideational fluency by a medium effect size (Cohen's d = .33, p = .39). As an alternative explanation to empathy, the found creativity effect may be attributed to priming that retrieves related memory items and thereby facilitates idea generation. As practical implications, design thinking practitioners can expect more original ideas and overcome design fixation if they brainstorm on a persona which is modelled in a concise and consistent way that caters to understanding the user need.

Journal of Engineering, Design and Technology, 2017

Purpose This paper aims to present a conceptual framework of how software teams can leverage the ... more Purpose This paper aims to present a conceptual framework of how software teams can leverage the implicit information of implemented acceptance tests to cater to the needs of decision makers. The research questions on this framework were how business stakeholders can receive project status information in an intuitive way and how this framework can guarantee the traceability of tests to requirements. Design/methodology/approach The conceptual framework delineates the design of an acceptance test framework in three aspects: how the requirements model reflects the evolving states of requirement maturity over a project, how the acceptance test model becomes synchronized with the requirements model without a traceability matrix and how the acceptance test model communicates business value to the decision makers. Findings In an industrial case study, the presented framework yielded the positive effects of intuitive understanding by business stakeholders, high test coverage of requirements...

The Design Journal, 2019

This paper investigates how nonexperts perceive digital design, and which psychological dimension... more This paper investigates how nonexperts perceive digital design, and which psychological dimensions are underlying this perception of design. It thus constructs a measurement instrument to analyse user response to online displayed design and to predict design preference. Study 1 let non

Lecture Notes in Business Information Processing, 2009

Summary. Rising interest on social-psychological effects of agile prac-tices necessitate the deve... more Summary. Rising interest on social-psychological effects of agile prac-tices necessitate the development of appropriate measurement instru-ments for future quantitative studies. This study has constructed such instruments for eight agile practices, namely iteration planning, itera-tive ...

Neural style transfer is a popular deep learning algorithm to generate images to mimic human arti... more Neural style transfer is a popular deep learning algorithm to generate images to mimic human artistry. This work applies the psychological method of the two-alternative forced choice (2afc) task to measure aesthetic preferences for neural style generated images. Portrait photos of three popular celebrities were generated by varying three parameters of neural style transfer in five configuration levels. Participants had to choose the image they preferred aesthetically from all pairwise combinations of configurations per style. The rate of being chosen was calculated for each neural style transfer configuration level. The findings show a differentiated picture of aesthetic preferences. On the one side, they indicate that people prefer images rendered with 500 iterations and a learning rate of 2e1, i.e. configurations that allow them to recognize the structure of the portrait image despite the stylization. On the other side, aesthetic preferences peak for two distinctly different conte...

Manche Menschen mit betont selbstbewusstem Auftreten scheinen Gefallen daran zu finden, andere zu... more Manche Menschen mit betont selbstbewusstem Auftreten scheinen Gefallen daran zu finden, andere zu erniedrigen. Wieso haben sie das nötig? Die jüngste psychologische Forschung zeigt: Tief drinnen ist es mit dem Selbstbewusstsein dieser Leute nicht allzu weit her.

How does good teamwork emerge? Can we control mechanisms of teamwork? The author has analyzed... more How does good teamwork emerge?
Can we control mechanisms of teamwork?
The author has analyzed these questions in a study involving 227 participants of 55 software development teams. First, he empirically confirmed his teamwork model based on innovation research, goal setting and control theory. Second, he measured the impact of a wide selection of agile practices on these teamwork mechanisms. Third, he explained these impacts based on a thorough review of current psychological research.
This book is intended for people working in agile contexts as they will gain insight into the complexity of how «good teamwork» emerges. This insight on team dynamics may also prove valuable for upper management for calibrating agile practices and «soft factors», thus increasing the effectiveness of software teams.

2nd International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2020, 2020

This paper investigates how accurately the prediction of being an introvert vs. extrovert can be ... more This paper investigates how accurately the prediction of being an introvert vs. extrovert can be made with less than ten predictors. The study is based on a previous data collection of 7161 respondents of a survey on 91 personality and 3 demographic items. The results show that it is possible to effectively reduce the size of this measurement instrument from 94 to 10 features with a performance loss of only 1%, achieving an accuracy of 73.81% on unseen data. Class imbalance correction methods like SMOTE or ADASYN showed considerable improvement on the validation set but only minor performance improvement on the testing set.

2nd International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2020, 2020

This paper analyzes how candidate choice prediction improves by different psychological predictor... more This paper analyzes how candidate choice prediction improves by different psychological predictors. To investigate this question, it collected an original survey dataset featuring the popular TV series "Game of Thrones". The respondents answered which character they anticipated to win in the final episode of the series, and explained their choice of the final candidate in free text from which sentiments were extracted. These sentiments were compared to feature sets derived from candidate likeability and candidate personality ratings. In our benchmarking of 10-fold cross-validation in 100 repetitions, all feature sets except the likeability ratings yielded a 10-11% improvement in accuracy on the holdout set over the base model. Treating the class imbalance with synthetic minority oversampling (SMOTE) increased holdout set performance by 20-34% but surprisingly not testing set performance. Taken together, our study provides a quantified estimation of the additional predictive value of psychological predictors. Likeability ratings were clearly outperformed by the feature sets based on personality, emotional valence, and basic emotions.