Feed-O-Meter (2026)

1. Summary of the Research

This paper is a collaboration between four researchers in KAIST’s Department of Industrial Design and one in its School of Computing. It was published in volume 208 (January 2026) of Elsevier’s International Journal of Human-Computer Studies (IJHCS).^{¹Lim, H., Choi, D., Choi, D. E., Nam, S., & Hong, H. (2026). Feed-O-Meter: Investigating AI-generated mentee personas as interactive agents for scaffolding design feedback practice. International Journal of Human-Computer Studies, 208, 103687. doi:10.1016/j.ijhcs.2025.103687} The China Computer Federation (CCF) classifies IJHCS as a Category A journal alongside ACM TOCHI in its recommended list for human-computer interaction and ubiquitous computing.^{²China Computer Federation (CCF). Recommended journals in human-computer interaction and ubiquitous computing.}

DxD Lab, led by corresponding author Professor Hwajung Hong, connects design, interaction, and data. Its research aims to extend human capabilities and build an inclusive society through human–AI collaboration.^{³DxD Lab. Lab introduction and research aims · KAIST Department of Industrial Design. DxD Lab introduction.} Feed-O-Meter can be read as an application of this direction to education in design feedback.

Feed-O-Meter is a system in which the user acts as a mentor, giving feedback to an AI mentee playing the role of a novice design student. The Feedback Reflection Interface (FRI) presents evaluation information, the AI mentee’s facial expressions and inner thoughts, and supports follow-up questions. Users reflect on how they give feedback by observing the AI mentee’s responses and changes to its ideas.

The research asks how students perceive the experience of giving feedback to an AI mentee (RQ1), and how FRI affects their feedback skills (RQ2). These questions operate at the micro level of individual interaction and performance, which is appropriate for exploring a system’s initial effects. Twenty-four Korean undergraduate and graduate students in design experienced both conditions, with and without FRI, for 20 minutes each. This within-participant comparison, with participants assigned different condition orders, has the advantage of reducing individual differences and order effects.

A feature of the method is that it examines both performance and experience by combining expert evaluation with surveys and interviews. Declarative feedback in the FRI condition received significantly higher scores for specificity, justification, and actionability, but there was no significant difference in the quality of interrogative feedback or the overall helpfulness of the session. Mean specificity was 4.64 in the FRI condition and 4.34 in the baseline condition (without FRI) on a seven-point scale, a difference of 0.30 points. Statistical significance does not imply a large educational improvement or broader development in critical judgment.

2. Critique

The criteria I took from reading 논문, 쓰다 were the importance of the question, its relationship to prior research, and the progression of the argument. In Feed-O-Meter, the importance of the question lies in an educational problem: students do not get enough practice giving feedback because they feel the pressure of evaluation. In relation to prior work, the paper builds on research into peer feedback and AI role-play, extending the discussion to a form of practice in which users reflect on their own feedback while teaching a novice mentee. Its progression from the problem to system design, user evaluation, and limitations allows the reader to compare the proposal with its supporting evidence. The paper is valuable in exploring the possibilities of feedback education through the roles of humans and AI.

Another strength is the release of the project’s source code and instructions for running it. These give subsequent researchers a basis for examining its structure and extending it to other experimental conditions.^{⁴Lim, H. Feed-O-Meter. Public source code and instructions on GitHub.} This also connects to the idea of making research available for scrutiny by a community.

A Research Perspective on Differences in Responses and Educational Backgrounds

Of the 24 participants, 22 (91.7%) majored in industrial design, and all 12 evaluators held at least a master’s degree in industrial design. The participants’ university affiliations were not disclosed, so an affiliation with KAIST cannot be assumed. Still, it is necessary to examine the design culture underlying the study’s design and evaluation, while distinguishing the concentration of participants and evaluators in one discipline from the KAIST researchers’ educational and research environment.

KAIST’s Department of Industrial Design belongs to the College of Engineering and emphasizes the combination of design with engineering and technical knowledge. Its published curriculum includes foundations in science and engineering, design engineering, interaction prototyping, and software prototyping.^{⁵KAIST. College of Engineering introduction to the Department of Industrial Design · Department of Industrial Design undergraduate curriculum.} I want to ask whether such experience may have influenced the role given to AI and the criteria for improvement in feedback.

Participant P9 interpreted Alex’s inability to understand ‘technical limitations’ as a characteristic of a novice designer. This was, of course, one participant’s criterion for judging proficiency; it does not establish how the group as a whole thinks. It would therefore be more appropriate to ask how familiar problems and ways of critiquing were reflected in the application and interpretation of the criteria than to conclude that there was an engineering bias.

A Research Perspective on the Criteria for Good Feedback

Even users familiar with how the technology works may adjust what they say in response to an AI’s facial expressions. Users educated to value interpretation in the humanities and arts, and the context of relationships, may respond differently to the same persona. These educational experiences can coexist, so a distinction between the humanities and sciences alone cannot predict a person’s tendency to empathize. The paper includes before-and-after measures of feedback self-efficacy, but I could not determine differences in responses according to educational experience, prior perceptions of AI, or tendencies to anthropomorphize it.

What caught my attention was P18’s statement that, despite thinking Alex’s idea was unrealistic, they softened their wording for fear of hurting its feelings. The authors also discuss the possibility that the human-like AI mentee’s responses made participants hesitate to ask critical questions. What I want to ask further is how an increase in consideration can be distinguished from an improvement in critical judgment.

Alex smiles when the tone of feedback is classified as positive and looks sad when it is classified as negative, while the quality score is reflected in the liveliness of its eyes. Critical content and a negative tone are different. If a participant softens the wording while identifying the problem accurately, consideration and critique coexist. If a necessary point is left out, the critique is missing. Adding retrospective interviews to conversation logs to identify which problems participants noticed but did not mention, and having experts assess the validity of those points, could have helped distinguish the two cases.

Because FRI provides facial expressions, inner thoughts, evaluation information, and follow-up questions together, it is difficult to determine the independent effect of facial expressions. P21’s comment that they came to give answers that would earn high scores rather than address the mentee’s needs also deserves attention. A follow-up experiment could hold the other functions constant and compare four conditions that cross the presence or absence of emotional expression with the presence or absence of quality evaluation displays.

The Context of Industry Support and the Application of the Research

The paper discloses support from the National Research Foundation of Korea, LG AI Research, and Elice in its acknowledgments. In 2023, Elice also entered into an industry–university agreement on generative AI educational technology and platforms with a KAIST research team that included Professor Hwajung Hong.^{⁶Elice (June 21, 2023). Elice Group and KAIST sign an industry–university research agreement on generative AI education models.} This collaboration can provide context for understanding the educational AI research setting in which the study was conducted.

However, the disclosed support and collaboration alone do not establish whether or how a company influenced this paper’s research design or interpretation of results. Explaining the supporting organizations’ roles alongside the scope of the researchers’ decision-making could help readers understand the conditions under which the research was conducted and its results more specifically.

A Practical Perspective on Coordinating Intentions across Roles

In organizations, design feedback should help planners, designers, and developers understand one another’s intentions and constraints and act on that understanding. Even within the same service, different roles may give different priorities to conversion rates, user experience, feasibility, and maintenance. The paper’s structure of feedback, revision, and review has practical potential as an exercise in explaining the grounds for a judgment. This is, however, a proposal to extend individual learning by design students to the meso level of collaboration within teams and organizations. The authors also acknowledge that 20 minutes of experience per condition is insufficient to establish long-term educational effects. Whether actual colleagues understand one another’s intentions and reach agreement, and whether misunderstandings and rework decrease, would need to be tested separately.

3. Questions

Research Questions

First, how should design feedback research address criteria for good feedback that vary across educational cultures? How did this study examine the possibility that the sample’s concentration in industrial design and the KAIST researchers’ backgrounds were reflected in its design and evaluation? For follow-up research, I propose collecting information in advance on institutional affiliation, actual experiences of education and critique, AI use, and tendencies to anthropomorphize AI, then comparing responses across different educational backgrounds.

Second, what should be measured to evaluate both consideration and the accuracy of critique in educational AI research? Would FRI’s effects still be evaluated positively if cases in which only the wording became gentler were distinguished from those in which valid criticism was omitted? What evidence would be needed to determine whether that difference was due to the AI’s emotional expressions or to behavior aimed simply at matching evaluation scores?

Third, in educational AI research supported by companies, how could the roles of supporting organizations and the researchers’ independence be explained in ways that help readers interpret the findings? One approach could be to distinguish their roles at each stage, such as setting the research questions, analysis, and manuscript review.

Practical Questions

Fourth, what conditions would be needed for the feedback practice in this study to lead to skills in design collaboration within organizations? How could AI reveal and help coordinate cases in which people understand the same words differently, and cases in which they understand the meaning but have different priorities according to their roles? Could its practical value be tested by observing changes in understanding intentions, implementing agreements, and rework in real tasks involving planners, developers, and designers?

The value of Feed-O-Meter lies in prompting people to reflect on their own feedback through the experience of teaching AI. To develop this possibility, it would be necessary to clarify the conditions under which consideration and accurate critique can coexist, differences in evaluation across educational backgrounds, the research context, and the paths through which this practice might carry over to professional work.

References

Thumbnail: Alex from Feed-O-Meter, © 2024 Hyunseung-Lim · MIT notice.