Kosei Horikawa

About

I am a Ph.D. student at the Software Design and Analysis Laboratory, Nara Institute of Science and Technology (NAIST).

Aiming at software quality assurance in the AI era, I study software testing, refactoring, bug prediction, and software development with AI agents. In particular, I empirically analyze how refactoring test code affects quality and maintainability, and examine the effectiveness and challenges of code generated and changed by AI agents. Through this research, I aim to realize development processes in which humans and AI collaborate to continuously build reliable software.

I actively engage in international collaboration and have conducted research visits at Radboud University (the Netherlands), Hangzhou Dianzi University (China), and Queen's University (Canada). Since August 2026, I have been a Visiting Research Student at the SAIL & MCIS Lab, Queen's University.

Research Interests

  • Agentic Software Engineering
  • Mining Software Repositories
  • Refactoring & Software Testing

International Conferences & Journals

  1. TOSEM'26Agentic Refactoring: An Empirical Study of AI Coding Agents

    Kosei Horikawa, Hao Li, Yutaro Kashiwa, Bram Adams, Hajimu Iida, Ahmed E. Hassan

    ACM Transactions on Software Engineering and Methodology

    arXivDOIPDF
    ABS

    Agentic coding tools, such as OpenAI Codex, Claude Code, and Cursor, are changing how software is developed. These AI-powered systems function as autonomous teammates capable of planning and executing complex development tasks. Agents have become active participants in refactoring, a key practice for improving internal code quality without altering observable behavior. Despite their increasing adoption, there is little empirical understanding of how agentic refactoring is used in practice, how it compares to human-driven refactoring, and what impact it has on code quality. To address this gap, we present a large-scale study of AI agent-generated refactorings in real-world open-source Java projects, analyzing 51,412 refactoring instances across 12,256 pull requests and 14,998 commits based on the AIDev dataset. We find that refactoring is a common and intentional activity, with agents explicitly targeting refactoring in 26.1% of commits. The subsequent analyses of refactoring types, purposes, and code quality focus on the 3,907 commits (32,611 instances) with explicit refactoring intent. High-level refactorings form the largest category overall; however, agentic refactoring contains a slightly larger share of low-level, consistency-oriented edits than human refactoring (27.6% vs. 24.4%), and its two most frequent operations are Rename Variable (7.6%) and Change Variable Type (6.8%). Additionally, the stated motivations behind agentic refactoring focus overwhelmingly on internal quality concerns (52.5% maintainability and 28.1% readability), although the measured metric improvements from these frequent low-level refactorings remain limited. Finally, code quality metrics show that agentic refactoring yields small but statistically significant improvements in structural metrics, particularly for medium-level changes, reducing class size and complexity.

    BibTeX
    @article{horikawa2026tosem,
      title = {{Agentic Refactoring: An Empirical Study of AI Coding Agents}},
      author = {Kosei Horikawa and Hao Li and Yutaro Kashiwa and Bram Adams and Hajimu Iida and Ahmed E. Hassan},
      journal = {ACM Transactions on Software Engineering and Methodology},
      year = {2026},
      doi = {10.1145/3859242},
      eprint = {2511.04824},
      archivePrefix = {arXiv}
    }
  2. ESEM'26Is Self-Admitted Technical Debt Tested? An Empirical Study of Coverage, Co-change, and Impact

    Suzuka Yoshimoto, Kosei Horikawa, Daniel Feitosa, Yutaro Kashiwa, Hajimu Iida

    ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

    arXivPDF
    ABS

    When developers write a TODO or FIXME comment, they are explicitly admitting that the code is suboptimal: a built-in warning that this logic deserves extra scrutiny. Yet it is an open question whether Self-Admitted Technical Debt (SATD) actually receives that scrutiny in the form of software testing. We aim to characterize the relationship between SATD and testing across three dimensions: the extent to which SATD-affected code is covered by existing tests, whether developers synchronize test additions with debt resolution, and whether such testing affects the long-term observability of resulting defects. For that, we conducted an empirical study on eight open-source Java projects, analyzing test coverage of 784 SATD instances identified in the latest releases and performing a longitudinal examination of 5,175 SATD removal events. Our results show that while 60.7% of SATD-affected code is covered by existing test suites, developers rarely synchronize test modifications with debt resolution; manual inspection confirms that only 3.4% of SATD removal commits include new tests specifically targeting the resolved debt (vs. 12.5% that co-add tests in the same commit). Longitudinal analysis further suggests that SATD resolutions exhibit nearly identical localized bug induction rates within short-to-medium-term windows regardless of test modifications. However, over a longer, unrestricted observation window, a slight divergence emerges where the test-added group reaches a higher cumulative defect alignment probability (6.32% vs. 4.37%), a counterintuitive trend potentially driven by the selective testing of inherently complex components. Developers treat SATD repayment as an ordinary code change rather than as a high-risk maintenance activity: most debt removals proceed without targeted verification, despite the developer's own prior flag that the code is suboptimal.

    BibTeX
    @inproceedings{yoshimoto2026esem,
      title = {{Is Self-Admitted Technical Debt Tested? An Empirical Study of Coverage, Co-change, and Impact}},
      author = {Suzuka Yoshimoto and Kosei Horikawa and Daniel Feitosa and Yutaro Kashiwa and Hajimu Iida},
      booktitle = {ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)},
      year = {2026},
      eprint = {2609.13485},
      archivePrefix = {arXiv}
    }
  3. ICSME'26What Are Developers Actually Discussing When Visual Regression Tests Fail?

    Miku Watanabe, Kosei Horikawa, Brittany Reid, Yutaro Kashiwa, Hajimu Iida

    The 42nd International Conference on Software Maintenance and Evolution (ICSME)

    arXivPDF
    ABS

    Visual Regression Tests (VRTs) are widely adopted as a mechanism for detecting unintended visual changes in user interfaces. By design, VRTs operate on rendered pixel output, and the prevailing assumption is that they catch stylistic regressions such as layout shifts, color mismatches, and font alterations. We conduct an empirical analysis of 307 pull requests (PRs) from 103 GitHub repositories that incorporate VRT results via Chromatic, comparing them against 299 PRs that contain image attachments but no VRT (Visual PRs). Quantitatively, VRT-PRs show no significant acceptance-rate difference, but exhibit a 3.8 times longer median resolution time, 10 times more discussion comments, and 1.75 to 4.5 times larger code changes than Visual PRs. VRT results are typically shared around the midpoint of the review process, sustaining ongoing discussion rather than serving only as a final check. Through a card-sorting analysis of 189 VRT-flagged issues, we identify seven defect categories assigned to the analyzed issues: Layout (39.7\%), Appearance (27.5\%), Color (14.8\%), Text (9.5\%), State (6.9\%), Test (6.3\%), and Image (4.2\%). The three most frequent categories are stylistic, while approximately 18.5\% of analyzed issues (35/189) involve non-stylistic origins, including undefined component state (13 cases), content disappearance (17 cases across multiple categories), and visually imperceptible regressions (5 cases). We further document cases in which VRT detected visual regressions originating from code changes in seemingly unrelated files, exposing non-local effects that no targeted test would have been written to catch. These observations indicate that, in addition to its primary role as a stylistic checker, VRT functions as a secondary detector of unintended consequences of code changes, with implications for how VRT should be integrated into the maintenance toolchain.

    BibTeX
    @inproceedings{watanabe2026icsme,
      title = {{What Are Developers Actually Discussing When Visual Regression Tests Fail?}},
      author = {Miku Watanabe and Kosei Horikawa and Brittany Reid and Yutaro Kashiwa and Hajimu Iida},
      booktitle = {The 42nd International Conference on Software Maintenance and Evolution (ICSME)},
      year = {2026},
      eprint = {2608.07020},
      archivePrefix = {arXiv}
    }

Domestic Workshops (non-refereed)

  1. SS'24テストコード固有のリファクタリングが及ぼす影響調査にむけて

    堀川 康生, 馬渕 航, 柏 祐太郎, 藤原 賢二, 飯田 元

    電子情報通信学会 ソフトウェアサイエンス研究会 (SS)

    BibTeX
    @inproceedings{paper2024ss,
      title = {{テストコード固有のリファクタリングが及ぼす影響調査にむけて}},
      author = {堀川, 康生 and 馬渕, 航 and 柏, 祐太郎 and 藤原, 賢二 and 飯田, 元},
      booktitle = {電子情報通信学会 ソフトウェアサイエンス研究会 (SS)},
      year = {2024}
    }
  2. SIGSS'23テストコードのリファクタリングがテスト実行時間へ及ぼす影響調査にむけて

    堀川 康生, 福本 大介, 柏 祐太郎, 市川 嘉裕, 飯田 元

    ソフトウェアサイエンス研究会(SIGSS)

    BibTeX
    @inproceedings{paper2023sigss,
      title = {{テストコードのリファクタリングがテスト実行時間へ及ぼす影響調査にむけて}},
      author = {堀川, 康生 and 福本, 大介 and 柏, 祐太郎 and 市川, 嘉裕 and 飯田, 元},
      booktitle = {ソフトウェアサイエンス研究会(SIGSS)},
      year = {2023}
    }

Grants

  • 2026.10 – 2029.03ACT-X (Principal Investigator)Establishing a Test Quality Assurance Foundation for AI-Driven Development · JPMJAX26MH
  • 2026.04 – 2029.03NAIST Granite-AI (Support for Innovative Doctoral Students in Next-Generation AI) (Principal Investigator)Establishing Bug Prediction Techniques to Support Quality Assurance in the Era of Collaboration with AI Agents · JPMJBS2423

Awards

  • 2025.09Best International Poster Award, Software Engineering Symposium (SES)Understanding Refactoring in Test Code: An Empirical Study

Service

Program Committee

  • MSR'27The 24th International Mining Software Repositories Conference (MSR'27), Mining Challenge Track
  • TRUST'26The 1st International Workshop on Trustworthy and Responsible aUtonomous SysTems (TRUST)

Contact

horikawa.kosei.hk1 [at] naist.ac.jp

8916-5 Takayama-cho, Ikoma, Nara 630-0192, Japan