Kosei Horikawa

Publications

Listed in reverse chronological order.

CONFConferenceJOURJournalCitation counts: Google Scholar

International Conferences & Journals

2026

  1. TOSEM'26Agentic Refactoring: An Empirical Study of AI Coding Agents

    Kosei Horikawa, Hao Li, Yutaro Kashiwa, Bram Adams, Hajimu Iida, Ahmed E. Hassan

    ACM Transactions on Software Engineering and Methodology

    arXivDOIPDF
    ABS

    Agentic coding tools, such as OpenAI Codex, Claude Code, and Cursor, are changing how software is developed. These AI-powered systems function as autonomous teammates capable of planning and executing complex development tasks. Agents have become active participants in refactoring, a key practice for improving internal code quality without altering observable behavior. Despite their increasing adoption, there is little empirical understanding of how agentic refactoring is used in practice, how it compares to human-driven refactoring, and what impact it has on code quality. To address this gap, we present a large-scale study of AI agent-generated refactorings in real-world open-source Java projects, analyzing 51,412 refactoring instances across 12,256 pull requests and 14,998 commits based on the AIDev dataset. We find that refactoring is a common and intentional activity, with agents explicitly targeting refactoring in 26.1% of commits. The subsequent analyses of refactoring types, purposes, and code quality focus on the 3,907 commits (32,611 instances) with explicit refactoring intent. High-level refactorings form the largest category overall; however, agentic refactoring contains a slightly larger share of low-level, consistency-oriented edits than human refactoring (27.6% vs. 24.4%), and its two most frequent operations are Rename Variable (7.6%) and Change Variable Type (6.8%). Additionally, the stated motivations behind agentic refactoring focus overwhelmingly on internal quality concerns (52.5% maintainability and 28.1% readability), although the measured metric improvements from these frequent low-level refactorings remain limited. Finally, code quality metrics show that agentic refactoring yields small but statistically significant improvements in structural metrics, particularly for medium-level changes, reducing class size and complexity.

    BibTeX
    @article{horikawa2026tosem,
      title = {{Agentic Refactoring: An Empirical Study of AI Coding Agents}},
      author = {Kosei Horikawa and Hao Li and Yutaro Kashiwa and Bram Adams and Hajimu Iida and Ahmed E. Hassan},
      journal = {ACM Transactions on Software Engineering and Methodology},
      year = {2026},
      doi = {10.1145/3859242},
      eprint = {2511.04824},
      archivePrefix = {arXiv}
    }
  2. ESEM'26Is Self-Admitted Technical Debt Tested? An Empirical Study of Coverage, Co-change, and Impact

    Suzuka Yoshimoto, Kosei Horikawa, Daniel Feitosa, Yutaro Kashiwa, Hajimu Iida

    ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

    arXivPDF
    ABS

    When developers write a TODO or FIXME comment, they are explicitly admitting that the code is suboptimal: a built-in warning that this logic deserves extra scrutiny. Yet it is an open question whether Self-Admitted Technical Debt (SATD) actually receives that scrutiny in the form of software testing. We aim to characterize the relationship between SATD and testing across three dimensions: the extent to which SATD-affected code is covered by existing tests, whether developers synchronize test additions with debt resolution, and whether such testing affects the long-term observability of resulting defects. For that, we conducted an empirical study on eight open-source Java projects, analyzing test coverage of 784 SATD instances identified in the latest releases and performing a longitudinal examination of 5,175 SATD removal events. Our results show that while 60.7% of SATD-affected code is covered by existing test suites, developers rarely synchronize test modifications with debt resolution; manual inspection confirms that only 3.4% of SATD removal commits include new tests specifically targeting the resolved debt (vs. 12.5% that co-add tests in the same commit). Longitudinal analysis further suggests that SATD resolutions exhibit nearly identical localized bug induction rates within short-to-medium-term windows regardless of test modifications. However, over a longer, unrestricted observation window, a slight divergence emerges where the test-added group reaches a higher cumulative defect alignment probability (6.32% vs. 4.37%), a counterintuitive trend potentially driven by the selective testing of inherently complex components. Developers treat SATD repayment as an ordinary code change rather than as a high-risk maintenance activity: most debt removals proceed without targeted verification, despite the developer's own prior flag that the code is suboptimal.

    BibTeX
    @inproceedings{yoshimoto2026esem,
      title = {{Is Self-Admitted Technical Debt Tested? An Empirical Study of Coverage, Co-change, and Impact}},
      author = {Suzuka Yoshimoto and Kosei Horikawa and Daniel Feitosa and Yutaro Kashiwa and Hajimu Iida},
      booktitle = {ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)},
      year = {2026},
      eprint = {2609.13485},
      archivePrefix = {arXiv}
    }
  3. ICSME'26What Are Developers Actually Discussing When Visual Regression Tests Fail?

    Miku Watanabe, Kosei Horikawa, Brittany Reid, Yutaro Kashiwa, Hajimu Iida

    The 42nd International Conference on Software Maintenance and Evolution (ICSME)

    arXivPDF
    ABS

    Visual Regression Tests (VRTs) are widely adopted as a mechanism for detecting unintended visual changes in user interfaces. By design, VRTs operate on rendered pixel output, and the prevailing assumption is that they catch stylistic regressions such as layout shifts, color mismatches, and font alterations. We conduct an empirical analysis of 307 pull requests (PRs) from 103 GitHub repositories that incorporate VRT results via Chromatic, comparing them against 299 PRs that contain image attachments but no VRT (Visual PRs). Quantitatively, VRT-PRs show no significant acceptance-rate difference, but exhibit a 3.8 times longer median resolution time, 10 times more discussion comments, and 1.75 to 4.5 times larger code changes than Visual PRs. VRT results are typically shared around the midpoint of the review process, sustaining ongoing discussion rather than serving only as a final check. Through a card-sorting analysis of 189 VRT-flagged issues, we identify seven defect categories assigned to the analyzed issues: Layout (39.7\%), Appearance (27.5\%), Color (14.8\%), Text (9.5\%), State (6.9\%), Test (6.3\%), and Image (4.2\%). The three most frequent categories are stylistic, while approximately 18.5\% of analyzed issues (35/189) involve non-stylistic origins, including undefined component state (13 cases), content disappearance (17 cases across multiple categories), and visually imperceptible regressions (5 cases). We further document cases in which VRT detected visual regressions originating from code changes in seemingly unrelated files, exposing non-local effects that no targeted test would have been written to catch. These observations indicate that, in addition to its primary role as a stylistic checker, VRT functions as a secondary detector of unintended consequences of code changes, with implications for how VRT should be integrated into the maintenance toolchain.

    BibTeX
    @inproceedings{watanabe2026icsme,
      title = {{What Are Developers Actually Discussing When Visual Regression Tests Fail?}},
      author = {Miku Watanabe and Kosei Horikawa and Brittany Reid and Yutaro Kashiwa and Hajimu Iida},
      booktitle = {The 42nd International Conference on Software Maintenance and Evolution (ICSME)},
      year = {2026},
      eprint = {2608.07020},
      archivePrefix = {arXiv}
    }
  4. MSR'26Do AI Agents Really Improve Code Readability?

    Kyogo Horikawa, Kosei Horikawa, Yutaro Kashiwa, Hidetake Uwano, Hajimu Iida

    The 23rd International Mining Software Repositories Conference (MSR 2026)

    arXivPDF
    ABS

    Code readability is fundamental to software quality and maintainability. Poor readability extends development time, increases bug-inducing risks, and contributes to technical debt. With the rapid advancement of Large Language Models, AI agent-based approaches have emerged as a promising paradigm for automated refactoring, capable of decomposing complex tasks through autonomous planning and execution. While prior studies have examined refactoring by AI agents, these analyses cover all forms of refactoring, including performance optimization and structural improvement. As a result, the extent to which AI agent-based refactoring specifically improves code readability remains unclear. This study investigates the impact of AI agent-based refactoring on code readability. We extracted commits containing readability-related keywords from the AIDev dataset and analyzed changes in readability metrics before and after each commit, covering 403 commits evaluated using multiple quantitative metrics. Our results indicate that AI agents primarily target logic complexity (42.4%) and documentation improvements (24.2%) rather than surface-level aspects like naming conventions or formatting. However, contrary to expectations, readability-focused commits often degraded traditional quality metrics: the Maintainability Index decreased in 56.1% of commits, while Cyclomatic Complexity increased in 42.7%.

    BibTeX
    @inproceedings{horikawa2026msr,
      title = {{Do AI Agents Really Improve Code Readability?}},
      author = {Kyogo Horikawa and Kosei Horikawa and Yutaro Kashiwa and Hidetake Uwano and Hajimu Iida},
      booktitle = {The 23rd International Mining Software Repositories Conference (MSR 2026)},
      year = {2026},
      eprint = {2603.13723},
      archivePrefix = {arXiv}
    }
  5. MSR'26Testing with AI Agents: An Empirical Study of Test Generation Frequency, Quality, and Coverage

    Suzuka Yoshimoto, Shun Fujita, Kosei Horikawa, Daniel Feitosa, Yutaro Kashiwa, Hajimu Iida

    The 23rd International Mining Software Repositories Conference (MSR 2026)

    arXivPDF
    ABS

    Agent-based coding tools have transformed software development practices. Unlike prompt-based approaches that require developers to manually integrate generated code, these agent-based tools autonomously interact with repositories to create, modify, and execute code, including test generation. While many developers have adopted agent-based coding tools, little is known about how these tools generate tests in real-world development scenarios or how AI-generated tests compare to human-written ones. This study presents an empirical analysis of test generation by agent-based coding tools using the AIDev dataset. We extracted 2,232 commits containing test-related changes and investigated three aspects: the frequency of test additions, the structural characteristics of the generated tests, and their impact on code coverage. Our findings reveal that (i) AI authored 16.4% of all commits adding tests in real-world repositories, (ii) AI-generated test methods exhibit distinct structural patterns, featuring longer code and a higher density of assertions while maintaining lower cyclomatic complexity through linear logic, and (iii) AI-generated tests contribute to code coverage comparable to human-written tests, frequently achieving positive coverage gains across several projects.

    BibTeX
    @inproceedings{yoshimoto2026msr,
      title = {{Testing with AI Agents: An Empirical Study of Test Generation Frequency, Quality, and Coverage}},
      author = {Suzuka Yoshimoto and Shun Fujita and Kosei Horikawa and Daniel Feitosa and Yutaro Kashiwa and Hajimu Iida},
      booktitle = {The 23rd International Mining Software Repositories Conference (MSR 2026)},
      year = {2026},
      eprint = {2603.13724},
      archivePrefix = {arXiv}
    }

2025

  1. ESEM'25An Empirical Investigation into Maintenance of Load Testing Scripts.

    Ibuki Nakamura, Kosei Horikawa, Brittany Reid, Yutaro Kashiwa, Hajimu Iida

    The ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2025)

    DOI
    BibTeX
    @inproceedings{nakamura2025esem,
      title = {{An Empirical Investigation into Maintenance of Load Testing Scripts.}},
      author = {Ibuki Nakamura and Kosei Horikawa and Brittany Reid and Yutaro Kashiwa and Hajimu Iida},
      booktitle = {The ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2025)},
      year = {2025},
      doi = {10.1109/ESEM64174.2025.00031}
    }
  2. ICSME'25How Does Test Code Differ from Production Code in Terms of Refactoring? An Empirical Study.

    Kosei Horikawa, Yutaro Kashiwa, Bin Lin, Kenji Fujiwara, Hajimu Iida

    The 41st International Conference on Software Maintenance and Evolution (ICSME 2025)

    DOI
    BibTeX
    @inproceedings{horikawa2025icsme,
      title = {{How Does Test Code Differ from Production Code in Terms of Refactoring? An Empirical Study.}},
      author = {Kosei Horikawa and Yutaro Kashiwa and Bin Lin and Kenji Fujiwara and Hajimu Iida},
      booktitle = {The 41st International Conference on Software Maintenance and Evolution (ICSME 2025)},
      year = {2025},
      doi = {10.1109/ICSME64153.2025.00079}
    }

Domestic Workshops (non-refereed)

2024

  1. SS'24テストコード固有のリファクタリングが及ぼす影響調査にむけて

    堀川 康生, 馬渕 航, 柏 祐太郎, 藤原 賢二, 飯田 元

    電子情報通信学会 ソフトウェアサイエンス研究会 (SS)

    BibTeX
    @inproceedings{paper2024ss,
      title = {{テストコード固有のリファクタリングが及ぼす影響調査にむけて}},
      author = {堀川, 康生 and 馬渕, 航 and 柏, 祐太郎 and 藤原, 賢二 and 飯田, 元},
      booktitle = {電子情報通信学会 ソフトウェアサイエンス研究会 (SS)},
      year = {2024}
    }

2023

  1. SIGSS'23テストコードのリファクタリングがテスト実行時間へ及ぼす影響調査にむけて

    堀川 康生, 福本 大介, 柏 祐太郎, 市川 嘉裕, 飯田 元

    ソフトウェアサイエンス研究会(SIGSS)

    BibTeX
    @inproceedings{paper2023sigss,
      title = {{テストコードのリファクタリングがテスト実行時間へ及ぼす影響調査にむけて}},
      author = {堀川, 康生 and 福本, 大介 and 柏, 祐太郎 and 市川, 嘉裕 and 飯田, 元},
      booktitle = {ソフトウェアサイエンス研究会(SIGSS)},
      year = {2023}
    }