Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese   Publishing type：Research paper (conference, symposium, etc.)  

J-GLOBAL

researchmap

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (conference, symposium, etc.)  

J-GLOBAL

researchmap

Large language models are now integrated into many scientific workflows, accelerating data analysis, hypothesis generation, and design space exploration. In parallel with this growth, there is a growing need to carefully evaluate whether models accurately capture domain-specific knowledge and notation, since general-purpose benchmarks rarely reflect these requirements. This gap is especially clear in quantum science, which features non-intuitive phenomena and requires advanced mathematics. In this study, we introduce QuantumBench, a benchmark for the quantum domain that systematically examine how well LLMs understand and can be applied to this non-intuitive field. Using publicly available materials, we compiled approximately 800 questions with their answers spanning nine areas related to quantum science and organized them into an eight-option multiple-choice dataset. With this benchmark, we evaluate several existing LLMs and analyze their performance in the quantum domain, including sensitivity to changes in question format. QuantumBench is the first LLM evaluation dataset built for the quantum domain, and it is intended to guide the effective use of LLMs in quantum research.

arXiv

researchmap

Other Link： https://arxiv.org/pdf/2511.00092v1

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.48550/arXiv.2503.23714

researchmap

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.48550/arXiv.2404.11262

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Institute of Electrical and Electronics Engineers (IEEE)  

DOI： 10.1109/access.2025.3556762

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Authorship：Lead author   Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (conference, symposium, etc.)  

Why do we build local large language models (LLMs)? What should a local LLM learn from the target language? Which abilities can be transferred from other languages? Do language-specific scaling laws exist? To explore these research questions, we evaluated 35 Japanese, English, and multilingual LLMs on 19 evaluation benchmarks for Japanese and English, taking Japanese as a local language. Adopting an observational approach, we analyzed correlations of benchmark scores, and conducted principal component analysis (PCA) on the scores to derive \textit{ability factors} of local LLMs. We found that training on English text can improve the scores of academic subjects in Japanese (JMMLU). In addition, it is unnecessary to specifically train on Japanese text to enhance abilities for solving Japanese code generation, arithmetic reasoning, commonsense, and reading comprehension tasks. In contrast, training on Japanese text could improve question-answering tasks about Japanese knowledge and English-Japanese translation, which indicates that abilities for solving these two tasks can be regarded as \textit{Japanese abilities} for LLMs. Furthermore, we confirmed that the Japanese abilities scale with the computational budget for Japanese text.

DOI： 10.48550/arXiv.2412.14471

arXiv

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：The Association for Natural Language Processing  

Aspect sentiment quad prediction (ASQP) analyzes the aspect terms, opinion terms, sentiment polarity, and aspect categories in a text. One challenge in this task is the scarcity of data owing to the high annotation cost. Data augmentation techniques are commonly used to address this issue. However, existing approaches simply rewrite texts in the training data, restricting the semantic diversity of the generated data and impairing the quality due to the inconsistency between text and quads. To address these limitations, we augment quads and train a quads-to-text model to generate corresponding texts. Furthermore, we designed novel strategies to filter out low-quality data and balance the sample difficulty distribution of the augmented dataset. Empirical studies on two ASQP datasets demonstrate that our method outperforms other data augmentation methods and achieves state-of-the-art performance on the benchmarks.

DOI： 10.5715/jnlp.31.1523

CiNii Research

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/starsem/2023

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (scientific journal)   Publisher：The Association for Natural Language Processing  

Document-level relation extraction (DocRE) is the task of identifying all relations between each entity pair in a document. Evidence, a set of sentences containing enough clues for deciding the relation between an entity pair, has been shown to benefit relation extraction. Previous works tackle Evidence Retrieval (ER) and DocRE as separate tasks, while this work propose to incorporate ER directly into the DocRE model. Specifically, we guide the self attention mechanism to assign higher weights on evidence when encoding entity pairs. In this way we obtain contextualized representations focused on evidence. We further propose to learn ER on massive data without evidence annotations from automatically-generated evidence. Experimental results show that our approach exhibits state-of-the-art performance on DocRED and Re-DocRED, two popular benchmarks for DocRE, in both DocRE and ER.

DOI： 10.5715/jnlp.31.105

CiNii Research

researchmap

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ELRA and ICCL  

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/coling/2024

Language：Japanese   Publishing type：Research paper (conference, symposium, etc.)  

J-GLOBAL

researchmap

Authorship：Lead author   Language：Japanese  

J-GLOBAL

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：Association for Computational Linguistics  

DOI： 10.18653/v1/2023.eacl-main.145

researchmap

Other Link： https://www.wikidata.org/entity/Q130600373

Authorship：Lead author   Publishing type：Research paper (international conference proceedings)   Publisher：Association for Computational Linguistics  

DOI： 10.18653/v1/2023.mrl-1.11

researchmap

Language：Japanese  

J-GLOBAL

researchmap

Authorship：Lead author   Language：Japanese  

J-GLOBAL

researchmap

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：Association for Computational Linguistics (ACL)  

DOI： 10.18653/v1/2022.spnlp-1.2

CiNii Research

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/acl-spnlp/2022

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：De La Salle University  

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/paclic/2022

Authorship：Lead author   Language：English   Publishing type：Research paper (scientific journal)   Publisher：The Association for Natural Language Processing  

In this study, we propose a method designed to extract named entities and relations from unstructured text based on table representations. To extract named entities, the proposed method computes representations for entity mentions and long-range dependencies using contextualized representations without hand-crafted features or complex neural network architectures. To extract relations, it applies a tensor dot product to predict all relation labels simultaneously without considering dependencies among relation labels. These advancements significantly simplify the proposed model and the associated algorithm for the extraction of named entities and relations. Despite its simplicity, the experimental results demonstrate that the proposed approach outperformed the state of the-art methods on multiple datasets. Compared with existing table-filling approaches, the proposed method achieved high performance solely by independently predicting the relation labels. In addition, we found that incorporating dependencies of relation labels into the system obtained little performance gain, indicating the effectiveness and sufficiency of the tensor dot-product mechanism for relation extraction in the proposed architecture. Experimental analyses were also performed to explore the benefits of joint training with named entity recognition in relation extraction in our design. We concluded that joint training with named entity recognition assists relation extraction to improve the span-level representation of entities.

DOI： 10.5715/jnlp.29.187

DOI： 10.48550/arxiv.2010.07522

CiNii Research

researchmap

Other Link： https://dblp.uni-trier.de/db/journals/corr/corr2010.html#abs-2010-07522

Authorship：Lead author   Language：Japanese  

J-GLOBAL

researchmap

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (conference, symposium, etc.)  

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：Association for Computational Linguistics  

DOI： 10.18653/v1/2020.acl-main.607

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/acl/2020