WEKO3
アイテム
Improving Collective MPI-IO Using Topology-Aware Stepwise Data Aggregation with I/O Throttling
https://nied-repo.bosai.go.jp/records/6401
https://nied-repo.bosai.go.jp/records/6401aeb81831-5d88-463f-8182-d9ecf706c2ba
| Item type | researchmap(1) | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 公開日 | 2023-09-20 | |||||||||||||||||
| タイトル | ||||||||||||||||||
| 言語 | en | |||||||||||||||||
| タイトル | Improving Collective MPI-IO Using Topology-Aware Stepwise Data Aggregation with I/O Throttling | |||||||||||||||||
| 言語 | ||||||||||||||||||
| 言語 | eng | |||||||||||||||||
| 著者 |
Yuichi Tsujita
× Yuichi Tsujita
× Atsushi Hori
× Toyohisa Kameyama
× Atsuya Uno
× Fumiyoshi Shoji
× Yutaka Ishikawa
|
|||||||||||||||||
| 抄録 | ||||||||||||||||||
| 内容記述タイプ | Other | |||||||||||||||||
| 内容記述 | MPI-IO has been used in an internal I/O interface layer of HDF5 or PnetCDF, where collective MPI-IO plays a big role in parallel I/O to manage a huge scale of scientific data. However, existing collective MPI-IO optimization named two-phase I/O has not been tuned enough for recent supercomputers consisting of mesh/torus interconnects and a huge scale of parallel file systems due to lack of topology-awareness in data transfers and optimization for parallel file systems. In this paper, we propose I/O throttling and topology-aware stepwise data aggregation in two-phase I/O of ROMIO, which is a representative MPI-IO library, in order to improve collective MPI-IO performance even if we have multiple processes per compute node. Throttling I/O requests going to a target file system mitigates I/O request contention, and consequently I/O performance improvements are achieved in file access phase of two-phase I/O. Topology-aware aggregator layout with paying attention to multiple aggregators per compute node alleviates contention in data aggregation phase of two-phase I/O. In addition, stepwise data aggregation improves data aggregation performance. HPIO benchmark results on the K computer indicate that the proposed optimization has achieved up to about 73% and 39% improvements in write performance compared with the original implementation using 12,288 and 24,576 processes on 3,072 and 6,144 compute nodes, respectively. | |||||||||||||||||
| 言語 | en | |||||||||||||||||
| 書誌情報 |
en : PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE COMPUTING IN ASIA-PACIFIC REGION (HPC ASIA 2018) p. 12-23, 発行日 2018 |
|||||||||||||||||
| 出版者 | ||||||||||||||||||
| 言語 | en | |||||||||||||||||
| 出版者 | ASSOC COMPUTING MACHINERY | |||||||||||||||||
| DOI | ||||||||||||||||||
| 関連識別子 | 10.1145/3149457.3149464 | |||||||||||||||||
