Monitoring Memory Behaviors and Mitigating NUMA Drawbacks on Tiered NVM Systems (original) (raw)

Abstract

Non-Volatile Memory with byte-addressability invites a new paradigm to access persistent data directly. However, this paradigm brings new challenges to the Non-Uniform Memory Access (NUMA) architecture. Since data accesses cross NUMA node can incur significant performance loss, and, traditionally, OS moves data to the NUMA node where the process accessing it locates to reduce the access latency. However, we find challenges when migrating data on NVM, which motivates us to migrate the process instead. We propose SysMon-N, an OS-level sampling module, to obtain access information about NVM in low overhead. Furthermore, we propose N-Policy to utilize the data collected by SysMon-N to guide process migration. We evaluate SysMon-N and N-Policy on off-the-shelf NVM devices. The experimental results show that they provide 5.9% to \(3.62\times \) bandwidth improvement in the case where cross-node memory accesses happen.

This project is supported by the National Key Research and Development Program of China under Grant No.2017YFB1001602 and the NSFC under grants No.61502452, 61902206, 61702286. This work originates from L. Liu’s series of studies in ISCA, PACT, TPDS, TC, etc. [5,6,7,8,9,10,11,12] on memory systems conducted in Sys-Inventor Lab. More details refer to Sys-Inventor Lab - https://liulei-sys-inventor.github.io.

References

  1. Direct Access for files. https://www.kernel.org/doc/Documentation/filesystems/dax.txt
  2. Fio - Flexible I/O tester. https://fio.readthedocs.io/en/latest/
  3. Persistent Memory Concepts. https://docs.pmem.io/ndctl-user-guide/concepts
  4. Rao, D.S., et al., System software for persistent memory. In: EuroSys (2014)
    Google Scholar
  5. Xie, M., et al.: Sysmon: monitoring memory behaviorsvia OS approach. In: APPT (2017)
    Google Scholar
  6. L. Liu, et al.: Hierarchical hybrid memory management in OS for tiered memory systems. In: IEEE TPDS (2019)
    Google Scholar
  7. X. Li, et al.: Thinking about a new mechanism for huge page management. In: APSys (2019)
    Google Scholar
  8. Liu, L., et al.: Going vertical in memory management: handling multiplicity by multi-policy. In: ISCA (2014) (revised version)
    Google Scholar
  9. Liu, L., et al.: BPM/BPM+: software-based dynamic memory partitioning mechanisms for mitigating DRAM Bank-/channel-level interferences in multicore systems. In: ACM TACO (2014) (revised version)
    Google Scholar
  10. Liu, L., et al.: A software memory partition approach for eliminating bank-level interference in multicore systems. In: PACT (2012) (revised version)
    Google Scholar
  11. Liu, L., et al.: Rethinking memory management in modern operating system: horizontal, vertical or random?. In: IEEE Trans. Comput. (TC) (2016)
    Google Scholar
  12. Liu, L., et al.: Memos: a full hierarchy hybrid memory management framework. In: ICCD (2016)
    Google Scholar
  13. Chen, S., et al.: Efficient GPU NVRAM persistence with helper warps. In: DAC (2019)
    Google Scholar
  14. Lv, F., Cui, H.-M., Wang, L., Liu, L., Wu, C.-G., Feng, X.-B., Yew, P.-C.: Dynamic I/O-Aware scheduling for batch-mode applications on chip multiprocessor systems of cluster platforms. J. Comput. Sci. Technol. 29(1), 21–37 (2014). https://doi.org/10.1007/s11390-013-1409-2
    Article Google Scholar

Download references

Author information

Authors and Affiliations

  1. Sys-Inventor Lab, Beijing, China
    Shengjie Yang, Xinyu Li, Xinglei Dou & Lei Liu
  2. SKLCA, ICT, CAS, Beijing, China
    Shengjie Yang, Xinyu Li, Xinglei Dou, Li Chen & Lei Liu
  3. Nankai, Tianjin, China
    Xiaoli Gong
  4. AMS, PLA & Tsinghua, Beijing, China
    Hao Liu

Authors

  1. Shengjie Yang
  2. Xinyu Li
  3. Xinglei Dou
  4. Xiaoli Gong
  5. Hao Liu
  6. Li Chen
  7. Lei Liu

Corresponding author

Correspondence toLei Liu .

Editor information

Editors and Affiliations

  1. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
    Xin He
  2. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
    En Shao
  3. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
    Guangming Tan

Rights and permissions

© 2021 IFIP International Federation for Information Processing

About this paper

Cite this paper

Yang, S. et al. (2021). Monitoring Memory Behaviors and Mitigating NUMA Drawbacks on Tiered NVM Systems. In: He, X., Shao, E., Tan, G. (eds) Network and Parallel Computing. NPC 2020. Lecture Notes in Computer Science(), vol 12639. Springer, Cham. https://doi.org/10.1007/978-3-030-79478-1\_33

Download citation

Keywords

Publish with us