Abstract
As the main topic in modern scientific computing and machine learning tasks, matrix equation solving is suffering high computational latency and tremendous power consumption due to the frequent data movement in traditional von Neumann computers. Although the in-memory computing paradigms have shown the potential to accelerate the execution of solving matrix equations, the existing memristive matrix equation solvers are still limited by the low system versatility and low computation precision of the memristor arrays. In this work, we demonstrate a hybrid architecture for accurate, as well as efficient, matrix equation solving problems, where the memristive crossbar arrays are used for the parallel vector-matrix multiplication and the digital computer for accuracy. The linear neural-network solving (NNS) method is adopted here and its versatility for various types of matrix equations is proved. The weight-slice computation method is developed to perform the analog matrix multiplication with high efficiency and high robustness in the array. The solution results confirmed that typical matrix equations can be solved by this memristive matrix equation solver with high accuracy. Further performance benchmarking demonstrates that the generalized memristive matrix equation solver has low solving time-complexity while outperforming the state-of-the-art CMOS and in-memory processors.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ran A C M, Reurings M C B. A fixed point theorem in partially ordered sets and some applications to matrix equations. Proc Amer Math Soc, 2004, 132: 1435–1443
Tan L, Kothapalli S, Chen L X, et al. A survey of power and energy efficient techniques for high performance numerical linear algebra operations. Parall Comput, 2014, 40: 559–573
Golub G H, van Loan C F. Matrix Computations. 4th ed. Baltimore: Johns Hopkins University Press, 2013
Sebastian A, Le Gallo M, Khaddam-Aljameh R, et al. Memory devices and applications for in-memory computing. Nat Nanotechnol, 2020, 15: 529–544
Chen J, Li J C, Li Y, et al. Multiply accumulate operations in memristor crossbar arrays for analog computing. J Semicond, 2021, 42: 013104
Hu M, Graves C E, Li C, et al. Memristor-based analog computation and neural network classification with a dot product engine. Adv Mater, 2018, 30: 1705914
Le Gallo M, Sebastian A, Mathis R, et al. Mixed-precision in-memory computing. Nat Electron, 2018, 1: 246–253
Zidan M A, Jeong Y J, Lee J, et al. A general memristor-based partial differential equation solver. Nat Electron, 2018, 1: 411–420
Sun Z, Pedretti G, Ambrosi E, et al. Solving matrix equations in one step with cross-point resistive arrays. Proc Natl Acad Sci USA, 2019, 116: 4123–4128
Li J C, Zhou H J, Li Y, et al. Self-adaptive matrix equation solving in analog memory array 1. In: Proceedings of the 5th IEEE Electron Devices Technology & Manufacturing Conference (EDTM), 2021
Micikevicius P, Narang S, Alben J, et al. Mixed precision training. 2017. ArXiv:1710.03740
Li C, Belkin D, Li Y N, et al. Efficient and self-adaptive in-situ learning in multilayer memristor neural networks. Nat Commun, 2018, 9: 2385
Sun Z, Pedretti G, Mannocci P, et al. Time complexity of in-memory solution of linear systems. IEEE Trans Electron Dev, 2020, 67: 2945–2951
Cai F X, Correll J M, Lee S H, et al. A fully integrated reprogrammable memristor-CMOS system for efficient multiply-accumulate operations. Nat Electron, 2019, 2: 290–299
Cheng L, Li J C, Zheng H X, et al. In-memory hamming weight calculation in a 1T1R memristive array. Adv Electron Mater, 2020, 6: 2000457
Sheridan P M, Cai F X, Du C, et al. Sparse coding with memristor networks. Nat Nanotech, 2017, 12: 784–789
Higham N J. Accuracy and Stability of Numerical Algorithms. Philadelphia: Society for Industrial and Applied Mathematics, 2002
Benzi M, Meyer C D, Tuma M. A sparse approximate inverse preconditioner for the conjugate gradient method. SIAM J Sci Comput, 1996, 17: 1135–1149
Sheng X P, Su Y F, Chen G L. An iterative method for the minimum norm-least square solution of the matrix equation ATXB + BTXTA = D (in Chinese). Numer Math J Chinese Univ, 2008, 4: 301–307
Weisberg S. Applied Linear Regression. Hoboken: John Wiley & Sons, 2005
Sun Z, Pedretti G, Bricalli A, et al. One-step regression and classification with cross-point resistive memory arrays. Sci Adv, 2020, 6: 2378
Meng X R, Saunders M A, Mahoney M W. LSRN: a parallel iterative solver for strongly over-or underdetermined systems. SIAM J Sci Comput, 2014, 36: 95–118
Mamaghanian H, Khaled N, Atienza D, et al. Compressed sensing for real-time energy-efficient ECG compression on wireless body sensor nodes. IEEE Trans Biomed Eng, 2011, 58: 2456–2466
Cannon J R. The One-dimensional Heat Equation. Cambridge: Cambridge University Press, 1984
Sun Z, Huang R. Time complexity of in-memory matrix-vector multiplication. IEEE Trans Circ Syst II, 2021, 68: 2785–2789
Wang K, Liu Z J, Lin Y J, et al. HAQ: hardware-aware automated quantization with mixed precision. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. 8612–8620
Huang X D, Li Y, Li H Y, et al. Forming-free, fast, uniform, and high endurance resistive switching from cryogenic to high temperatures in W/AlOx/Al2O3/Pt bilayer memristor. IEEE Electron Dev Lett, 2020, 41: 549–552
Berdan R, Marukame T, Ota K, et al. Low-power linear computation using nonlinear ferroelectric tunnel junction memristors. Nat Electron, 2020, 3: 259–266
Jouppi N P, Young C, Patil N, et al. In-datacenter performance analysis of a tensor processing unit. In: Proceedings of the 44th Annual International Symposium on Computer Architecture, 2017
Xu R Y, Liu B, Yuan J. Digitally calibrated 768-kS/s 10-b minimum-size SAR ADC array with dithering. IEEE J Solid-State Circ, 2012, 47: 2129–2140
Li T L, Sakai S, Kawada S, et al. A column-parallel hybrid analog-to-digital converter using successive-approximation-register and single-slope architectures with error correction for complementary metal oxide silicon image sensors. Jpn J Appl Phys, 2013, 52: 1–7
Acknowledgements
This work was partly supported by National Key R&D Program of China (Grant No. 2019YFB2205100), National Natural Science Foundation of China (Grant Nos. 61874164, 61841404), Hubei Key Laboratory of Advanced Memories, and Hubei Engineering Research Center on Microelectronics.
Author information
Authors and Affiliations
Corresponding author
Additional information
Supporting information
Appendixes A and B. The supporting information is available online at info.scichina.com and link.springer.com. The supporting materials are published as submitted, without typesetting or editing. The responsibility for scientific accuracy and content remains entirely with the authors.
Rights and permissions
About this article
Cite this article
Li, J., Zhou, H., Li, Y. et al. A memristive neural network based matrix equation solver with high versatility and high energy efficiency. Sci. China Inf. Sci. 66, 122402 (2023). https://doi.org/10.1007/s11432-021-3374-x
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11432-021-3374-x