Iterative Compression-Decimation Scheme for Tensor Network Optimization (original) (raw)
Related papers
Tensor network method for reversible classical computation
Physical review. E, 2018
We develop a tensor network technique that can solve universal reversible classical computational problems, formulated as vertex models on a square lattice [Nat. Commun. 8, 15303 (2017)2041-172310.1038/ncomms15303]. By encoding the truth table of each vertex constraint in a tensor, the total number of solutions compatible with partial inputs and outputs at the boundary can be represented as the full contraction of a tensor network. We introduce an iterative compression-decimation (ICD) scheme that performs this contraction efficiently. The ICD algorithm first propagates local constraints to longer ranges via repeated contraction-decomposition sweeps over all lattice bonds, thus achieving compression on a given length scale. It then decimates the lattice via coarse-graining tensor contractions. Repeated iterations of these two steps gradually collapse the tensor network and ultimately yield the exact tensor trace for large systems, without the need for manual control of tensor dimens...
Memory-Constrained Data Locality Optimization for Tensor Contractions
Lecture Notes in Computer Science, 2004
The accurate modeling of the electronic structure of atoms and molecules involves computationally intensive tensor contractions over large multidimensional arrays. Efficient computation of these contractions usually requires the generation of temporary intermediate arrays. These intermediates could be extremely large, requiring their storage on disk. However, the intermediates can often be generated and used in batches through appropriate loop fusion transformations. To optimize the performance of such computations a combination of loop fusion and loop tiling is required, so that the cost of disk I/O is minimized. In this paper, we address the memory-constrained data-locality optimization problem in the context of this class of computations. We develop an optimization framework to search among a space of fusion and tiling choices to minimize the data movement overhead. The effectiveness of the developed optimization approach is demonstrated on a computation representative of a component used in quantum chemistry suites.
Quantum Compression of Tensor Network States
New Journal of Physics
We design quantum compression algorithms for parametric families of tensor network states. We first establish an upper bound on the amount of memory needed to store an arbitrary state from a given state family. The bound is determined by the minimum cut of a suitable flow network, and is related to the flow of information from the manifold of parameters that specify the states to the physical systems in which the states are embodied. For given network topology and given edge dimensions, our upper bound is tight when all edge dimensions are powers of the same integer. When this condition is not met, the bound is optimal up to a multiplicative factor smaller than 1.585. We then provide a compression algorithm for general state families, and show that the algorithm runs in polynomial time for matrix product states.
Memory minimization for tensor contractions using integer linear programming
2006
This paper presents a technique for memory optimization for a class of computations that arises in the field of correlated electronic structure methods such as coupled cluster and configuration interaction methods in quantum chemistry. In this class of computations, loop computations perform a multi-dimensional sum of product of input arrays. There are many different ways to get the same final results that differ in the required number of arithmetic operations required. In addition, for a given number of arithmetic operations, different expressions of the loop have different memory requirements. Loop fusion is a plausible solution for reducing memory usage. By fusing loops between producer loop nest and consumer loop nest, the required storage of intermediate array is reduced by the range of the fused loop. Because resultant loops have to be legal after fusion, some loops can not be fused at the same time. In this paper, we have developed a novel integer linear programming (ILP) formulation that is shown to be highly effective on a number of test cases producing the optimal solutions using very small execution times. The main idea in the ILP formulation is the encoding of legality rules for loop fusion of a special class of loops using logical constraints over binary decision variables and a highly effective approximation of memory usage.
Optimization at the boundary of the tensor network variety
Physical Review B, 2021
Tensor network states form a variational ansatz class widely used, both analytically and numerically, in the study of quantum many-body systems. It is known that if the underlying graph contains a cycle, e.g., as in projected entangled pair states, then the set of tensor network states of given bond dimension is not closed. Its closure is the tensor network variety. Recent work has shown that states on the boundary of this variety can yield more efficient representations for states of physical interest, but it remained unclear how to systematically find and optimize over such representations. We address this issue by defining an ansatz class of states that includes states at the boundary of the tensor network variety of given bond dimension. We show how to optimize over this class in order to find ground states of local Hamiltonians by only slightly modifying standard algorithms and code for tensor networks. We apply this method to different models and observe favorable energies and runtimes when compared with standard tensor network methods.
Compression Protocols for Tensor Network States
2019
We show that tensor network states of n identical quantum systems can be faithfully compressed into O(log n) memory qubits. For a given parametric family of tensor network states, our technique is based on a partition of the tensor network into constant and variable terms. In our compression protocols, the prefactor in the logarithmic scaling O(log n) is determined by a minimal cut between the physical systems and variable terms in the network, which can be interpreted as a maximum information flow from the free parameters to the output quantum state. The logarithmic scaling O(log n) is generally optimal, while the determination of the optimal prefactor remains an open problem.
Tensor network decompositions in the presence of a global symmetry
Tensor network decompositions offer an efficient description of certain many-body states of a lattice system and are the basis of a wealth of numerical simulation algorithms. We discuss how to incorporate a global symmetry, given by a compact, completely reducible group G, in tensor network decompositions and algorithms. This is achieved by considering tensors that are invariant under the action of the group G. Each symmetric tensor decomposes into two types of tensors: degeneracy tensors, containing all the degrees of freedom, and structural tensors, which only depend on the symmetry group. In numerical calculations, the use of symmetric tensors ensures the preservation of the symmetry, allows selection of a specific symmetry sector, and significantly reduces computational costs. On the other hand, the resulting tensor network can be interpreted as a superposition of exponentially many spin networks. Spin networks are used extensively in loop quantum gravity, where they represent states of quantum geometry. Our work highlights their importance in the context of tensor network algorithms as well, thus setting the stage for cross-fertilization between these two areas of research. Locality and symmetry are pivotal concepts in the formulation of physical theories. In a quantum many-body system, locality implies that the dynamics are governed by a Hamiltonian H that decomposes as the sum of terms involving only a small number of particles and whose strength decays with the distance between the particles. In turn, a symmetry of the Hamiltonian H allows us to organize the kinematic space of the theory according to the irreducible representations of the symmetry group. Both symmetry and locality can be exploited to obtain a more compact description of many-body states and to reduce computational costs in numerical simulations. In the case of symmetries, this has long been understood. Space symmetries, such as invariance under translations or rotations, as well as internal symmetries, such as particle number conservation or spin isotropy, divide the Hilbert space of the theory into sectors labeled by quantum numbers or charges. The Hamiltonian H is by definition block-diagonal in these sectors. If, for instance, the ground state is known to have zero momentum, it can be obtained by just diagonalizing the (comparatively small) zero-momentum block of H. In recent times, the far-reaching implications of locality for our ability to describe many-body systems have also started to unfold. The local character of the Hamiltonian H limits the amount of entanglement that low-energy states may have, and in a lattice system, restrictions on entanglement can be exploited to succinctly describe these states with a tensor network (TN) decomposition. Examples of TN decomposi-tions include matrix product states (MPS's) [1], projected entangled-pair states [2], and the multiscale entanglement renormalization ansatz (MERA) [3]. It is important to note that in a lattice made of N sites, where the Hilbert space dimension grows exponentially with N , TN decompositions often offer an efficient description (with costs that scale roughly as N). This allows for scalable simulations of quantum lattice systems, even in cases that are beyond the reach of standard Monte Carlo sampling techniques. As an example, the MERA has been recently used to investigate ground states of frustrated antiferromagnets [4]. In this article we investigate how to incorporate a global symmetry into a TN, so as to be able to simultaneously exploit both the locality and the symmetries of physical Hamiltonians to describe many-body states. Specifically, in order to represent a symmetric state that has a limited amount of entanglement, we use a TN made of symmetric tensors. This leads to an approximate, efficient decomposition that preserves the symmetry exactly. Moreover, a more compressed description is obtained by breaking each symmetric tensor into several degeneracy tensors (containing all the degrees of freedom of the original tensor) and structural tensors (completely fixed by the symmetry). This decomposition leads to a substantial reduction in computational costs and reveals a connection between TN algorithms and the formalism of spin networks [5] used in loop quantum gravity [6]. In the case of an MPS, global symmetries have already been studied by many authors (see, e.g., [1,7]) in the context of both one-dimensional quantum systems and two-dimensional (2D) classical systems. An MPS is a trivalent TN (i.e., each tensor has at most three indices) and symmetries are comparatively easy to characterize. The present analysis applies to the more challenging case of a generic TN decomposition (where tensors typically have more than three indices). We consider a lattice L made of N sites, where each site is described by a complex vector space V of finite dimension d. A pure state | ∈ V ⊗N of the lattice can be expanded as
Foundations and Trends® in Machine Learning, 2016
Modern applications in engineering and data science are increasingly based on multidimensional data of exceedingly high volume, variety, and structural richness. However, standard machine learning algorithms typically scale exponentially with data volume and complexity of cross-modal couplings-the so called curse of dimensionalitywhich is prohibitive to the analysis of large-scale, multi-modal and multi-relational datasets. Given that such data are often efficiently represented as multiway arrays or tensors, it is therefore timely and valuable for the multidisciplinary machine learning and data analytic communities to review low-rank tensor decompositions and tensor networks as emerging tools for dimensionality reduction and large scale optimization problems. Our particular emphasis is on elucidating that, by virtue of the underlying low-rank approximations, tensor networks have the ability to alleviate the curse of dimensionality in a number of applied areas. In Part 1 of this monograph we provide innovative solutions to low-rank tensor network decompositions and easy to interpret graphical representations of the mathematical operations on tensor networks. Such a conceptual insight allows for seamless migration of ideas from the flat-view matrices to tensor network operations and vice versa, and provides a platform for further developments, practical applications, and non-Euclidean extensions. It also permits the introduction of various tensor network operations without an explicit notion of mathematical expressions, which may be beneficial for many research communities that do not directly rely on multilinear algebra. Our focus is on the Tucker and tensor train (TT) decompositions and their extensions, and on demonstrating the ability of tensor networks to provide linearly or even super-linearly (e.g., logarithmically) scalable solutions, as illustrated in detail in Part 2 of this monograph.
Global communication optimization for tensor contraction expressions under memory constraints
Proceedings International Parallel and Distributed Processing Symposium, 2003
The accurate modeling of the electronic structure of atoms and molecules involves computationally intensive tensor contractions involving large multi-dimensional arrays. The efficient computation of complex tensor contractions usually requires the generation of temporary intermediate arrays. These intermediates could be extremely large, but they can often be generated and used in batches through appropriate loop fusion transformations. To optimize the performance of such computations on parallel computers, the total amount of interprocessor communication must be minimized, subject to the available memory on each processor. In this paper, we address the memory-constrained communication minimization problem in the context of this class of computations. Based on a framework that models the relationship between loop fusion and memory usage, we develop an approach to identify the best combination of loop fusion and data partitioning that minimizes inter-processor communication cost without exceeding the per-processor memory limit. The effectiveness of the developed optimization approach is demonstrated on a computation representative of a component used in quantum chemistry suites.
Efficient variational contraction of two-dimensional tensor networks with a non-trivial unit cell
Quantum, 2020
Tensor network states provide an efficient class of states that faithfully capture strongly correlated quantum models and systems in classical statistical mechanics. While tensor networks can now be seen as becoming standard tools in the description of such complex many-body systems, close to optimal variational principles based on such states are less obvious to come by. In this work, we generalize a recently proposed variational uniform matrix product state algorithm for capturing one-dimensional quantum lattices in the thermodynamic limit, to the study of regular two-dimensional tensor networks with a non-trivial unit cell. A key property of the algorithm is a computational effort that scales linearly rather than exponentially in the size of the unit cell. We demonstrate the performance of our approach on the computation of the classical partition functions of the antiferromagnetic Ising model and interacting dimers on the square lattice, as well as of a quantum doped resonating ...