MLIR: lib/Dialect/SparseTensor/Transforms/Sparsification.cpp File Reference (original) (raw)
Go to the source code of this file.
| Functions | |
|---|---|
| static bool | isInvariantAffine (AffineExpr a, LoopId curr, bool &isCurrentLoop) |
| Returns true iff affine expression is invariant. More... | |
| static bool | findAffine (Merger &merger, TensorId tid, Level lvl, AffineExpr a, LevelType lt, bool setLvlFormat=true) |
| Helper method to inspect affine expressions. More... | |
| static bool | findDepIdxSet (Merger &merger, TensorId tensor, Level lvl, AffineExpr a, LevelType lt, bool isSubExp=false, int64_t coefficient=1) |
| Helper method to inspect affine expressions for index variable reduction based codegen. More... | |
| static unsigned | getNumNonTrivialIdxExpOnSparseLvls (AffineMap map, Value tensor) |
| Gets the total number of compound affine expressions in the getMatchingIndexingMap for the given tensor. More... | |
| static unsigned | getNumNonTrivialIdxExpOnSparseLvls (linalg::GenericOp op) |
| Gets the total number of sparse levels with compound affine expressions, summed over all operands of the GenericOp. More... | |
| static bool | hasNonTrivialAffineOnSparseOut (linalg::GenericOp op) |
| static bool | findSparseAnnotations (CodegenEnv &env, bool idxReducBased) |
| Helper method to inspect sparse encodings in the tensor types. More... | |
| static void | genBuffers (CodegenEnv &env, OpBuilder &builder) |
| Local bufferization of all dense and sparse data structures. More... | |
| static Value | genIndex (CodegenEnv &env, OpOperand *t) |
| Generates index for load/store on sparse tensor. More... | |
| static Value | genSubscript (CodegenEnv &env, OpBuilder &builder, OpOperand *t, SmallVectorImpl< Value > &args) |
| Generates subscript for load/store on a dense or sparse tensor. More... | |
| static Value | genInsertionLoad (CodegenEnv &env, OpBuilder &builder, OpOperand *t) |
| Generates insertion code to implement dynamic tensor load. More... | |
| static Value | genInsertionLoadReduce (CodegenEnv &env, OpBuilder &builder, OpOperand *t) |
| Generates insertion code to implement dynamic tensor load for reduction. More... | |
| static Value | genConditionalInsert (Location loc, OpBuilder &builder, Value cond, Value sparseOut, ValueRange ivs, Value v) |
| static void | genInsertionStore (CodegenEnv &env, OpBuilder &builder, OpOperand *t, Value rhs) |
| Generates insertion code to implement dynamic tensor store. More... | |
| static Value | genTensorLoad (CodegenEnv &env, OpBuilder &builder, ExprId exp) |
| Generates a load on a dense or sparse tensor. More... | |
| static void | genTensorStore (CodegenEnv &env, OpBuilder &builder, ExprId exp, Value rhs) |
| Generates a store on a dense or sparse tensor. More... | |
| static Value | genInvariantValue (CodegenEnv &env, ExprId exp) |
| Generates an invariant value. More... | |
| static Value | relinkBranch (CodegenEnv &env, RewriterBase &rewriter, Block *block, Value e) |
| Semi-ring branches are simply inlined by the sparsifier. More... | |
| static Value | genExp (CodegenEnv &env, RewriterBase &rewriter, ExprId e) |
| Recursively generates tensor expression. More... | |
| static void | genInvariants (CodegenEnv &env, OpBuilder &builder, ExprId exp, LoopId curr, bool isStart) |
| Hoists loop invariant tensor loads for which indices have been exhausted. More... | |
| static void | genExpand (CodegenEnv &env, OpBuilder &builder, LoopId curr, bool isStart) |
| Generates an expanded access pattern in innermost dimension. More... | |
| static bool | isParallelFor (CodegenEnv &env, bool isOuter, bool isSparse) |
| Returns parallelization strategy. More... | |
| static bool | shouldTryParallize (CodegenEnv &env, LoopId curr, ArrayRef< TensorLevel > tidLvls) |
| Whether or not the current loop being generated should be parallized (if possible) according to the configuration. More... | |
| static Operation * | genCoIteration (CodegenEnv &env, OpBuilder &builder, ArrayRef< TensorLevel > tidLvls, unsigned numCases, bool tryParallel, bool needsUniv) |
| Emit a loop to coiterate over the list of tensor levels. More... | |
| static Operation * | genLoop (CodegenEnv &env, OpBuilder &builder, LoopId curr, unsigned numCases, bool needsUniv, ArrayRef< TensorLevel > tidLvls) |
| Generates a for-loop or a while-loop, depending on whether it implements singleton iteration or co-iteration over the given conjunction. More... | |
| static void | finalizeWhileOp (CodegenEnv &env, OpBuilder &builder, bool needsUniv) |
| Generates the induction structure for a while-loop. More... | |
| static void | genCoIterationCase (CodegenEnv &env, OpBuilder &builder, unsigned caseIdx, LatPointId allCase, LatPointId curCase, MutableArrayRef< Value > reduc) |
| Generates a case region in the coiterate operation. More... | |
| static scf::IfOp | genIf (CodegenEnv &env, OpBuilder &builder, LoopId curr, LatPointId p) |
| Generates a single if-statement within a while-loop. More... | |
| static void | endIf (CodegenEnv &env, OpBuilder &builder, scf::IfOp ifOp, Value redInput, Value cntInput, Value insInput, Value validIns) |
| Generates end of true branch of if-statement within a while-loop. More... | |
| static bool | getAllTidLvlsInLatPoints (CodegenEnv &env, LatPointId li, LoopId curr, llvm::function_ref< void(TensorLevel, AffineExpr)> callback) |
| static bool | startLoopSeq (CodegenEnv &env, OpBuilder &builder, ExprId exp, LoopId curr, LatSetId lts) |
| Starts a loop sequence at given level. More... | |
| static void | genConstantDenseAddressFromLevel (CodegenEnv &env, OpBuilder &builder, TensorId tid, Level startLvl) |
| static void | genInitConstantDenseAddress (CodegenEnv &env, RewriterBase &rewriter) |
| static bool | translateBitsToTidLvlPairs (CodegenEnv &env, LatPointId li, LoopId curr, SmallVectorImpl< TensorLevel > &tidLvls, SmallVectorImpl< std::pair< TensorLevel, AffineExpr >> &affineTidLvls) |
| Returns true if the lattice bit can be iterated by a for loop. More... | |
| static std::pair< Operation *, bool > | startLoop (CodegenEnv &env, OpBuilder &builder, LoopId curr, LatPointId li, unsigned numCases, bool needsUniv) |
| Starts a single loop in current sequence. More... | |
| static bool | endLoop (CodegenEnv &env, RewriterBase &rewriter, Operation *loop, LatPointId li, bool needsUniv, bool isSingleCond) |
| Ends a single loop in current sequence. Returns new values for needsUniv. More... | |
| static void | endLoopSeq (CodegenEnv &env, OpBuilder &builder, unsigned exp, unsigned at) |
| Ends a loop sequence at given level. More... | |
| static void | genStmt (CodegenEnv &env, RewriterBase &rewriter, ExprId exp, LoopId curr) |
| Recursively generates code while computing iteration lattices in order to manage the complexity of implementing co-iteration over unions and intersections of sparse iterations spaces. More... | |
| static void | genResult (CodegenEnv &env, RewriterBase &rewriter) |
| Converts the result computed by the sparse kernel into the required form. More... | |
◆ endIf()
Generates end of true branch of if-statement within a while-loop.
Definition at line 974 of file Sparsification.cpp.
References mlir::sparse_tensor::constantI1(), mlir::OpBuilder::create(), mlir::sparse_tensor::CodegenEnv::getExpandCount(), mlir::sparse_tensor::CodegenEnv::getInsertionChain(), mlir::sparse_tensor::CodegenEnv::getReduc(), mlir::sparse_tensor::CodegenEnv::isExpand(), mlir::sparse_tensor::CodegenEnv::isReduc(), mlir::sparse_tensor::CodegenEnv::isValidLexInsert(), mlir::sparse_tensor::CodegenEnv::op(), mlir::OpBuilder::setInsertionPointToStart(), mlir::sparse_tensor::CodegenEnv::updateExpandCount(), mlir::sparse_tensor::CodegenEnv::updateInsertionChain(), mlir::sparse_tensor::CodegenEnv::updateReduc(), and mlir::sparse_tensor::CodegenEnv::updateValidLexInsert().
Referenced by genStmt().
◆ endLoop()
◆ endLoopSeq()
| static void endLoopSeq ( CodegenEnv & env, OpBuilder & builder, unsigned exp, unsigned at ) | static |
|---|
◆ finalizeWhileOp()
Generates the induction structure for a while-loop.
Definition at line 867 of file Sparsification.cpp.
References mlir::OpBuilder::create(), mlir::get(), mlir::sparse_tensor::CodegenEnv::getExpandCount(), mlir::OpBuilder::getInsertionBlock(), mlir::sparse_tensor::CodegenEnv::getInsertionChain(), mlir::sparse_tensor::LoopEmitter::getLoopEmitterLoopAttrName(), mlir::Block::getParentOp(), mlir::sparse_tensor::CodegenEnv::getReduc(), mlir::sparse_tensor::CodegenEnv::getValidLexInsert(), mlir::sparse_tensor::CodegenEnv::isExpand(), mlir::sparse_tensor::CodegenEnv::isReduc(), mlir::sparse_tensor::CodegenEnv::isValidLexInsert(), mlir::sparse_tensor::CodegenEnv::op(), mlir::OpBuilder::setInsertionPointAfter(), mlir::sparse_tensor::CodegenEnv::updateExpandCount(), mlir::sparse_tensor::CodegenEnv::updateInsertionChain(), mlir::sparse_tensor::CodegenEnv::updateReduc(), and mlir::sparse_tensor::CodegenEnv::updateValidLexInsert().
Referenced by endLoop().
◆ findAffine()
◆ findDepIdxSet()
Helper method to inspect affine expressions for index variable reduction based codegen.
It finds the dependent index set for all tensor levels in the current expression we are generating.
For example, when handling A[i+j][j+k], we build the two way mapping in merger between (tensor, level) pairs and their dependent index variable set: A_0 <=> [i, j] and A_1 <=> [j, k]
It rejects cases (returns false) 1st, when the same index is used more than once, e.g., A[i+j][i] 2nd, when multiplication is used in the non-trivial index expression. 3rd, when a constant operand is used in the non-trivial index expression.
TODO: constant should be easy to handle.
Definition at line 119 of file Sparsification.cpp.
References mlir::Add, mlir::Constant, mlir::DimId, mlir::AffineExpr::getKind(), mlir::sparse_tensor::Merger::getLvlType(), mlir::sparse_tensor::Merger::hasDependentLvl(), mlir::sparse_tensor::isUndefLT(), mlir::sparse_tensor::Merger::makeLoopId(), mlir::Mul, mlir::sparse_tensor::Merger::setLevelAndType(), and mlir::sparse_tensor::Merger::setLoopDependentTensorLevel().
Referenced by findSparseAnnotations().
◆ findSparseAnnotations()
| static bool findSparseAnnotations ( CodegenEnv & env, bool idxReducBased ) | static |
|---|
◆ genBuffers()
Local bufferization of all dense and sparse data structures.
Generates buffer for the output tensor. Note that all sparse kernels assume that when all elements are written to (viz. x(i) = y(i) * z(i)), the output buffer is already initialized to all zeroes and only nonzeroes values are computed and written out. For updates (viz. x(i) += y(i) * z(i)), only nonzeroes values are used for the updates and no assumption on the original contents of the output buffer is necessary.
Definition at line 287 of file Sparsification.cpp.
References mlir::sparse_tensor::constantZero(), mlir::OpBuilder::create(), mlir::sparse_tensor::CodegenEnv::emitter(), mlir::IROperand< DerivedT, IRValueT >::get(), mlir::getElementTypeOrSelf(), mlir::sparse_tensor::getSparseTensorEncoding(), mlir::Value::getType(), mlir::getValueOrCreateConstantIndexOp(), mlir::sparse_tensor::LoopEmitter::initializeLoopEmit(), and mlir::sparse_tensor::CodegenEnv::op().
◆ genCoIteration()
◆ genCoIterationCase()
◆ genConditionalInsert()
◆ genConstantDenseAddressFromLevel()
Definition at line 1148 of file Sparsification.cpp.
References mlir::sparse_tensor::CodegenEnv::emitter(), mlir::IROperand< DerivedT, IRValueT >::get(), mlir::OpOperand::getOperandNumber(), mlir::sparse_tensor::getSparseTensorEncoding(), mlir::Value::getType(), mlir::sparse_tensor::LoopEmitter::locateLvlAtAffineAddress(), mlir::sparse_tensor::CodegenEnv::makeTensorId(), mlir::sparse_tensor::CodegenEnv::makeTensorLevel(), and mlir::sparse_tensor::CodegenEnv::op().
Referenced by genInitConstantDenseAddress(), and startLoop().
◆ genExp()
Recursively generates tensor expression.
Definition at line 627 of file Sparsification.cpp.
References mlir::sparse_tensor::Merger::buildExp(), mlir::sparse_tensor::TensorExp::children, mlir::sparse_tensor::constantZero(), mlir::sparse_tensor::TensorExp::Children::e0, mlir::sparse_tensor::TensorExp::Children::e1, mlir::sparse_tensor::CodegenEnv::endCustomReduc(), mlir::sparse_tensor::CodegenEnv::exp(), genInvariantValue(), genTensorLoad(), mlir::sparse_tensor::CodegenEnv::getLoopVar(), mlir::Value::getParentBlock(), mlir::Value::getType(), mlir::sparse_tensor::TensorExp::kBinary, mlir::sparse_tensor::TensorExp::kBinaryBranch, mlir::sparse_tensor::TensorExp::kind, kind, mlir::sparse_tensor::detail::kInvalidId, mlir::sparse_tensor::TensorExp::kInvariant, mlir::sparse_tensor::TensorExp::kLoopVar, mlir::sparse_tensor::TensorExp::kReduce, mlir::sparse_tensor::TensorExp::kSelect, mlir::sparse_tensor::TensorExp::kSynZero, mlir::sparse_tensor::TensorExp::kTensor, mlir::sparse_tensor::TensorExp::kUnary, mlir::sparse_tensor::TensorExp::loop, mlir::sparse_tensor::CodegenEnv::merger(), mlir::sparse_tensor::CodegenEnv::op(), relinkBranch(), mlir::sparse_tensor::Merger::setExprValue(), and mlir::sparse_tensor::CodegenEnv::startCustomReduc().
Referenced by genStmt().
◆ genExpand()
◆ genIf()
Generates a single if-statement within a while-loop.
Definition at line 927 of file Sparsification.cpp.
References mlir::sparse_tensor::constantI1(), mlir::OpBuilder::create(), mlir::sparse_tensor::CodegenEnv::emitter(), mlir::sparse_tensor::Merger::foreachTensorLoopId(), mlir::sparse_tensor::LoopEmitter::getCoord(), mlir::Builder::getIndexType(), mlir::sparse_tensor::CodegenEnv::getInsertionChain(), mlir::sparse_tensor::CodegenEnv::getLoopVar(), mlir::sparse_tensor::CodegenEnv::getReduc(), mlir::sparse_tensor::getSparseTensorType(), mlir::Value::getType(), mlir::sparse_tensor::CodegenEnv::getValidLexInsert(), mlir::sparse_tensor::LevelType::hasDenseSemantic(), mlir::sparse_tensor::LevelType::hasSparseSemantic(), mlir::sparse_tensor::CodegenEnv::isExpand(), mlir::sparse_tensor::CodegenEnv::isReduc(), mlir::sparse_tensor::isUndefLT(), mlir::sparse_tensor::CodegenEnv::isValidLexInsert(), mlir::sparse_tensor::Merger::loop(), mlir::sparse_tensor::CodegenEnv::merger(), mlir::sparse_tensor::CodegenEnv::op(), and mlir::OpBuilder::setInsertionPointToStart().
Referenced by genStmt().
◆ genIndex()
◆ genInitConstantDenseAddress()
◆ genInsertionLoad()
◆ genInsertionLoadReduce()
◆ genInsertionStore()
Generates insertion code to implement dynamic tensor store.
Definition at line 426 of file Sparsification.cpp.
References mlir::sparse_tensor::constantI1(), mlir::sparse_tensor::constantIndex(), mlir::OpBuilder::create(), mlir::sparse_tensor::CodegenEnv::emitter(), genConditionalInsert(), genIndex(), mlir::sparse_tensor::genIsNonzero(), mlir::sparse_tensor::CodegenEnv::getCurrentDepth(), mlir::sparse_tensor::CodegenEnv::getExpandAdded(), mlir::sparse_tensor::CodegenEnv::getExpandCount(), mlir::sparse_tensor::CodegenEnv::getExpandFilled(), mlir::sparse_tensor::CodegenEnv::getExpandValues(), mlir::Builder::getIndexType(), mlir::sparse_tensor::CodegenEnv::getInsertionChain(), mlir::sparse_tensor::LoopEmitter::getLoopIVsRange(), mlir::sparse_tensor::CodegenEnv::getValidLexInsert(), mlir::sparse_tensor::hasAnySparseType(), mlir::sparse_tensor::CodegenEnv::isExpand(), mlir::sparse_tensor::CodegenEnv::isValidLexInsert(), mlir::sparse_tensor::CodegenEnv::op(), mlir::OpBuilder::setInsertionPointAfter(), mlir::OpBuilder::setInsertionPointToStart(), mlir::sparse_tensor::CodegenEnv::updateExpandCount(), and mlir::sparse_tensor::CodegenEnv::updateInsertionChain().
Referenced by genTensorStore().
◆ genInvariants()
Hoists loop invariant tensor loads for which indices have been exhausted.
Definition at line 686 of file Sparsification.cpp.
References mlir::sparse_tensor::TensorExp::children, mlir::sparse_tensor::Merger::clearExprValue(), mlir::sparse_tensor::constantI1(), mlir::sparse_tensor::TensorExp::Children::e0, mlir::sparse_tensor::TensorExp::Children::e1, mlir::sparse_tensor::CodegenEnv::endCustomReduc(), mlir::sparse_tensor::CodegenEnv::endReduc(), mlir::sparse_tensor::CodegenEnv::endValidLexInsert(), mlir::sparse_tensor::CodegenEnv::exp(), genTensorLoad(), genTensorStore(), mlir::IROperand< DerivedT, IRValueT >::get(), mlir::sparse_tensor::CodegenEnv::getCustomRedId(), mlir::sparse_tensor::getSparseTensorType(), mlir::sparse_tensor::CodegenEnv::hasSparseOutput(), mlir::sparse_tensor::CodegenEnv::isCustomReduc(), isInvariantAffine(), mlir::sparse_tensor::CodegenEnv::isReduc(), mlir::sparse_tensor::TensorExp::kind, mlir::sparse_tensor::detail::kInvalidId, mlir::sparse_tensor::TensorExp::kInvariant, mlir::sparse_tensor::TensorExp::kLoopVar, mlir::sparse_tensor::TensorExp::kReduce, mlir::sparse_tensor::TensorExp::kSynZero, mlir::sparse_tensor::TensorExp::kTensor, mlir::sparse_tensor::CodegenEnv::merger(), mlir::sparse_tensor::CodegenEnv::op(), mlir::sparse_tensor::Merger::setExprValue(), mlir::sparse_tensor::CodegenEnv::startCustomReduc(), mlir::sparse_tensor::CodegenEnv::startReduc(), mlir::sparse_tensor::CodegenEnv::startValidLexInsert(), and mlir::sparse_tensor::TensorExp::tensor.
Referenced by endLoopSeq(), and startLoopSeq().
◆ genInvariantValue()
◆ genLoop()
◆ genResult()
Converts the result computed by the sparse kernel into the required form.
Definition at line 1363 of file Sparsification.cpp.
References mlir::sparse_tensor::CodegenEnv::emitter(), mlir::IROperand< DerivedT, IRValueT >::get(), mlir::sparse_tensor::CodegenEnv::getInsertionChain(), mlir::sparse_tensor::Merger::getOutTensorID(), mlir::sparse_tensor::getSparseTensorEncoding(), mlir::Value::getType(), mlir::sparse_tensor::LoopEmitter::getValBuffer(), mlir::sparse_tensor::CodegenEnv::merger(), mlir::sparse_tensor::CodegenEnv::op(), and mlir::RewriterBase::replaceOpWithNewOp().
◆ genStmt()
Recursively generates code while computing iteration lattices in order to manage the complexity of implementing co-iteration over unions and intersections of sparse iterations spaces.
Definition at line 1270 of file Sparsification.cpp.
References mlir::sparse_tensor::Merger::buildLattices(), mlir::OpBuilder::create(), endIf(), endLoop(), endLoopSeq(), mlir::sparse_tensor::LatPoint::exp, genCoIterationCase(), mlir::sparse_tensor::CodegenEnv::generatingSparseIterator(), genExp(), genIf(), mlir::sparse_tensor::CodegenEnv::genLoopBoundary(), genTensorStore(), mlir::sparse_tensor::CodegenEnv::getCurrentDepth(), mlir::sparse_tensor::CodegenEnv::getExpandCount(), mlir::sparse_tensor::CodegenEnv::getInsertionChain(), mlir::sparse_tensor::CodegenEnv::getLoopNum(), mlir::sparse_tensor::CodegenEnv::getReduc(), mlir::sparse_tensor::CodegenEnv::getValidLexInsert(), mlir::sparse_tensor::CodegenEnv::lat(), mlir::sparse_tensor::Merger::latGT(), mlir::sparse_tensor::CodegenEnv::merger(), mlir::sparse_tensor::CodegenEnv::op(), mlir::sparse_tensor::Merger::optimizeSet(), mlir::sparse_tensor::CodegenEnv::set(), startLoop(), and startLoopSeq().
◆ genSubscript()
Generates subscript for load/store on a dense or sparse tensor.
Definition at line 349 of file Sparsification.cpp.
References mlir::sparse_tensor::CodegenEnv::emitter(), mlir::sparse_tensor::LoopEmitter::genAffine(), mlir::IROperand< DerivedT, IRValueT >::get(), mlir::OpOperand::getOperandNumber(), mlir::sparse_tensor::getSparseTensorType(), mlir::sparse_tensor::LoopEmitter::getValBuffer(), mlir::sparse_tensor::LoopEmitter::getValPosits(), mlir::kSparseIterator, mlir::sparse_tensor::CodegenEnv::makeTensorId(), mlir::sparse_tensor::CodegenEnv::op(), mlir::sparse_tensor::CodegenEnv::options(), and mlir::SparsificationOptions::sparseEmitStrategy.
Referenced by genTensorLoad(), genTensorStore(), and relinkBranch().
◆ genTensorLoad()
Generates a load on a dense or sparse tensor.
Definition at line 499 of file Sparsification.cpp.
References mlir::OpBuilder::create(), mlir::sparse_tensor::CodegenEnv::exp(), genInsertionLoad(), genInsertionLoadReduce(), genSubscript(), mlir::sparse_tensor::genValFromAttr(), mlir::IROperand< DerivedT, IRValueT >::get(), mlir::sparse_tensor::getSparseTensorType(), mlir::Value::getType(), mlir::sparse_tensor::CodegenEnv::isCustomReduc(), mlir::sparse_tensor::CodegenEnv::isSparseOutput(), mlir::kSparseIterator, mlir::sparse_tensor::CodegenEnv::op(), mlir::sparse_tensor::CodegenEnv::options(), mlir::SparsificationOptions::sparseEmitStrategy, mlir::sparse_tensor::TensorExp::tensor, and mlir::sparse_tensor::TensorExp::val.
Referenced by genExp(), and genInvariants().
◆ genTensorStore()
Generates a store on a dense or sparse tensor.
Definition at line 531 of file Sparsification.cpp.
References mlir::sparse_tensor::Merger::clearExprValue(), mlir::OpBuilder::create(), mlir::sparse_tensor::CodegenEnv::exp(), genInsertionStore(), genSubscript(), mlir::sparse_tensor::CodegenEnv::getInsertionChain(), mlir::Value::getType(), mlir::sparse_tensor::CodegenEnv::isReduc(), mlir::sparse_tensor::CodegenEnv::isSparseOutput(), mlir::sparse_tensor::TensorExp::kBinary, mlir::sparse_tensor::TensorExp::kind, mlir::sparse_tensor::TensorExp::kReduce, mlir::sparse_tensor::TensorExp::kSelect, mlir::sparse_tensor::TensorExp::kUnary, mlir::sparse_tensor::CodegenEnv::merger(), mlir::sparse_tensor::CodegenEnv::op(), mlir::OpBuilder::setInsertionPointAfter(), mlir::OpBuilder::setInsertionPointToStart(), mlir::sparse_tensor::CodegenEnv::updateInsertionChain(), mlir::sparse_tensor::CodegenEnv::updateReduc(), and mlir::sparse_tensor::TensorExp::val.
Referenced by genInvariants(), and genStmt().
◆ getAllTidLvlsInLatPoints()
Definition at line 1004 of file Sparsification.cpp.
References mlir::sparse_tensor::Merger::foreachTensorLoopId(), mlir::IROperand< DerivedT, IRValueT >::get(), mlir::sparse_tensor::CodegenEnv::getCurrentDepth(), mlir::sparse_tensor::Merger::getLvl(), mlir::sparse_tensor::Merger::getOutTensorID(), mlir::sparse_tensor::getSparseTensorType(), mlir::sparse_tensor::Merger::getSynTensorID(), mlir::sparse_tensor::LevelType::hasDenseSemantic(), mlir::sparse_tensor::isDenseLT(), isInvariantAffine(), mlir::sparse_tensor::isUndefLT(), mlir::sparse_tensor::isUniqueLT(), mlir::kSparseIterator, mlir::sparse_tensor::CodegenEnv::lat(), mlir::sparse_tensor::CodegenEnv::lt(), mlir::sparse_tensor::CodegenEnv::makeTensorLevel(), mlir::sparse_tensor::CodegenEnv::merger(), mlir::sparse_tensor::CodegenEnv::op(), mlir::sparse_tensor::CodegenEnv::options(), mlir::sparse_tensor::LatPoint::simple, and mlir::SparsificationOptions::sparseEmitStrategy.
Referenced by startLoopSeq(), and translateBitsToTidLvlPairs().
◆ getNumNonTrivialIdxExpOnSparseLvls() [1/2]
| static unsigned getNumNonTrivialIdxExpOnSparseLvls ( AffineMap map, Value tensor ) | static |
|---|
◆ getNumNonTrivialIdxExpOnSparseLvls() [2/2]
| static unsigned getNumNonTrivialIdxExpOnSparseLvls ( linalg::GenericOp op) | static |
|---|
◆ hasNonTrivialAffineOnSparseOut()
| static bool hasNonTrivialAffineOnSparseOut ( linalg::GenericOp op) | static |
|---|
◆ isInvariantAffine()
| static bool isInvariantAffine ( AffineExpr a, LoopId curr, bool & isCurrentLoop ) | static |
|---|
◆ isParallelFor()
| static bool isParallelFor ( CodegenEnv & env, bool isOuter, bool isSparse ) | static |
|---|
◆ relinkBranch()
Semi-ring branches are simply inlined by the sparsifier.
Prior analysis has verified that all computations are "local" to the inlined branch or otherwise invariantly defined outside the loop nest, with the exception of index computations, which need to be relinked to actual inlined cloned code.
Definition at line 593 of file Sparsification.cpp.
References mlir::OpBuilder::create(), genSubscript(), mlir::IROperand< DerivedT, IRValueT >::get(), mlir::Value::getDefiningOp(), mlir::sparse_tensor::CodegenEnv::getLoopVar(), mlir::sparse_tensor::getSparseTensorType(), mlir::sparse_tensor::SparseTensorType::hasEncoding(), mlir::sparse_tensor::CodegenEnv::makeLoopId(), mlir::sparse_tensor::CodegenEnv::makeTensorId(), mlir::RewriterBase::modifyOpInPlace(), mlir::sparse_tensor::CodegenEnv::op(), and mlir::OpBuilder::setInsertionPoint().
Referenced by genExp().
◆ shouldTryParallize()
◆ startLoop()
◆ startLoopSeq()
Starts a loop sequence at given level.
Returns true if the universal loop index must be maintained at this level.
Definition at line 1116 of file Sparsification.cpp.
References mlir::sparse_tensor::CodegenEnv::emitter(), mlir::sparse_tensor::LoopEmitter::enterNewLoopSeq(), genExpand(), genInvariants(), getAllTidLvlsInLatPoints(), mlir::sparse_tensor::CodegenEnv::getLoopVar(), mlir::sparse_tensor::Merger::hasAnySparse(), mlir::sparse_tensor::CodegenEnv::lat(), mlir::sparse_tensor::CodegenEnv::merger(), mlir::sparse_tensor::CodegenEnv::op(), mlir::sparse_tensor::CodegenEnv::set(), and mlir::sparse_tensor::LatPoint::simple.
Referenced by genStmt().