LLVM: llvm::EpilogueVectorizerMainLoop Class Reference (original) (raw)

A specialized derived class of inner loop vectorizer that performs vectorization of main loops in the process of vectorizing loops and their epilogues. More...

Public Member Functions
EpilogueVectorizerMainLoop (Loop *OrigLoop, PredicatedScalarEvolution &PSE, LoopInfo *LI, DominatorTree *DT, const TargetTransformInfo *TTI, AssumptionCache *AC, EpilogueLoopVectorizationInfo &EPI, LoopVectorizationCostModel *CM, GeneratedRTChecks &Check, VPlan &Plan)
BasicBlock * createVectorizedLoopSkeleton () final
Implements the interface for creating a vectorized skeleton using the main loop strategy (i.e., the first pass of VPlan execution).
Public Member Functions inherited from llvm::InnerLoopAndEpilogueVectorizer
InnerLoopAndEpilogueVectorizer (Loop *OrigLoop, PredicatedScalarEvolution &PSE, LoopInfo *LI, DominatorTree *DT, const TargetTransformInfo *TTI, AssumptionCache *AC, EpilogueLoopVectorizationInfo &EPI, LoopVectorizationCostModel *CM, GeneratedRTChecks &Checks, VPlan &Plan, ElementCount VecWidth, ElementCount MinProfitableTripCount, unsigned UnrollFactor)
Public Member Functions inherited from llvm::InnerLoopVectorizer
InnerLoopVectorizer (Loop *OrigLoop, PredicatedScalarEvolution &PSE, LoopInfo *LI, DominatorTree *DT, const TargetTransformInfo *TTI, AssumptionCache *AC, ElementCount VecWidth, unsigned UnrollFactor, LoopVectorizationCostModel *CM, GeneratedRTChecks &RTChecks, VPlan &Plan)
virtual ~InnerLoopVectorizer ()=default
void fixVectorizedLoop (VPTransformState &State)
Fix the vectorized code, taking care of header phi's, and more.
void fixNonInductionPHIs (VPTransformState &State)
Fix the non-induction PHIs in Plan.
Value * getTripCount () const
Returns the original loop trip count.
void setTripCount (Value *TC)
Used to set the trip count after ILV's construction and after the preheader block has been executed.
Protected Member Functions
void introduceCheckBlockInVPlan (BasicBlock *CheckIRBB)
Introduces a new VPIRBasicBlock for CheckIRBB to Plan between the vector preheader and its predecessor, also connecting the new block to the scalar preheader.
Value * createIterationCountCheck (BasicBlock *VectorPH, ElementCount VF, unsigned UF) const
BasicBlock * emitIterationCountCheck (BasicBlock *VectorPH, BasicBlock *Bypass, bool ForEpilogue)
Emits an iteration count bypass check once for the main loop (when ForEpilogue is false) and once for the epilogue loop (when ForEpilogue is true).
void printDebugTracesAtStart () override
Allow subclasses to override and print debug traces before/after vplan execution, when trace information is requested.
void printDebugTracesAtEnd () override
Protected Member Functions inherited from llvm::InnerLoopVectorizer
BasicBlock * createScalarPreheader (StringRef Prefix)
Create and return a new IR basic block for the scalar preheader whose name is prefixed with Prefix.
Additional Inherited Members
Public Attributes inherited from llvm::InnerLoopAndEpilogueVectorizer
EpilogueLoopVectorizationInfo & EPI
Holds and updates state information required to vectorize the main loop and its epilogue in two separate passes.
Protected Attributes inherited from llvm::InnerLoopAndEpilogueVectorizer
ElementCount MinProfitableTripCount
Protected Attributes inherited from llvm::InnerLoopVectorizer
Loop * OrigLoop
The original loop.
PredicatedScalarEvolution & PSE
A wrapper around ScalarEvolution used to add runtime SCEV checks.
LoopInfo * LI
Loop Info.
DominatorTree * DT
Dominator Tree.
const TargetTransformInfo * TTI
Target Transform Info.
AssumptionCache * AC
Assumption Cache.
ElementCount VF
The vectorization SIMD factor to use.
unsigned UF
The vectorization unroll factor to use.
IRBuilder Builder
The builder that we use.
Value * TripCount = nullptr
Trip count of the original loop.
LoopVectorizationCostModel * Cost
The profitablity analysis.
GeneratedRTChecks & RTChecks
Structure to hold information about generated runtime checks, responsible for cleaning the checks, if vectorization turns out unprofitable.
VPlan & Plan
VPBasicBlock * VectorPHVPBB
The vector preheader block of Plan, used as target for check blocks introduced during skeleton creation.

A specialized derived class of inner loop vectorizer that performs vectorization of main loops in the process of vectorizing loops and their epilogues.

Definition at line 655 of file LoopVectorize.cpp.

◆ createIterationCountCheck()

Definition at line 2304 of file LoopVectorize.cpp.

References llvm::ScalarEvolution::applyLoopGuards(), llvm::InnerLoopVectorizer::Builder, llvm::cast(), llvm::Count, llvm::createStepForVF(), llvm::DataAndControlFlowWithoutRuntimeCheck, llvm::BasicBlock::getContext(), llvm::BasicBlock::getDataLayout(), llvm::CmpInst::getInversePredicate(), getMask(), llvm::ScalarEvolution::getSCEV(), llvm::BasicBlock::getTerminator(), llvm::InnerLoopVectorizer::getTripCount(), llvm::CmpInst::ICMP_ULE, llvm::CmpInst::ICMP_ULT, isIndvarOverflowCheckKnownFalse(), llvm::ScalarEvolution::isKnownPredicate(), llvm::InnerLoopAndEpilogueVectorizer::MinProfitableTripCount, llvm::None, llvm::InnerLoopVectorizer::OrigLoop, P, llvm::InnerLoopVectorizer::PSE, llvm::InnerLoopVectorizer::UF, and llvm::InnerLoopVectorizer::VF.

Referenced by emitIterationCountCheck().

◆ createVectorizedLoopSkeleton()

BasicBlock * EpilogueVectorizerMainLoop::createVectorizedLoopSkeleton ( ) finalvirtual

◆ emitIterationCountCheck()

Emits an iteration count bypass check once for the main loop (when ForEpilogue is false) and once for the epilogue loop (when ForEpilogue is true).

Definition at line 7536 of file LoopVectorize.cpp.

References assert(), llvm::Count, llvm::BranchInst::Create(), createIterationCountCheck(), llvm::InnerLoopAndEpilogueVectorizer::EPI, llvm::ElementCount::getFixed(), llvm::BasicBlock::getTerminator(), llvm::InnerLoopVectorizer::getTripCount(), llvm::hasBranchWeightMD(), introduceCheckBlockInVPlan(), llvm::InnerLoopVectorizer::LI, MinItersBypassWeights, llvm::InnerLoopAndEpilogueVectorizer::MinProfitableTripCount, llvm::InnerLoopVectorizer::OrigLoop, llvm::ReplaceInstWithInst(), replaceVPBBWithIRVPBB(), llvm::setBranchWeights(), llvm::Value::setName(), llvm::SplitBlock(), and llvm::InnerLoopVectorizer::VectorPHVPBB.

Referenced by createVectorizedLoopSkeleton().

◆ introduceCheckBlockInVPlan()

void EpilogueVectorizerMainLoop::introduceCheckBlockInVPlan ( BasicBlock * CheckIRBB) protected

Introduces a new VPIRBasicBlock for CheckIRBB to Plan between the vector preheader and its predecessor, also connecting the new block to the scalar preheader.

Definition at line 2279 of file LoopVectorize.cpp.

References assert(), llvm::cast(), llvm::VPBlockUtils::connectBlocks(), llvm::VPBlockBase::getNumPredecessors(), llvm::VPBlockBase::getNumSuccessors(), llvm::VPBlockBase::getSuccessors(), llvm::VPBlockUtils::insertOnEdge(), llvm::isa(), llvm::InnerLoopVectorizer::Plan, llvm::VPBlockBase::swapSuccessors(), and llvm::InnerLoopVectorizer::VectorPHVPBB.

Referenced by emitIterationCountCheck().

◆ printDebugTracesAtEnd()

void EpilogueVectorizerMainLoop::printDebugTracesAtEnd ( ) overrideprotectedvirtual

◆ printDebugTracesAtStart()

void EpilogueVectorizerMainLoop::printDebugTracesAtStart ( ) overrideprotectedvirtual

The documentation for this class was generated from the following file: