LLVM: llvm::InnerLoopAndEpilogueVectorizer Class Reference (original) (raw)

An extension of the inner loop vectorizer that creates a skeleton for a vectorized loop that has its epilogue (residual) also vectorized. More...

Public Member Functions
	InnerLoopAndEpilogueVectorizer (Loop *OrigLoop, PredicatedScalarEvolution &PSE, LoopInfo *LI, DominatorTree *DT, const TargetLibraryInfo *TLI, const TargetTransformInfo *TTI, AssumptionCache *AC, OptimizationRemarkEmitter *ORE, EpilogueLoopVectorizationInfo &EPI, LoopVectorizationLegality *LVL, llvm::LoopVectorizationCostModel *CM, BlockFrequencyInfo *BFI, ProfileSummaryInfo *PSI, GeneratedRTChecks &Checks, VPlan &Plan)
BasicBlock *	createVectorizedLoopSkeleton (const SCEV2ValueTy &ExpandedSCEVs) final
	Create a new empty loop that will contain vectorized instructions later on, while the old loop will be used as the scalar remainder.
virtual BasicBlock *	createEpilogueVectorizedLoopSkeleton (const SCEV2ValueTy &ExpandedSCEVs)=0
	The interface for creating a vectorized skeleton using one of two different strategies, each corresponding to one execution of the vplan as described above.
Public Member Functions inherited from llvm::InnerLoopVectorizer
	InnerLoopVectorizer (Loop *OrigLoop, PredicatedScalarEvolution &PSE, LoopInfo *LI, DominatorTree *DT, const TargetLibraryInfo *TLI, const TargetTransformInfo *TTI, AssumptionCache *AC, OptimizationRemarkEmitter *ORE, ElementCount VecWidth, ElementCount MinProfitableTripCount, unsigned UnrollFactor, LoopVectorizationLegality *LVL, LoopVectorizationCostModel *CM, BlockFrequencyInfo *BFI, ProfileSummaryInfo *PSI, GeneratedRTChecks &RTChecks, VPlan &Plan)
virtual	~InnerLoopVectorizer ()=default
virtual BasicBlock *	createVectorizedLoopSkeleton (const SCEV2ValueTy &ExpandedSCEVs)
	Create a new empty loop that will contain vectorized instructions later on, while the old loop will be used as the scalar remainder.
void	fixVectorizedLoop (VPTransformState &State)
	Fix the vectorized code, taking care of header phi's, and more.
bool	areSafetyChecksAdded ()
void	scalarizeInstruction (const Instruction *Instr, VPReplicateRecipe *RepRecipe, const VPLane &Lane, VPTransformState &State)
	A helper function to scalarize a single Instruction in the innermost loop.
void	fixNonInductionPHIs (VPTransformState &State)
	Fix the non-induction PHIs in Plan.
Value *	getTripCount () const
	Returns the original loop trip count.
void	setTripCount (Value *TC)
	Used to set the trip count after ILV's construction and after the preheader block has been executed.
Value *	getInductionAdditionalBypassValue (PHINode *OrigPhi) const
	induction header phi.
BasicBlock *	getAdditionalBypassBlock () const
	Return the additional bypass block which targets the scalar loop by skipping the epilogue loop after completing the main loop.

Additional Inherited Members
Protected Member Functions inherited from llvm::InnerLoopVectorizer
void	sinkScalarOperands (Instruction *PredInst)
	Iteratively sink the scalarized operands of a predicated instruction into the block that was created for it.
Value *	getOrCreateVectorTripCount (BasicBlock *InsertBlock)
	Returns (and creates if needed) the trip count of the widened loop.
void	emitIterationCountCheck (BasicBlock *Bypass)
	Emit a bypass check to see if the vector trip count is zero, including if it overflows.
BasicBlock *	emitSCEVChecks (BasicBlock *Bypass)
	Emit a bypass check to see if all of the SCEV assumptions we've had to make are correct.
BasicBlock *	emitMemRuntimeChecks (BasicBlock *Bypass)
	Emit bypass checks to check any memory assumptions we may have made.
void	createVectorLoopSkeleton (StringRef Prefix)
	Emit basic blocks (prefixed with Prefix) for the iteration check, vector loop preheader, middle block and scalar preheader.
void	createInductionAdditionalBypassValues (const SCEV2ValueTy &ExpandedSCEVs, Value *MainVectorTripCount)
	Create and record the values for induction variables to resume coming from the additional bypass block.
virtual void	printDebugTracesAtStart ()
	Allow subclasses to override and print debug traces before/after vplan execution, when trace information is requested.
virtual void	printDebugTracesAtEnd ()
void	introduceCheckBlockInVPlan (BasicBlock *CheckIRBB)
	Introduces a new VPIRBasicBlock for CheckIRBB to Plan between the vector preheader and its predecessor, also connecting the new block to the scalar preheader.
Protected Attributes inherited from llvm::InnerLoopVectorizer
Loop *	OrigLoop
	The original loop.
PredicatedScalarEvolution &	PSE
	A wrapper around ScalarEvolution used to add runtime SCEV checks.
LoopInfo *	LI
	Loop Info.
DominatorTree *	DT
	Dominator Tree.
const TargetLibraryInfo *	TLI
	Target Library Info.
const TargetTransformInfo *	TTI
	Target Transform Info.
AssumptionCache *	AC
	Assumption Cache.
OptimizationRemarkEmitter *	ORE
	Interface to emit optimization remarks.
ElementCount	VF
	The vectorization SIMD factor to use.
ElementCount	MinProfitableTripCount
unsigned	UF
	The vectorization unroll factor to use.
IRBuilder	Builder
	The builder that we use.
BasicBlock *	LoopVectorPreHeader
	The vector-loop preheader.
BasicBlock *	LoopScalarPreHeader
	The scalar-loop preheader.
BasicBlock *	LoopMiddleBlock
	Middle Block between the vector and the scalar.
SmallVector< BasicBlock *, 4 >	LoopBypassBlocks
	A list of all bypass blocks. The first block is the entry of the loop.
SmallVector< Instruction *, 4 >	PredicatedInstructions
	Store instructions that were predicated.
Value *	TripCount = nullptr
	Trip count of the original loop.
Value *	VectorTripCount = nullptr
	Trip count of the widened loop (TripCount - TripCount % (VF*UF))
LoopVectorizationLegality *	Legal
	The legality analysis.
LoopVectorizationCostModel *	Cost
	The profitablity analysis.
bool	AddedSafetyChecks = false
BlockFrequencyInfo *	BFI
	BFI and PSI are used to check for profile guided size optimizations.
ProfileSummaryInfo *	PSI
bool	OptForSizeBasedOnProfile
GeneratedRTChecks &	RTChecks
	Structure to hold information about generated runtime checks, responsible for cleaning the checks, if vectorization turns out unprofitable.
DenseMap< PHINode *, Value * >	Induction2AdditionalBypassValue
	Mapping of induction phis to their additional bypass values.
BasicBlock *	AdditionalBypassBlock = nullptr
	The additional bypass block which conditionally skips over the epilogue loop after executing the main loop.
VPlan &	Plan
VPBlockBase *	VectorPHVPB
	The vector preheader block of Plan, used as target for check blocks introduced during skeleton creation.

An extension of the inner loop vectorizer that creates a skeleton for a vectorized loop that has its epilogue (residual) also vectorized.

The idea is to run the vplan on a given loop twice, firstly to setup the skeleton and vectorize the main loop, and secondly to complete the skeleton from the first step and vectorize the epilogue. This is achieved by deriving two concrete strategy classes from this base class and invoking them in succession from the loop vectorizer planner.

Definition at line 719 of file LoopVectorize.cpp.

◆ createEpilogueVectorizedLoopSkeleton()

◆ createVectorizedLoopSkeleton()

Create a new empty loop that will contain vectorized instructions later on, while the old loop will be used as the scalar remainder.

Control flow is generated around the vectorized (and scalar epilogue) loops consisting of various checks and bypasses. Return the pre-header block of the new loop. In the case of epilogue vectorization, this function is overriden to handle the more complex control flow around the loops. ExpandedSCEVs is used to look up SCEV expansions for expressions needed during skeleton creation.

Reimplemented from llvm::InnerLoopVectorizer.

Definition at line 737 of file LoopVectorize.cpp.

References createEpilogueVectorizedLoopSkeleton().

◆ EPI

The documentation for this class was generated from the following file:

• lib/Transforms/Vectorize/LoopVectorize.cpp