InterleavedAccessPass.cpp File Reference (original) (raw)

Go to the source code of this file.

Macros
#define	DEBUG_TYPE "interleaved-access"

Functions
	INITIALIZE_PASS_BEGIN (InterleavedAccess, DEBUG_TYPE, "Lower interleaved memory accesses to target specific intrinsics", false, false) INITIALIZE_PASS_END(InterleavedAccess
static bool	isDeInterleaveMask (ArrayRef< int > Mask, unsigned &Factor, unsigned &Index, unsigned MaxFactor, unsigned NumLoadElements)
	Check if the mask is a DE-interleave mask for an interleaved load.
static bool	isReInterleaveMask (ShuffleVectorInst *SVI, unsigned &Factor, unsigned MaxFactor)
	Check if the mask can be used in an interleaved store.
static Value *	getMaskOperand (IntrinsicInst *II)
static std::pair< Value *, APInt >	getMask (Value *WideMask, unsigned Factor, ElementCount LeafValueEC)
static std::pair< Value *, APInt >	getMask (Value WideMask, unsigned Factor, VectorType LeafValueTy)
static void	getGapMask (const Constant &MaskConst, unsigned Factor, unsigned LeafMaskLen, APInt &GapMask)

Variables
static cl::opt< bool >	LowerInterleavedAccesses ("lower-interleaved-accesses", cl::desc("Enable lowering interleaved accesses to intrinsics"), cl::init(true), cl::Hidden)
	DEBUG_TYPE
Lower interleaved memory accesses to target specific	intrinsics
Lower interleaved memory accesses to target specific	false

◆ DEBUG_TYPE

#define DEBUG_TYPE "interleaved-access"

◆ getGapMask()

◆ getMask() [1/2]

Definition at line 588 of file InterleavedAccessPass.cpp.

References llvm::all_of(), llvm::CallingConv::C, llvm::cast(), llvm::dyn_cast(), llvm::enumerate(), F, llvm::ConstantVector::get(), llvm::VectorType::get(), llvm::APInt::getAllOnes(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getFixedValue(), getGapMask(), llvm::Type::getInt1Ty(), llvm::getInterleaveIntrinsicFactor(), getMask(), llvm::ConstantVector::getSplat(), llvm::IRBuilder(), llvm::isa(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::isFixed(), llvm::ShuffleVectorInst::isInterleaveMask(), and llvm::Splat.

Referenced by llvm::VPlanTransforms::addMinimumIterationCheck(), llvm::VPInterleaveRecipe::clone(), llvm::VPReplicateRecipe::clone(), llvm::VPWidenLoadRecipe::clone(), llvm::VPWidenStoreRecipe::clone(), combineConcatVectorOps(), llvm::EpilogueVectorizerMainLoop::createIterationCountCheck(), getMask(), getMask(), llvm::VPInterleaveEVLRecipe::getNumStoreOperands(), llvm::VPInterleaveRecipe::getNumStoreOperands(), lowerShuffleAsLanePermuteAndRepeatedMask(), and llvm::VPBlendRecipe::printRecipe().

◆ getMask() [2/2]

◆ getMaskOperand()

◆ INITIALIZE_PASS_BEGIN()

INITIALIZE_PASS_BEGIN	(	InterleavedAccess	,
DEBUG_TYPE	,
"Lower interleaved memory accesses to target specific intrinsics"	,
false	,
false	)

◆ isDeInterleaveMask()

◆ isReInterleaveMask()

Check if the mask can be used in an interleaved store.

It checks for a more general pattern than the RE-interleave mask. I.e. <x, y, ... z, x+1, y+1, ...z+1, x+2, y+2, ...z+2, ...> E.g. For a Factor of 2 (LaneLen=4): <4, 32, 5, 33, 6, 34, 7, 35> E.g. For a Factor of 3 (LaneLen=4): <4, 32, 16, 5, 33, 17, 6, 34, 18, 7, 35, 19> E.g. For a Factor of 4 (LaneLen=2): <8, 2, 12, 4, 9, 3, 13, 5>

The particular case of an RE-interleave mask is: I.e. <0, LaneLen, ... , LaneLen*(Factor - 1), 1, LaneLen + 1, ...> E.g. For a Factor of 2 (LaneLen=4): <0, 4, 1, 5, 2, 6, 3, 7>

Definition at line 241 of file InterleavedAccessPass.cpp.

References llvm::ShuffleVectorInst::getShuffleMask(), and llvm::ShuffleVectorInst::isInterleave().

◆ DEBUG_TYPE

◆ false

Lower interleaved memory accesses to target specific false

◆ intrinsics

Lower interleaved memory accesses to target specific intrinsics

◆ LowerInterleavedAccesses

cl::opt< bool > LowerInterleavedAccesses("lower-interleaved-accesses", cl::desc("Enable lowering interleaved accesses to intrinsics"), cl::init(true), cl::Hidden) ( "lower-interleaved-accesses" , cl::desc("Enable lowering interleaved accesses to intrinsics") , cl::init(true) , cl::Hidden )	static