VPRecipeBuilder.h Source File (original) (raw)

9#ifndef LLVM_TRANSFORMS_VECTORIZE_VPRECIPEBUILDER_H

10#define LLVM_TRANSFORMS_VECTORIZE_VPRECIPEBUILDER_H

17namespace llvm {

52 Loop *OrigLoop;

100

101

102

105

106

107

108

110

111

112

113

115

116

117

118

119

122

123

124

125

126

127

128

129

130

131

132 bool getScaledReductions(

134 SmallVectorImpl<std::pair<PartialReductionChain, unsigned>> &Chains);

135

136public:

143 : Plan(Plan), OrigLoop(OrigLoop), TLI(TLI), TTI(TTI), Legal(Legal),

144 CM(CM), PSE(PSE), Builder(Builder), BlockMaskCache(BlockMaskCache) {}

145

147 auto It = ScaledReductionMap.find(ExitInst);

148 return It == ScaledReductionMap.end() ? std::nullopt

149 : std::make_optional(It->second);

150 }

151

152

153

155

156

157

159

160

161

163 unsigned ScaleFactor);

164

165

167 assert(!Ingredient2Recipe.contains(I) &&

168 "Cannot reset recipe for instruction.");

169 Ingredient2Recipe[I] = R;

170 }

171

172

173

175 return BlockMaskCache.lookup(VPBB);

176 }

177

178

180 assert(Ingredient2Recipe.count(I) &&

181 "Recording this ingredients recipe was not requested");

182 assert(Ingredient2Recipe[I] != nullptr &&

183 "Ingredient doesn't have a recipe");

184 return Ingredient2Recipe[I];

185 }

186

187

188

189

191

194 if (auto *R = Ingredient2Recipe.lookup(I))

195 return R->getVPSingleValue();

196 }

197 return Plan.getOrAddLiveIn(V);

198 }

199

201 for (auto &[_, V] : BlockMaskCache) {

202 if (auto *New = Old2New.lookup(V)) {

203 V->replaceAllUsesWith(New);

204 V = New;

205 }

206 }

207 }

208};

209}

210

211#endif

assert(UImm &&(UImm !=~static_cast< T >(0)) &&"Invalid immediate!")

This file defines the DenseMap class.

This file provides a LoopVectorizationPlanner class.

ConstantRange Range(APInt(BitWidth, Low), APInt(BitWidth, High))

This file contains the declarations of the Vectorization Plan base classes:

ValueT lookup(const_arg_type_t< KeyT > Val) const

lookup - Return the entry for the specified key, or a default constructed value if no such entry exis...

LoopVectorizationCostModel - estimates the expected speedups due to vectorization.

LoopVectorizationLegality checks if it is legal to vectorize a loop, and to what vectorization factor...

Represents a single loop in the control flow graph.

An interface layer with SCEV used to manage how we see SCEV expressions for values in the context of ...

This class consists of common code factored out of the SmallVector class to reduce code duplication b...

This is a 'vector' (really, a variable-sized array), optimized for the case when the array is small.

Provides information about what library functions are available for the current target.

This pass provides access to the codegen interfaces that are needed for IR-level transformations.

VPBasicBlock serves as the leaf of the Hierarchical Control-Flow Graph.

VPlan-based builder utility analogous to IRBuilder.

A recipe representing a sequence of load -> update -> store as part of a histogram operation.

This is a concrete Recipe that models a single VPlan-level instruction.

VPRecipeBase is a base class modeling a sequence of one or more output IR instructions.

VPRecipeBase * tryToCreateWidenRecipe(VPSingleDefRecipe *R, VFRange &Range)

Create and return a widened recipe for R if one can be created within the given VF Range.

VPValue * getBlockInMask(VPBasicBlock *VPBB) const

Returns the entry mask for block VPBB or null if the mask is all-true.

Definition VPRecipeBuilder.h:174

VPRecipeBuilder(VPlan &Plan, Loop *OrigLoop, const TargetLibraryInfo *TLI, const TargetTransformInfo *TTI, LoopVectorizationLegality *Legal, LoopVectorizationCostModel &CM, PredicatedScalarEvolution &PSE, VPBuilder &Builder, DenseMap< VPBasicBlock *, VPValue * > &BlockMaskCache)

Definition VPRecipeBuilder.h:137

VPValue * getVPValueOrAddLiveIn(Value *V)

Definition VPRecipeBuilder.h:192

VPRecipeBase * tryToCreatePartialReduction(VPInstruction *Reduction, unsigned ScaleFactor)

Create and return a partial reduction recipe for a reduction instruction along with binary operation ...

std::optional< unsigned > getScalingForReduction(const Instruction *ExitInst)

Definition VPRecipeBuilder.h:146

void collectScaledReductions(VFRange &Range)

Find all possible partial reductions in the loop and track all of those that are valid so recipes can...

void setRecipe(Instruction *I, VPRecipeBase *R)

Set the recipe created for given ingredient.

Definition VPRecipeBuilder.h:166

VPReplicateRecipe * handleReplication(VPInstruction *VPI, VFRange &Range)

Build a VPReplicationRecipe for VPI.

VPRecipeBase * getRecipe(Instruction *I)

Return the recipe created for given ingredient.

Definition VPRecipeBuilder.h:179

void updateBlockMaskCache(DenseMap< VPValue *, VPValue * > &Old2New)

Definition VPRecipeBuilder.h:200

VPReplicateRecipe replicates a given instruction producing multiple scalar copies of the original sca...

VPSingleDef is a base class for recipes for modeling a sequence of one or more output IR that define ...

This is the base class of the VPlan Def/Use graph, used for modeling the data flow into,...

A recipe for handling phi nodes of integer and floating-point inductions, producing their vector valu...

A common base class for widening memory operations.

VPWidenRecipe is a recipe for producing a widened instruction using the opcode and operands of the re...

VPlan models a candidate for vectorization, encoding various decisions take to produce efficient outp...

LLVM Value Representation.

This is an optimization pass for GlobalISel generic memory operations.

decltype(auto) dyn_cast(const From &Val)

dyn_cast - Return the argument parameter cast to the specified type.

This holds details about a histogram operation – a load -> update -> store sequence where each lane i...

PartialReductionChain(Instruction *Reduction, Instruction *ExtendA, Instruction *ExtendB, Instruction *ExtendUser)

Definition VPRecipeBuilder.h:31

Instruction * ExtendB

Definition VPRecipeBuilder.h:40

Instruction * ExtendUser

The user of the extends that is then reduced.

Definition VPRecipeBuilder.h:43

Instruction * Reduction

The top-level binary operation that forms the reduction to a scalar after the loop body.

Definition VPRecipeBuilder.h:37

Instruction * ExtendA

The extension of each of the inner binary operation's operands.

Definition VPRecipeBuilder.h:39

A range of powers-of-2 vectorization factors with fixed start and adjustable end.