LLVM: llvm::PeelingModuloScheduleExpander Class Reference (original) (raw)

A reimplementation of ModuloScheduleExpander. More...

#include "[llvm/CodeGen/ModuloSchedule.h](ModuloSchedule%5F8h%5Fsource.html)"

Protected Member Functions
void rewriteKernel ()
Converts BB from the original loop body to the rewritten, pipelined steady-state.
MachineBasicBlock * peelKernel (LoopPeelDirection LPD)
Peels one iteration of the rewritten kernel (BB) in the specified direction.
void filterInstructions (MachineBasicBlock *MB, int MinStage)
void moveStageBetweenBlocks (MachineBasicBlock *DestBB, MachineBasicBlock *SourceBB, unsigned Stage)
void peelPrologAndEpilogs ()
Peel the kernel forwards and backwards to produce prologs and epilogs, and stitch them together.
Register getEquivalentRegisterIn (Register Reg, MachineBasicBlock *BB)
All prolog and epilog blocks are clones of the kernel, so any produced register in one block has an corollary in all other blocks.
void rewriteUsesOf (MachineInstr *MI)
Change all users of MI, if MI is predicated out (LiveStages[MI->getParent()] == false).
void fixupBranches ()
Insert branches between prologs, kernel and epilogs.
MachineBasicBlock * CreateLCSSAExitingBlock ()
Create a poor-man's LCSSA by cloning only the PHIs from the kernel block to a block dominated by all prologs and epilogs.
unsigned getStage (MachineInstr *MI)
Helper to get the stage of an instruction in the schedule.
Register getPhiCanonicalReg (MachineInstr *CanonicalPhi, MachineInstr *Phi)
Helper function to find the right canonical register for a phi instruction coming from a peeled out prologue.
Protected Attributes
ModuloSchedule & Schedule
MachineFunction & MF
const TargetSubtargetInfo & ST
MachineRegisterInfo & MRI
const TargetInstrInfo * TII = nullptr
LiveIntervals * LIS = nullptr
MachineBasicBlock * BB = nullptr
The original loop block that gets rewritten in-place.
MachineBasicBlock * Preheader = nullptr
The original loop preheader.
SmallVector< MachineBasicBlock *, 4 > Prologs
All prolog and epilog blocks.
SmallVector< MachineBasicBlock *, 4 > Epilogs
DenseMap< MachineBasicBlock *, BitVector > LiveStages
For every block, the stages that are produced.
DenseMap< MachineBasicBlock *, BitVector > AvailableStages
For every block, the stages that are available.
DenseMap< MachineInstr *, unsigned > PhiNodeLoopIteration
When peeling the epilogue keep track of the distance between the phi nodes and the kernel.
DenseMap< MachineInstr *, MachineInstr * > CanonicalMIs
CanonicalMIs and BlockMIs form a bidirectional map between any of the loop kernel clones.
DenseMap< std::pair< MachineBasicBlock *, MachineInstr * >, MachineInstr * > BlockMIs
std::deque< MachineBasicBlock * > PeeledFront
State passed from peelKernel to peelPrologAndEpilogs().
std::deque< MachineBasicBlock * > PeeledBack
SmallVector< MachineInstr *, 4 > IllegalPhisToDelete
Illegal phis that need to be deleted once we re-link stages.
std::unique_ptr< TargetInstrInfo::PipelinerLoopInfo > LoopInfo
Target loop info before kernel peeling.

A reimplementation of ModuloScheduleExpander.

It works by generating a standalone kernel loop and peeling out the prologs and epilogs.

Definition at line 283 of file ModuloSchedule.h.

◆ CreateLCSSAExitingBlock()

MachineBasicBlock * PeelingModuloScheduleExpander::CreateLCSSAExitingBlock ( ) protected

Create a poor-man's LCSSA by cloning only the PHIs from the kernel block to a block dominated by all prologs and epilogs.

This allows us to treat the loop exiting block as any other kernel clone.

Definition at line 1869 of file ModuloSchedule.cpp.

References llvm::MachineInstrBuilder::addMBB(), llvm::MachineInstrBuilder::addReg(), llvm::MachineBasicBlock::addSuccessor(), assert(), BB, BlockMIs, llvm::BuildMI(), CanonicalMIs, Cond, llvm::dwarf_linker::DebugLoc, MF, MI, MRI, TBB, TII, and Uses.

Referenced by peelPrologAndEpilogs().

◆ expand()

void PeelingModuloScheduleExpander::expand ( )

◆ filterInstructions()

void PeelingModuloScheduleExpander::filterInstructions ( MachineBasicBlock * MB, int MinStage ) protected

Definition at line 1620 of file ModuloSchedule.cpp.

References assert(), llvm::SmallVectorImpl< T >::emplace_back(), getEquivalentRegisterIn(), llvm::MachineBasicBlock::getFirstInstrTerminator(), llvm::MachineBasicBlock::getFirstNonPHI(), getStage(), I, LIS, MI, MRI, llvm::Sub, and UseMI.

Referenced by peelPrologAndEpilogs().

◆ fixupBranches()

void PeelingModuloScheduleExpander::fixupBranches ( ) protected

Insert branches between prologs, kernel and epilogs.

Definition at line 1963 of file ModuloSchedule.cpp.

References Cond, llvm::dbgs(), llvm::dwarf_linker::DebugLoc, Epilog, Epilogs, LLVM_DEBUG, LoopInfo, P, llvm::MachineBasicBlock::phis(), Prolog, Prologs, Schedule, and TII.

Referenced by expand().

◆ getEquivalentRegisterIn()

◆ getPhiCanonicalReg()

Helper function to find the right canonical register for a phi instruction coming from a peeled out prologue.

Definition at line 1735 of file ModuloSchedule.cpp.

References assert(), llvm::MachineOperand::getMBB(), llvm::MachineInstr::getNumOperands(), llvm::MachineInstr::getOperand(), llvm::MachineInstr::getParent(), llvm::MachineOperand::getReg(), I, llvm::MachineInstr::isPHI(), MRI, PhiNodeLoopIteration, and std::swap().

Referenced by peelPrologAndEpilogs().

◆ getStage()

◆ moveStageBetweenBlocks()

Definition at line 1649 of file ModuloSchedule.cpp.

References llvm::MachineInstrBuilder::addMBB(), llvm::MachineInstrBuilder::addReg(), assert(), BlockMIs, llvm::BuildMI(), CanonicalMIs, llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::end(), llvm::MachineBasicBlock::end(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::find(), llvm::MachineBasicBlock::getFirstNonPHI(), llvm::MachineInstr::getOperand(), llvm::MachineOperand::getReg(), getStage(), I, llvm::MachineBasicBlock::insert(), llvm::MachineOperand::isReg(), llvm::make_early_inc_range(), llvm::make_range(), MF, MI, MRI, P, PhiNodeLoopIteration, llvm::MachineBasicBlock::phis(), llvm::MachineBasicBlock::pred_begin(), llvm::SmallVectorTemplateBase< T, bool >::push_back(), llvm::MachineOperand::setMBB(), llvm::MachineOperand::setReg(), and TII.

Referenced by peelPrologAndEpilogs().

◆ peelKernel()

Peels one iteration of the rewritten kernel (BB) in the specified direction.

Definition at line 1604 of file ModuloSchedule.cpp.

References BB, llvm::MachineBasicBlock::begin(), BlockMIs, CanonicalMIs, I, llvm::LPD_Front, MRI, PeeledBack, PeeledFront, llvm::PeelSingleBlockLoop(), and TII.

Referenced by peelPrologAndEpilogs().

◆ peelPrologAndEpilogs()

void PeelingModuloScheduleExpander::peelPrologAndEpilogs ( ) protected

Peel the kernel forwards and backwards to produce prologs and epilogs, and stitch them together.

Definition at line 1752 of file ModuloSchedule.cpp.

References llvm::append_range(), assert(), AvailableStages, B(), BB, CanonicalMIs, CreateLCSSAExitingBlock(), llvm::MachineOperand::CreateMBB(), llvm::MachineOperand::CreateReg(), Epilogs, filterInstructions(), getEquivalentRegisterIn(), getPhiCanonicalReg(), I, IllegalPhisToDelete, llvm::MachineInstr::isPHI(), LIS, LiveStages, llvm::LPD_Back, llvm::LPD_Front, MI, moveStageBetweenBlocks(), MRI, PeeledBack, PeeledFront, peelKernel(), PhiNodeLoopIteration, llvm::MachineBasicBlock::pred_begin(), Prologs, llvm::SmallVectorTemplateBase< T, bool >::push_back(), llvm::reverse(), rewriteUsesOf(), and Schedule.

Referenced by expand(), and validateAgainstModuloScheduleExpander().

◆ rewriteKernel()

void PeelingModuloScheduleExpander::rewriteKernel ( ) protected

◆ rewriteUsesOf()

void PeelingModuloScheduleExpander::rewriteUsesOf ( MachineInstr * MI) protected

Change all users of MI, if MI is predicated out (LiveStages[MI->getParent()] == false).

Definition at line 1920 of file ModuloSchedule.cpp.

References assert(), AvailableStages, llvm::SmallVectorImpl< T >::emplace_back(), getEquivalentRegisterIn(), getStage(), IllegalPhisToDelete, LIS, LiveStages, MI, MRI, llvm::Sub, and UseMI.

Referenced by peelPrologAndEpilogs().

◆ validateAgainstModuloScheduleExpander()

void PeelingModuloScheduleExpander::validateAgainstModuloScheduleExpander ( )

Runs ModuloScheduleExpander and treats it as a golden input to validate aspects of the code generated by PeelingModuloScheduleExpander.

Definition at line 2027 of file ModuloSchedule.cpp.

References assert(), BB, llvm::MachineBasicBlock::begin(), llvm::ModuloScheduleExpander::cleanup(), llvm::SmallVectorImpl< T >::emplace_back(), llvm::errs(), llvm::ModuloScheduleExpander::expand(), llvm::Failed(), llvm::raw_ostream::flush(), llvm::ModuloScheduleExpander::getRewrittenKernel(), llvm::SmallPtrSetImpl< PtrType >::insert(), LIS, MF, MRI, peelPrologAndEpilogs(), Preheader, llvm::MachineBasicBlock::print(), llvm::report_fatal_error(), and Schedule.

◆ AvailableStages

◆ BB

◆ BlockMIs

◆ CanonicalMIs

◆ Epilogs

◆ IllegalPhisToDelete

◆ LIS

LiveIntervals* llvm::PeelingModuloScheduleExpander::LIS = nullptr protected

◆ LiveStages

◆ LoopInfo

◆ MF

◆ MRI

◆ PeeledBack

◆ PeeledFront

◆ PhiNodeLoopIteration

◆ Preheader

◆ Prologs

◆ Schedule

◆ ST

◆ TII

The documentation for this class was generated from the following files: