Fennel: ExecStreamGraph Class Reference (original) (raw)

An ExecStreamGraph is a directed graph representing dataflow among ExecStreams. More...

#include <[ExecStreamGraph.h](ExecStreamGraph%5F8h-source.html)>

Inheritance diagram for ExecStreamGraph:

List of all members.

Public Member Functions
virtual ~ExecStreamGraph ()
ExecStreamScheduler * getScheduler () const
**Returns:**pointer to executing scheduler, or null if there is none.
SharedDynamicParamManager getDynamicParamManager ()
**Returns:**reference to the DynamicParamManager for this graph.
virtual void setTxn (SharedLogicalTxn pTxn)=0
Sets the transaction within which this graph should execute.
virtual void setErrorTarget (SharedErrorTarget pErrorTarget)=0
Sets the ErrorTarget to which this graph's streams should send row errors.
virtual void setScratchSegment (SharedSegment pScratchSegment)=0
Sets the ScratchSegment from which this graph's streams should allocate memory buffers.
virtual void setResourceGovernor (SharedExecStreamGovernor pResourceGovernor)=0
Sets the global exec stream governor.
virtual SharedLogicalTxn getTxn ()=0
**Returns:**the transaction within which this graph is executing
virtual TxnId getTxnId ()=0
**Returns:**the transaction ID for this graph
virtual void enableDummyTxnId (bool enabled)=0
Controls whether it is OK to call getTxnId without first calling setTxn.
virtual SharedExecStreamGovernor getResourceGovernor ()=0
**Returns:**exec stream governor
virtual void addStream (SharedExecStream pStream)=0
Adds a stream to this graph.
virtual void removeStream (ExecStreamId)=0
Removes a stream from the graph: deletes the edges, and puts the vertex on a free list to be reallocated.
virtual void addDataflow (ExecStreamId producerId, ExecStreamId consumerId, bool isImplicit=false)=0
Defines a dataflow relationship between two streams in this graph.
virtual void addOutputDataflow (ExecStreamId producerId)=0
Defines a dataflow representing external output produced by this graph.
virtual void addInputDataflow (ExecStreamId consumerId)=0
Defines a dataflow representing external input consumed by this graph.
virtual void mergeFrom (ExecStreamGraph &src)=0
Adds all the vertices and edges from another graph.
virtual void mergeFrom (ExecStreamGraph &src, std::vector< ExecStreamId > const &nodes)=0
Adds a subgraph, taken (removed) from another graph.
virtual SharedExecStream findStream (std::string name)=0
Finds a stream by name.
virtual SharedExecStream findLastStream (std::string name, uint iOutput)=0
Finds last stream known for name.
virtual void interposeStream (std::string name, uint iOutput, ExecStreamId interposedId)=0
Interposes an adapter stream.
virtual void prepare (ExecStreamScheduler &scheduler)=0
Prepares this graph for execution.
virtual void open ()=0
Opens execution on this graph.
virtual SharedExecStream getStream (ExecStreamId id)=0
Translates a stream ID to a stream pointer.
virtual uint getInputCount (ExecStreamId streamId)=0
Determines number of explicit input flows consumed by a stream.
virtual uint getOutputCount (ExecStreamId streamId)=0
Determines number of explicit output flows produced by a stream.
virtual SharedExecStream getStreamInput (ExecStreamId streamId, uint iInput)=0
Accesses a stream's input.
virtual SharedExecStreamBufAccessor getStreamInputAccessor (ExecStreamId streamId, uint iInput)=0
Accesses a stream's input accessor.
virtual SharedExecStream getStreamOutput (ExecStreamId streamId, uint iOutput)=0
Accesses a stream's output.
virtual SharedExecStreamBufAccessor getStreamOutputAccessor (ExecStreamId streamId, uint iOutput)=0
Accesses a stream's output accessor.
virtual std::vector< SharedExecStream > getSortedStreams ()=0
Gets streams, sorted topologically.
virtual int getStreamCount ()=0
**Returns:**the number of streams in the graph; can only be called after prepare.
virtual int getDataflowCount ()=0
**Returns:**the number of dataflows (edges) in the graph; can only be called after prepare.
virtual void renderGraphviz (std::ostream &dotStream)=0
Renders the graph in the .dot format defined by Graphviz.
virtual bool isAcyclic ()=0
**Returns:**true if graph has no cycles
virtual void closeProducers (ExecStreamId streamId)=0
Closes the producers of a stream with a given id.
virtual void declareDynamicParamWriter (ExecStreamId streamId, DynamicParamId dynamicParamId)=0
Declares that a given stream writes a given dynamic parameter.
virtual void declareDynamicParamReader (ExecStreamId streamId, DynamicParamId dynamicParamId)=0
Declares that a given stream reads a given dynamic parameter.
virtual const std::vector< ExecStreamId > & getDynamicParamWriters (DynamicParamId dynamicParamId)=0
Returns a list of stream ids that write a given dynamic parameter.
virtual const std::vector< ExecStreamId > & getDynamicParamReaders (DynamicParamId dynamicParamId)=0
Returns a list of stream ids that read a given dynamic parameter.
bool isClosed () const
**Returns:**whether the object has been closed
void close ()
Closes this object, releasing any unallocated resources.
Static Public Member Functions
static SharedExecStreamGraph newExecStreamGraph ()
Constructs a new ExecStreamGraph.
Protected Member Functions
ExecStreamGraph ()
virtual void closeImpl ()=0
Must be implemented by derived class to release any resources.
Protected Attributes
ExecStreamScheduler * pScheduler
A Scheduler responsible for executing streams in this graph.
SharedDynamicParamManager pDynamicParamManager
Manager that handles dynamic parameters for this graph.
bool needsClose
Friends
class ExecStreamScheduler

Detailed Description

An ExecStreamGraph is a directed graph representing dataflow among ExecStreams.

For more information, see ExecStreamDesign.

A stream is always a node is a stream graph, but over its lifetime it may be moved from one graph to another. Specifically, it may be prepared in one graph (see ExecStreamGraphEmbro), executed in another, and finally closed & deleted in a third graph.

A stream has a permanent, unique name. These names are used later to find the streams. When a stream is added to a graph it is assigned an ExecStreamId. This identifier is later used to work with the stream. If the stream is moved to another graph, it obtains a new ExecStreamId.

Definition at line 59 of file ExecStreamGraph.h.

Constructor & Destructor Documentation

| ExecStreamGraph::ExecStreamGraph | ( | | ) | [explicit, protected] | | -------------------------------- | - | | - | ----------------------- |

| ExecStreamGraph::~ExecStreamGraph | ( | | ) | [virtual] | | ---------------------------------- | - | | - | ----------- |

Member Function Documentation

Sets the transaction within which this graph should execute.

The transaction is reset whenever the graph is closed.

Parameters:

Implemented in ExecStreamGraphImpl.

virtual void ExecStreamGraph::setErrorTarget ( SharedErrorTarget pErrorTarget ) [pure virtual]
virtual void ExecStreamGraph::setScratchSegment ( SharedSegment pScratchSegment ) [pure virtual]

Sets the ScratchSegment from which this graph's streams should allocate memory buffers.

Parameters:

pScratchSegment scratch segment

Implemented in ExecStreamGraphImpl.

Sets the global exec stream governor.

Parameters:

pResourceGovernor exec stream governor

Implemented in ExecStreamGraphImpl.

| virtual TxnId ExecStreamGraph::getTxnId | ( | | ) | [pure virtual] | | --------------------------------------- | - | | - | ---------------- |

virtual void ExecStreamGraph::enableDummyTxnId ( bool enabled ) [pure virtual]

Controls whether it is OK to call getTxnId without first calling setTxn.

Normally, this is a bad idea (since in that case getTxnId will return FIRST_TXN_ID as a dummy, which could lead to concurrency problems), but for non-transactional unit tests, this can be useful. Default is disabled.

Parameters:

enabled whether dummy txn ID's are enabled

Implemented in ExecStreamGraphImpl.

virtual void ExecStreamGraph::addStream ( SharedExecStream pStream ) [pure virtual]

| virtual void ExecStreamGraph::removeStream | ( | ExecStreamId | | ) | [pure virtual] | | ------------------------------------------ | - | -------------------------------------------------------------------------- | | - | ---------------- |

Removes a stream from the graph: deletes the edges, and puts the vertex on a free list to be reallocated.

Does not free the ExecStream or its ExecStreamBufAccessors.

Implemented in ExecStreamGraphImpl.

virtual void ExecStreamGraph::addDataflow ( ExecStreamId producerId,
ExecStreamId consumerId,
bool isImplicit = false
) [pure virtual]

Defines a dataflow relationship between two streams in this graph.

Parameters:

producerId ID of producer stream in this graph
consumerId ID of consumer stream in this graph
isImplicit false (the default) if the edge represents direct dataflow; true if the edge represents an implicit dataflow dependency

Implemented in ExecStreamGraphImpl.

virtual void ExecStreamGraph::addOutputDataflow ( ExecStreamId producerId ) [pure virtual]
virtual void ExecStreamGraph::addInputDataflow ( ExecStreamId consumerId ) [pure virtual]

Defines a dataflow representing external input consumed by this graph.

Parameters:

consumerId ID of consumer stream in this graph

Implemented in ExecStreamGraphImpl.

virtual void ExecStreamGraph::mergeFrom ( ExecStreamGraph & src ) [pure virtual]

Adds all the vertices and edges from another graph.

Assumes the graphs are disjoint, and that both have been prepared. The two graphs are both open, or else both closed.

Parameters:

src the other graph, which is left empty.

Implemented in ExecStreamGraphImpl.

Adds a subgraph, taken (removed) from another graph.

(Slower than mergeFrom(ExecStreamGraph&), which merges its entire source). Assumes the graphs are disjoint, and that both have been prepared. The two graphs are both open, or else both closed.

Parameters:

src the source graph
nodes identifies source nodes.

Implemented in ExecStreamGraphImpl.

virtual SharedExecStream ExecStreamGraph::findStream ( std::string name ) [pure virtual]

Finds a stream by name.

Parameters:

name name of stream to find

Returns:

stream found

Implemented in ExecStreamGraphImpl.

virtual SharedExecStream ExecStreamGraph::findLastStream ( std::string name,
uint iOutput
) [pure virtual]

Finds last stream known for name.

May be original stream or an adapter.

Parameters:

name name of stream to find
iOutput ordinal of output arc

Returns:

stream found

Implemented in ExecStreamGraphImpl.

virtual void ExecStreamGraph::interposeStream ( std::string name,
uint iOutput,
ExecStreamId interposedId
) [pure virtual]

Interposes an adapter stream.

In the process, creates a dataflow from last stream associated with name to the adapter stream.

Parameters:

name name of stream to adapt
iOutput ordinal of output of stream
interposedId ID of adapter stream within this graph

Implemented in ExecStreamGraphImpl.

Prepares this graph for execution.

Only called once (before first open) after all streams and dataflows have been defined.

Parameters:

Implemented in ExecStreamGraphImpl.

| virtual void ExecStreamGraph::open | ( | | ) | [pure virtual] | | ---------------------------------- | - | | - | ---------------- |

Opens execution on this graph.

A graph may be repeatedly closed and then reopened.

Implemented in ExecStreamGraphImpl.

virtual uint ExecStreamGraph::getInputCount ( ExecStreamId streamId ) [pure virtual]

Determines number of explicit input flows consumed by a stream.

Parameters:

Returns:

input count

Implemented in ExecStreamGraphImpl.

virtual uint ExecStreamGraph::getOutputCount ( ExecStreamId streamId ) [pure virtual]

Determines number of explicit output flows produced by a stream.

Parameters:

Returns:

output count

Implemented in ExecStreamGraphImpl.

Accesses a stream's output.

Parameters:

streamId ID of stream
iOutput 0-based output explicit flow ordinal

Returns:

downstream consumer

Implemented in ExecStreamGraphImpl.

Accesses a stream's output accessor.

Parameters:

streamId ID of stream
iOutput 0-based output explicit flow ordinal

Returns:

accessor used by downstream consumer

Implemented in ExecStreamGraphImpl.

| virtual std::vector<SharedExecStream> ExecStreamGraph::getSortedStreams | ( | | ) | [pure virtual] | | ---------------------------------------------------------------------------------------------------------------------------------- | - | | - | ---------------- |

| virtual int ExecStreamGraph::getStreamCount | ( | | ) | [pure virtual] | | ------------------------------------------- | - | | - | ---------------- |

Returns:

the number of streams in the graph; can only be called after prepare.

Implemented in ExecStreamGraphImpl.

| virtual int ExecStreamGraph::getDataflowCount | ( | | ) | [pure virtual] | | --------------------------------------------- | - | | - | ---------------- |

Returns:

the number of dataflows (edges) in the graph; can only be called after prepare.

Implemented in ExecStreamGraphImpl.

virtual void ExecStreamGraph::renderGraphviz ( std::ostream & dotStream ) [pure virtual]

Renders the graph in the .dot format defined by Graphviz.

Parameters:

dotStream ostream on which to write .dot representation

Implemented in ExecStreamGraphImpl.

| virtual bool ExecStreamGraph::isAcyclic | ( | | ) | [pure virtual] | | --------------------------------------- | - | | - | ---------------- |

virtual void ExecStreamGraph::closeProducers ( ExecStreamId streamId ) [pure virtual]

Closes the producers of a stream with a given id.

Parameters:

streamId stream id of the stream whose producers will be closed

Implemented in ExecStreamGraphImpl.

virtual void ExecStreamGraph::declareDynamicParamWriter ( ExecStreamId streamId,
DynamicParamId dynamicParamId
) [pure virtual]
virtual void ExecStreamGraph::declareDynamicParamReader ( ExecStreamId streamId,
DynamicParamId dynamicParamId
) [pure virtual]
virtual const std::vector<ExecStreamId>& ExecStreamGraph::getDynamicParamWriters ( DynamicParamId dynamicParamId ) [pure virtual]

Returns a list of stream ids that write a given dynamic parameter.

Parameters:

dynamicParamId Dynamic parameter id

Returns:

List of ids of streams that write the parameter

Implemented in ExecStreamGraphImpl.

virtual const std::vector<ExecStreamId>& ExecStreamGraph::getDynamicParamReaders ( DynamicParamId dynamicParamId ) [pure virtual]

| virtual void ClosableObject::closeImpl | ( | | ) | [protected, pure virtual, inherited] | | -------------------------------------- | - | | - | -------------------------------------- |

Must be implemented by derived class to release any resources.

Implemented in CacheImpl< PageT, VictimPolicyT >, ByteArrayInputStream, ByteArrayOutputStream, ByteOutputStream, CheckpointThread, Database, BarrierExecStream, DoubleBufferExecStream, ExecStream, ExecStreamGraphImpl, MockResourceExecStream, ReshapeExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReader, SegBufferReaderExecStream, SegBufferWriter, SegBufferWriterExecStream, JavaSinkExecStream, JavaTransformExecStream, FlatFileBuffer, FlatFileExecStreamImpl, BTreeExecStream, BTreeInsertExecStream, BTreePrefetchSearchExecStream, BTreeReadExecStream, BTreeSearchExecStream, FtrsTableWriterExecStream, LhxAggExecStream, LhxJoinExecStream, LbmBitOpExecStream, LbmChopperExecStream, LbmGeneratorExecStream, LbmIntersectExecStream, LbmMinusExecStream, LbmSplicerExecStream, LbmUnionExecStream, LcsClusterAppendExecStream, LcsRowScanBaseExecStream, LcsRowScanExecStream, DelegatingSegment, DynamicDelegatingSegment, ScratchSegment, SegInputStream, Segment, SegOutputStream, SegPageBackupRestoreDevice, SegStream, SegStreamAllocation, SpillOutputStream, and ExternalSortExecStreamImpl.

Referenced by ClosableObject::close().

| bool ClosableObject::isClosed | ( | | ) | const [inline, inherited] | | ----------------------------- | - | | - | --------------------------- |

Returns:

whether the object has been closed

Definition at line 58 of file ClosableObject.h.

| void ClosableObject::close | ( | | ) | [inherited] | | -------------------------- | - | | - | ------------- |

Closes this object, releasing any unallocated resources.

Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.

Definition at line 39 of file ClosableObject.cpp.

References ClosableObject::closeImpl(), and ClosableObject::needsClose.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), FlatFileBuffer::open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().

Member Data Documentation

A Scheduler responsible for executing streams in this graph.

(Can be null if the current graph is only building streams, not executing them.) Note that we don't use a weak_ptr for this because it needs to be accessed frequently during execution, and the extra locking overhead would be frivolous.

Definition at line 73 of file ExecStreamGraph.h.

Referenced by getScheduler().

The documentation for this class was generated from the following files:

Generated on Mon Jun 22 04:00:30 2009 for Fennel by doxygen 1.5.1