Fennel: LcsRowScanExecStreamTest Class Reference (original) (raw)

Testcase for scanning multiple clusters. More...

Inheritance diagram for LcsRowScanExecStreamTest:

List of all members.

Public Member Functions
LcsRowScanExecStreamTest ()
void testCaseSetUp ()
Equivalent to JUnit TestCase.setUp; this is called before each test case method is invoked.
void testCaseTearDown ()
Equivalent to JUnit TestCase.tearDown; this is called after each test case method is invoked.
void testScans ()
void testScanOnEmptyCluster ()
Create an empty cluster with 1 column.
void testScanPastEndOfCluster ()
Create a cluster with only a single row.
void testCompressedFiltering ()
void testBernoulliSampling ()
Configure Bernoulli sampling, with a specific seed and verify that the expected number of rows are returned.
void testSystemSampling ()
Configure system sampling, with a specific clump size and verify that the expected number of rows are returned.
virtual void openStorage (DeviceMode openMode)
virtual void openSegmentStorage (DeviceMode openMode)
virtual void openRandomSegment ()
SharedSegment createLinearDeviceSegment (DeviceId deviceId, uint nPages)
void closeLinearSegment ()
void closeRandomSegment ()
void closeVersionedRandomSegment ()
void closeSnapshotRandomSegment ()
virtual void closeStorage ()
Cache & getCache ()
virtual SharedCache newCache ()
SharedRandomAccessDevice openDevice (std::string devName, DeviceMode openMode, uint nDevicePages, DeviceId deviceId)
void closeDevice (DeviceId deviceId, SharedRandomAccessDevice &pDevice)
TestSuite * releaseTestSuite ()
void beforeTestCase (std::string testCaseName)
void afterTestCase (std::string testCaseName)
virtual void notifyTrace (std::string source, TraceLevel level, std::string message)
Receives notification when a trace event occurs.
virtual TraceLevel getSourceTraceLevel (std::string source)
Gets the level at which a particular source should be traced.
Static Public Member Functions
static void readParams (int argc, char **argv)
Parses the command line.
Static Public Attributes
static ParamName paramTestSuiteName
static ParamName paramTraceFileName
static ParamName paramDictionaryFileName
static ParamName paramTraceLevel
static ParamName paramStatsFileName
static ParamName paramTraceStdout
static ParamName paramDegreeOfParallelism
static ConfigMap configMap
Configuration parameters.
Protected Types
enum VictimPolicy { victimTwoQ, victimLRU, victimRandom }
The available victim policy implementations. More...
Protected Member Functions
void loadClusters (uint nRows, uint nCols, uint nClusters, bool compressed)
Serially loads nClusters, each cluster containing nCols and nRows.
void loadOneCluster (uint nRows, uint nCols, int colStart, BTreeDescriptor &bTreeDescriptor, bool compressed)
Loads a single cluster with nCols columns and nRows rows.
void testScanCols (uint nRows, uint nCols, uint nClusters, TupleProjection proj, uint skipRows, uint expectedNumRows)
Reads rows from clusters.
void testFilterCols (uint nRows, uint nCols, uint nClusters, TupleProjection proj, uint skipRows, uint expectedNumRows, bool compressed)
Filter rows from clusters.
void setSearchKey (char lowerDirective, char upperDirective, uint64_t lowerVal, uint64_t upperVal, PBuffer inputBuf, uint &offset, TupleAccessor &inputTupleAccessor, TupleData &inputTupleData)
void testSampleScanCols (uint nRows, uint nRowsActual, uint nCols, uint nClusters, TupleProjection proj, uint skipRows, TableSamplingMode mode, float rate, int seed, uint clumps, uint expectedNumRows)
Sample rows from clusters.
int generateBitmaps (uint nRows, uint skipRows, TupleDescriptor const &bitmapTupleDesc, PBuffer pBuf)
Generate bitmaps to pass as input into row scan exec stream.
void produceEntry (LbmEntry &lbmEntry, TupleAccessor &bitmapTupleAccessor, PBuffer pBuf, int &bufSize)
SharedExecStream prepareSourceGraph (ExecStreamEmbryo &sourceStreamEmbryo)
Defines and prepares a graph consisting of one source stream.
SharedExecStream prepareTransformGraph (ExecStreamEmbryo &sourceStreamEmbryo, ExecStreamEmbryo &transformStreamEmbryo)
Defines and prepares a graph consisting of one source stream and one transform stream.
SharedExecStream prepareTransformGraph (ExecStreamEmbryo &sourceStreamEmbryo, std::vector< ExecStreamEmbryo > &transforms)
Defines and prepares a graph consisting of one source stream and one or multiple transform streams.
SharedExecStream prepareConfluenceGraph (ExecStreamEmbryo &sourceStreamEmbryo1, ExecStreamEmbryo &sourceStreamEmbryo2, ExecStreamEmbryo &confluenceStreamEmbryo)
Defines and prepares a graph consisting of two source streams and one confluence stream.
SharedExecStream prepareConfluenceGraph (std::vector< ExecStreamEmbryo > &sourceStreamEmbryos, ExecStreamEmbryo &confluenceStreamEmbryo)
Defines and prepares a graph consisting of a list of source streams and one confluence stream.
SharedExecStream prepareConfluenceGraph (std::vector< std::vector< ExecStreamEmbryo > > &sourceStreamEmbryosList, ExecStreamEmbryo &confluenceStreamEmbryo)
Defines and prepares a graph consisting of one or more source streams and one confluence stream.
SharedExecStream prepareConfluenceTransformGraph (ExecStreamEmbryo &sourceStreamEmbryo1, ExecStreamEmbryo &sourceStreamEmbryo2, ExecStreamEmbryo &confluenceStreamEmbryo, ExecStreamEmbryo &transformStreamEmbryo)
Defines and prepares a graph consisting of two source streams, one confluence stream, and one transform stream.
SharedExecStream prepareDAG (ExecStreamEmbryo &srcStreamEmbryo, ExecStreamEmbryo &splitterStreamEmbryo, std::vector< ExecStreamEmbryo > &interStreamEmbryos, ExecStreamEmbryo &destStreamEmbryo, bool createSink=true, bool saveSrc=true)
Defines and prepares a graph consisting of a source, a splitter, and one or more parallel transform streams which flow together into a confluence stream.
SharedExecStream prepareDAG (ExecStreamEmbryo &srcStreamEmbryo, ExecStreamEmbryo &splitterStreamEmbryo, std::vector< std::vector< ExecStreamEmbryo > > &interStreamEmbryosList, ExecStreamEmbryo &destStreamEmbryo, bool createSink=true, bool saveSrc=true)
Defines and prepares a graph consisting of a source, a splitter, and one or more parallel transform streams which flow together into a confluence stream.
void verifyOutput (ExecStream &stream, uint nRowsExpected, MockProducerExecStreamGenerator &verifier, bool stopEarly=false)
Executes the prepared stream graph and verifies that its output matches that produced by a value generator.
void verifyConstantOutput (ExecStream &stream, const TupleData &expectedTuple, uint nRowsExpected)
Executes the prepared stream graph and verifies that all output tuples matche an expected and given one.
void verifyBufferedOutput (ExecStream &stream, TupleDescriptor outputTupleDesc, uint nRowsExpected, PBuffer expectedBuffer)
Executes the prepared stream graph and verifies the resultant tuples against a set of tuples supplied in an input buffer.
void resetExecStreamTest ()
Reset stream graph so multiple iterations of a method can be called within a single testcase.
virtual void tearDownExecStreamTest ()
ExecStream-specific handler called from testCaseTearDown.
virtual SharedExecStreamGraph newStreamGraph ()
Creates a stream graph.
virtual SharedExecStreamGraphEmbryo newStreamGraphEmbryo (SharedExecStreamGraph)
Creates an embryo for a stream graph.
virtual ExecStreamScheduler * newScheduler ()
Creates a scheduler.
virtual ExecStreamGovernor * newResourceGovernor (ExecStreamResourceKnobs const &knobSettings, ExecStreamResourceQuantity const &resourcesAvailable)
Creates the resource governor.
void snooze (uint nSeconds)
Protected Attributes
StandardTypeDescriptorFactory stdTypeFactory
TupleAttributeDescriptor attrDesc_int64
TupleAttributeDescriptor attrDesc_bitmap
TupleAttributeDescriptor attrDesc_char1
uint bitmapColSize
vector< boost::shared_ptr< BTreeDescriptor > > bTreeClusters
SharedExecStreamGraph pGraph
SharedExecStreamGraphEmbryo pGraphEmbryo
SharedExecStreamScheduler pScheduler
SharedExecStreamGovernor pResourceGovernor
SharedCacheAccessor pCacheAccessor
SharedSegmentFactory pSegmentFactory
SharedSegment pLinearSegment
Segment supporting linear page allocation.
SharedSegment pRandomSegment
(Optional) segment supporting random page allocation.
SharedSegment pVersionedRandomSegment
(Optional) segment supporting versioned random page allocation.
SharedSegment pSnapshotRandomSegment
(Optional) segment supporting snapshot random page allocation.
VictimPolicy victimPolicy
VictimPolicy to instantiate.
CacheParams cacheParams
Parameters for cache initialization.
SharedCache pCache
Cache instance being tested.
SharedRandomAccessDevice pRandomAccessDevice
The default cached device.
uint nMemPages
Size of cache in memory pages.
uint nDiskPages
Size of device in disk pages.
uint cbPageFull
Disk page size.
DeviceId dataDeviceId
Fixed ID to assign to data device.
TestSuite * pTestSuite
Boost test suite.
boost::shared_ptr< TestBase > pTestObj
std::ofstream traceStream
Output file stream for tracing.
StrictMutex traceMutex
Protects traceStream.
std::string testName
Name of test.
TraceLevel traceLevel
Level at which to trace test execution.
FileStatsTarget statsTarget
Output for stats.
StatsTimer statsTimer
Timer for stats collection.
bool traceStdout
Copy trace output to stdout.
bool traceFile
Copy trace output to file.
TestCaseGroup defaultTests
TestCaseGroup extraTests
Static Protected Attributes
static const uint NDUPS = 20
static const uint DefaultCacheReservePercent = 5
static const uint DefaultConcurrentStatements = 4
static bool runAll
Run all test cases, including the extra tests.
static std::string runSingle
Run only the test case of this name.

Detailed Description

Testcase for scanning multiple clusters.

Note that LcsClusterAppendExecStreamTest also has some tests for scans, but those only test single cluster scans

Definition at line 51 of file LcsRowScanExecStreamTest.cpp.


Member Enumeration Documentation

The available victim policy implementations.

Enumerator:

| victimTwoQ | | | -------------- | | | victimLRU | | | victimRandom | |

Definition at line 44 of file CacheTestBase.h.


Constructor & Destructor Documentation

| LcsRowScanExecStreamTest::LcsRowScanExecStreamTest | ( | | ) | [inline, explicit] | | -------------------------------------------------- | - | | - | -------------------- |


Member Function Documentation

void LcsRowScanExecStreamTest::loadClusters ( uint nRows,
uint nCols,
uint nClusters,
bool compressed
) [protected]
void LcsRowScanExecStreamTest::loadOneCluster ( uint nRows,
uint nCols,
int colStart,
BTreeDescriptor & bTreeDescriptor,
bool compressed
) [protected]

Loads a single cluster with nCols columns and nRows rows.

Each column will have a different sequence of values, as follows: column0 - colStart, colStart+1, ..., colStart+nRows-1 column1 - colStart+1, colStart+2, ..., colStart+nRows column2 - colStart+2, colStart+3, ..., colStart+nRows+1 ...

Definition at line 256 of file LcsRowScanExecStreamTest.cpp.

References attrDesc_int64, BTreeBuilder::createEmptyRoot(), BTreeAccessBase::getRootPageId(), ExecStreamEmbryo::getStream(), ExecStreamEmbryo::init(), LcsClusterAppendExecStreamParams::inputProj, BTreeParams::keyProj, BTreeDescriptor::keyProjection, NDUPS, MockProducerExecStreamParams::nRows, NULL_PAGE_ID, SingleOutputExecStreamParams::outputTupleDesc, BTreeDescriptor::pageOwnerId, BTreeParams::pageOwnerId, CacheTestBase::pCache, SegmentAccessor::pCacheAccessor, ExecStreamParams::pCacheAccessor, MockProducerExecStreamParams::pGenerator, SegStorageTestBase::pRandomSegment, ExecStreamUnitTestBase::prepareTransformGraph(), BTreeParams::pRootMap, SegmentAccessor::pSegment, BTreeParams::pSegment, SegStorageTestBase::pSegmentFactory, BTreeParams::rootPageId, BTreeDescriptor::rootPageId, BTreeParams::rootPageIdParamId, ExecStreamParams::scratchAccessor, BTreeDescriptor::segmentAccessor, BTreeDescriptor::segmentId, BTreeParams::segmentId, BTreeParams::tupleDesc, BTreeDescriptor::tupleDescriptor, and ExecStreamUnitTestBase::verifyOutput().

Referenced by loadClusters(), and testScanPastEndOfCluster().

Reads rows from clusters.

Assumes clusters have been loaded by loadClusters/loadOneCluster.

Parameters:

nRows total number of rows in the clusters
nCols number of columns in each cluster
nClusters number of clusters
proj columns to be projected
skipRows read every "skipRows" rows
expectedNumRows expected number of rows in scan result; normally should be the same as nRows unless skipping rows or testing exception cases

Definition at line 343 of file LcsRowScanExecStreamTest.cpp.

References attrDesc_bitmap, attrDesc_int64, bitmapColSize, bTreeClusters, ValuesExecStreamParams::bufSize, FixedBuffer, generateBitmaps(), ExecStreamEmbryo::getStream(), LcsRowScanExecStreamParams::hasExtraFilter, ExecStreamEmbryo::init(), LcsRowScanExecStreamParams::isFullScan, LcsRowScanBaseExecStreamParams::lcsClusterScanDefs, max(), LcsRowScanBaseExecStreamParams::outputProj, SingleOutputExecStreamParams::outputTupleDesc, ExecStreamUnitTestBase::prepareTransformGraph(), ValuesExecStreamParams::pTupleBuffer, SAMPLING_OFF, LcsRowScanExecStreamParams::samplingMode, and ExecStreamUnitTestBase::verifyOutput().

Referenced by testScanOnEmptyCluster(), testScanPastEndOfCluster(), and testScans().

00350 { 00351
00352 00353 ValuesExecStreamParams valuesParams; 00354 boost::shared_array pBuffer; 00355 ExecStreamEmbryo valuesStreamEmbryo; 00356 LcsRowScanExecStreamParams scanParams; 00357 00358 scanParams.hasExtraFilter = false; 00359 scanParams.samplingMode = SAMPLING_OFF; 00360 00361
00362
00363
00364 valuesParams.outputTupleDesc.push_back(attrDesc_int64); 00365 valuesParams.outputTupleDesc.push_back(attrDesc_bitmap); 00366 valuesParams.outputTupleDesc.push_back(attrDesc_bitmap); 00367 00368
00369
00370
00371 uint bufferSize = std::max( 00372 16, (int) (nRows / 8 + nRows / bitmapColSize * 8)); 00373 pBuffer.reset(new FixedBuffer[bufferSize]); 00374 valuesParams.pTupleBuffer = pBuffer; 00375 00376 if (nRows > 0) { 00377 valuesParams.bufSize = generateBitmaps( 00378 nRows, skipRows, valuesParams.outputTupleDesc, pBuffer.get()); 00379 assert(valuesParams.bufSize <= bufferSize); 00380 scanParams.isFullScan = false; 00381 } else { 00382 scanParams.isFullScan = true; 00383 valuesParams.bufSize = 0; 00384 } 00385 valuesStreamEmbryo.init(new ValuesExecStream(), valuesParams); 00386 valuesStreamEmbryo.getStream()->setName("ValuesExecStream"); 00387 00388
00389
00390 00391 for (uint i = 0; i < nClusters; i++) { 00392 struct LcsClusterScanDef clusterScanDef; 00393 00394 for (uint j = 0; j < nCols; j++) { 00395 clusterScanDef.clusterTupleDesc.push_back(attrDesc_int64); 00396 } 00397 00398 clusterScanDef.pSegment = bTreeClusters[i]->segmentAccessor.pSegment; 00399 clusterScanDef.pCacheAccessor = 00400 bTreeClusters[i]->segmentAccessor.pCacheAccessor; 00401 clusterScanDef.tupleDesc = bTreeClusters[i]->tupleDescriptor; 00402 clusterScanDef.keyProj = bTreeClusters[i]->keyProjection; 00403 clusterScanDef.rootPageId = bTreeClusters[i]->rootPageId; 00404 clusterScanDef.segmentId = bTreeClusters[i]->segmentId; 00405 clusterScanDef.pageOwnerId = bTreeClusters[i]->pageOwnerId; 00406 00407 scanParams.lcsClusterScanDefs.push_back(clusterScanDef); 00408 } 00409 00410
00411 scanParams.outputProj = proj; 00412 for (uint i = 0; i < proj.size(); i++) { 00413 scanParams.outputTupleDesc.push_back(attrDesc_int64); 00414 } 00415 00416 ExecStreamEmbryo scanStreamEmbryo; 00417 scanStreamEmbryo.init(new LcsRowScanExecStream(), scanParams); 00418 scanStreamEmbryo.getStream()->setName("RowScanExecStream"); 00419 SharedExecStream pOutputStream; 00420 00421 pOutputStream = 00422 prepareTransformGraph(valuesStreamEmbryo, scanStreamEmbryo); 00423 00424
00425 00426 vector<boost::shared_ptr<ColumnGenerator > > columnGenerators; 00427 for (uint i = 0; i < proj.size(); i++) { 00428 SharedInt64ColumnGenerator col = 00429 SharedInt64ColumnGenerator( 00430 new SeqColumnGenerator( 00431 proj[i], 00432 skipRows)); 00433 columnGenerators.push_back(col); 00434 } 00435 00436 CompositeExecStreamGenerator resultGenerator(columnGenerators); 00437 verifyOutput(*pOutputStream, expectedNumRows, resultGenerator); 00438 }

void LcsRowScanExecStreamTest::testFilterCols ( uint nRows,
uint nCols,
uint nClusters,
TupleProjection proj,
uint skipRows,
uint expectedNumRows,
bool compressed
) [protected]

Filter rows from clusters.

Assumes clusters have been loaded by loadClusters/loadOneCluster.

Parameters:

nRows total number of rows in the clusters
nCols number of columns in each cluster
nClusters number of clusters
proj columns to be projected
skipRows read every "skipRows" rows
expectedNumRows expected number of rows in scan result; normally should be the same as nRows unless skipping rows or testing exception cases
compressed testing compressed bitmap optimization

Definition at line 772 of file LcsRowScanExecStreamTest.cpp.

References attrDesc_bitmap, attrDesc_char1, attrDesc_int64, bitmapColSize, bTreeClusters, ValuesExecStreamParams::bufSize, TupleAccessor::compute(), FixedBuffer, generateBitmaps(), TupleAccessor::getMaxByteCount(), ExecStreamEmbryo::getStream(), LcsRowScanExecStreamParams::hasExtraFilter, ExecStreamEmbryo::init(), LcsRowScanExecStreamParams::isFullScan, LcsRowScanBaseExecStreamParams::lcsClusterScanDefs, max(), NDUPS, StandardTypeDescriptorFactory::newDataType(), LcsRowScanBaseExecStreamParams::outputProj, SingleOutputExecStreamParams::outputTupleDesc, ExecStreamUnitTestBase::prepareConfluenceGraph(), ValuesExecStreamParams::pTupleBuffer, LcsRowScanExecStreamParams::residualFilterCols, SAMPLING_OFF, LcsRowScanExecStreamParams::samplingMode, setSearchKey(), STANDARD_TYPE_INT_64, stdTypeFactory, and ExecStreamUnitTestBase::verifyOutput().

Referenced by testCompressedFiltering(), and testScans().

00780 { 00781
00782 00783 ValuesExecStreamParams valuesParams; 00784 boost::shared_array pBuffer; 00785 ExecStreamEmbryo valuesStreamEmbryo; 00786 LcsRowScanExecStreamParams scanParams; 00787 00788 scanParams.hasExtraFilter = true; 00789 scanParams.samplingMode = SAMPLING_OFF; 00790 00791
00792
00793
00794 valuesParams.outputTupleDesc.push_back(attrDesc_int64); 00795 valuesParams.outputTupleDesc.push_back(attrDesc_bitmap); 00796 valuesParams.outputTupleDesc.push_back(attrDesc_bitmap); 00797 00798 00799
00800
00801
00802 uint bufferSize = std::max( 00803 16, (int) (nRows / 8 + nRows / bitmapColSize * 8)); 00804 pBuffer.reset(new FixedBuffer[bufferSize]); 00805 valuesParams.pTupleBuffer = pBuffer; 00806 00807 if (nRows > 0) { 00808 valuesParams.bufSize = generateBitmaps( 00809 nRows, skipRows, valuesParams.outputTupleDesc, pBuffer.get()); 00810 assert(valuesParams.bufSize <= bufferSize); 00811 scanParams.isFullScan = false; 00812 } else { 00813 scanParams.isFullScan = true; 00814 valuesParams.bufSize = 0; 00815 } 00816 valuesStreamEmbryo.init(new ValuesExecStream(), valuesParams); 00817 valuesStreamEmbryo.getStream()->setName("ValuesExecStream"); 00818 00819
00820
00821
00822
00823
00824
00825
00826
00827 00828 TupleAttributeDescriptor attrDesc_nullableInt64 = 00829 TupleAttributeDescriptor( 00830 stdTypeFactory.newDataType(STANDARD_TYPE_INT_64), 00831 true, sizeof(uint64_t)); 00832 00833 valuesParams.outputTupleDesc.resize(0); 00834 TupleDescriptor inputTupleDesc; 00835 for (uint i = 0; i < 2; i++) { 00836 inputTupleDesc.push_back(attrDesc_char1); 00837 inputTupleDesc.push_back(attrDesc_nullableInt64); 00838 valuesParams.outputTupleDesc.push_back(attrDesc_char1); 00839 valuesParams.outputTupleDesc.push_back(attrDesc_nullableInt64); 00840 } 00841 TupleData inputTupleData(inputTupleDesc); 00842 TupleAccessor inputTupleAccessor; 00843 inputTupleAccessor.compute(inputTupleDesc); 00844 00845 uint nInputTuples = 3; 00846 boost::shared_array inputBuffer; 00847 inputBuffer.reset( 00848 new FixedBuffer[nInputTuples * inputTupleAccessor.getMaxByteCount()]); 00849 00850 PBuffer inputBuf = inputBuffer.get(); 00851 uint offset = 0; 00852 00853 setSearchKey( 00854 '-', ')', 0, 1000, inputBuf, offset, inputTupleAccessor, 00855 inputTupleData); 00856 setSearchKey( 00857 '[', '+', 2000, 0, inputBuf, offset, inputTupleAccessor, 00858 inputTupleData); 00859 00860 TupleData inputTupleData1(inputTupleDesc); 00861 boost::shared_array inputBuffer1; 00862 inputBuffer1.reset( 00863 new FixedBuffer[nInputTuples * inputTupleAccessor.getMaxByteCount()]); 00864 PBuffer inputBuf1 = inputBuffer1.get(); 00865 uint offset1 = 0; 00866 00867 setSearchKey( 00868 '[', ')', 500 + nCols, 2999 + nCols, inputBuf1, offset1, 00869 inputTupleAccessor, 00870 inputTupleData1); 00871 setSearchKey( 00872 '[', ']', 2999 + nCols, 2999 + nCols, inputBuf1, offset1, 00873 inputTupleAccessor, inputTupleData1); 00874 00875 TupleData inputTupleData2(inputTupleDesc); 00876 boost::shared_array inputBuffer2; 00877 inputBuffer2.reset( 00878 new FixedBuffer[nInputTuples * inputTupleAccessor.getMaxByteCount()]); 00879 PBuffer inputBuf2 = inputBuffer2.get(); 00880 uint offset2 = 0; 00881 00882 setSearchKey( 00883 '(', '+', 1500+2nCols, 0, inputBuf2, offset2, inputTupleAccessor, 00884 inputTupleData1); 00885 00886 valuesParams.pTupleBuffer = inputBuffer; 00887 valuesParams.bufSize = offset; 00888 00889 ExecStreamEmbryo valuesStreamEmbryo1, valuesStreamEmbryo2, 00890 valuesStreamEmbryo3; 00891 valuesStreamEmbryo1.init(new ValuesExecStream(), valuesParams); 00892 valuesStreamEmbryo1.getStream()->setName("ValuesExecStream1"); 00893 00894 valuesParams.pTupleBuffer = inputBuffer1; 00895 valuesParams.bufSize = offset1; 00896 valuesStreamEmbryo2.init(new ValuesExecStream(), valuesParams); 00897 valuesStreamEmbryo2.getStream()->setName("ValuesExecStream2"); 00898 00899 valuesParams.pTupleBuffer = inputBuffer2; 00900 valuesParams.bufSize = offset2; 00901 valuesStreamEmbryo3.init(new ValuesExecStream(), valuesParams); 00902 valuesStreamEmbryo3.getStream()->setName("ValuesExecStream3"); 00903 00904
00905
00906 00907 for (uint i = 0; i < nClusters; i++) { 00908 struct LcsClusterScanDef clusterScanDef; 00909 00910 for (uint j = 0; j < nCols; j++) { 00911 clusterScanDef.clusterTupleDesc.push_back(attrDesc_int64); 00912 } 00913 00914 clusterScanDef.pSegment = bTreeClusters[i]->segmentAccessor.pSegment; 00915 clusterScanDef.pCacheAccessor = 00916 bTreeClusters[i]->segmentAccessor.pCacheAccessor; 00917 clusterScanDef.tupleDesc = bTreeClusters[i]->tupleDescriptor; 00918 clusterScanDef.keyProj = bTreeClusters[i]->keyProjection; 00919 clusterScanDef.rootPageId = bTreeClusters[i]->rootPageId; 00920 clusterScanDef.segmentId = bTreeClusters[i]->segmentId; 00921 clusterScanDef.pageOwnerId = bTreeClusters[i]->pageOwnerId; 00922 00923 scanParams.lcsClusterScanDefs.push_back(clusterScanDef); 00924 } 00925 00926
00927 scanParams.outputProj = proj; 00928 for (uint i = 0; i < proj.size(); i++) { 00929 scanParams.outputTupleDesc.push_back(attrDesc_int64); 00930 } 00931 scanParams.residualFilterCols.push_back(0); 00932 scanParams.residualFilterCols.push_back(nCols); 00933 scanParams.residualFilterCols.push_back(2*nCols); 00934 00935 ExecStreamEmbryo scanStreamEmbryo; 00936 scanStreamEmbryo.init(new LcsRowScanExecStream(), scanParams); 00937 scanStreamEmbryo.getStream()->setName("RowScanExecStream"); 00938 SharedExecStream pOutputStream; 00939 00940 std::vector sources; 00941 sources.push_back(valuesStreamEmbryo); 00942 sources.push_back(valuesStreamEmbryo1); 00943 sources.push_back(valuesStreamEmbryo2); 00944 sources.push_back(valuesStreamEmbryo3); 00945 00946 pOutputStream = 00947 prepareConfluenceGraph(sources, scanStreamEmbryo); 00948 00949
00950 00951 vector<boost::shared_ptr<ColumnGenerator > > columnGenerators; 00952 offset = (int) ceil(2000.0 / skipRows) * skipRows; 00953 for (uint i = 0; i < proj.size(); i++) { 00954 SharedInt64ColumnGenerator col = 00955 SharedInt64ColumnGenerator( 00956 compressed ? 00957 (Int64ColumnGenerator
) new MixedDupColumnGenerator( 00958 NDUPS, proj[i] + 2000,500) : 00959 new SeqColumnGenerator(proj[i] + offset, skipRows)); 00960 columnGenerators.push_back(col); 00961 } 00962 00963 00964 CompositeExecStreamGenerator resultGenerator(columnGenerators); 00965 verifyOutput(*pOutputStream, expectedNumRows, resultGenerator); 00966 }

void LcsRowScanExecStreamTest::testSampleScanCols ( uint nRows,
uint nRowsActual,
uint nCols,
uint nClusters,
TupleProjection proj,
uint skipRows,
TableSamplingMode mode,
float rate,
int seed,
uint clumps,
uint expectedNumRows
) [protected]

Sample rows from clusters.

Assumes clusters have been loaded by loadClusters/loadOneCluster.

Parameters:

nRows total number of rows in the clusters; if set to 0, full table scans should be done
nRowsActual actual total number of rows in the clusters
nCols number of columns in each cluster
nClusters number of clusters
proj columns to be projected
skipRows read every "skipRows" rows
mode sampling mode (SAMPLING_BERNOULLI or SAMPLING_SYSTEM)
rate sampling rate
seed Bernoulli sampling RNG seed
clumps number of system sampling clumps
expectedNumRows expected number of rows in scan result; normally should be the same as nRows unless skipping rows or testing exception cases

Definition at line 969 of file LcsRowScanExecStreamTest.cpp.

References attrDesc_bitmap, attrDesc_int64, bitmapColSize, bTreeClusters, ValuesExecStreamParams::bufSize, FixedBuffer, generateBitmaps(), ExecStreamEmbryo::getStream(), LcsRowScanExecStreamParams::hasExtraFilter, ExecStreamEmbryo::init(), LcsRowScanExecStreamParams::isFullScan, LcsRowScanBaseExecStreamParams::lcsClusterScanDefs, max(), LcsRowScanBaseExecStreamParams::outputProj, SingleOutputExecStreamParams::outputTupleDesc, ExecStreamUnitTestBase::prepareTransformGraph(), ValuesExecStreamParams::pTupleBuffer, SAMPLING_BERNOULLI, SAMPLING_SYSTEM, LcsRowScanExecStreamParams::samplingClumps, LcsRowScanExecStreamParams::samplingIsRepeatable, LcsRowScanExecStreamParams::samplingMode, LcsRowScanExecStreamParams::samplingRate, LcsRowScanExecStreamParams::samplingRepeatableSeed, LcsRowScanExecStreamParams::samplingRowCount, and ExecStreamUnitTestBase::verifyOutput().

Referenced by testBernoulliSampling(), and testSystemSampling().

00981 { 00982
00983 00984 ValuesExecStreamParams valuesParams; 00985 boost::shared_array pBuffer; 00986 ExecStreamEmbryo valuesStreamEmbryo; 00987 LcsRowScanExecStreamParams scanParams; 00988 00989 scanParams.hasExtraFilter = false; 00990 00991
00992
00993
00994 valuesParams.outputTupleDesc.push_back(attrDesc_int64); 00995 valuesParams.outputTupleDesc.push_back(attrDesc_bitmap); 00996 valuesParams.outputTupleDesc.push_back(attrDesc_bitmap); 00997 00998 uint nRowsInternal = (mode == SAMPLING_SYSTEM) ? 0 : nRows; 00999 01000
01001
01002
01003
01004 uint bufferSize = std::max( 01005 16, (int) (nRowsInternal / 8 + nRowsInternal / bitmapColSize * 8)); 01006 pBuffer.reset(new FixedBuffer[bufferSize]); 01007 valuesParams.pTupleBuffer = pBuffer; 01008 01009 if (nRowsInternal > 0) { 01010 valuesParams.bufSize = generateBitmaps( 01011 nRowsInternal, skipRows, valuesParams.outputTupleDesc, 01012 pBuffer.get()); 01013 assert(valuesParams.bufSize <= bufferSize); 01014 scanParams.isFullScan = false; 01015 } else { 01016 scanParams.isFullScan = true; 01017 valuesParams.bufSize = 0; 01018 } 01019 valuesStreamEmbryo.init(new ValuesExecStream(), valuesParams); 01020 valuesStreamEmbryo.getStream()->setName("ValuesExecStream"); 01021 01022
01023
01024 01025 for (uint i = 0; i < nClusters; i++) { 01026 struct LcsClusterScanDef clusterScanDef; 01027 01028 for (uint j = 0; j < nCols; j++) { 01029 clusterScanDef.clusterTupleDesc.push_back(attrDesc_int64); 01030 } 01031 01032 clusterScanDef.pSegment = bTreeClusters[i]->segmentAccessor.pSegment; 01033 clusterScanDef.pCacheAccessor = 01034 bTreeClusters[i]->segmentAccessor.pCacheAccessor; 01035 clusterScanDef.tupleDesc = bTreeClusters[i]->tupleDescriptor; 01036 clusterScanDef.keyProj = bTreeClusters[i]->keyProjection; 01037 clusterScanDef.rootPageId = bTreeClusters[i]->rootPageId; 01038 clusterScanDef.segmentId = bTreeClusters[i]->segmentId; 01039 clusterScanDef.pageOwnerId = bTreeClusters[i]->pageOwnerId; 01040 01041 scanParams.lcsClusterScanDefs.push_back(clusterScanDef); 01042 } 01043 01044
01045 scanParams.outputProj = proj; 01046 for (uint i = 0; i < proj.size(); i++) { 01047 scanParams.outputTupleDesc.push_back(attrDesc_int64); 01048 } 01049 01050 01051
01052 scanParams.samplingMode = mode; 01053 scanParams.samplingRate = rate; 01054 scanParams.samplingIsRepeatable = true; 01055 scanParams.samplingRepeatableSeed = seed; 01056 scanParams.samplingClumps = clumps; 01057 scanParams.samplingRowCount = nRowsActual; 01058 01059 ExecStreamEmbryo scanStreamEmbryo; 01060 scanStreamEmbryo.init(new LcsRowScanExecStream(), scanParams); 01061 scanStreamEmbryo.getStream()->setName("RowScanExecStream"); 01062 SharedExecStream pOutputStream; 01063 01064 pOutputStream = 01065 prepareTransformGraph(valuesStreamEmbryo, scanStreamEmbryo); 01066 01067
01068 01069 vector<boost::shared_ptr<ColumnGenerator > > columnGenerators; 01070 for (uint i = 0; i < proj.size(); i++) { 01071 SharedInt64ColumnGenerator col = 01072 SharedInt64ColumnGenerator( 01073 new SeqColumnGenerator( 01074 proj[i], 01075 skipRows)); 01076 columnGenerators.push_back(col); 01077 } 01078 01079 boost::shared_ptr baseResultGenerator( 01080 new CompositeExecStreamGenerator(columnGenerators)); 01081 01082 if (mode == SAMPLING_BERNOULLI) { 01083 BernoulliSamplingExecStreamGenerator resultGenerator( 01084 baseResultGenerator, 01085 rate, 01086 seed, 01087 proj.size()); 01088 01089 verifyOutput(*pOutputStream, expectedNumRows, resultGenerator); 01090 } else { 01091 SystemSamplingExecStreamGenerator resultGenerator( 01092 baseResultGenerator, 01093 rate, 01094 nRows, 01095 proj.size(), 01096 clumps); 01097 01098 verifyOutput(*pOutputStream, expectedNumRows, resultGenerator); 01099 } 01100 }

Generate bitmaps to pass as input into row scan exec stream.

Parameters:

nRows number of rows in table
skipRows generate rids every "skipRows" rows; i.e., if skipRows == 1, there are no gaps in the rids
bitmapTupleDesc tuple descriptor for bitmap segment
pBuf buffer where bitmap segment tuples will be marshalled

Returns:

size of the buffer containing the marshalled tuples

Definition at line 440 of file LcsRowScanExecStreamTest.cpp.

References bitmapColSize, TupleAccessor::compute(), FixedBuffer, LbmEntry::getScratchBufferSize(), LbmEntry::init(), produceEntry(), LbmEntry::setEntryTuple(), and LbmEntry::setRID().

Referenced by testFilterCols(), testSampleScanCols(), and testScanCols().

00443 { 00444 int bufSize = 0; 00445 LbmEntry lbmEntry; 00446 boost::scoped_array entryBuf; 00447 TupleAccessor bitmapTupleAccessor; 00448 LcsRid rid = LcsRid(0); 00449 00450 TupleData bitmapTupleData(bitmapTupleDesc); 00451 bitmapTupleData[0].pData = (PConstBuffer) &rid; 00452 bitmapTupleData[1].pData = NULL; 00453 bitmapTupleData[1].cbData = 0; 00454 bitmapTupleData[2].pData = NULL; 00455 bitmapTupleData[2].cbData = 0; 00456 00457 bitmapTupleAccessor.compute(bitmapTupleDesc); 00458 00459
00460 uint scratchBufSize = LbmEntry::getScratchBufferSize(bitmapColSize); 00461 entryBuf.reset(new FixedBuffer[scratchBufSize]); 00462 lbmEntry.init(entryBuf.get(), NULL, scratchBufSize, bitmapTupleDesc); 00463 lbmEntry.setEntryTuple(bitmapTupleData); 00464 00465
00466 for (rid = LcsRid(skipRows); rid < LcsRid(nRows); rid += skipRows) { 00467 if (!lbmEntry.setRID(LcsRid(rid))) { 00468
00469
00470 produceEntry(lbmEntry, bitmapTupleAccessor, pBuf, bufSize); 00471 lbmEntry.setEntryTuple(bitmapTupleData); 00472 } 00473 } 00474
00475 produceEntry(lbmEntry, bitmapTupleAccessor, pBuf, bufSize); 00476 00477 return bufSize; 00478 }

void LcsRowScanExecStreamTest::produceEntry ( LbmEntry & lbmEntry,
TupleAccessor & bitmapTupleAccessor,
PBuffer pBuf,
int & bufSize
) [protected]

| void LcsRowScanExecStreamTest::testCaseSetUp | ( | | ) | [virtual] | | -------------------------------------------- | - | | - | ----------- |

Equivalent to JUnit TestCase.setUp; this is called before each test case method is invoked.

Default is no-op.

Reimplemented from ExecStreamUnitTestBase.

Definition at line 1102 of file LcsRowScanExecStreamTest.cpp.

References attrDesc_bitmap, attrDesc_char1, attrDesc_int64, bitmapColSize, StandardTypeDescriptorFactory::newDataType(), SegStorageTestBase::pRandomSegment, STANDARD_TYPE_CHAR, STANDARD_TYPE_INT_64, STANDARD_TYPE_VARBINARY, stdTypeFactory, and ExecStreamUnitTestBase::testCaseSetUp().

| void LcsRowScanExecStreamTest::testCaseTearDown | ( | | ) | [virtual] | | ----------------------------------------------- | - | | - | ----------- |

| void LcsRowScanExecStreamTest::testScans | ( | | ) | | ---------------------------------------- | - | | - |

Definition at line 489 of file LcsRowScanExecStreamTest.cpp.

References loadClusters(), ExecStreamUnitTestBase::resetExecStreamTest(), testFilterCols(), and testScanCols().

Referenced by LcsRowScanExecStreamTest().

00490 { 00491
00492
00493
00494
00495
00496 00497 uint nRows = 50000; 00498 uint nCols = 12; 00499 uint nClusters = 3; 00500 TupleProjection proj; 00501 00502 loadClusters(nRows, nCols, nClusters, false); 00503
00504
00505 00506
00507 for (uint i = 0; i < nClusters; i++) { 00508 for (uint j = 0; j < nCols; j++) { 00509 proj.push_back(i * nCols + j); 00510 } 00511 } 00512 testScanCols(nRows, nCols, nClusters, proj, 1, nRows); 00513 resetExecStreamTest(); 00514 00515
00516 proj.clear(); 00517 proj.push_back(22); 00518 proj.push_back(10); 00519 proj.push_back(12); 00520 proj.push_back(26); 00521 proj.push_back(1); 00522 proj.push_back(35); 00523 proj.push_back(15); 00524 proj.push_back(5); 00525 proj.push_back(17); 00526 proj.push_back(30); 00527 proj.push_back(4); 00528 proj.push_back(20); 00529 proj.push_back(7); 00530 proj.push_back(13); 00531 00532 testScanCols(nRows, nCols, nClusters, proj, 1, nRows); 00533 resetExecStreamTest(); 00534 00535
00536 testScanCols( 00537 nRows, nCols, nClusters, proj, 7, (int) ceil((double) nRows / 7)); 00538 resetExecStreamTest(); 00539 00540 00541
00542 testScanCols( 00543 nRows, nCols, nClusters, proj, 37, (int) ceil((double) nRows / 37)); 00544 resetExecStreamTest(); 00545 00546
00547 testScanCols(0, nCols, nClusters, proj, 1, nRows); 00548 00549 resetExecStreamTest(); 00550 00551
00552 for (uint i = 0; i < nClusters; i++) { 00553 for (uint j = 0; j < nCols; j++) { 00554 proj.push_back(i * nCols + j); 00555 } 00556 } 00557 testFilterCols(nRows, nCols, nClusters, proj, 1, 1000, false); 00558 00559 resetExecStreamTest(); 00560 00561
00562 proj.resize(0); 00563 for (uint i = 0; i < nClusters; i++) { 00564 for (uint j = 0; j < nCols; j++) { 00565 if (!(i < 2 && (j == 0 || j == 1))) { 00566 proj.push_back(i * nCols + j); 00567 } 00568 } 00569 } 00570 testFilterCols(nRows, nCols, nClusters, proj, 1, 1000, false); 00571 00572 resetExecStreamTest(); 00573 00574
00575 proj.resize(0); 00576 for (uint i = 0; i < nClusters - 1; i++) { 00577 for (uint j = 0; j < nCols; j++) { 00578 proj.push_back(i * nCols + j); 00579 } 00580 } 00581 testFilterCols( 00582 nRows, nCols, nClusters, proj, 7, 1000 / 7 + 1, false); 00583 }

| void LcsRowScanExecStreamTest::testScanOnEmptyCluster | ( | | ) | | ----------------------------------------------------- | - | | - |

Create an empty cluster with 1 column.

Try reading a rid from it

Definition at line 640 of file LcsRowScanExecStreamTest.cpp.

References attrDesc_int64, bTreeClusters, BTreeBuilder::createEmptyRoot(), BTreeAccessBase::getRootPageId(), BTreeDescriptor::keyProjection, NULL_PAGE_ID, CacheTestBase::pCache, SegmentAccessor::pCacheAccessor, SegStorageTestBase::pRandomSegment, SegmentAccessor::pSegment, BTreeDescriptor::rootPageId, BTreeDescriptor::segmentAccessor, testScanCols(), and BTreeDescriptor::tupleDescriptor.

Referenced by LcsRowScanExecStreamTest().

| void LcsRowScanExecStreamTest::testScanPastEndOfCluster | ( | | ) | | ------------------------------------------------------- | - | | - |

| void LcsRowScanExecStreamTest::testCompressedFiltering | ( | | ) | | ------------------------------------------------------ | - | | - |

Definition at line 585 of file LcsRowScanExecStreamTest.cpp.

References loadClusters(), NDUPS, ExecStreamUnitTestBase::resetExecStreamTest(), and testFilterCols().

Referenced by LcsRowScanExecStreamTest().

00586 { 00587
00588
00589
00590
00591
00592 00593 uint nRows = 50000; 00594 uint nCols = 12; 00595 uint nClusters = 3; 00596 TupleProjection proj; 00597 00598
00599
00600 loadClusters(nRows, nCols, nClusters, true); 00601 00602
00603 proj.resize(0); 00604 for (uint i = 0; i < nClusters; i++) { 00605 for (uint j = 0; j < nCols; j++) { 00606 proj.push_back(i * nCols + j); 00607 } 00608 } 00609 testFilterCols(nRows, nCols, nClusters, proj, 1, 500*NDUPS+500, true); 00610 00611 resetExecStreamTest(); 00612 00613
00614 proj.resize(0); 00615 for (uint i = 0; i < nClusters; i++) { 00616 for (uint j = 0; j < nCols; j++) { 00617 if (!(i < 2 && (j == 0 || j == 1))) { 00618 proj.push_back(i * nCols + j); 00619 } 00620 } 00621 } 00622 testFilterCols(nRows, nCols, nClusters, proj, 1, 500*NDUPS+500, true); 00623 00624 resetExecStreamTest(); 00625 00626
00627 proj.resize(0); 00628 for (uint i = 0; i < nClusters - 1; i++) { 00629 for (uint j = 0; j < nCols; j++) { 00630 proj.push_back(i * nCols + j); 00631 } 00632 } 00633 testFilterCols(nRows, nCols, nClusters, proj, 1, 500*NDUPS+500, true); 00634 }

| void LcsRowScanExecStreamTest::testBernoulliSampling | ( | | ) | | ---------------------------------------------------- | - | | - |

Configure Bernoulli sampling, with a specific seed and verify that the expected number of rows are returned.

Definition at line 688 of file LcsRowScanExecStreamTest.cpp.

References loadClusters(), ExecStreamUnitTestBase::resetExecStreamTest(), SAMPLING_BERNOULLI, and testSampleScanCols().

Referenced by LcsRowScanExecStreamTest().

00689 { 00690 uint nRows = 50000; 00691 uint nCols = 12; 00692 uint nClusters = 3; 00693 TupleProjection proj; 00694 00695 int seed = 19721212; 00696 float rate = 0.1; 00697 TableSamplingMode mode = SAMPLING_BERNOULLI; 00698 00699 loadClusters(nRows, nCols, nClusters, false); 00700
00701
00702 00703
00704 for (uint i = 0; i < nClusters; i++) { 00705 for (uint j = 0; j < nCols; j++) { 00706 proj.push_back(i * nCols + j); 00707 } 00708 } 00709 00710
00711 testSampleScanCols( 00712 0, nRows, nCols, nClusters, proj, 1, mode, rate, seed, 0, 4938); 00713 resetExecStreamTest(); 00714 00715
00716 testSampleScanCols( 00717 nRows, nRows, nCols, nClusters, proj, 2, mode, rate, seed, 0, 2489); 00718 resetExecStreamTest(); 00719 }

| void LcsRowScanExecStreamTest::testSystemSampling | ( | | ) | | ------------------------------------------------- | - | | - |

Configure system sampling, with a specific clump size and verify that the expected number of rows are returned.

Definition at line 726 of file LcsRowScanExecStreamTest.cpp.

References loadClusters(), ExecStreamUnitTestBase::resetExecStreamTest(), SAMPLING_SYSTEM, and testSampleScanCols().

Referenced by LcsRowScanExecStreamTest().

00727 { 00728 uint nRows = 50000; 00729 uint nCols = 12; 00730 uint nClusters = 3; 00731 TupleProjection proj; 00732 00733 TableSamplingMode mode = SAMPLING_SYSTEM; 00734 00735 loadClusters(nRows, nCols, nClusters, false); 00736
00737
00738 00739
00740 for (uint i = 0; i < nClusters; i++) { 00741 for (uint j = 0; j < nCols; j++) { 00742 proj.push_back(i * nCols + j); 00743 } 00744 } 00745 00746 testSampleScanCols( 00747 nRows, nRows, nCols, nClusters, proj, 1, mode, 0.1, -1, 10, 5000); 00748 resetExecStreamTest(); 00749 00750 testSampleScanCols( 00751 nRows, nRows, nCols, nClusters, proj, 1, mode, 1.0, -1, 10, 50000); 00752 resetExecStreamTest(); 00753 00754 testSampleScanCols( 00755 nRows, nRows, nCols, nClusters, proj, 1, mode, 0.33333, -1, 10, 16670); 00756 resetExecStreamTest(); 00757 }

Defines and prepares a graph consisting of one source stream and one transform stream.

Parameters:

sourceStreamEmbryo embryonic source stream which produces tuples
transformStreamEmbryo embryonic transform stream which processes tuples produced by sourceStreamEmbryo

Returns:

output buffer stream

Definition at line 48 of file ExecStreamUnitTestBase.cpp.

Referenced by LcsClusterReplaceExecStreamTest::loadCluster(), loadOneCluster(), ExecStreamUnitTestBase::prepareSourceGraph(), LcsClusterReplaceExecStreamTest::replaceCluster(), LcsMultiClusterAppendTest::scanCols(), LbmSplicerExecStreamTest::spliceInput(), ExecStreamTestSuite::testBTreeInsertExecStream(), CollectExecStreamTestSuite::testCollectCollectUncollectUncollect(), CollectExecStreamTestSuite::testCollectInts(), CollectExecStreamTestSuite::testCollectUncollect(), CalcExecStreamTestSuite::testConstant(), ExecStreamTestSuite::testCopyExecStream(), ExecStreamTestSuite::testCountAggExecStream(), LhxAggExecStreamTest::testCountImpl(), ExecStreamTestSuite::testDoubleBufferExecStream(), ExecStreamTestSuite::testGroupAggExecStreamNrows(), LhxAggExecStreamTest::testGroupCountImpl(), ExternalSortExecStreamTest::testImpl(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), LcsClusterAppendExecStreamTest::testLoadSingleCol(), LbmNormalizerExecStreamTest::testNormalizer(), ExecStreamTestSuite::testReshapeExecStream(), testSampleScanCols(), testScanCols(), LbmSearchTest::testScanIdx(), LcsClusterAppendExecStreamTest::testScanMultiCol(), LcsClusterAppendExecStreamTest::testScanSingleCol(), ExecStreamTestSuite::testScratchBufferExecStream(), ExecStreamTestSuite::testSegBufferExecStream(), ExecStreamTestSuite::testSingleValueAggExecStream(), LhxAggExecStreamTest::testSingleValueImpl(), LbmSortedAggExecStreamTest::testSortedAgg(), ExecStreamTestSuite::testSumAggExecStream(), LhxAggExecStreamTest::testSumImpl(), LbmUnionExecStreamTest::testUnion(), and LcsClusterReplaceExecStreamTest::verifyCluster().

00051 { 00052 std::vector transforms; 00053 transforms.push_back(transformStreamEmbryo); 00054 return prepareTransformGraph(sourceStreamEmbryo, transforms); 00055 }

Defines and prepares a graph consisting of one source stream and one or multiple transform streams.

Parameters:

sourceStreamEmbryo embryonic source stream which produces tuples
transforms embryonic transform streams which process tuples produced by sourceStreamEmbryo or a child stream

Returns:

output buffer stream

Definition at line 57 of file ExecStreamUnitTestBase.cpp.

References BUFPROV_PRODUCER, ExecStreamEmbryo::getStream(), ExecStreamUnitTestBase::pGraph, and ExecStreamUnitTestBase::pGraphEmbryo.

00060 { 00061 pGraphEmbryo->saveStreamEmbryo(sourceStreamEmbryo); 00062 std::vector::iterator it; 00063 00064
00065 for (it = transforms.begin(); it != transforms.end(); ++it) { 00066 pGraphEmbryo->saveStreamEmbryo(*it); 00067 } 00068 00069
00070 ExecStreamEmbryo& previousStream = sourceStreamEmbryo; 00071 for (it = transforms.begin(); it != transforms.end(); ++it) { 00072 pGraphEmbryo->addDataflow( 00073 previousStream.getStream()->getName(), 00074 (*it).getStream()->getName()); 00075 previousStream = *it; 00076 } 00077 00078 SharedExecStream pAdaptedStream = 00079 pGraphEmbryo->addAdapterFor( 00080 previousStream.getStream()->getName(), 00081 0, 00082 BUFPROV_PRODUCER); 00083 pGraph->addOutputDataflow(pAdaptedStream->getStreamId()); 00084 00085 pGraphEmbryo->prepareGraph(shared_from_this(), ""); 00086 return pAdaptedStream; 00087 }

Defines and prepares a graph consisting of two source streams and one confluence stream.

Parameters:

sourceStreamEmbryo1 embryonic source stream which produces tuples
sourceStreamEmbryo2 embryonic source stream which produces tuples
confluenceStreamEmbryo embryonic confluence stream which processes tuples produced by the sourceStreamEmbryos

Returns:

output buffer stream

Definition at line 89 of file ExecStreamUnitTestBase.cpp.

Referenced by ExecStreamUnitTestBase::prepareConfluenceGraph(), ExecStreamTestSuite::testCartesianJoinExecStream(), CorrelationJoinExecStreamTestSuite::testCorrelationJoin(), testFilterCols(), ExecStreamGovernorTest::testGovernor(), LhxJoinExecStreamTest::testImpl(), LbmIntersectExecStreamTest::testIntersect(), ExecStreamTestSuite::testMergeExecStream(), LbmMinusExecStreamTest::testMinus(), and ExecStreamTestSuite::testNestedLoopJoinExecStream().

00093 { 00094 std::vector sourceStreamEmbryos; 00095 sourceStreamEmbryos.push_back(sourceStreamEmbryo1); 00096 sourceStreamEmbryos.push_back(sourceStreamEmbryo2); 00097 return prepareConfluenceGraph(sourceStreamEmbryos, confluenceStreamEmbryo); 00098 }

Defines and prepares a graph consisting of a list of source streams and one confluence stream.

Parameters:

sourceStreamEmbryos list of embryonic source streams that produce tuples
confluenceStreamEmbryo embryonic confluence stream which processes tuples produced by the sourceStreamEmbryos

Returns:

output buffer stream

Definition at line 155 of file ExecStreamUnitTestBase.cpp.

References ExecStreamUnitTestBase::prepareConfluenceGraph().

00158 { 00159 std::vector<std::vector > sourceStreamEmbryosList; 00160 std::vector::iterator it; 00161 std::vector sourceStreamList; 00162 for (it = sourceStreamEmbryos.begin(); it != sourceStreamEmbryos.end(); 00163 it++) 00164 { 00165 sourceStreamList.clear(); 00166 sourceStreamList.push_back(*it); 00167 sourceStreamEmbryosList.push_back(sourceStreamList); 00168 } 00169 00170 return 00171 prepareConfluenceGraph(sourceStreamEmbryosList, confluenceStreamEmbryo); 00172 }

Defines and prepares a graph consisting of one or more source streams and one confluence stream.

Each source stream can be a list of streams.

Parameters:

sourceStreamEmbryosList list of embryonic source streams which produce tuples
confluenceStreamEmbryo embryonic confluence stream which processes tuples produced by the source streams

Returns:

output buffer stream

Definition at line 174 of file ExecStreamUnitTestBase.cpp.

References BUFPROV_PRODUCER, ExecStreamEmbryo::getStream(), ExecStreamUnitTestBase::pGraph, and ExecStreamUnitTestBase::pGraphEmbryo.

00177 { 00178 pGraphEmbryo->saveStreamEmbryo(confluenceStreamEmbryo); 00179 00180 for (int i = 0; i < sourceStreamEmbryosList.size(); i++) { 00181 for (int j = 0; j < sourceStreamEmbryosList[i].size(); j++) { 00182 pGraphEmbryo->saveStreamEmbryo(sourceStreamEmbryosList[i][j]); 00183 } 00184 00185
00186 for (int j = 1; j < sourceStreamEmbryosList[i].size(); j++) { 00187 pGraphEmbryo->addDataflow( 00188 sourceStreamEmbryosList[i][j - 1].getStream()->getName(), 00189 sourceStreamEmbryosList[i][j].getStream()->getName()); 00190 } 00191 pGraphEmbryo->addDataflow( 00192 sourceStreamEmbryosList[i].back().getStream()->getName(), 00193 confluenceStreamEmbryo.getStream()->getName()); 00194 } 00195 00196 SharedExecStream pAdaptedStream = 00197 pGraphEmbryo->addAdapterFor( 00198 confluenceStreamEmbryo.getStream()->getName(), 0, 00199 BUFPROV_PRODUCER); 00200 pGraph->addOutputDataflow( 00201 pAdaptedStream->getStreamId()); 00202 00203 pGraphEmbryo->prepareGraph(shared_from_this(), ""); 00204 00205 return pAdaptedStream; 00206 }

Defines and prepares a graph consisting of two source streams, one confluence stream, and one transform stream.

Parameters:

sourceStreamEmbryo1 embryonic source stream which produces tuples
sourceStreamEmbryo2 embryonic source stream which produces tuples
confluenceStreamEmbryo embryonic confluence stream which processes tuples produced by the sourceStreamEmbryos
transformStreamEmbryo embryonic transform streams which process tuples produced by a child stream

Returns:

output buffer stream

Definition at line 100 of file ExecStreamUnitTestBase.cpp.

References BUFPROV_PRODUCER, ExecStreamEmbryo::getStream(), ExecStreamUnitTestBase::pGraph, and ExecStreamUnitTestBase::pGraphEmbryo.

Referenced by LhxJoinExecStreamTest::testImpl(), and LbmMinusExecStreamTest::testRestartingMinus().

00105 { 00106 std::vector sourceStreamEmbryos; 00107 sourceStreamEmbryos.push_back(sourceStreamEmbryo1); 00108 sourceStreamEmbryos.push_back(sourceStreamEmbryo2); 00109 00110 std::vector::iterator it; 00111 00112 for (it = sourceStreamEmbryos.begin(); it != sourceStreamEmbryos.end(); 00113 ++it) 00114 { 00115 pGraphEmbryo->saveStreamEmbryo(*it); 00116 } 00117 pGraphEmbryo->saveStreamEmbryo(confluenceStreamEmbryo); 00118 00119 for (it = sourceStreamEmbryos.begin(); it != sourceStreamEmbryos.end(); 00120 ++it) 00121 { 00122 pGraphEmbryo->addDataflow( 00123 (*it).getStream()->getName(), 00124 confluenceStreamEmbryo.getStream()->getName()); 00125 } 00126 00127 std::vector transforms; 00128 transforms.push_back(transformStreamEmbryo); 00129 ExecStreamEmbryo& previousStream = confluenceStreamEmbryo; 00130 00131
00132 for (it = transforms.begin(); it != transforms.end(); ++it) { 00133 pGraphEmbryo->saveStreamEmbryo(*it); 00134 } 00135 00136 for (it = transforms.begin(); it != transforms.end(); ++it) { 00137 pGraphEmbryo->addDataflow( 00138 previousStream.getStream()->getName(), 00139 (*it).getStream()->getName()); 00140 previousStream = *it; 00141 } 00142 00143 00144 SharedExecStream pAdaptedStream = 00145 pGraphEmbryo->addAdapterFor( 00146 previousStream.getStream()->getName(), 00147 0, 00148 BUFPROV_PRODUCER); 00149 pGraph->addOutputDataflow(pAdaptedStream->getStreamId()); 00150 00151 pGraphEmbryo->prepareGraph(shared_from_this(), ""); 00152 return pAdaptedStream; 00153 }

Defines and prepares a graph consisting of a source, a splitter, and one or more parallel transform streams which flow together into a confluence stream.

Parameters:

srcStreamEmbryo embryonic source stream which produces tuples
splitterStreamEmbryo embryonic SplitterExecStream which produces tuples for multiple consumers
interStreamEmbryos embryonic intermediate streams which transform tuples; each stream consists of a single embryo
destStreamEmbryo embryonic confluence stream which processes tuples produced by the interStreamEmbryos
createSink if true (the default), creates a final output sink in the stream graph
saveSrc if true (the default), save the source in the stream graph; if false, the save has already been done

Returns:

output buffer stream or null stream if createSink is false

Definition at line 208 of file ExecStreamUnitTestBase.cpp.

Referenced by LcsMultiClusterAppendTest::loadClusters(), LbmSearchTest::loadTableAndIndex(), LbmLoadBitmapTest::testLoad(), ExecStreamTestSuite::testMergeImplicitPullInputs(), ExecStreamTestSuite::testSegBufferReaderWriterExecStream(), and ExecStreamTestSuite::testSplitterPlusBarrier().

00215 { 00216 std::vector<std::vector > listOfList; 00217 00218
00219
00220
00221 for (uint i = 0; i < interStreamEmbryos.size(); i++) { 00222 std::vector interStreamEmbryoList; 00223 00224 interStreamEmbryoList.push_back(interStreamEmbryos[i]); 00225 listOfList.push_back(interStreamEmbryoList); 00226 } 00227 return prepareDAG( 00228 srcStreamEmbryo, splitterStreamEmbryo, listOfList, destStreamEmbryo, 00229 createSink, saveSrc); 00230 }

Defines and prepares a graph consisting of a source, a splitter, and one or more parallel transform streams which flow together into a confluence stream.

Parameters:

srcStreamEmbryo embryonic source stream which produces tuples
splitterStreamEmbryo embryonic SplitterExecStream which produces tuples for multiple consumers
interStreamEmbryosList one or more embryonic intermediate streams which transform tuples; each stream can have one more embryos
destStreamEmbryo embryonic confluence stream which processes tuples produced by the interStreamEmbryos
createSink if true (the default), creates a final output sink in the stream graph
saveSrc if true (the default), save the source in the stream graph; if false, the save has already been done

Returns:

output buffer stream or null stream if createSink is false

Definition at line 232 of file ExecStreamUnitTestBase.cpp.

References BUFPROV_PRODUCER, ExecStreamEmbryo::getStream(), ExecStreamUnitTestBase::pGraph, and ExecStreamUnitTestBase::pGraphEmbryo.

00239 { 00240 if (saveSrc) { 00241 pGraphEmbryo->saveStreamEmbryo(srcStreamEmbryo); 00242 } 00243 pGraphEmbryo->saveStreamEmbryo(splitterStreamEmbryo); 00244 00245
00246 for (int i = 0; i < interStreamEmbryos.size(); i++) { 00247 for (int j = 0; j < interStreamEmbryos[i].size(); j++) { 00248 pGraphEmbryo->saveStreamEmbryo(interStreamEmbryos[i][j]); 00249 } 00250 00251
00252 for (int j = 1; j < interStreamEmbryos[i].size(); j++) { 00253 pGraphEmbryo->addDataflow( 00254 interStreamEmbryos[i][j - 1].getStream()->getName(), 00255 interStreamEmbryos[i][j].getStream()->getName()); 00256 } 00257 } 00258 00259 pGraphEmbryo->saveStreamEmbryo(destStreamEmbryo); 00260 00261 pGraphEmbryo->addDataflow( 00262 srcStreamEmbryo.getStream()->getName(), 00263 splitterStreamEmbryo.getStream()->getName()); 00264 00265
00266 for (int i = 0; i < interStreamEmbryos.size(); i++) { 00267 pGraphEmbryo->addDataflow( 00268 splitterStreamEmbryo.getStream()->getName(), 00269 interStreamEmbryos[i][0].getStream()->getName()); 00270 pGraphEmbryo->addDataflow( 00271 interStreamEmbryos[i].back().getStream()->getName(), 00272 destStreamEmbryo.getStream()->getName()); 00273 } 00274 00275 SharedExecStream pAdaptedStream; 00276 00277 if (createSink) { 00278 pAdaptedStream = pGraphEmbryo->addAdapterFor( 00279 destStreamEmbryo.getStream()->getName(), 0, 00280 BUFPROV_PRODUCER); 00281 pGraph->addOutputDataflow(pAdaptedStream->getStreamId()); 00282 00283 pGraphEmbryo->prepareGraph(shared_from_this(), ""); 00284 } 00285 00286 return pAdaptedStream; 00287 }

Executes the prepared stream graph and verifies that its output matches that produced by a value generator.

Parameters:

stream output stream from which to read
nRowsExpected number of rows expected
verifier generator for expected values
stopEarly if true, stop once nRowsExpected have been fetched, even if more rows are available; this can be used for simulating the cleanup effect of an error in the middle of execution

Definition at line 325 of file ExecStreamUnitTestBase.cpp.

References TupleData::compute(), EXECBUF_EOS, MockProducerExecStreamGenerator::generateValue(), ExecStreamUnitTestBase::pGraph, ExecStreamTestBase::pResourceGovernor, and ExecStreamTestBase::pScheduler.

Referenced by LcsClusterReplaceExecStreamTest::loadCluster(), LcsMultiClusterAppendTest::loadClusters(), loadOneCluster(), LbmSearchTest::loadTableAndIndex(), LcsClusterReplaceExecStreamTest::replaceCluster(), LcsMultiClusterAppendTest::scanCols(), LbmSplicerExecStreamTest::spliceInput(), ExecStreamTestSuite::testBTreeInsertExecStream(), CollectExecStreamTestSuite::testCollectCollectUncollectUncollect(), CollectExecStreamTestSuite::testCollectUncollect(), CorrelationJoinExecStreamTestSuite::testCorrelationJoin(), ExecStreamTestSuite::testCountAggExecStream(), LhxAggExecStreamTest::testCountImpl(), ExecStreamTestSuite::testDoubleBufferExecStream(), testFilterCols(), ExecStreamTestSuite::testGroupAggExecStreamNrows(), LhxAggExecStreamTest::testGroupCountImpl(), LhxJoinExecStreamTest::testImpl(), ExternalSortExecStreamTest::testImpl(), LbmLoadBitmapTest::testLoad(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), LcsClusterAppendExecStreamTest::testLoadSingleCol(), ExecStreamTestSuite::testMergeImplicitPullInputs(), ExecStreamTestSuite::testNestedLoopJoinExecStream(), LbmNormalizerExecStreamTest::testNormalizer(), ExecStreamTestSuite::testReshapeExecStream(), LbmMinusExecStreamTest::testRestartingMinus(), testSampleScanCols(), testScanCols(), LcsClusterAppendExecStreamTest::testScanMultiCol(), LcsClusterAppendExecStreamTest::testScanSingleCol(), ExecStreamTestSuite::testScratchBufferExecStream(), ExecStreamTestSuite::testSingleValueAggExecStream(), LhxAggExecStreamTest::testSingleValueImpl(), LbmSortedAggExecStreamTest::testSortedAgg(), ExecStreamTestSuite::testSplitterPlusBarrier(), ExecStreamTestSuite::testSumAggExecStream(), LhxAggExecStreamTest::testSumImpl(), and LcsClusterReplaceExecStreamTest::verifyCluster().

00330 { 00331
00332 00333 pResourceGovernor->requestResources(*pGraph); 00334 pGraph->open(); 00335 pScheduler->start(); 00336 uint nRows = 0; 00337 for (;;) { 00338 ExecStreamBufAccessor &bufAccessor = 00339 pScheduler->readStream(stream); 00340 if (bufAccessor.getState() == EXECBUF_EOS) { 00341 break; 00342 } 00343 BOOST_REQUIRE(bufAccessor.isConsumptionPossible()); 00344 const uint nCol = 00345 bufAccessor.getConsumptionTupleAccessor().size(); 00346 BOOST_REQUIRE(nCol == bufAccessor.getTupleDesc().size()); 00347 BOOST_REQUIRE(nCol >= 1); 00348 TupleData inputTuple; 00349 inputTuple.compute(bufAccessor.getTupleDesc()); 00350 for (;;) { 00351 if (!bufAccessor.demandData()) { 00352 break; 00353 } 00354 BOOST_REQUIRE(nRows < nRowsExpected); 00355 bufAccessor.unmarshalTuple(inputTuple); 00356 for (int col = 0; col < nCol; ++col) { 00357 int64_t actualValue = 00358 *reinterpret_cast<int64_t const *>(inputTuple[col].pData); 00359 int64_t expectedValue = generator.generateValue(nRows, col); 00360 if (actualValue != expectedValue) { 00361 std::cout << "(Row, Col) = (" << nRows << ", " << col <<")" 00362 << std::endl; 00363 BOOST_CHECK_EQUAL(expectedValue,actualValue); 00364 return; 00365 } 00366 } 00367 bufAccessor.consumeTuple(); 00368 ++nRows; 00369 if (stopEarly && nRows == nRowsExpected) { 00370 return; 00371 } 00372 } 00373 } 00374 BOOST_CHECK_EQUAL(nRowsExpected,nRows); 00375 }

void ExecStreamUnitTestBase::verifyConstantOutput ( ExecStream & stream,
const TupleData & expectedTuple,
uint nRowsExpected
) [protected, inherited]

Executes the prepared stream graph and verifies that all output tuples matche an expected and given one.

Parameters:

stream output stream from which to read
expectedTuple
nRowsExpected

Definition at line 377 of file ExecStreamUnitTestBase.cpp.

References TupleData::compute(), EXECBUF_EOS, ExecStreamUnitTestBase::pGraph, ExecStreamTestBase::pResourceGovernor, TuplePrinter::print(), and ExecStreamTestBase::pScheduler.

Referenced by ExecStreamTestSuite::testCartesianJoinExecStream(), CollectExecStreamTestSuite::testCollectInts(), CalcExecStreamTestSuite::testConstant(), ExecStreamTestSuite::testCopyExecStream(), ExecStreamGovernorTest::testGovernor(), ExecStreamTestSuite::testMergeExecStream(), ExecStreamTestSuite::testSegBufferExecStream(), and ExecStreamTestSuite::testSegBufferReaderWriterExecStream().

00381 { 00382
00383 00384 pResourceGovernor->requestResources(*pGraph); 00385 pGraph->open(); 00386 pScheduler->start(); 00387 uint nRows = 0; 00388 for (;;) { 00389 ExecStreamBufAccessor &bufAccessor = 00390 pScheduler->readStream(stream); 00391 if (bufAccessor.getState() == EXECBUF_EOS) { 00392 break; 00393 } 00394 BOOST_REQUIRE(bufAccessor.isConsumptionPossible()); 00395 00396 if (!bufAccessor.demandData()) { 00397 break; 00398 } 00399 BOOST_REQUIRE(nRows < nRowsExpected); 00400 00401 TupleData actualTuple; 00402 actualTuple.compute(bufAccessor.getTupleDesc()); 00403 bufAccessor.unmarshalTuple(actualTuple); 00404 00405 int c = bufAccessor.getTupleDesc().compareTuples( 00406 expectedTuple, actualTuple); 00407 bufAccessor.consumeTuple(); 00408 ++nRows; 00409 if (c) { 00410 #if 1 00411 TupleDescriptor statusDesc = bufAccessor.getTupleDesc(); 00412 TuplePrinter tuplePrinter; 00413 tuplePrinter.print(std::cout, statusDesc, actualTuple); 00414 tuplePrinter.print(std::cout, statusDesc, expectedTuple); 00415 std::cout << std::endl; 00416 #endif 00417 BOOST_CHECK_EQUAL(0,c); 00418 break; 00419 } 00420 } 00421 BOOST_CHECK_EQUAL(nRowsExpected, nRows); 00422 }

Executes the prepared stream graph and verifies the resultant tuples against a set of tuples supplied in an input buffer.

Parameters:

stream output stream from which to read
outputTupleDesc descriptor of expected output tuple
nRowsExpected number of rows expected
expectedBuffer buffer containing expected tuples

Definition at line 424 of file ExecStreamUnitTestBase.cpp.

References TupleDescriptor::compareTuples(), TupleData::compute(), TupleAccessor::compute(), EXECBUF_EOS, TupleAccessor::getCurrentByteCount(), ExecStreamUnitTestBase::pGraph, ExecStreamTestBase::pResourceGovernor, ExecStreamTestBase::pScheduler, TupleAccessor::setCurrentTupleBuf(), and TupleAccessor::unmarshal().

Referenced by LbmIntersectExecStreamTest::testIntersect(), LbmMinusExecStreamTest::testMinus(), LbmSearchTest::testScanIdx(), and LbmUnionExecStreamTest::testUnion().

00429 { 00430
00431 00432 TupleAccessor expectedOutputAccessor; 00433 expectedOutputAccessor.compute(outputTupleDesc); 00434 TupleData expectedTuple(outputTupleDesc); 00435 uint bufOffset = 0; 00436 pResourceGovernor->requestResources(*pGraph); 00437 pGraph->open(); 00438 pScheduler->start(); 00439 uint nRows = 0; 00440 for (;;) { 00441 ExecStreamBufAccessor &bufAccessor = 00442 pScheduler->readStream(stream); 00443 if (bufAccessor.getState() == EXECBUF_EOS) { 00444 break; 00445 } 00446 BOOST_REQUIRE(bufAccessor.getTupleDesc() == outputTupleDesc); 00447 BOOST_REQUIRE(bufAccessor.isConsumptionPossible()); 00448 const uint nCol = 00449 bufAccessor.getConsumptionTupleAccessor().size(); 00450 BOOST_REQUIRE(nCol == bufAccessor.getTupleDesc().size()); 00451 BOOST_REQUIRE(nCol >= 1); 00452 TupleData inputTuple; 00453 inputTuple.compute(bufAccessor.getTupleDesc()); 00454 for (;;) { 00455 if (!bufAccessor.demandData()) { 00456 break; 00457 } 00458 BOOST_REQUIRE(nRows < nRowsExpected); 00459 bufAccessor.unmarshalTuple(inputTuple); 00460 expectedOutputAccessor.setCurrentTupleBuf( 00461 expectedBuffer + bufOffset); 00462 expectedOutputAccessor.unmarshal(expectedTuple); 00463 int c = outputTupleDesc.compareTuples(inputTuple, expectedTuple); 00464 if (c) { 00465 std::cout << "(Row) = (" << nRows << ")" 00466 << " -- Tuples don't match"<< std::endl; 00467 BOOST_CHECK_EQUAL(0,c); 00468 return; 00469 } 00470 bufAccessor.consumeTuple(); 00471 bufOffset += expectedOutputAccessor.getCurrentByteCount(); 00472 ++nRows; 00473 } 00474 } 00475 BOOST_CHECK_EQUAL(nRowsExpected,nRows); 00476 }

| void ExecStreamUnitTestBase::resetExecStreamTest | ( | | ) | [protected, inherited] | | ------------------------------------------------ | - | | - | ------------------------ |

Reset stream graph so multiple iterations of a method can be called within a single testcase.

Definition at line 305 of file ExecStreamUnitTestBase.cpp.

References ExecStreamTestBase::newScheduler(), ExecStreamTestBase::newStreamGraph(), ExecStreamTestBase::newStreamGraphEmbryo(), ExecStreamUnitTestBase::pGraph, ExecStreamUnitTestBase::pGraphEmbryo, ExecStreamTestBase::pResourceGovernor, ExecStreamTestBase::pScheduler, and ExecStreamUnitTestBase::tearDownExecStreamTest().

Referenced by loadClusters(), LbmLoadBitmapTest::testAppend(), testBernoulliSampling(), LcsClusterReplaceExecStreamTest::testClusterReplace(), testCompressedFiltering(), LbmSplicerExecStreamTest::testLER5968(), LbmSplicerExecStreamTest::testLER6473(), LcsMultiClusterAppendTest::testLoad(), LcsClusterAppendExecStreamTest::testMultiColConstNewRoot(), LcsClusterAppendExecStreamTest::testMultiColConstOldRoot(), LcsClusterAppendExecStreamTest::testMultiColNoDupNewRoot(), LcsClusterAppendExecStreamTest::testMultiColNoDupOldRoot(), LcsClusterAppendExecStreamTest::testMultiColStairNewRoot(), LcsClusterAppendExecStreamTest::testMultiColStairOldRoot(), LbmSearchTest::testMultipleRanges(), ExecStreamGovernorTest::testReturnResources(), LbmSearchTest::testScanIdx(), testScanPastEndOfCluster(), testScans(), LcsClusterAppendExecStreamTest::testSingleColConstNewRoot(), LcsClusterAppendExecStreamTest::testSingleColConstOldRoot(), LcsClusterAppendExecStreamTest::testSingleColNoDupNewRoot(), LcsClusterAppendExecStreamTest::testSingleColNoDupOldRoot(), LcsClusterAppendExecStreamTest::testSingleColStairNewRoot(), LcsClusterAppendExecStreamTest::testSingleColStairOldRoot(), LbmSplicerExecStreamTest::testSpliceRids(), LbmSplicerExecStreamTest::testSpliceWithKeys(), and testSystemSampling().

| void ExecStreamUnitTestBase::tearDownExecStreamTest | ( | | ) | [protected, virtual, inherited] | | --------------------------------------------------- | - | | - | --------------------------------- |

void SegStorageTestBase::openStorage ( DeviceMode openMode ) [virtual, inherited]

Reimplemented from CacheTestBase.

Reimplemented in SegmentTestBase.

Definition at line 31 of file SegStorageTestBase.cpp.

References SegStorageTestBase::openSegmentStorage(), and CacheTestBase::openStorage().

Referenced by SegStorageTestBase::openRandomSegment(), SegmentTestBase::openStorage(), LhxHashTableTest::testCaseSetUp(), ExecStreamTestBase::testCaseSetUp(), BTreeTest::testCaseSetUp(), BTreeReadersTest::testCaseSetUp(), LbmEntryTest::testCaseSetUp(), SegPageIterTest::testIter(), SegPageEntryIterTest::testIter(), SegStreamTest::testMarkReset(), SegStreamTest::testReadSeg(), LogicalTxnTest::testTxn(), SegStreamTest::testWriteSeg(), and SegStreamTest::testWriteSpillAndRead().

void SegStorageTestBase::openSegmentStorage ( DeviceMode openMode ) [virtual, inherited]

| void SegStorageTestBase::openRandomSegment | ( | | ) | [virtual, inherited] | | ------------------------------------------ | - | | - | ---------------------- |

SharedSegment SegStorageTestBase::createLinearDeviceSegment ( DeviceId deviceId,
uint nPages
) [inherited]

| void SegStorageTestBase::closeLinearSegment | ( | | ) | [inherited] | | ------------------------------------------- | - | | - | ------------- |

| void SegStorageTestBase::closeRandomSegment | ( | | ) | [inherited] | | ------------------------------------------- | - | | - | ------------- |

| void SegStorageTestBase::closeVersionedRandomSegment | ( | | ) | [inherited] | | ---------------------------------------------------- | - | | - | ------------- |

| void SegStorageTestBase::closeSnapshotRandomSegment | ( | | ) | [inherited] | | --------------------------------------------------- | - | | - | ------------- |

| void SegStorageTestBase::closeStorage | ( | | ) | [virtual, inherited] | | ------------------------------------- | - | | - | ---------------------- |

Reimplemented from CacheTestBase.

Reimplemented in SnapshotSegmentTestBase, and VersionedSegmentTest.

Definition at line 102 of file SegStorageTestBase.cpp.

References SegStorageTestBase::closeLinearSegment(), SegStorageTestBase::closeRandomSegment(), CacheTestBase::closeStorage(), and SegStorageTestBase::closeVersionedRandomSegment().

Referenced by VersionedSegmentTest::closeStorage(), SnapshotSegmentTestBase::closeStorage(), SegStorageTestBase::openRandomSegment(), SegPageIterTest::testIter(), SegPageEntryIterTest::testIter(), SegStreamTest::testMarkReset(), SegStreamTest::testReadSeg(), SegmentTestBase::testSingleThread(), SegStreamTest::testWriteSeg(), and SegStreamTest::testWriteSpillAndRead().

| Cache & CacheTestBase::getCache | ( | | ) | [inherited] | | -------------------------------------------------- | - | | - | ------------- |

| SharedCache CacheTestBase::newCache | ( | | ) | [virtual, inherited] | | --------------------------------------------------------------------------------------------- | - | | - | ---------------------- |

void TestBase::snooze ( uint nSeconds ) [protected, inherited]
void TestBase::readParams ( int argc,
char ** argv
) [static, inherited]

Parses the command line.

format: [-v] [-t TEST | -all] {param=val}* [CONFIGFILE | -] Normally, the test program runs the default test cases. With the option "-all", runs the extra test cases as well. With the option "-t TEST", runs only the single test case named TEST. CONFIGFILE is read to load configuration parameters. Configuration parameters can also be set ad hoc, from the command line, as pairs name=val. These take precedence.

Definition at line 108 of file TestBase.cpp.

References TestBase::configMap, ConfigMap::dumpParams(), ConfigMap::isParamSet(), ConfigMap::mergeFrom(), TestBase::paramDictionaryFileName, ConfigMap::readParams(), TestBase::runAll, TestBase::runSingle, ConfigMap::setStringParam(), and verbose.

00109 { 00110 bool verbose = false; 00111 ConfigMap adhocMap; 00112 00113 for (int i = 1; i < argc; ++i) { 00114 std::string arg = argv[i]; 00115 if (argv[i][0] == '-') { 00116 if (arg == "-v") { 00117 verbose = true; 00118 } else if (arg == "-") { 00119 configMap.readParams(std::cin); 00120 } else if (arg == "-all") { 00121 runAll = true; 00122 } else if (arg == "-t") {
00123 permAssert(i + 1 < argc); 00124 runSingle = argv[++i]; 00125 } else if (arg[1] == 't') { 00126 runSingle = arg.substr(2); 00127 } 00128 } else { 00129 int i = arg.find("="); 00130 if ((0 < i) && (i < arg.size())) { 00131
00132 std::string key = arg.substr(0,i); 00133 std::string val = arg.substr(i + 1); 00134 adhocMap.setStringParam(key,val); 00135 } else { 00136
00137 std::ifstream configFile(arg.c_str()); 00138 assert(configFile.good()); 00139 configMap.readParams(configFile); 00140 } 00141 } 00142 } 00143 configMap.mergeFrom(adhocMap); 00144 00145
00146
00147 if (configMap.isParamSet(paramDictionaryFileName)) { 00148 std::string dictFileName = "dictWords"; 00149 configMap.setStringParam(paramDictionaryFileName,dictFileName); 00150 } 00151 00152 if (verbose) { 00153 configMap.dumpParams(std::cout); 00154 } 00155 }

| TestSuite * TestBase::releaseTestSuite | ( | | ) | [inherited] | | ---------------------------------------------------------------------------------------------- | - | | - | ------------- |

void TestBase::beforeTestCase ( std::string testCaseName ) [inherited]
void TestBase::afterTestCase ( std::string testCaseName ) [inherited]
void TestBase::notifyTrace ( std::string source,
TraceLevel level,
std::string message
) [virtual, inherited]
TraceLevel TestBase::getSourceTraceLevel ( std::string source ) [virtual, inherited]

Member Data Documentation

Definition at line 46 of file ExecStreamUnitTestBase.h.

Referenced by ExecStreamUnitTestBase::prepareConfluenceGraph(), ExecStreamUnitTestBase::prepareConfluenceTransformGraph(), ExecStreamUnitTestBase::prepareDAG(), ExecStreamUnitTestBase::prepareTransformGraph(), ExecStreamUnitTestBase::resetExecStreamTest(), LbmSplicerExecStreamTest::spliceInput(), ExecStreamUnitTestBase::tearDownExecStreamTest(), ExecStreamTestSuite::testBTreeInsertExecStream(), ExecStreamUnitTestBase::testCaseSetUp(), CorrelationJoinExecStreamTestSuite::testCorrelationJoin(), LhxJoinExecStreamTest::testImpl(), ExternalSortExecStreamTest::testImpl(), LbmLoadBitmapTest::testLoad(), ExecStreamTestSuite::testReshapeExecStream(), LbmSearchTest::testScanIdx(), ExecStreamUnitTestBase::verifyBufferedOutput(), ExecStreamUnitTestBase::verifyConstantOutput(), FlatFileExecStreamTest::verifyOutput(), and ExecStreamUnitTestBase::verifyOutput().

Definition at line 51 of file ExecStreamTestBase.h.

Referenced by ExecStreamUnitTestBase::resetExecStreamTest(), ExternalSortExecStreamTest::testCaseSetUp(), ExecStreamUnitTestBase::testCaseSetUp(), ExecStreamTestBase::testCaseSetUp(), ExecStreamGovernorTest::testCaseSetUp(), ExecStreamTestBase::testCaseTearDown(), ExecStreamGovernorTest::testMinGreaterAllocation(), ExecStreamUnitTestBase::verifyBufferedOutput(), ExecStreamUnitTestBase::verifyConstantOutput(), FlatFileExecStreamTest::verifyOutput(), and ExecStreamUnitTestBase::verifyOutput().

Definition at line 40 of file SegStorageTestBase.h.

Referenced by BackupRestoreTest::backup(), SegStorageTestBase::createLinearDeviceSegment(), LogicalTxnTest::createRecoveryLog(), LbmSearchTest::initBTreeExecStreamParam(), LbmLoadBitmapTest::initBTreeExecStreamParam(), LcsClusterReplaceExecStreamTest::initClusterAppendParams(), LbmExecStreamTestBase::initSorterExecStream(), LcsMultiClusterAppendTest::loadClusters(), loadOneCluster(), LbmSearchTest::loadTableAndIndex(), LbmMinusExecStreamTest::newMinusStream(), ExecStreamTestBase::newStreamGraphEmbryo(), SegStorageTestBase::openRandomSegment(), VersionedSegmentTest::openSegmentStorage(), SnapshotSegmentTestBase::openSegmentStorage(), LinearViewSegmentTest::openSegmentStorage(), BackupRestoreTest::restore(), SegStorageTestBase::SegStorageTestBase(), LbmSplicerExecStreamTest::spliceInput(), ExecStreamTestSuite::testBTreeInsertExecStream(), LhxHashTableTest::testCaseSetUp(), LcsClusterReplaceExecStreamTest::testClusterReplace(), LhxAggExecStreamTest::testCountImpl(), ExecStreamTestSuite::testDoubleBufferExecStream(), ExecStreamGovernorTest::testGovernor(), LhxAggExecStreamTest::testGroupCountImpl(), LhxJoinExecStreamTest::testImpl(), ExternalSortExecStreamTest::testImpl(), BTreeTest::testInserts(), LbmIntersectExecStreamTest::testIntersect(), LbmEntryTest::testldb35(), LbmEntryTest::testler5920(), LbmLoadBitmapTest::testLoad(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), LcsClusterAppendExecStreamTest::testLoadSingleCol(), LbmEntryTest::testMergeEntry(), BTreeTest::testMultiKeySearches(), BTreeTest::testScan(), ExecStreamTestSuite::testScratchBufferExecStream(), LhxAggExecStreamTest::testSingleValueImpl(), FlatFileExecStreamTest::testStream(), LhxAggExecStreamTest::testSumImpl(), LogicalTxnTest::testTxn(), SnapshotSegmentTest::testUncommittedReads(), LbmUnionExecStreamTest::testUnion(), SegStreamTest::testWriteSpillAndRead(), and LbmEntryTest::testZeroBytes().

Segment supporting linear page allocation.

Definition at line 45 of file SegStorageTestBase.h.

Referenced by SegStorageTestBase::closeLinearSegment(), LogicalTxnTest::createRecoveryLog(), SegmentTestBase::lockPage(), SegStorageTestBase::openRandomSegment(), VersionedSegmentTest::openSegmentStorage(), SnapshotSegmentTestBase::openSegmentStorage(), SegStorageTestBase::openSegmentStorage(), LinearViewSegmentTest::openSegmentStorage(), SegmentTestBase::openStorage(), SegmentTestBase::prefetchPage(), SnapshotSegmentTestBase::testAllocateAll(), LinearViewSegmentTest::testAllocateAll(), VersionedSegmentTest::testCheckpoint(), SegmentTestBase::testCheckpoint(), SegPageIterTest::testIter(), SegPageEntryIterTest::testIter(), SegStreamTest::testMarkReset(), SegStreamTest::testReadSeg(), VersionedSegmentTest::testRecovery(), LogicalTxnTest::testTxn(), SnapshotSegmentTest::testUncommittedReads(), SegStreamTest::testWriteSeg(), and VersionedSegmentTest::verifyPage().

(Optional) segment supporting random page allocation.

Definition at line 50 of file SegStorageTestBase.h.

Referenced by SegStorageTestBase::closeRandomSegment(), SnapshotSegmentTestBase::closeStorage(), LbmSplicerExecStreamTest::createBTree(), LbmSplicerExecStreamTest::initBTreeParam(), LbmSearchTest::initBTreeParam(), LbmLoadBitmapTest::initBTreeParam(), LbmLoadBitmapTest::initBTreeTupleDesc(), LcsClusterReplaceExecStreamTest::initClusterAppendParams(), LbmExecStreamTestBase::initSorterExecStream(), LcsClusterReplaceExecStreamTest::loadCluster(), LcsMultiClusterAppendTest::loadClusters(), loadOneCluster(), LbmSearchTest::loadTableAndIndex(), SegStorageTestBase::openRandomSegment(), SnapshotSegmentTestBase::openSegmentStorage(), LinearViewSegmentTest::openSegmentStorage(), SegmentTestBase::openStorage(), SegmentTestBase::testAllocate(), RandomAllocationSegmentTest::testAllocateAndDeallocate(), ExecStreamTestSuite::testBTreeInsertExecStream(), BTreeTest::testBulkLoad(), LhxHashTableTest::testCaseSetUp(), BTreeTest::testCaseSetUp(), BTreeReadersTest::testCaseSetUp(), testCaseSetUp(), LcsMultiClusterAppendTest::testCaseSetUp(), LcsClusterReplaceExecStreamTest::testCaseSetUp(), LcsClusterAppendExecStreamTest::testCaseSetUp(), LbmExecStreamTestBase::testCaseSetUp(), LcsClusterReplaceExecStreamTest::testClusterReplace(), LhxAggExecStreamTest::testCountImpl(), SegmentTestBase::testDeallocate(), LhxAggExecStreamTest::testGroupCountImpl(), LhxJoinExecStreamTest::testImpl(), ExternalSortExecStreamTest::testImpl(), BTreeTest::testInserts(), LbmLoadBitmapTest::testLoad(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), LcsClusterAppendExecStreamTest::testLoadSingleCol(), ExecStreamTestSuite::testMergeImplicitPullInputs(), BTreeTest::testMultiKeySearches(), BTreeReadersTest::testReaders(), testScanOnEmptyCluster(), ExecStreamTestSuite::testSegBufferExecStream(), ExecStreamTestSuite::testSegBufferReaderWriterExecStream(), LhxAggExecStreamTest::testSingleValueImpl(), LhxAggExecStreamTest::testSumImpl(), and LbmUnionExecStreamTest::testUnion().

Cache instance being tested.

Definition at line 63 of file CacheTestBase.h.

Referenced by BackupRestoreTest::backup(), CacheTestBase::closeDevice(), CacheTestBase::closeStorage(), LbmSplicerExecStreamTest::createBTree(), SegStorageTestBase::createLinearDeviceSegment(), LogicalTxnTest::createRecoveryLog(), BackupRestoreTest::createSnapshotData(), BackupRestoreTest::executeSnapshotTxn(), CacheTestBase::getCache(), LbmSearchTest::initBTreeExecStreamParam(), LbmLoadBitmapTest::initBTreeExecStreamParam(), LbmSearchTest::initBTreeParam(), LbmLoadBitmapTest::initBTreeParam(), LcsClusterReplaceExecStreamTest::initClusterAppendParams(), LbmExecStreamTestBase::initSorterExecStream(), LcsClusterReplaceExecStreamTest::loadCluster(), LcsMultiClusterAppendTest::loadClusters(), loadOneCluster(), LbmSearchTest::loadTableAndIndex(), SegmentTestBase::lockPage(), LbmMinusExecStreamTest::newMinusStream(), ExecStreamTestBase::newStreamGraphEmbryo(), CacheTestBase::openDevice(), CacheTestBase::openStorage(), LcsClusterReplaceExecStreamTest::replaceCluster(), BackupRestoreTest::restore(), LbmSplicerExecStreamTest::spliceInput(), SegmentTestBase::testAllocate(), RandomAllocationSegmentTest::testAllocateAndDeallocate(), BackupRestoreTest::testBackupCleanup(), ExecStreamTestSuite::testBTreeInsertExecStream(), BTreeTest::testBulkLoad(), LhxHashTableTest::testCaseSetUp(), ExecStreamUnitTestBase::testCaseSetUp(), BTreeTest::testCaseSetUp(), BTreeReadersTest::testCaseSetUp(), LhxAggExecStreamTest::testCountImpl(), SegmentTestBase::testDeallocate(), SnapshotSegmentTest::testDeallocateOld(), ExecStreamTestSuite::testDoubleBufferExecStream(), ExecStreamGovernorTest::testGovernor(), LhxAggExecStreamTest::testGroupCountImpl(), BackupRestoreTest::testHeaderBackupRestore(), LhxJoinExecStreamTest::testImpl(), ExternalSortExecStreamTest::testImpl(), BTreeTest::testInserts(), LbmIntersectExecStreamTest::testIntersect(), SegPageIterTest::testIter(), SegPageEntryIterTest::testIter(), CacheTest::testLargeCacheInit(), CacheTest::testLargeCacheRequest(), LbmEntryTest::testldb35(), LbmEntryTest::testler5920(), LbmLoadBitmapTest::testLoad(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), LcsClusterAppendExecStreamTest::testLoadSingleCol(), SegStreamTest::testMarkReset(), LbmEntryTest::testMergeEntry(), BTreeTest::testMultiKeySearches(), CacheTest::testQuotaCacheAccessor(), BTreeReadersTest::testReaders(), SegStreamTest::testReadSeg(), BTreeTest::testScan(), testScanOnEmptyCluster(), ExecStreamTestSuite::testScratchBufferExecStream(), LhxAggExecStreamTest::testSingleValueImpl(), FlatFileExecStreamTest::testStream(), LhxAggExecStreamTest::testSumImpl(), LogicalTxnTest::testTxn(), LogicalTxnTest::testTxnIdSequence(), LbmUnionExecStreamTest::testUnion(), SegStreamTest::testWriteSeg(), SegStreamTest::testWriteSpillAndRead(), LbmEntryTest::testZeroBytes(), and BackupRestoreTest::verifySnapshotData().

Size of device in disk pages.

Definition at line 78 of file CacheTestBase.h.

Referenced by CacheTestBase::CacheTestBase(), LinearViewSegmentTest::LinearViewSegmentTest(), CacheTest::makeBlockId(), VersionedSegmentTest::openSegmentStorage(), SnapshotSegmentTestBase::openSegmentStorage(), LinearViewSegmentTest::openSegmentStorage(), CacheTestBase::openStorage(), SegStreamTest::SegStreamTest(), SnapshotSegmentTestBase::SnapshotSegmentTestBase(), SnapshotSegmentTest::testDeallocateOld(), PagingTestBase::testPrefetch(), PagingTestBase::testPrefetchBatch(), PagingTestBase::testRandomOp(), SnapshotSegmentTest::testRollback(), PagingTestBase::testSequentialOp(), and PagingTestBase::testSkipOp().

Configuration parameters.

The reason this is static is so that no constructor parameters (which burden virtual bases) are needed.

Definition at line 155 of file TestBase.h.

Referenced by TestBase::afterTestCase(), TestBase::beforeTestCase(), BTreeTxnTest::BTreeTxnTest(), CacheTestBase::CacheTestBase(), BackupRestoreTest::createSnapshotData(), DatabaseTest::DatabaseTest(), TestOptionsTest::extra(), DatabaseTest::loadDatabase(), SparseBitmapTest::openStorage(), PagingTestBase::PagingTestBase(), ParallelExecStreamSchedulerTest::ParallelExecStreamSchedulerTest(), RandomAccessFileDeviceTest::RandomAccessFileDeviceTest(), TestBase::readParams(), SegStorageTestBase::SegStorageTestBase(), TestOptionsTest::test1(), TestOptionsTest::test2(), BackupRestoreTest::testBackupCleanup(), TestBase::TestBase(), BTreeTxnTest::testCaseSetUp(), BTreeTxnTest::testCheckpoint(), DatabaseTest::testCreateEmpty(), DatabaseTest::testForceTxns(), BackupRestoreTest::testHeaderBackupRestore(), SegPageEntryIterTest::testIter(), SegStreamTest::testRead(), BTreeTxnTest::testTxns(), SegStreamTest::testWrite(), ThreadedTestBase::ThreadedTestBase(), and TestBase::~TestBase().


The documentation for this class was generated from the following file:


Generated on Mon Jun 22 04:00:38 2009 for Fennel by doxygen 1.5.1