Fennel: LcsRowScanExecStreamTest Class Reference (original) (raw)
Testcase for scanning multiple clusters. More...
Inheritance diagram for LcsRowScanExecStreamTest:

| Public Member Functions | |
|---|---|
| LcsRowScanExecStreamTest () | |
| void | testCaseSetUp () |
| Equivalent to JUnit TestCase.setUp; this is called before each test case method is invoked. | |
| void | testCaseTearDown () |
| Equivalent to JUnit TestCase.tearDown; this is called after each test case method is invoked. | |
| void | testScans () |
| void | testScanOnEmptyCluster () |
| Create an empty cluster with 1 column. | |
| void | testScanPastEndOfCluster () |
| Create a cluster with only a single row. | |
| void | testCompressedFiltering () |
| void | testBernoulliSampling () |
| Configure Bernoulli sampling, with a specific seed and verify that the expected number of rows are returned. | |
| void | testSystemSampling () |
| Configure system sampling, with a specific clump size and verify that the expected number of rows are returned. | |
| virtual void | openStorage (DeviceMode openMode) |
| virtual void | openSegmentStorage (DeviceMode openMode) |
| virtual void | openRandomSegment () |
| SharedSegment | createLinearDeviceSegment (DeviceId deviceId, uint nPages) |
| void | closeLinearSegment () |
| void | closeRandomSegment () |
| void | closeVersionedRandomSegment () |
| void | closeSnapshotRandomSegment () |
| virtual void | closeStorage () |
| Cache & | getCache () |
| virtual SharedCache | newCache () |
| SharedRandomAccessDevice | openDevice (std::string devName, DeviceMode openMode, uint nDevicePages, DeviceId deviceId) |
| void | closeDevice (DeviceId deviceId, SharedRandomAccessDevice &pDevice) |
| TestSuite * | releaseTestSuite () |
| void | beforeTestCase (std::string testCaseName) |
| void | afterTestCase (std::string testCaseName) |
| virtual void | notifyTrace (std::string source, TraceLevel level, std::string message) |
| Receives notification when a trace event occurs. | |
| virtual TraceLevel | getSourceTraceLevel (std::string source) |
| Gets the level at which a particular source should be traced. | |
| Static Public Member Functions | |
| static void | readParams (int argc, char **argv) |
| Parses the command line. | |
| Static Public Attributes | |
| static ParamName | paramTestSuiteName |
| static ParamName | paramTraceFileName |
| static ParamName | paramDictionaryFileName |
| static ParamName | paramTraceLevel |
| static ParamName | paramStatsFileName |
| static ParamName | paramTraceStdout |
| static ParamName | paramDegreeOfParallelism |
| static ConfigMap | configMap |
| Configuration parameters. | |
| Protected Types | |
| enum | VictimPolicy { victimTwoQ, victimLRU, victimRandom } |
| The available victim policy implementations. More... | |
| Protected Member Functions | |
| void | loadClusters (uint nRows, uint nCols, uint nClusters, bool compressed) |
| Serially loads nClusters, each cluster containing nCols and nRows. | |
| void | loadOneCluster (uint nRows, uint nCols, int colStart, BTreeDescriptor &bTreeDescriptor, bool compressed) |
| Loads a single cluster with nCols columns and nRows rows. | |
| void | testScanCols (uint nRows, uint nCols, uint nClusters, TupleProjection proj, uint skipRows, uint expectedNumRows) |
| Reads rows from clusters. | |
| void | testFilterCols (uint nRows, uint nCols, uint nClusters, TupleProjection proj, uint skipRows, uint expectedNumRows, bool compressed) |
| Filter rows from clusters. | |
| void | setSearchKey (char lowerDirective, char upperDirective, uint64_t lowerVal, uint64_t upperVal, PBuffer inputBuf, uint &offset, TupleAccessor &inputTupleAccessor, TupleData &inputTupleData) |
| void | testSampleScanCols (uint nRows, uint nRowsActual, uint nCols, uint nClusters, TupleProjection proj, uint skipRows, TableSamplingMode mode, float rate, int seed, uint clumps, uint expectedNumRows) |
| Sample rows from clusters. | |
| int | generateBitmaps (uint nRows, uint skipRows, TupleDescriptor const &bitmapTupleDesc, PBuffer pBuf) |
| Generate bitmaps to pass as input into row scan exec stream. | |
| void | produceEntry (LbmEntry &lbmEntry, TupleAccessor &bitmapTupleAccessor, PBuffer pBuf, int &bufSize) |
| SharedExecStream | prepareSourceGraph (ExecStreamEmbryo &sourceStreamEmbryo) |
| Defines and prepares a graph consisting of one source stream. | |
| SharedExecStream | prepareTransformGraph (ExecStreamEmbryo &sourceStreamEmbryo, ExecStreamEmbryo &transformStreamEmbryo) |
| Defines and prepares a graph consisting of one source stream and one transform stream. | |
| SharedExecStream | prepareTransformGraph (ExecStreamEmbryo &sourceStreamEmbryo, std::vector< ExecStreamEmbryo > &transforms) |
| Defines and prepares a graph consisting of one source stream and one or multiple transform streams. | |
| SharedExecStream | prepareConfluenceGraph (ExecStreamEmbryo &sourceStreamEmbryo1, ExecStreamEmbryo &sourceStreamEmbryo2, ExecStreamEmbryo &confluenceStreamEmbryo) |
| Defines and prepares a graph consisting of two source streams and one confluence stream. | |
| SharedExecStream | prepareConfluenceGraph (std::vector< ExecStreamEmbryo > &sourceStreamEmbryos, ExecStreamEmbryo &confluenceStreamEmbryo) |
| Defines and prepares a graph consisting of a list of source streams and one confluence stream. | |
| SharedExecStream | prepareConfluenceGraph (std::vector< std::vector< ExecStreamEmbryo > > &sourceStreamEmbryosList, ExecStreamEmbryo &confluenceStreamEmbryo) |
| Defines and prepares a graph consisting of one or more source streams and one confluence stream. | |
| SharedExecStream | prepareConfluenceTransformGraph (ExecStreamEmbryo &sourceStreamEmbryo1, ExecStreamEmbryo &sourceStreamEmbryo2, ExecStreamEmbryo &confluenceStreamEmbryo, ExecStreamEmbryo &transformStreamEmbryo) |
| Defines and prepares a graph consisting of two source streams, one confluence stream, and one transform stream. | |
| SharedExecStream | prepareDAG (ExecStreamEmbryo &srcStreamEmbryo, ExecStreamEmbryo &splitterStreamEmbryo, std::vector< ExecStreamEmbryo > &interStreamEmbryos, ExecStreamEmbryo &destStreamEmbryo, bool createSink=true, bool saveSrc=true) |
| Defines and prepares a graph consisting of a source, a splitter, and one or more parallel transform streams which flow together into a confluence stream. | |
| SharedExecStream | prepareDAG (ExecStreamEmbryo &srcStreamEmbryo, ExecStreamEmbryo &splitterStreamEmbryo, std::vector< std::vector< ExecStreamEmbryo > > &interStreamEmbryosList, ExecStreamEmbryo &destStreamEmbryo, bool createSink=true, bool saveSrc=true) |
| Defines and prepares a graph consisting of a source, a splitter, and one or more parallel transform streams which flow together into a confluence stream. | |
| void | verifyOutput (ExecStream &stream, uint nRowsExpected, MockProducerExecStreamGenerator &verifier, bool stopEarly=false) |
| Executes the prepared stream graph and verifies that its output matches that produced by a value generator. | |
| void | verifyConstantOutput (ExecStream &stream, const TupleData &expectedTuple, uint nRowsExpected) |
| Executes the prepared stream graph and verifies that all output tuples matche an expected and given one. | |
| void | verifyBufferedOutput (ExecStream &stream, TupleDescriptor outputTupleDesc, uint nRowsExpected, PBuffer expectedBuffer) |
| Executes the prepared stream graph and verifies the resultant tuples against a set of tuples supplied in an input buffer. | |
| void | resetExecStreamTest () |
| Reset stream graph so multiple iterations of a method can be called within a single testcase. | |
| virtual void | tearDownExecStreamTest () |
| ExecStream-specific handler called from testCaseTearDown. | |
| virtual SharedExecStreamGraph | newStreamGraph () |
| Creates a stream graph. | |
| virtual SharedExecStreamGraphEmbryo | newStreamGraphEmbryo (SharedExecStreamGraph) |
| Creates an embryo for a stream graph. | |
| virtual ExecStreamScheduler * | newScheduler () |
| Creates a scheduler. | |
| virtual ExecStreamGovernor * | newResourceGovernor (ExecStreamResourceKnobs const &knobSettings, ExecStreamResourceQuantity const &resourcesAvailable) |
| Creates the resource governor. | |
| void | snooze (uint nSeconds) |
| Protected Attributes | |
| StandardTypeDescriptorFactory | stdTypeFactory |
| TupleAttributeDescriptor | attrDesc_int64 |
| TupleAttributeDescriptor | attrDesc_bitmap |
| TupleAttributeDescriptor | attrDesc_char1 |
| uint | bitmapColSize |
| vector< boost::shared_ptr< BTreeDescriptor > > | bTreeClusters |
| SharedExecStreamGraph | pGraph |
| SharedExecStreamGraphEmbryo | pGraphEmbryo |
| SharedExecStreamScheduler | pScheduler |
| SharedExecStreamGovernor | pResourceGovernor |
| SharedCacheAccessor | pCacheAccessor |
| SharedSegmentFactory | pSegmentFactory |
| SharedSegment | pLinearSegment |
| Segment supporting linear page allocation. | |
| SharedSegment | pRandomSegment |
| (Optional) segment supporting random page allocation. | |
| SharedSegment | pVersionedRandomSegment |
| (Optional) segment supporting versioned random page allocation. | |
| SharedSegment | pSnapshotRandomSegment |
| (Optional) segment supporting snapshot random page allocation. | |
| VictimPolicy | victimPolicy |
| VictimPolicy to instantiate. | |
| CacheParams | cacheParams |
| Parameters for cache initialization. | |
| SharedCache | pCache |
| Cache instance being tested. | |
| SharedRandomAccessDevice | pRandomAccessDevice |
| The default cached device. | |
| uint | nMemPages |
| Size of cache in memory pages. | |
| uint | nDiskPages |
| Size of device in disk pages. | |
| uint | cbPageFull |
| Disk page size. | |
| DeviceId | dataDeviceId |
| Fixed ID to assign to data device. | |
| TestSuite * | pTestSuite |
| Boost test suite. | |
| boost::shared_ptr< TestBase > | pTestObj |
| std::ofstream | traceStream |
| Output file stream for tracing. | |
| StrictMutex | traceMutex |
| Protects traceStream. | |
| std::string | testName |
| Name of test. | |
| TraceLevel | traceLevel |
| Level at which to trace test execution. | |
| FileStatsTarget | statsTarget |
| Output for stats. | |
| StatsTimer | statsTimer |
| Timer for stats collection. | |
| bool | traceStdout |
| Copy trace output to stdout. | |
| bool | traceFile |
| Copy trace output to file. | |
| TestCaseGroup | defaultTests |
| TestCaseGroup | extraTests |
| Static Protected Attributes | |
| static const uint | NDUPS = 20 |
| static const uint | DefaultCacheReservePercent = 5 |
| static const uint | DefaultConcurrentStatements = 4 |
| static bool | runAll |
| Run all test cases, including the extra tests. | |
| static std::string | runSingle |
| Run only the test case of this name. |
Detailed Description
Testcase for scanning multiple clusters.
Note that LcsClusterAppendExecStreamTest also has some tests for scans, but those only test single cluster scans
Definition at line 51 of file LcsRowScanExecStreamTest.cpp.
Member Enumeration Documentation
The available victim policy implementations.
Enumerator:
| victimTwoQ | | | -------------- | | | victimLRU | | | victimRandom | |
Definition at line 44 of file CacheTestBase.h.
Constructor & Destructor Documentation
| LcsRowScanExecStreamTest::LcsRowScanExecStreamTest | ( | | ) | [inline, explicit] | | -------------------------------------------------- | - | | - | -------------------- |
Member Function Documentation
| void LcsRowScanExecStreamTest::loadClusters | ( | uint | nRows, |
|---|---|---|---|
| uint | nCols, | ||
| uint | nClusters, | ||
| bool | compressed | ||
| ) | [protected] |
| void LcsRowScanExecStreamTest::loadOneCluster | ( | uint | nRows, |
|---|---|---|---|
| uint | nCols, | ||
| int | colStart, | ||
| BTreeDescriptor & | bTreeDescriptor, | ||
| bool | compressed | ||
| ) | [protected] |
Loads a single cluster with nCols columns and nRows rows.
Each column will have a different sequence of values, as follows: column0 - colStart, colStart+1, ..., colStart+nRows-1 column1 - colStart+1, colStart+2, ..., colStart+nRows column2 - colStart+2, colStart+3, ..., colStart+nRows+1 ...
Definition at line 256 of file LcsRowScanExecStreamTest.cpp.
References attrDesc_int64, BTreeBuilder::createEmptyRoot(), BTreeAccessBase::getRootPageId(), ExecStreamEmbryo::getStream(), ExecStreamEmbryo::init(), LcsClusterAppendExecStreamParams::inputProj, BTreeParams::keyProj, BTreeDescriptor::keyProjection, NDUPS, MockProducerExecStreamParams::nRows, NULL_PAGE_ID, SingleOutputExecStreamParams::outputTupleDesc, BTreeDescriptor::pageOwnerId, BTreeParams::pageOwnerId, CacheTestBase::pCache, SegmentAccessor::pCacheAccessor, ExecStreamParams::pCacheAccessor, MockProducerExecStreamParams::pGenerator, SegStorageTestBase::pRandomSegment, ExecStreamUnitTestBase::prepareTransformGraph(), BTreeParams::pRootMap, SegmentAccessor::pSegment, BTreeParams::pSegment, SegStorageTestBase::pSegmentFactory, BTreeParams::rootPageId, BTreeDescriptor::rootPageId, BTreeParams::rootPageIdParamId, ExecStreamParams::scratchAccessor, BTreeDescriptor::segmentAccessor, BTreeDescriptor::segmentId, BTreeParams::segmentId, BTreeParams::tupleDesc, BTreeDescriptor::tupleDescriptor, and ExecStreamUnitTestBase::verifyOutput().
Referenced by loadClusters(), and testScanPastEndOfCluster().
Reads rows from clusters.
Assumes clusters have been loaded by loadClusters/loadOneCluster.
Parameters:
| nRows | total number of rows in the clusters |
|---|---|
| nCols | number of columns in each cluster |
| nClusters | number of clusters |
| proj | columns to be projected |
| skipRows | read every "skipRows" rows |
| expectedNumRows | expected number of rows in scan result; normally should be the same as nRows unless skipping rows or testing exception cases |
Definition at line 343 of file LcsRowScanExecStreamTest.cpp.
References attrDesc_bitmap, attrDesc_int64, bitmapColSize, bTreeClusters, ValuesExecStreamParams::bufSize, FixedBuffer, generateBitmaps(), ExecStreamEmbryo::getStream(), LcsRowScanExecStreamParams::hasExtraFilter, ExecStreamEmbryo::init(), LcsRowScanExecStreamParams::isFullScan, LcsRowScanBaseExecStreamParams::lcsClusterScanDefs, max(), LcsRowScanBaseExecStreamParams::outputProj, SingleOutputExecStreamParams::outputTupleDesc, ExecStreamUnitTestBase::prepareTransformGraph(), ValuesExecStreamParams::pTupleBuffer, SAMPLING_OFF, LcsRowScanExecStreamParams::samplingMode, and ExecStreamUnitTestBase::verifyOutput().
Referenced by testScanOnEmptyCluster(), testScanPastEndOfCluster(), and testScans().
00350 {
00351
00352
00353 ValuesExecStreamParams valuesParams;
00354 boost::shared_array pBuffer;
00355 ExecStreamEmbryo valuesStreamEmbryo;
00356 LcsRowScanExecStreamParams scanParams;
00357
00358 scanParams.hasExtraFilter = false;
00359 scanParams.samplingMode = SAMPLING_OFF;
00360
00361
00362
00363
00364 valuesParams.outputTupleDesc.push_back(attrDesc_int64);
00365 valuesParams.outputTupleDesc.push_back(attrDesc_bitmap);
00366 valuesParams.outputTupleDesc.push_back(attrDesc_bitmap);
00367
00368
00369
00370
00371 uint bufferSize = std::max(
00372 16, (int) (nRows / 8 + nRows / bitmapColSize * 8));
00373 pBuffer.reset(new FixedBuffer[bufferSize]);
00374 valuesParams.pTupleBuffer = pBuffer;
00375
00376 if (nRows > 0) {
00377 valuesParams.bufSize = generateBitmaps(
00378 nRows, skipRows, valuesParams.outputTupleDesc, pBuffer.get());
00379 assert(valuesParams.bufSize <= bufferSize);
00380 scanParams.isFullScan = false;
00381 } else {
00382 scanParams.isFullScan = true;
00383 valuesParams.bufSize = 0;
00384 }
00385 valuesStreamEmbryo.init(new ValuesExecStream(), valuesParams);
00386 valuesStreamEmbryo.getStream()->setName("ValuesExecStream");
00387
00388
00389
00390
00391 for (uint i = 0; i < nClusters; i++) {
00392 struct LcsClusterScanDef clusterScanDef;
00393
00394 for (uint j = 0; j < nCols; j++) {
00395 clusterScanDef.clusterTupleDesc.push_back(attrDesc_int64);
00396 }
00397
00398 clusterScanDef.pSegment = bTreeClusters[i]->segmentAccessor.pSegment;
00399 clusterScanDef.pCacheAccessor =
00400 bTreeClusters[i]->segmentAccessor.pCacheAccessor;
00401 clusterScanDef.tupleDesc = bTreeClusters[i]->tupleDescriptor;
00402 clusterScanDef.keyProj = bTreeClusters[i]->keyProjection;
00403 clusterScanDef.rootPageId = bTreeClusters[i]->rootPageId;
00404 clusterScanDef.segmentId = bTreeClusters[i]->segmentId;
00405 clusterScanDef.pageOwnerId = bTreeClusters[i]->pageOwnerId;
00406
00407 scanParams.lcsClusterScanDefs.push_back(clusterScanDef);
00408 }
00409
00410
00411 scanParams.outputProj = proj;
00412 for (uint i = 0; i < proj.size(); i++) {
00413 scanParams.outputTupleDesc.push_back(attrDesc_int64);
00414 }
00415
00416 ExecStreamEmbryo scanStreamEmbryo;
00417 scanStreamEmbryo.init(new LcsRowScanExecStream(), scanParams);
00418 scanStreamEmbryo.getStream()->setName("RowScanExecStream");
00419 SharedExecStream pOutputStream;
00420
00421 pOutputStream =
00422 prepareTransformGraph(valuesStreamEmbryo, scanStreamEmbryo);
00423
00424
00425
00426 vector<boost::shared_ptr<ColumnGenerator > > columnGenerators;
00427 for (uint i = 0; i < proj.size(); i++) {
00428 SharedInt64ColumnGenerator col =
00429 SharedInt64ColumnGenerator(
00430 new SeqColumnGenerator(
00431 proj[i],
00432 skipRows));
00433 columnGenerators.push_back(col);
00434 }
00435
00436 CompositeExecStreamGenerator resultGenerator(columnGenerators);
00437 verifyOutput(*pOutputStream, expectedNumRows, resultGenerator);
00438 }
| void LcsRowScanExecStreamTest::testFilterCols | ( | uint | nRows, |
|---|---|---|---|
| uint | nCols, | ||
| uint | nClusters, | ||
| TupleProjection | proj, | ||
| uint | skipRows, | ||
| uint | expectedNumRows, | ||
| bool | compressed | ||
| ) | [protected] |
Filter rows from clusters.
Assumes clusters have been loaded by loadClusters/loadOneCluster.
Parameters:
| nRows | total number of rows in the clusters |
|---|---|
| nCols | number of columns in each cluster |
| nClusters | number of clusters |
| proj | columns to be projected |
| skipRows | read every "skipRows" rows |
| expectedNumRows | expected number of rows in scan result; normally should be the same as nRows unless skipping rows or testing exception cases |
| compressed | testing compressed bitmap optimization |
Definition at line 772 of file LcsRowScanExecStreamTest.cpp.
References attrDesc_bitmap, attrDesc_char1, attrDesc_int64, bitmapColSize, bTreeClusters, ValuesExecStreamParams::bufSize, TupleAccessor::compute(), FixedBuffer, generateBitmaps(), TupleAccessor::getMaxByteCount(), ExecStreamEmbryo::getStream(), LcsRowScanExecStreamParams::hasExtraFilter, ExecStreamEmbryo::init(), LcsRowScanExecStreamParams::isFullScan, LcsRowScanBaseExecStreamParams::lcsClusterScanDefs, max(), NDUPS, StandardTypeDescriptorFactory::newDataType(), LcsRowScanBaseExecStreamParams::outputProj, SingleOutputExecStreamParams::outputTupleDesc, ExecStreamUnitTestBase::prepareConfluenceGraph(), ValuesExecStreamParams::pTupleBuffer, LcsRowScanExecStreamParams::residualFilterCols, SAMPLING_OFF, LcsRowScanExecStreamParams::samplingMode, setSearchKey(), STANDARD_TYPE_INT_64, stdTypeFactory, and ExecStreamUnitTestBase::verifyOutput().
Referenced by testCompressedFiltering(), and testScans().
00780 {
00781
00782
00783 ValuesExecStreamParams valuesParams;
00784 boost::shared_array pBuffer;
00785 ExecStreamEmbryo valuesStreamEmbryo;
00786 LcsRowScanExecStreamParams scanParams;
00787
00788 scanParams.hasExtraFilter = true;
00789 scanParams.samplingMode = SAMPLING_OFF;
00790
00791
00792
00793
00794 valuesParams.outputTupleDesc.push_back(attrDesc_int64);
00795 valuesParams.outputTupleDesc.push_back(attrDesc_bitmap);
00796 valuesParams.outputTupleDesc.push_back(attrDesc_bitmap);
00797
00798
00799
00800
00801
00802 uint bufferSize = std::max(
00803 16, (int) (nRows / 8 + nRows / bitmapColSize * 8));
00804 pBuffer.reset(new FixedBuffer[bufferSize]);
00805 valuesParams.pTupleBuffer = pBuffer;
00806
00807 if (nRows > 0) {
00808 valuesParams.bufSize = generateBitmaps(
00809 nRows, skipRows, valuesParams.outputTupleDesc, pBuffer.get());
00810 assert(valuesParams.bufSize <= bufferSize);
00811 scanParams.isFullScan = false;
00812 } else {
00813 scanParams.isFullScan = true;
00814 valuesParams.bufSize = 0;
00815 }
00816 valuesStreamEmbryo.init(new ValuesExecStream(), valuesParams);
00817 valuesStreamEmbryo.getStream()->setName("ValuesExecStream");
00818
00819
00820
00821
00822
00823
00824
00825
00826
00827
00828 TupleAttributeDescriptor attrDesc_nullableInt64 =
00829 TupleAttributeDescriptor(
00830 stdTypeFactory.newDataType(STANDARD_TYPE_INT_64),
00831 true, sizeof(uint64_t));
00832
00833 valuesParams.outputTupleDesc.resize(0);
00834 TupleDescriptor inputTupleDesc;
00835 for (uint i = 0; i < 2; i++) {
00836 inputTupleDesc.push_back(attrDesc_char1);
00837 inputTupleDesc.push_back(attrDesc_nullableInt64);
00838 valuesParams.outputTupleDesc.push_back(attrDesc_char1);
00839 valuesParams.outputTupleDesc.push_back(attrDesc_nullableInt64);
00840 }
00841 TupleData inputTupleData(inputTupleDesc);
00842 TupleAccessor inputTupleAccessor;
00843 inputTupleAccessor.compute(inputTupleDesc);
00844
00845 uint nInputTuples = 3;
00846 boost::shared_array inputBuffer;
00847 inputBuffer.reset(
00848 new FixedBuffer[nInputTuples * inputTupleAccessor.getMaxByteCount()]);
00849
00850 PBuffer inputBuf = inputBuffer.get();
00851 uint offset = 0;
00852
00853 setSearchKey(
00854 '-', ')', 0, 1000, inputBuf, offset, inputTupleAccessor,
00855 inputTupleData);
00856 setSearchKey(
00857 '[', '+', 2000, 0, inputBuf, offset, inputTupleAccessor,
00858 inputTupleData);
00859
00860 TupleData inputTupleData1(inputTupleDesc);
00861 boost::shared_array inputBuffer1;
00862 inputBuffer1.reset(
00863 new FixedBuffer[nInputTuples * inputTupleAccessor.getMaxByteCount()]);
00864 PBuffer inputBuf1 = inputBuffer1.get();
00865 uint offset1 = 0;
00866
00867 setSearchKey(
00868 '[', ')', 500 + nCols, 2999 + nCols, inputBuf1, offset1,
00869 inputTupleAccessor,
00870 inputTupleData1);
00871 setSearchKey(
00872 '[', ']', 2999 + nCols, 2999 + nCols, inputBuf1, offset1,
00873 inputTupleAccessor, inputTupleData1);
00874
00875 TupleData inputTupleData2(inputTupleDesc);
00876 boost::shared_array inputBuffer2;
00877 inputBuffer2.reset(
00878 new FixedBuffer[nInputTuples * inputTupleAccessor.getMaxByteCount()]);
00879 PBuffer inputBuf2 = inputBuffer2.get();
00880 uint offset2 = 0;
00881
00882 setSearchKey(
00883 '(', '+', 1500+2nCols, 0, inputBuf2, offset2, inputTupleAccessor,
00884 inputTupleData1);
00885
00886 valuesParams.pTupleBuffer = inputBuffer;
00887 valuesParams.bufSize = offset;
00888
00889 ExecStreamEmbryo valuesStreamEmbryo1, valuesStreamEmbryo2,
00890 valuesStreamEmbryo3;
00891 valuesStreamEmbryo1.init(new ValuesExecStream(), valuesParams);
00892 valuesStreamEmbryo1.getStream()->setName("ValuesExecStream1");
00893
00894 valuesParams.pTupleBuffer = inputBuffer1;
00895 valuesParams.bufSize = offset1;
00896 valuesStreamEmbryo2.init(new ValuesExecStream(), valuesParams);
00897 valuesStreamEmbryo2.getStream()->setName("ValuesExecStream2");
00898
00899 valuesParams.pTupleBuffer = inputBuffer2;
00900 valuesParams.bufSize = offset2;
00901 valuesStreamEmbryo3.init(new ValuesExecStream(), valuesParams);
00902 valuesStreamEmbryo3.getStream()->setName("ValuesExecStream3");
00903
00904
00905
00906
00907 for (uint i = 0; i < nClusters; i++) {
00908 struct LcsClusterScanDef clusterScanDef;
00909
00910 for (uint j = 0; j < nCols; j++) {
00911 clusterScanDef.clusterTupleDesc.push_back(attrDesc_int64);
00912 }
00913
00914 clusterScanDef.pSegment = bTreeClusters[i]->segmentAccessor.pSegment;
00915 clusterScanDef.pCacheAccessor =
00916 bTreeClusters[i]->segmentAccessor.pCacheAccessor;
00917 clusterScanDef.tupleDesc = bTreeClusters[i]->tupleDescriptor;
00918 clusterScanDef.keyProj = bTreeClusters[i]->keyProjection;
00919 clusterScanDef.rootPageId = bTreeClusters[i]->rootPageId;
00920 clusterScanDef.segmentId = bTreeClusters[i]->segmentId;
00921 clusterScanDef.pageOwnerId = bTreeClusters[i]->pageOwnerId;
00922
00923 scanParams.lcsClusterScanDefs.push_back(clusterScanDef);
00924 }
00925
00926
00927 scanParams.outputProj = proj;
00928 for (uint i = 0; i < proj.size(); i++) {
00929 scanParams.outputTupleDesc.push_back(attrDesc_int64);
00930 }
00931 scanParams.residualFilterCols.push_back(0);
00932 scanParams.residualFilterCols.push_back(nCols);
00933 scanParams.residualFilterCols.push_back(2*nCols);
00934
00935 ExecStreamEmbryo scanStreamEmbryo;
00936 scanStreamEmbryo.init(new LcsRowScanExecStream(), scanParams);
00937 scanStreamEmbryo.getStream()->setName("RowScanExecStream");
00938 SharedExecStream pOutputStream;
00939
00940 std::vector sources;
00941 sources.push_back(valuesStreamEmbryo);
00942 sources.push_back(valuesStreamEmbryo1);
00943 sources.push_back(valuesStreamEmbryo2);
00944 sources.push_back(valuesStreamEmbryo3);
00945
00946 pOutputStream =
00947 prepareConfluenceGraph(sources, scanStreamEmbryo);
00948
00949
00950
00951 vector<boost::shared_ptr<ColumnGenerator > > columnGenerators;
00952 offset = (int) ceil(2000.0 / skipRows) * skipRows;
00953 for (uint i = 0; i < proj.size(); i++) {
00954 SharedInt64ColumnGenerator col =
00955 SharedInt64ColumnGenerator(
00956 compressed ?
00957 (Int64ColumnGenerator) new MixedDupColumnGenerator(
00958 NDUPS, proj[i] + 2000,500) :
00959 new SeqColumnGenerator(proj[i] + offset, skipRows));
00960 columnGenerators.push_back(col);
00961 }
00962
00963
00964 CompositeExecStreamGenerator resultGenerator(columnGenerators);
00965 verifyOutput(*pOutputStream, expectedNumRows, resultGenerator);
00966 }
| void LcsRowScanExecStreamTest::testSampleScanCols | ( | uint | nRows, |
|---|---|---|---|
| uint | nRowsActual, | ||
| uint | nCols, | ||
| uint | nClusters, | ||
| TupleProjection | proj, | ||
| uint | skipRows, | ||
| TableSamplingMode | mode, | ||
| float | rate, | ||
| int | seed, | ||
| uint | clumps, | ||
| uint | expectedNumRows | ||
| ) | [protected] |
Sample rows from clusters.
Assumes clusters have been loaded by loadClusters/loadOneCluster.
Parameters:
| nRows | total number of rows in the clusters; if set to 0, full table scans should be done |
|---|---|
| nRowsActual | actual total number of rows in the clusters |
| nCols | number of columns in each cluster |
| nClusters | number of clusters |
| proj | columns to be projected |
| skipRows | read every "skipRows" rows |
| mode | sampling mode (SAMPLING_BERNOULLI or SAMPLING_SYSTEM) |
| rate | sampling rate |
| seed | Bernoulli sampling RNG seed |
| clumps | number of system sampling clumps |
| expectedNumRows | expected number of rows in scan result; normally should be the same as nRows unless skipping rows or testing exception cases |
Definition at line 969 of file LcsRowScanExecStreamTest.cpp.
References attrDesc_bitmap, attrDesc_int64, bitmapColSize, bTreeClusters, ValuesExecStreamParams::bufSize, FixedBuffer, generateBitmaps(), ExecStreamEmbryo::getStream(), LcsRowScanExecStreamParams::hasExtraFilter, ExecStreamEmbryo::init(), LcsRowScanExecStreamParams::isFullScan, LcsRowScanBaseExecStreamParams::lcsClusterScanDefs, max(), LcsRowScanBaseExecStreamParams::outputProj, SingleOutputExecStreamParams::outputTupleDesc, ExecStreamUnitTestBase::prepareTransformGraph(), ValuesExecStreamParams::pTupleBuffer, SAMPLING_BERNOULLI, SAMPLING_SYSTEM, LcsRowScanExecStreamParams::samplingClumps, LcsRowScanExecStreamParams::samplingIsRepeatable, LcsRowScanExecStreamParams::samplingMode, LcsRowScanExecStreamParams::samplingRate, LcsRowScanExecStreamParams::samplingRepeatableSeed, LcsRowScanExecStreamParams::samplingRowCount, and ExecStreamUnitTestBase::verifyOutput().
Referenced by testBernoulliSampling(), and testSystemSampling().
00981 {
00982
00983
00984 ValuesExecStreamParams valuesParams;
00985 boost::shared_array pBuffer;
00986 ExecStreamEmbryo valuesStreamEmbryo;
00987 LcsRowScanExecStreamParams scanParams;
00988
00989 scanParams.hasExtraFilter = false;
00990
00991
00992
00993
00994 valuesParams.outputTupleDesc.push_back(attrDesc_int64);
00995 valuesParams.outputTupleDesc.push_back(attrDesc_bitmap);
00996 valuesParams.outputTupleDesc.push_back(attrDesc_bitmap);
00997
00998 uint nRowsInternal = (mode == SAMPLING_SYSTEM) ? 0 : nRows;
00999
01000
01001
01002
01003
01004 uint bufferSize = std::max(
01005 16, (int) (nRowsInternal / 8 + nRowsInternal / bitmapColSize * 8));
01006 pBuffer.reset(new FixedBuffer[bufferSize]);
01007 valuesParams.pTupleBuffer = pBuffer;
01008
01009 if (nRowsInternal > 0) {
01010 valuesParams.bufSize = generateBitmaps(
01011 nRowsInternal, skipRows, valuesParams.outputTupleDesc,
01012 pBuffer.get());
01013 assert(valuesParams.bufSize <= bufferSize);
01014 scanParams.isFullScan = false;
01015 } else {
01016 scanParams.isFullScan = true;
01017 valuesParams.bufSize = 0;
01018 }
01019 valuesStreamEmbryo.init(new ValuesExecStream(), valuesParams);
01020 valuesStreamEmbryo.getStream()->setName("ValuesExecStream");
01021
01022
01023
01024
01025 for (uint i = 0; i < nClusters; i++) {
01026 struct LcsClusterScanDef clusterScanDef;
01027
01028 for (uint j = 0; j < nCols; j++) {
01029 clusterScanDef.clusterTupleDesc.push_back(attrDesc_int64);
01030 }
01031
01032 clusterScanDef.pSegment = bTreeClusters[i]->segmentAccessor.pSegment;
01033 clusterScanDef.pCacheAccessor =
01034 bTreeClusters[i]->segmentAccessor.pCacheAccessor;
01035 clusterScanDef.tupleDesc = bTreeClusters[i]->tupleDescriptor;
01036 clusterScanDef.keyProj = bTreeClusters[i]->keyProjection;
01037 clusterScanDef.rootPageId = bTreeClusters[i]->rootPageId;
01038 clusterScanDef.segmentId = bTreeClusters[i]->segmentId;
01039 clusterScanDef.pageOwnerId = bTreeClusters[i]->pageOwnerId;
01040
01041 scanParams.lcsClusterScanDefs.push_back(clusterScanDef);
01042 }
01043
01044
01045 scanParams.outputProj = proj;
01046 for (uint i = 0; i < proj.size(); i++) {
01047 scanParams.outputTupleDesc.push_back(attrDesc_int64);
01048 }
01049
01050
01051
01052 scanParams.samplingMode = mode;
01053 scanParams.samplingRate = rate;
01054 scanParams.samplingIsRepeatable = true;
01055 scanParams.samplingRepeatableSeed = seed;
01056 scanParams.samplingClumps = clumps;
01057 scanParams.samplingRowCount = nRowsActual;
01058
01059 ExecStreamEmbryo scanStreamEmbryo;
01060 scanStreamEmbryo.init(new LcsRowScanExecStream(), scanParams);
01061 scanStreamEmbryo.getStream()->setName("RowScanExecStream");
01062 SharedExecStream pOutputStream;
01063
01064 pOutputStream =
01065 prepareTransformGraph(valuesStreamEmbryo, scanStreamEmbryo);
01066
01067
01068
01069 vector<boost::shared_ptr<ColumnGenerator > > columnGenerators;
01070 for (uint i = 0; i < proj.size(); i++) {
01071 SharedInt64ColumnGenerator col =
01072 SharedInt64ColumnGenerator(
01073 new SeqColumnGenerator(
01074 proj[i],
01075 skipRows));
01076 columnGenerators.push_back(col);
01077 }
01078
01079 boost::shared_ptr baseResultGenerator(
01080 new CompositeExecStreamGenerator(columnGenerators));
01081
01082 if (mode == SAMPLING_BERNOULLI) {
01083 BernoulliSamplingExecStreamGenerator resultGenerator(
01084 baseResultGenerator,
01085 rate,
01086 seed,
01087 proj.size());
01088
01089 verifyOutput(*pOutputStream, expectedNumRows, resultGenerator);
01090 } else {
01091 SystemSamplingExecStreamGenerator resultGenerator(
01092 baseResultGenerator,
01093 rate,
01094 nRows,
01095 proj.size(),
01096 clumps);
01097
01098 verifyOutput(*pOutputStream, expectedNumRows, resultGenerator);
01099 }
01100 }
Generate bitmaps to pass as input into row scan exec stream.
Parameters:
| nRows | number of rows in table |
|---|---|
| skipRows | generate rids every "skipRows" rows; i.e., if skipRows == 1, there are no gaps in the rids |
| bitmapTupleDesc | tuple descriptor for bitmap segment |
| pBuf | buffer where bitmap segment tuples will be marshalled |
Returns:
size of the buffer containing the marshalled tuples
Definition at line 440 of file LcsRowScanExecStreamTest.cpp.
References bitmapColSize, TupleAccessor::compute(), FixedBuffer, LbmEntry::getScratchBufferSize(), LbmEntry::init(), produceEntry(), LbmEntry::setEntryTuple(), and LbmEntry::setRID().
Referenced by testFilterCols(), testSampleScanCols(), and testScanCols().
00443 {
00444 int bufSize = 0;
00445 LbmEntry lbmEntry;
00446 boost::scoped_array entryBuf;
00447 TupleAccessor bitmapTupleAccessor;
00448 LcsRid rid = LcsRid(0);
00449
00450 TupleData bitmapTupleData(bitmapTupleDesc);
00451 bitmapTupleData[0].pData = (PConstBuffer) &rid;
00452 bitmapTupleData[1].pData = NULL;
00453 bitmapTupleData[1].cbData = 0;
00454 bitmapTupleData[2].pData = NULL;
00455 bitmapTupleData[2].cbData = 0;
00456
00457 bitmapTupleAccessor.compute(bitmapTupleDesc);
00458
00459
00460 uint scratchBufSize = LbmEntry::getScratchBufferSize(bitmapColSize);
00461 entryBuf.reset(new FixedBuffer[scratchBufSize]);
00462 lbmEntry.init(entryBuf.get(), NULL, scratchBufSize, bitmapTupleDesc);
00463 lbmEntry.setEntryTuple(bitmapTupleData);
00464
00465
00466 for (rid = LcsRid(skipRows); rid < LcsRid(nRows); rid += skipRows) {
00467 if (!lbmEntry.setRID(LcsRid(rid))) {
00468
00469
00470 produceEntry(lbmEntry, bitmapTupleAccessor, pBuf, bufSize);
00471 lbmEntry.setEntryTuple(bitmapTupleData);
00472 }
00473 }
00474
00475 produceEntry(lbmEntry, bitmapTupleAccessor, pBuf, bufSize);
00476
00477 return bufSize;
00478 }
| void LcsRowScanExecStreamTest::produceEntry | ( | LbmEntry & | lbmEntry, |
|---|---|---|---|
| TupleAccessor & | bitmapTupleAccessor, | ||
| PBuffer | pBuf, | ||
| int & | bufSize | ||
| ) | [protected] |
| void LcsRowScanExecStreamTest::testCaseSetUp | ( | | ) | [virtual] | | -------------------------------------------- | - | | - | ----------- |
Equivalent to JUnit TestCase.setUp; this is called before each test case method is invoked.
Default is no-op.
Reimplemented from ExecStreamUnitTestBase.
Definition at line 1102 of file LcsRowScanExecStreamTest.cpp.
References attrDesc_bitmap, attrDesc_char1, attrDesc_int64, bitmapColSize, StandardTypeDescriptorFactory::newDataType(), SegStorageTestBase::pRandomSegment, STANDARD_TYPE_CHAR, STANDARD_TYPE_INT_64, STANDARD_TYPE_VARBINARY, stdTypeFactory, and ExecStreamUnitTestBase::testCaseSetUp().
| void LcsRowScanExecStreamTest::testCaseTearDown | ( | | ) | [virtual] | | ----------------------------------------------- | - | | - | ----------- |
| void LcsRowScanExecStreamTest::testScans | ( | | ) | | ---------------------------------------- | - | | - |
Definition at line 489 of file LcsRowScanExecStreamTest.cpp.
References loadClusters(), ExecStreamUnitTestBase::resetExecStreamTest(), testFilterCols(), and testScanCols().
Referenced by LcsRowScanExecStreamTest().
00490 {
00491
00492
00493
00494
00495
00496
00497 uint nRows = 50000;
00498 uint nCols = 12;
00499 uint nClusters = 3;
00500 TupleProjection proj;
00501
00502 loadClusters(nRows, nCols, nClusters, false);
00503
00504
00505
00506
00507 for (uint i = 0; i < nClusters; i++) {
00508 for (uint j = 0; j < nCols; j++) {
00509 proj.push_back(i * nCols + j);
00510 }
00511 }
00512 testScanCols(nRows, nCols, nClusters, proj, 1, nRows);
00513 resetExecStreamTest();
00514
00515
00516 proj.clear();
00517 proj.push_back(22);
00518 proj.push_back(10);
00519 proj.push_back(12);
00520 proj.push_back(26);
00521 proj.push_back(1);
00522 proj.push_back(35);
00523 proj.push_back(15);
00524 proj.push_back(5);
00525 proj.push_back(17);
00526 proj.push_back(30);
00527 proj.push_back(4);
00528 proj.push_back(20);
00529 proj.push_back(7);
00530 proj.push_back(13);
00531
00532 testScanCols(nRows, nCols, nClusters, proj, 1, nRows);
00533 resetExecStreamTest();
00534
00535
00536 testScanCols(
00537 nRows, nCols, nClusters, proj, 7, (int) ceil((double) nRows / 7));
00538 resetExecStreamTest();
00539
00540
00541
00542 testScanCols(
00543 nRows, nCols, nClusters, proj, 37, (int) ceil((double) nRows / 37));
00544 resetExecStreamTest();
00545
00546
00547 testScanCols(0, nCols, nClusters, proj, 1, nRows);
00548
00549 resetExecStreamTest();
00550
00551
00552 for (uint i = 0; i < nClusters; i++) {
00553 for (uint j = 0; j < nCols; j++) {
00554 proj.push_back(i * nCols + j);
00555 }
00556 }
00557 testFilterCols(nRows, nCols, nClusters, proj, 1, 1000, false);
00558
00559 resetExecStreamTest();
00560
00561
00562 proj.resize(0);
00563 for (uint i = 0; i < nClusters; i++) {
00564 for (uint j = 0; j < nCols; j++) {
00565 if (!(i < 2 && (j == 0 || j == 1))) {
00566 proj.push_back(i * nCols + j);
00567 }
00568 }
00569 }
00570 testFilterCols(nRows, nCols, nClusters, proj, 1, 1000, false);
00571
00572 resetExecStreamTest();
00573
00574
00575 proj.resize(0);
00576 for (uint i = 0; i < nClusters - 1; i++) {
00577 for (uint j = 0; j < nCols; j++) {
00578 proj.push_back(i * nCols + j);
00579 }
00580 }
00581 testFilterCols(
00582 nRows, nCols, nClusters, proj, 7, 1000 / 7 + 1, false);
00583 }
| void LcsRowScanExecStreamTest::testScanOnEmptyCluster | ( | | ) | | ----------------------------------------------------- | - | | - |
Create an empty cluster with 1 column.
Try reading a rid from it
Definition at line 640 of file LcsRowScanExecStreamTest.cpp.
References attrDesc_int64, bTreeClusters, BTreeBuilder::createEmptyRoot(), BTreeAccessBase::getRootPageId(), BTreeDescriptor::keyProjection, NULL_PAGE_ID, CacheTestBase::pCache, SegmentAccessor::pCacheAccessor, SegStorageTestBase::pRandomSegment, SegmentAccessor::pSegment, BTreeDescriptor::rootPageId, BTreeDescriptor::segmentAccessor, testScanCols(), and BTreeDescriptor::tupleDescriptor.
Referenced by LcsRowScanExecStreamTest().
| void LcsRowScanExecStreamTest::testScanPastEndOfCluster | ( | | ) | | ------------------------------------------------------- | - | | - |
| void LcsRowScanExecStreamTest::testCompressedFiltering | ( | | ) | | ------------------------------------------------------ | - | | - |
Definition at line 585 of file LcsRowScanExecStreamTest.cpp.
References loadClusters(), NDUPS, ExecStreamUnitTestBase::resetExecStreamTest(), and testFilterCols().
Referenced by LcsRowScanExecStreamTest().
00586 {
00587
00588
00589
00590
00591
00592
00593 uint nRows = 50000;
00594 uint nCols = 12;
00595 uint nClusters = 3;
00596 TupleProjection proj;
00597
00598
00599
00600 loadClusters(nRows, nCols, nClusters, true);
00601
00602
00603 proj.resize(0);
00604 for (uint i = 0; i < nClusters; i++) {
00605 for (uint j = 0; j < nCols; j++) {
00606 proj.push_back(i * nCols + j);
00607 }
00608 }
00609 testFilterCols(nRows, nCols, nClusters, proj, 1, 500*NDUPS+500, true);
00610
00611 resetExecStreamTest();
00612
00613
00614 proj.resize(0);
00615 for (uint i = 0; i < nClusters; i++) {
00616 for (uint j = 0; j < nCols; j++) {
00617 if (!(i < 2 && (j == 0 || j == 1))) {
00618 proj.push_back(i * nCols + j);
00619 }
00620 }
00621 }
00622 testFilterCols(nRows, nCols, nClusters, proj, 1, 500*NDUPS+500, true);
00623
00624 resetExecStreamTest();
00625
00626
00627 proj.resize(0);
00628 for (uint i = 0; i < nClusters - 1; i++) {
00629 for (uint j = 0; j < nCols; j++) {
00630 proj.push_back(i * nCols + j);
00631 }
00632 }
00633 testFilterCols(nRows, nCols, nClusters, proj, 1, 500*NDUPS+500, true);
00634 }
| void LcsRowScanExecStreamTest::testBernoulliSampling | ( | | ) | | ---------------------------------------------------- | - | | - |
Configure Bernoulli sampling, with a specific seed and verify that the expected number of rows are returned.
Definition at line 688 of file LcsRowScanExecStreamTest.cpp.
References loadClusters(), ExecStreamUnitTestBase::resetExecStreamTest(), SAMPLING_BERNOULLI, and testSampleScanCols().
Referenced by LcsRowScanExecStreamTest().
00689 {
00690 uint nRows = 50000;
00691 uint nCols = 12;
00692 uint nClusters = 3;
00693 TupleProjection proj;
00694
00695 int seed = 19721212;
00696 float rate = 0.1;
00697 TableSamplingMode mode = SAMPLING_BERNOULLI;
00698
00699 loadClusters(nRows, nCols, nClusters, false);
00700
00701
00702
00703
00704 for (uint i = 0; i < nClusters; i++) {
00705 for (uint j = 0; j < nCols; j++) {
00706 proj.push_back(i * nCols + j);
00707 }
00708 }
00709
00710
00711 testSampleScanCols(
00712 0, nRows, nCols, nClusters, proj, 1, mode, rate, seed, 0, 4938);
00713 resetExecStreamTest();
00714
00715
00716 testSampleScanCols(
00717 nRows, nRows, nCols, nClusters, proj, 2, mode, rate, seed, 0, 2489);
00718 resetExecStreamTest();
00719 }
| void LcsRowScanExecStreamTest::testSystemSampling | ( | | ) | | ------------------------------------------------- | - | | - |
Configure system sampling, with a specific clump size and verify that the expected number of rows are returned.
Definition at line 726 of file LcsRowScanExecStreamTest.cpp.
References loadClusters(), ExecStreamUnitTestBase::resetExecStreamTest(), SAMPLING_SYSTEM, and testSampleScanCols().
Referenced by LcsRowScanExecStreamTest().
00727 {
00728 uint nRows = 50000;
00729 uint nCols = 12;
00730 uint nClusters = 3;
00731 TupleProjection proj;
00732
00733 TableSamplingMode mode = SAMPLING_SYSTEM;
00734
00735 loadClusters(nRows, nCols, nClusters, false);
00736
00737
00738
00739
00740 for (uint i = 0; i < nClusters; i++) {
00741 for (uint j = 0; j < nCols; j++) {
00742 proj.push_back(i * nCols + j);
00743 }
00744 }
00745
00746 testSampleScanCols(
00747 nRows, nRows, nCols, nClusters, proj, 1, mode, 0.1, -1, 10, 5000);
00748 resetExecStreamTest();
00749
00750 testSampleScanCols(
00751 nRows, nRows, nCols, nClusters, proj, 1, mode, 1.0, -1, 10, 50000);
00752 resetExecStreamTest();
00753
00754 testSampleScanCols(
00755 nRows, nRows, nCols, nClusters, proj, 1, mode, 0.33333, -1, 10, 16670);
00756 resetExecStreamTest();
00757 }
Defines and prepares a graph consisting of one source stream and one transform stream.
Parameters:
| sourceStreamEmbryo | embryonic source stream which produces tuples |
|---|---|
| transformStreamEmbryo | embryonic transform stream which processes tuples produced by sourceStreamEmbryo |
Returns:
output buffer stream
Definition at line 48 of file ExecStreamUnitTestBase.cpp.
Referenced by LcsClusterReplaceExecStreamTest::loadCluster(), loadOneCluster(), ExecStreamUnitTestBase::prepareSourceGraph(), LcsClusterReplaceExecStreamTest::replaceCluster(), LcsMultiClusterAppendTest::scanCols(), LbmSplicerExecStreamTest::spliceInput(), ExecStreamTestSuite::testBTreeInsertExecStream(), CollectExecStreamTestSuite::testCollectCollectUncollectUncollect(), CollectExecStreamTestSuite::testCollectInts(), CollectExecStreamTestSuite::testCollectUncollect(), CalcExecStreamTestSuite::testConstant(), ExecStreamTestSuite::testCopyExecStream(), ExecStreamTestSuite::testCountAggExecStream(), LhxAggExecStreamTest::testCountImpl(), ExecStreamTestSuite::testDoubleBufferExecStream(), ExecStreamTestSuite::testGroupAggExecStreamNrows(), LhxAggExecStreamTest::testGroupCountImpl(), ExternalSortExecStreamTest::testImpl(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), LcsClusterAppendExecStreamTest::testLoadSingleCol(), LbmNormalizerExecStreamTest::testNormalizer(), ExecStreamTestSuite::testReshapeExecStream(), testSampleScanCols(), testScanCols(), LbmSearchTest::testScanIdx(), LcsClusterAppendExecStreamTest::testScanMultiCol(), LcsClusterAppendExecStreamTest::testScanSingleCol(), ExecStreamTestSuite::testScratchBufferExecStream(), ExecStreamTestSuite::testSegBufferExecStream(), ExecStreamTestSuite::testSingleValueAggExecStream(), LhxAggExecStreamTest::testSingleValueImpl(), LbmSortedAggExecStreamTest::testSortedAgg(), ExecStreamTestSuite::testSumAggExecStream(), LhxAggExecStreamTest::testSumImpl(), LbmUnionExecStreamTest::testUnion(), and LcsClusterReplaceExecStreamTest::verifyCluster().
00051 { 00052 std::vector transforms; 00053 transforms.push_back(transformStreamEmbryo); 00054 return prepareTransformGraph(sourceStreamEmbryo, transforms); 00055 }
Defines and prepares a graph consisting of one source stream and one or multiple transform streams.
Parameters:
| sourceStreamEmbryo | embryonic source stream which produces tuples |
|---|---|
| transforms | embryonic transform streams which process tuples produced by sourceStreamEmbryo or a child stream |
Returns:
output buffer stream
Definition at line 57 of file ExecStreamUnitTestBase.cpp.
References BUFPROV_PRODUCER, ExecStreamEmbryo::getStream(), ExecStreamUnitTestBase::pGraph, and ExecStreamUnitTestBase::pGraphEmbryo.
00060 {
00061 pGraphEmbryo->saveStreamEmbryo(sourceStreamEmbryo);
00062 std::vector::iterator it;
00063
00064
00065 for (it = transforms.begin(); it != transforms.end(); ++it) {
00066 pGraphEmbryo->saveStreamEmbryo(*it);
00067 }
00068
00069
00070 ExecStreamEmbryo& previousStream = sourceStreamEmbryo;
00071 for (it = transforms.begin(); it != transforms.end(); ++it) {
00072 pGraphEmbryo->addDataflow(
00073 previousStream.getStream()->getName(),
00074 (*it).getStream()->getName());
00075 previousStream = *it;
00076 }
00077
00078 SharedExecStream pAdaptedStream =
00079 pGraphEmbryo->addAdapterFor(
00080 previousStream.getStream()->getName(),
00081 0,
00082 BUFPROV_PRODUCER);
00083 pGraph->addOutputDataflow(pAdaptedStream->getStreamId());
00084
00085 pGraphEmbryo->prepareGraph(shared_from_this(), "");
00086 return pAdaptedStream;
00087 }
Defines and prepares a graph consisting of two source streams and one confluence stream.
Parameters:
| sourceStreamEmbryo1 | embryonic source stream which produces tuples |
|---|---|
| sourceStreamEmbryo2 | embryonic source stream which produces tuples |
| confluenceStreamEmbryo | embryonic confluence stream which processes tuples produced by the sourceStreamEmbryos |
Returns:
output buffer stream
Definition at line 89 of file ExecStreamUnitTestBase.cpp.
Referenced by ExecStreamUnitTestBase::prepareConfluenceGraph(), ExecStreamTestSuite::testCartesianJoinExecStream(), CorrelationJoinExecStreamTestSuite::testCorrelationJoin(), testFilterCols(), ExecStreamGovernorTest::testGovernor(), LhxJoinExecStreamTest::testImpl(), LbmIntersectExecStreamTest::testIntersect(), ExecStreamTestSuite::testMergeExecStream(), LbmMinusExecStreamTest::testMinus(), and ExecStreamTestSuite::testNestedLoopJoinExecStream().
00093 { 00094 std::vector sourceStreamEmbryos; 00095 sourceStreamEmbryos.push_back(sourceStreamEmbryo1); 00096 sourceStreamEmbryos.push_back(sourceStreamEmbryo2); 00097 return prepareConfluenceGraph(sourceStreamEmbryos, confluenceStreamEmbryo); 00098 }
Defines and prepares a graph consisting of a list of source streams and one confluence stream.
Parameters:
| sourceStreamEmbryos | list of embryonic source streams that produce tuples |
|---|---|
| confluenceStreamEmbryo | embryonic confluence stream which processes tuples produced by the sourceStreamEmbryos |
Returns:
output buffer stream
Definition at line 155 of file ExecStreamUnitTestBase.cpp.
References ExecStreamUnitTestBase::prepareConfluenceGraph().
00158 { 00159 std::vector<std::vector > sourceStreamEmbryosList; 00160 std::vector::iterator it; 00161 std::vector sourceStreamList; 00162 for (it = sourceStreamEmbryos.begin(); it != sourceStreamEmbryos.end(); 00163 it++) 00164 { 00165 sourceStreamList.clear(); 00166 sourceStreamList.push_back(*it); 00167 sourceStreamEmbryosList.push_back(sourceStreamList); 00168 } 00169 00170 return 00171 prepareConfluenceGraph(sourceStreamEmbryosList, confluenceStreamEmbryo); 00172 }
Defines and prepares a graph consisting of one or more source streams and one confluence stream.
Each source stream can be a list of streams.
Parameters:
| sourceStreamEmbryosList | list of embryonic source streams which produce tuples |
|---|---|
| confluenceStreamEmbryo | embryonic confluence stream which processes tuples produced by the source streams |
Returns:
output buffer stream
Definition at line 174 of file ExecStreamUnitTestBase.cpp.
References BUFPROV_PRODUCER, ExecStreamEmbryo::getStream(), ExecStreamUnitTestBase::pGraph, and ExecStreamUnitTestBase::pGraphEmbryo.
00177 {
00178 pGraphEmbryo->saveStreamEmbryo(confluenceStreamEmbryo);
00179
00180 for (int i = 0; i < sourceStreamEmbryosList.size(); i++) {
00181 for (int j = 0; j < sourceStreamEmbryosList[i].size(); j++) {
00182 pGraphEmbryo->saveStreamEmbryo(sourceStreamEmbryosList[i][j]);
00183 }
00184
00185
00186 for (int j = 1; j < sourceStreamEmbryosList[i].size(); j++) {
00187 pGraphEmbryo->addDataflow(
00188 sourceStreamEmbryosList[i][j - 1].getStream()->getName(),
00189 sourceStreamEmbryosList[i][j].getStream()->getName());
00190 }
00191 pGraphEmbryo->addDataflow(
00192 sourceStreamEmbryosList[i].back().getStream()->getName(),
00193 confluenceStreamEmbryo.getStream()->getName());
00194 }
00195
00196 SharedExecStream pAdaptedStream =
00197 pGraphEmbryo->addAdapterFor(
00198 confluenceStreamEmbryo.getStream()->getName(), 0,
00199 BUFPROV_PRODUCER);
00200 pGraph->addOutputDataflow(
00201 pAdaptedStream->getStreamId());
00202
00203 pGraphEmbryo->prepareGraph(shared_from_this(), "");
00204
00205 return pAdaptedStream;
00206 }
Defines and prepares a graph consisting of two source streams, one confluence stream, and one transform stream.
Parameters:
| sourceStreamEmbryo1 | embryonic source stream which produces tuples |
|---|---|
| sourceStreamEmbryo2 | embryonic source stream which produces tuples |
| confluenceStreamEmbryo | embryonic confluence stream which processes tuples produced by the sourceStreamEmbryos |
| transformStreamEmbryo | embryonic transform streams which process tuples produced by a child stream |
Returns:
output buffer stream
Definition at line 100 of file ExecStreamUnitTestBase.cpp.
References BUFPROV_PRODUCER, ExecStreamEmbryo::getStream(), ExecStreamUnitTestBase::pGraph, and ExecStreamUnitTestBase::pGraphEmbryo.
Referenced by LhxJoinExecStreamTest::testImpl(), and LbmMinusExecStreamTest::testRestartingMinus().
00105 {
00106 std::vector sourceStreamEmbryos;
00107 sourceStreamEmbryos.push_back(sourceStreamEmbryo1);
00108 sourceStreamEmbryos.push_back(sourceStreamEmbryo2);
00109
00110 std::vector::iterator it;
00111
00112 for (it = sourceStreamEmbryos.begin(); it != sourceStreamEmbryos.end();
00113 ++it)
00114 {
00115 pGraphEmbryo->saveStreamEmbryo(*it);
00116 }
00117 pGraphEmbryo->saveStreamEmbryo(confluenceStreamEmbryo);
00118
00119 for (it = sourceStreamEmbryos.begin(); it != sourceStreamEmbryos.end();
00120 ++it)
00121 {
00122 pGraphEmbryo->addDataflow(
00123 (*it).getStream()->getName(),
00124 confluenceStreamEmbryo.getStream()->getName());
00125 }
00126
00127 std::vector transforms;
00128 transforms.push_back(transformStreamEmbryo);
00129 ExecStreamEmbryo& previousStream = confluenceStreamEmbryo;
00130
00131
00132 for (it = transforms.begin(); it != transforms.end(); ++it) {
00133 pGraphEmbryo->saveStreamEmbryo(*it);
00134 }
00135
00136 for (it = transforms.begin(); it != transforms.end(); ++it) {
00137 pGraphEmbryo->addDataflow(
00138 previousStream.getStream()->getName(),
00139 (*it).getStream()->getName());
00140 previousStream = *it;
00141 }
00142
00143
00144 SharedExecStream pAdaptedStream =
00145 pGraphEmbryo->addAdapterFor(
00146 previousStream.getStream()->getName(),
00147 0,
00148 BUFPROV_PRODUCER);
00149 pGraph->addOutputDataflow(pAdaptedStream->getStreamId());
00150
00151 pGraphEmbryo->prepareGraph(shared_from_this(), "");
00152 return pAdaptedStream;
00153 }
Defines and prepares a graph consisting of a source, a splitter, and one or more parallel transform streams which flow together into a confluence stream.
Parameters:
| srcStreamEmbryo | embryonic source stream which produces tuples |
|---|---|
| splitterStreamEmbryo | embryonic SplitterExecStream which produces tuples for multiple consumers |
| interStreamEmbryos | embryonic intermediate streams which transform tuples; each stream consists of a single embryo |
| destStreamEmbryo | embryonic confluence stream which processes tuples produced by the interStreamEmbryos |
| createSink | if true (the default), creates a final output sink in the stream graph |
| saveSrc | if true (the default), save the source in the stream graph; if false, the save has already been done |
Returns:
output buffer stream or null stream if createSink is false
Definition at line 208 of file ExecStreamUnitTestBase.cpp.
Referenced by LcsMultiClusterAppendTest::loadClusters(), LbmSearchTest::loadTableAndIndex(), LbmLoadBitmapTest::testLoad(), ExecStreamTestSuite::testMergeImplicitPullInputs(), ExecStreamTestSuite::testSegBufferReaderWriterExecStream(), and ExecStreamTestSuite::testSplitterPlusBarrier().
00215 {
00216 std::vector<std::vector > listOfList;
00217
00218
00219
00220
00221 for (uint i = 0; i < interStreamEmbryos.size(); i++) {
00222 std::vector interStreamEmbryoList;
00223
00224 interStreamEmbryoList.push_back(interStreamEmbryos[i]);
00225 listOfList.push_back(interStreamEmbryoList);
00226 }
00227 return prepareDAG(
00228 srcStreamEmbryo, splitterStreamEmbryo, listOfList, destStreamEmbryo,
00229 createSink, saveSrc);
00230 }
Defines and prepares a graph consisting of a source, a splitter, and one or more parallel transform streams which flow together into a confluence stream.
Parameters:
| srcStreamEmbryo | embryonic source stream which produces tuples |
|---|---|
| splitterStreamEmbryo | embryonic SplitterExecStream which produces tuples for multiple consumers |
| interStreamEmbryosList | one or more embryonic intermediate streams which transform tuples; each stream can have one more embryos |
| destStreamEmbryo | embryonic confluence stream which processes tuples produced by the interStreamEmbryos |
| createSink | if true (the default), creates a final output sink in the stream graph |
| saveSrc | if true (the default), save the source in the stream graph; if false, the save has already been done |
Returns:
output buffer stream or null stream if createSink is false
Definition at line 232 of file ExecStreamUnitTestBase.cpp.
References BUFPROV_PRODUCER, ExecStreamEmbryo::getStream(), ExecStreamUnitTestBase::pGraph, and ExecStreamUnitTestBase::pGraphEmbryo.
00239 {
00240 if (saveSrc) {
00241 pGraphEmbryo->saveStreamEmbryo(srcStreamEmbryo);
00242 }
00243 pGraphEmbryo->saveStreamEmbryo(splitterStreamEmbryo);
00244
00245
00246 for (int i = 0; i < interStreamEmbryos.size(); i++) {
00247 for (int j = 0; j < interStreamEmbryos[i].size(); j++) {
00248 pGraphEmbryo->saveStreamEmbryo(interStreamEmbryos[i][j]);
00249 }
00250
00251
00252 for (int j = 1; j < interStreamEmbryos[i].size(); j++) {
00253 pGraphEmbryo->addDataflow(
00254 interStreamEmbryos[i][j - 1].getStream()->getName(),
00255 interStreamEmbryos[i][j].getStream()->getName());
00256 }
00257 }
00258
00259 pGraphEmbryo->saveStreamEmbryo(destStreamEmbryo);
00260
00261 pGraphEmbryo->addDataflow(
00262 srcStreamEmbryo.getStream()->getName(),
00263 splitterStreamEmbryo.getStream()->getName());
00264
00265
00266 for (int i = 0; i < interStreamEmbryos.size(); i++) {
00267 pGraphEmbryo->addDataflow(
00268 splitterStreamEmbryo.getStream()->getName(),
00269 interStreamEmbryos[i][0].getStream()->getName());
00270 pGraphEmbryo->addDataflow(
00271 interStreamEmbryos[i].back().getStream()->getName(),
00272 destStreamEmbryo.getStream()->getName());
00273 }
00274
00275 SharedExecStream pAdaptedStream;
00276
00277 if (createSink) {
00278 pAdaptedStream = pGraphEmbryo->addAdapterFor(
00279 destStreamEmbryo.getStream()->getName(), 0,
00280 BUFPROV_PRODUCER);
00281 pGraph->addOutputDataflow(pAdaptedStream->getStreamId());
00282
00283 pGraphEmbryo->prepareGraph(shared_from_this(), "");
00284 }
00285
00286 return pAdaptedStream;
00287 }
Executes the prepared stream graph and verifies that its output matches that produced by a value generator.
Parameters:
| stream | output stream from which to read |
|---|---|
| nRowsExpected | number of rows expected |
| verifier | generator for expected values |
| stopEarly | if true, stop once nRowsExpected have been fetched, even if more rows are available; this can be used for simulating the cleanup effect of an error in the middle of execution |
Definition at line 325 of file ExecStreamUnitTestBase.cpp.
References TupleData::compute(), EXECBUF_EOS, MockProducerExecStreamGenerator::generateValue(), ExecStreamUnitTestBase::pGraph, ExecStreamTestBase::pResourceGovernor, and ExecStreamTestBase::pScheduler.
Referenced by LcsClusterReplaceExecStreamTest::loadCluster(), LcsMultiClusterAppendTest::loadClusters(), loadOneCluster(), LbmSearchTest::loadTableAndIndex(), LcsClusterReplaceExecStreamTest::replaceCluster(), LcsMultiClusterAppendTest::scanCols(), LbmSplicerExecStreamTest::spliceInput(), ExecStreamTestSuite::testBTreeInsertExecStream(), CollectExecStreamTestSuite::testCollectCollectUncollectUncollect(), CollectExecStreamTestSuite::testCollectUncollect(), CorrelationJoinExecStreamTestSuite::testCorrelationJoin(), ExecStreamTestSuite::testCountAggExecStream(), LhxAggExecStreamTest::testCountImpl(), ExecStreamTestSuite::testDoubleBufferExecStream(), testFilterCols(), ExecStreamTestSuite::testGroupAggExecStreamNrows(), LhxAggExecStreamTest::testGroupCountImpl(), LhxJoinExecStreamTest::testImpl(), ExternalSortExecStreamTest::testImpl(), LbmLoadBitmapTest::testLoad(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), LcsClusterAppendExecStreamTest::testLoadSingleCol(), ExecStreamTestSuite::testMergeImplicitPullInputs(), ExecStreamTestSuite::testNestedLoopJoinExecStream(), LbmNormalizerExecStreamTest::testNormalizer(), ExecStreamTestSuite::testReshapeExecStream(), LbmMinusExecStreamTest::testRestartingMinus(), testSampleScanCols(), testScanCols(), LcsClusterAppendExecStreamTest::testScanMultiCol(), LcsClusterAppendExecStreamTest::testScanSingleCol(), ExecStreamTestSuite::testScratchBufferExecStream(), ExecStreamTestSuite::testSingleValueAggExecStream(), LhxAggExecStreamTest::testSingleValueImpl(), LbmSortedAggExecStreamTest::testSortedAgg(), ExecStreamTestSuite::testSplitterPlusBarrier(), ExecStreamTestSuite::testSumAggExecStream(), LhxAggExecStreamTest::testSumImpl(), and LcsClusterReplaceExecStreamTest::verifyCluster().
00330 {
00331
00332
00333 pResourceGovernor->requestResources(*pGraph);
00334 pGraph->open();
00335 pScheduler->start();
00336 uint nRows = 0;
00337 for (;;) {
00338 ExecStreamBufAccessor &bufAccessor =
00339 pScheduler->readStream(stream);
00340 if (bufAccessor.getState() == EXECBUF_EOS) {
00341 break;
00342 }
00343 BOOST_REQUIRE(bufAccessor.isConsumptionPossible());
00344 const uint nCol =
00345 bufAccessor.getConsumptionTupleAccessor().size();
00346 BOOST_REQUIRE(nCol == bufAccessor.getTupleDesc().size());
00347 BOOST_REQUIRE(nCol >= 1);
00348 TupleData inputTuple;
00349 inputTuple.compute(bufAccessor.getTupleDesc());
00350 for (;;) {
00351 if (!bufAccessor.demandData()) {
00352 break;
00353 }
00354 BOOST_REQUIRE(nRows < nRowsExpected);
00355 bufAccessor.unmarshalTuple(inputTuple);
00356 for (int col = 0; col < nCol; ++col) {
00357 int64_t actualValue =
00358 *reinterpret_cast<int64_t const *>(inputTuple[col].pData);
00359 int64_t expectedValue = generator.generateValue(nRows, col);
00360 if (actualValue != expectedValue) {
00361 std::cout << "(Row, Col) = (" << nRows << ", " << col <<")"
00362 << std::endl;
00363 BOOST_CHECK_EQUAL(expectedValue,actualValue);
00364 return;
00365 }
00366 }
00367 bufAccessor.consumeTuple();
00368 ++nRows;
00369 if (stopEarly && nRows == nRowsExpected) {
00370 return;
00371 }
00372 }
00373 }
00374 BOOST_CHECK_EQUAL(nRowsExpected,nRows);
00375 }
| void ExecStreamUnitTestBase::verifyConstantOutput | ( | ExecStream & | stream, |
|---|---|---|---|
| const TupleData & | expectedTuple, | ||
| uint | nRowsExpected | ||
| ) | [protected, inherited] |
Executes the prepared stream graph and verifies that all output tuples matche an expected and given one.
Parameters:
| stream | output stream from which to read |
|---|---|
| expectedTuple | |
| nRowsExpected |
Definition at line 377 of file ExecStreamUnitTestBase.cpp.
References TupleData::compute(), EXECBUF_EOS, ExecStreamUnitTestBase::pGraph, ExecStreamTestBase::pResourceGovernor, TuplePrinter::print(), and ExecStreamTestBase::pScheduler.
Referenced by ExecStreamTestSuite::testCartesianJoinExecStream(), CollectExecStreamTestSuite::testCollectInts(), CalcExecStreamTestSuite::testConstant(), ExecStreamTestSuite::testCopyExecStream(), ExecStreamGovernorTest::testGovernor(), ExecStreamTestSuite::testMergeExecStream(), ExecStreamTestSuite::testSegBufferExecStream(), and ExecStreamTestSuite::testSegBufferReaderWriterExecStream().
00381 {
00382
00383
00384 pResourceGovernor->requestResources(*pGraph);
00385 pGraph->open();
00386 pScheduler->start();
00387 uint nRows = 0;
00388 for (;;) {
00389 ExecStreamBufAccessor &bufAccessor =
00390 pScheduler->readStream(stream);
00391 if (bufAccessor.getState() == EXECBUF_EOS) {
00392 break;
00393 }
00394 BOOST_REQUIRE(bufAccessor.isConsumptionPossible());
00395
00396 if (!bufAccessor.demandData()) {
00397 break;
00398 }
00399 BOOST_REQUIRE(nRows < nRowsExpected);
00400
00401 TupleData actualTuple;
00402 actualTuple.compute(bufAccessor.getTupleDesc());
00403 bufAccessor.unmarshalTuple(actualTuple);
00404
00405 int c = bufAccessor.getTupleDesc().compareTuples(
00406 expectedTuple, actualTuple);
00407 bufAccessor.consumeTuple();
00408 ++nRows;
00409 if (c) {
00410 #if 1
00411 TupleDescriptor statusDesc = bufAccessor.getTupleDesc();
00412 TuplePrinter tuplePrinter;
00413 tuplePrinter.print(std::cout, statusDesc, actualTuple);
00414 tuplePrinter.print(std::cout, statusDesc, expectedTuple);
00415 std::cout << std::endl;
00416 #endif
00417 BOOST_CHECK_EQUAL(0,c);
00418 break;
00419 }
00420 }
00421 BOOST_CHECK_EQUAL(nRowsExpected, nRows);
00422 }
Executes the prepared stream graph and verifies the resultant tuples against a set of tuples supplied in an input buffer.
Parameters:
| stream | output stream from which to read |
|---|---|
| outputTupleDesc | descriptor of expected output tuple |
| nRowsExpected | number of rows expected |
| expectedBuffer | buffer containing expected tuples |
Definition at line 424 of file ExecStreamUnitTestBase.cpp.
References TupleDescriptor::compareTuples(), TupleData::compute(), TupleAccessor::compute(), EXECBUF_EOS, TupleAccessor::getCurrentByteCount(), ExecStreamUnitTestBase::pGraph, ExecStreamTestBase::pResourceGovernor, ExecStreamTestBase::pScheduler, TupleAccessor::setCurrentTupleBuf(), and TupleAccessor::unmarshal().
Referenced by LbmIntersectExecStreamTest::testIntersect(), LbmMinusExecStreamTest::testMinus(), LbmSearchTest::testScanIdx(), and LbmUnionExecStreamTest::testUnion().
00429 {
00430
00431
00432 TupleAccessor expectedOutputAccessor;
00433 expectedOutputAccessor.compute(outputTupleDesc);
00434 TupleData expectedTuple(outputTupleDesc);
00435 uint bufOffset = 0;
00436 pResourceGovernor->requestResources(*pGraph);
00437 pGraph->open();
00438 pScheduler->start();
00439 uint nRows = 0;
00440 for (;;) {
00441 ExecStreamBufAccessor &bufAccessor =
00442 pScheduler->readStream(stream);
00443 if (bufAccessor.getState() == EXECBUF_EOS) {
00444 break;
00445 }
00446 BOOST_REQUIRE(bufAccessor.getTupleDesc() == outputTupleDesc);
00447 BOOST_REQUIRE(bufAccessor.isConsumptionPossible());
00448 const uint nCol =
00449 bufAccessor.getConsumptionTupleAccessor().size();
00450 BOOST_REQUIRE(nCol == bufAccessor.getTupleDesc().size());
00451 BOOST_REQUIRE(nCol >= 1);
00452 TupleData inputTuple;
00453 inputTuple.compute(bufAccessor.getTupleDesc());
00454 for (;;) {
00455 if (!bufAccessor.demandData()) {
00456 break;
00457 }
00458 BOOST_REQUIRE(nRows < nRowsExpected);
00459 bufAccessor.unmarshalTuple(inputTuple);
00460 expectedOutputAccessor.setCurrentTupleBuf(
00461 expectedBuffer + bufOffset);
00462 expectedOutputAccessor.unmarshal(expectedTuple);
00463 int c = outputTupleDesc.compareTuples(inputTuple, expectedTuple);
00464 if (c) {
00465 std::cout << "(Row) = (" << nRows << ")"
00466 << " -- Tuples don't match"<< std::endl;
00467 BOOST_CHECK_EQUAL(0,c);
00468 return;
00469 }
00470 bufAccessor.consumeTuple();
00471 bufOffset += expectedOutputAccessor.getCurrentByteCount();
00472 ++nRows;
00473 }
00474 }
00475 BOOST_CHECK_EQUAL(nRowsExpected,nRows);
00476 }
| void ExecStreamUnitTestBase::resetExecStreamTest | ( | | ) | [protected, inherited] | | ------------------------------------------------ | - | | - | ------------------------ |
Reset stream graph so multiple iterations of a method can be called within a single testcase.
Definition at line 305 of file ExecStreamUnitTestBase.cpp.
References ExecStreamTestBase::newScheduler(), ExecStreamTestBase::newStreamGraph(), ExecStreamTestBase::newStreamGraphEmbryo(), ExecStreamUnitTestBase::pGraph, ExecStreamUnitTestBase::pGraphEmbryo, ExecStreamTestBase::pResourceGovernor, ExecStreamTestBase::pScheduler, and ExecStreamUnitTestBase::tearDownExecStreamTest().
Referenced by loadClusters(), LbmLoadBitmapTest::testAppend(), testBernoulliSampling(), LcsClusterReplaceExecStreamTest::testClusterReplace(), testCompressedFiltering(), LbmSplicerExecStreamTest::testLER5968(), LbmSplicerExecStreamTest::testLER6473(), LcsMultiClusterAppendTest::testLoad(), LcsClusterAppendExecStreamTest::testMultiColConstNewRoot(), LcsClusterAppendExecStreamTest::testMultiColConstOldRoot(), LcsClusterAppendExecStreamTest::testMultiColNoDupNewRoot(), LcsClusterAppendExecStreamTest::testMultiColNoDupOldRoot(), LcsClusterAppendExecStreamTest::testMultiColStairNewRoot(), LcsClusterAppendExecStreamTest::testMultiColStairOldRoot(), LbmSearchTest::testMultipleRanges(), ExecStreamGovernorTest::testReturnResources(), LbmSearchTest::testScanIdx(), testScanPastEndOfCluster(), testScans(), LcsClusterAppendExecStreamTest::testSingleColConstNewRoot(), LcsClusterAppendExecStreamTest::testSingleColConstOldRoot(), LcsClusterAppendExecStreamTest::testSingleColNoDupNewRoot(), LcsClusterAppendExecStreamTest::testSingleColNoDupOldRoot(), LcsClusterAppendExecStreamTest::testSingleColStairNewRoot(), LcsClusterAppendExecStreamTest::testSingleColStairOldRoot(), LbmSplicerExecStreamTest::testSpliceRids(), LbmSplicerExecStreamTest::testSpliceWithKeys(), and testSystemSampling().
| void ExecStreamUnitTestBase::tearDownExecStreamTest | ( | | ) | [protected, virtual, inherited] | | --------------------------------------------------- | - | | - | --------------------------------- |
| void SegStorageTestBase::openStorage | ( | DeviceMode | openMode | ) | [virtual, inherited] |
|---|
Reimplemented from CacheTestBase.
Reimplemented in SegmentTestBase.
Definition at line 31 of file SegStorageTestBase.cpp.
References SegStorageTestBase::openSegmentStorage(), and CacheTestBase::openStorage().
Referenced by SegStorageTestBase::openRandomSegment(), SegmentTestBase::openStorage(), LhxHashTableTest::testCaseSetUp(), ExecStreamTestBase::testCaseSetUp(), BTreeTest::testCaseSetUp(), BTreeReadersTest::testCaseSetUp(), LbmEntryTest::testCaseSetUp(), SegPageIterTest::testIter(), SegPageEntryIterTest::testIter(), SegStreamTest::testMarkReset(), SegStreamTest::testReadSeg(), LogicalTxnTest::testTxn(), SegStreamTest::testWriteSeg(), and SegStreamTest::testWriteSpillAndRead().
| void SegStorageTestBase::openSegmentStorage | ( | DeviceMode | openMode | ) | [virtual, inherited] |
|---|
| void SegStorageTestBase::openRandomSegment | ( | | ) | [virtual, inherited] | | ------------------------------------------ | - | | - | ---------------------- |
| SharedSegment SegStorageTestBase::createLinearDeviceSegment | ( | DeviceId | deviceId, |
|---|---|---|---|
| uint | nPages | ||
| ) | [inherited] |
| void SegStorageTestBase::closeLinearSegment | ( | | ) | [inherited] | | ------------------------------------------- | - | | - | ------------- |
| void SegStorageTestBase::closeRandomSegment | ( | | ) | [inherited] | | ------------------------------------------- | - | | - | ------------- |
| void SegStorageTestBase::closeVersionedRandomSegment | ( | | ) | [inherited] | | ---------------------------------------------------- | - | | - | ------------- |
| void SegStorageTestBase::closeSnapshotRandomSegment | ( | | ) | [inherited] | | --------------------------------------------------- | - | | - | ------------- |
| void SegStorageTestBase::closeStorage | ( | | ) | [virtual, inherited] | | ------------------------------------- | - | | - | ---------------------- |
Reimplemented from CacheTestBase.
Reimplemented in SnapshotSegmentTestBase, and VersionedSegmentTest.
Definition at line 102 of file SegStorageTestBase.cpp.
References SegStorageTestBase::closeLinearSegment(), SegStorageTestBase::closeRandomSegment(), CacheTestBase::closeStorage(), and SegStorageTestBase::closeVersionedRandomSegment().
Referenced by VersionedSegmentTest::closeStorage(), SnapshotSegmentTestBase::closeStorage(), SegStorageTestBase::openRandomSegment(), SegPageIterTest::testIter(), SegPageEntryIterTest::testIter(), SegStreamTest::testMarkReset(), SegStreamTest::testReadSeg(), SegmentTestBase::testSingleThread(), SegStreamTest::testWriteSeg(), and SegStreamTest::testWriteSpillAndRead().
| Cache & CacheTestBase::getCache | ( | | ) | [inherited] | | -------------------------------------------------- | - | | - | ------------- |
| SharedCache CacheTestBase::newCache | ( | | ) | [virtual, inherited] | | --------------------------------------------------------------------------------------------- | - | | - | ---------------------- |
| void TestBase::snooze | ( | uint | nSeconds | ) | [protected, inherited] |
|---|
| void TestBase::readParams | ( | int | argc, |
|---|---|---|---|
| char ** | argv | ||
| ) | [static, inherited] |
Parses the command line.
format: [-v] [-t TEST | -all] {param=val}* [CONFIGFILE | -] Normally, the test program runs the default test cases. With the option "-all", runs the extra test cases as well. With the option "-t TEST", runs only the single test case named TEST. CONFIGFILE is read to load configuration parameters. Configuration parameters can also be set ad hoc, from the command line, as pairs name=val. These take precedence.
Definition at line 108 of file TestBase.cpp.
References TestBase::configMap, ConfigMap::dumpParams(), ConfigMap::isParamSet(), ConfigMap::mergeFrom(), TestBase::paramDictionaryFileName, ConfigMap::readParams(), TestBase::runAll, TestBase::runSingle, ConfigMap::setStringParam(), and verbose.
00109 {
00110 bool verbose = false;
00111 ConfigMap adhocMap;
00112
00113 for (int i = 1; i < argc; ++i) {
00114 std::string arg = argv[i];
00115 if (argv[i][0] == '-') {
00116 if (arg == "-v") {
00117 verbose = true;
00118 } else if (arg == "-") {
00119 configMap.readParams(std::cin);
00120 } else if (arg == "-all") {
00121 runAll = true;
00122 } else if (arg == "-t") {
00123 permAssert(i + 1 < argc);
00124 runSingle = argv[++i];
00125 } else if (arg[1] == 't') {
00126 runSingle = arg.substr(2);
00127 }
00128 } else {
00129 int i = arg.find("=");
00130 if ((0 < i) && (i < arg.size())) {
00131
00132 std::string key = arg.substr(0,i);
00133 std::string val = arg.substr(i + 1);
00134 adhocMap.setStringParam(key,val);
00135 } else {
00136
00137 std::ifstream configFile(arg.c_str());
00138 assert(configFile.good());
00139 configMap.readParams(configFile);
00140 }
00141 }
00142 }
00143 configMap.mergeFrom(adhocMap);
00144
00145
00146
00147 if (.isParamSet(paramDictionaryFileName)) {
00148 std::string dictFileName = "dictWords";
00149 configMap.setStringParam(paramDictionaryFileName,dictFileName);
00150 }
00151
00152 if (verbose) {
00153 configMap.dumpParams(std::cout);
00154 }
00155 }
| TestSuite * TestBase::releaseTestSuite | ( | | ) | [inherited] | | ---------------------------------------------------------------------------------------------- | - | | - | ------------- |
| void TestBase::beforeTestCase | ( | std::string | testCaseName | ) | [inherited] |
|---|
| void TestBase::afterTestCase | ( | std::string | testCaseName | ) | [inherited] |
|---|
| void TestBase::notifyTrace | ( | std::string | source, |
|---|---|---|---|
| TraceLevel | level, | ||
| std::string | message | ||
| ) | [virtual, inherited] |
| TraceLevel TestBase::getSourceTraceLevel | ( | std::string | source | ) | [virtual, inherited] |
|---|
Member Data Documentation
Definition at line 46 of file ExecStreamUnitTestBase.h.
Referenced by ExecStreamUnitTestBase::prepareConfluenceGraph(), ExecStreamUnitTestBase::prepareConfluenceTransformGraph(), ExecStreamUnitTestBase::prepareDAG(), ExecStreamUnitTestBase::prepareTransformGraph(), ExecStreamUnitTestBase::resetExecStreamTest(), LbmSplicerExecStreamTest::spliceInput(), ExecStreamUnitTestBase::tearDownExecStreamTest(), ExecStreamTestSuite::testBTreeInsertExecStream(), ExecStreamUnitTestBase::testCaseSetUp(), CorrelationJoinExecStreamTestSuite::testCorrelationJoin(), LhxJoinExecStreamTest::testImpl(), ExternalSortExecStreamTest::testImpl(), LbmLoadBitmapTest::testLoad(), ExecStreamTestSuite::testReshapeExecStream(), LbmSearchTest::testScanIdx(), ExecStreamUnitTestBase::verifyBufferedOutput(), ExecStreamUnitTestBase::verifyConstantOutput(), FlatFileExecStreamTest::verifyOutput(), and ExecStreamUnitTestBase::verifyOutput().
Definition at line 51 of file ExecStreamTestBase.h.
Referenced by ExecStreamUnitTestBase::resetExecStreamTest(), ExternalSortExecStreamTest::testCaseSetUp(), ExecStreamUnitTestBase::testCaseSetUp(), ExecStreamTestBase::testCaseSetUp(), ExecStreamGovernorTest::testCaseSetUp(), ExecStreamTestBase::testCaseTearDown(), ExecStreamGovernorTest::testMinGreaterAllocation(), ExecStreamUnitTestBase::verifyBufferedOutput(), ExecStreamUnitTestBase::verifyConstantOutput(), FlatFileExecStreamTest::verifyOutput(), and ExecStreamUnitTestBase::verifyOutput().
Definition at line 40 of file SegStorageTestBase.h.
Referenced by BackupRestoreTest::backup(), SegStorageTestBase::createLinearDeviceSegment(), LogicalTxnTest::createRecoveryLog(), LbmSearchTest::initBTreeExecStreamParam(), LbmLoadBitmapTest::initBTreeExecStreamParam(), LcsClusterReplaceExecStreamTest::initClusterAppendParams(), LbmExecStreamTestBase::initSorterExecStream(), LcsMultiClusterAppendTest::loadClusters(), loadOneCluster(), LbmSearchTest::loadTableAndIndex(), LbmMinusExecStreamTest::newMinusStream(), ExecStreamTestBase::newStreamGraphEmbryo(), SegStorageTestBase::openRandomSegment(), VersionedSegmentTest::openSegmentStorage(), SnapshotSegmentTestBase::openSegmentStorage(), LinearViewSegmentTest::openSegmentStorage(), BackupRestoreTest::restore(), SegStorageTestBase::SegStorageTestBase(), LbmSplicerExecStreamTest::spliceInput(), ExecStreamTestSuite::testBTreeInsertExecStream(), LhxHashTableTest::testCaseSetUp(), LcsClusterReplaceExecStreamTest::testClusterReplace(), LhxAggExecStreamTest::testCountImpl(), ExecStreamTestSuite::testDoubleBufferExecStream(), ExecStreamGovernorTest::testGovernor(), LhxAggExecStreamTest::testGroupCountImpl(), LhxJoinExecStreamTest::testImpl(), ExternalSortExecStreamTest::testImpl(), BTreeTest::testInserts(), LbmIntersectExecStreamTest::testIntersect(), LbmEntryTest::testldb35(), LbmEntryTest::testler5920(), LbmLoadBitmapTest::testLoad(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), LcsClusterAppendExecStreamTest::testLoadSingleCol(), LbmEntryTest::testMergeEntry(), BTreeTest::testMultiKeySearches(), BTreeTest::testScan(), ExecStreamTestSuite::testScratchBufferExecStream(), LhxAggExecStreamTest::testSingleValueImpl(), FlatFileExecStreamTest::testStream(), LhxAggExecStreamTest::testSumImpl(), LogicalTxnTest::testTxn(), SnapshotSegmentTest::testUncommittedReads(), LbmUnionExecStreamTest::testUnion(), SegStreamTest::testWriteSpillAndRead(), and LbmEntryTest::testZeroBytes().
Segment supporting linear page allocation.
Definition at line 45 of file SegStorageTestBase.h.
Referenced by SegStorageTestBase::closeLinearSegment(), LogicalTxnTest::createRecoveryLog(), SegmentTestBase::lockPage(), SegStorageTestBase::openRandomSegment(), VersionedSegmentTest::openSegmentStorage(), SnapshotSegmentTestBase::openSegmentStorage(), SegStorageTestBase::openSegmentStorage(), LinearViewSegmentTest::openSegmentStorage(), SegmentTestBase::openStorage(), SegmentTestBase::prefetchPage(), SnapshotSegmentTestBase::testAllocateAll(), LinearViewSegmentTest::testAllocateAll(), VersionedSegmentTest::testCheckpoint(), SegmentTestBase::testCheckpoint(), SegPageIterTest::testIter(), SegPageEntryIterTest::testIter(), SegStreamTest::testMarkReset(), SegStreamTest::testReadSeg(), VersionedSegmentTest::testRecovery(), LogicalTxnTest::testTxn(), SnapshotSegmentTest::testUncommittedReads(), SegStreamTest::testWriteSeg(), and VersionedSegmentTest::verifyPage().
(Optional) segment supporting random page allocation.
Definition at line 50 of file SegStorageTestBase.h.
Referenced by SegStorageTestBase::closeRandomSegment(), SnapshotSegmentTestBase::closeStorage(), LbmSplicerExecStreamTest::createBTree(), LbmSplicerExecStreamTest::initBTreeParam(), LbmSearchTest::initBTreeParam(), LbmLoadBitmapTest::initBTreeParam(), LbmLoadBitmapTest::initBTreeTupleDesc(), LcsClusterReplaceExecStreamTest::initClusterAppendParams(), LbmExecStreamTestBase::initSorterExecStream(), LcsClusterReplaceExecStreamTest::loadCluster(), LcsMultiClusterAppendTest::loadClusters(), loadOneCluster(), LbmSearchTest::loadTableAndIndex(), SegStorageTestBase::openRandomSegment(), SnapshotSegmentTestBase::openSegmentStorage(), LinearViewSegmentTest::openSegmentStorage(), SegmentTestBase::openStorage(), SegmentTestBase::testAllocate(), RandomAllocationSegmentTest::testAllocateAndDeallocate(), ExecStreamTestSuite::testBTreeInsertExecStream(), BTreeTest::testBulkLoad(), LhxHashTableTest::testCaseSetUp(), BTreeTest::testCaseSetUp(), BTreeReadersTest::testCaseSetUp(), testCaseSetUp(), LcsMultiClusterAppendTest::testCaseSetUp(), LcsClusterReplaceExecStreamTest::testCaseSetUp(), LcsClusterAppendExecStreamTest::testCaseSetUp(), LbmExecStreamTestBase::testCaseSetUp(), LcsClusterReplaceExecStreamTest::testClusterReplace(), LhxAggExecStreamTest::testCountImpl(), SegmentTestBase::testDeallocate(), LhxAggExecStreamTest::testGroupCountImpl(), LhxJoinExecStreamTest::testImpl(), ExternalSortExecStreamTest::testImpl(), BTreeTest::testInserts(), LbmLoadBitmapTest::testLoad(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), LcsClusterAppendExecStreamTest::testLoadSingleCol(), ExecStreamTestSuite::testMergeImplicitPullInputs(), BTreeTest::testMultiKeySearches(), BTreeReadersTest::testReaders(), testScanOnEmptyCluster(), ExecStreamTestSuite::testSegBufferExecStream(), ExecStreamTestSuite::testSegBufferReaderWriterExecStream(), LhxAggExecStreamTest::testSingleValueImpl(), LhxAggExecStreamTest::testSumImpl(), and LbmUnionExecStreamTest::testUnion().
Cache instance being tested.
Definition at line 63 of file CacheTestBase.h.
Referenced by BackupRestoreTest::backup(), CacheTestBase::closeDevice(), CacheTestBase::closeStorage(), LbmSplicerExecStreamTest::createBTree(), SegStorageTestBase::createLinearDeviceSegment(), LogicalTxnTest::createRecoveryLog(), BackupRestoreTest::createSnapshotData(), BackupRestoreTest::executeSnapshotTxn(), CacheTestBase::getCache(), LbmSearchTest::initBTreeExecStreamParam(), LbmLoadBitmapTest::initBTreeExecStreamParam(), LbmSearchTest::initBTreeParam(), LbmLoadBitmapTest::initBTreeParam(), LcsClusterReplaceExecStreamTest::initClusterAppendParams(), LbmExecStreamTestBase::initSorterExecStream(), LcsClusterReplaceExecStreamTest::loadCluster(), LcsMultiClusterAppendTest::loadClusters(), loadOneCluster(), LbmSearchTest::loadTableAndIndex(), SegmentTestBase::lockPage(), LbmMinusExecStreamTest::newMinusStream(), ExecStreamTestBase::newStreamGraphEmbryo(), CacheTestBase::openDevice(), CacheTestBase::openStorage(), LcsClusterReplaceExecStreamTest::replaceCluster(), BackupRestoreTest::restore(), LbmSplicerExecStreamTest::spliceInput(), SegmentTestBase::testAllocate(), RandomAllocationSegmentTest::testAllocateAndDeallocate(), BackupRestoreTest::testBackupCleanup(), ExecStreamTestSuite::testBTreeInsertExecStream(), BTreeTest::testBulkLoad(), LhxHashTableTest::testCaseSetUp(), ExecStreamUnitTestBase::testCaseSetUp(), BTreeTest::testCaseSetUp(), BTreeReadersTest::testCaseSetUp(), LhxAggExecStreamTest::testCountImpl(), SegmentTestBase::testDeallocate(), SnapshotSegmentTest::testDeallocateOld(), ExecStreamTestSuite::testDoubleBufferExecStream(), ExecStreamGovernorTest::testGovernor(), LhxAggExecStreamTest::testGroupCountImpl(), BackupRestoreTest::testHeaderBackupRestore(), LhxJoinExecStreamTest::testImpl(), ExternalSortExecStreamTest::testImpl(), BTreeTest::testInserts(), LbmIntersectExecStreamTest::testIntersect(), SegPageIterTest::testIter(), SegPageEntryIterTest::testIter(), CacheTest::testLargeCacheInit(), CacheTest::testLargeCacheRequest(), LbmEntryTest::testldb35(), LbmEntryTest::testler5920(), LbmLoadBitmapTest::testLoad(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), LcsClusterAppendExecStreamTest::testLoadSingleCol(), SegStreamTest::testMarkReset(), LbmEntryTest::testMergeEntry(), BTreeTest::testMultiKeySearches(), CacheTest::testQuotaCacheAccessor(), BTreeReadersTest::testReaders(), SegStreamTest::testReadSeg(), BTreeTest::testScan(), testScanOnEmptyCluster(), ExecStreamTestSuite::testScratchBufferExecStream(), LhxAggExecStreamTest::testSingleValueImpl(), FlatFileExecStreamTest::testStream(), LhxAggExecStreamTest::testSumImpl(), LogicalTxnTest::testTxn(), LogicalTxnTest::testTxnIdSequence(), LbmUnionExecStreamTest::testUnion(), SegStreamTest::testWriteSeg(), SegStreamTest::testWriteSpillAndRead(), LbmEntryTest::testZeroBytes(), and BackupRestoreTest::verifySnapshotData().
Size of device in disk pages.
Definition at line 78 of file CacheTestBase.h.
Referenced by CacheTestBase::CacheTestBase(), LinearViewSegmentTest::LinearViewSegmentTest(), CacheTest::makeBlockId(), VersionedSegmentTest::openSegmentStorage(), SnapshotSegmentTestBase::openSegmentStorage(), LinearViewSegmentTest::openSegmentStorage(), CacheTestBase::openStorage(), SegStreamTest::SegStreamTest(), SnapshotSegmentTestBase::SnapshotSegmentTestBase(), SnapshotSegmentTest::testDeallocateOld(), PagingTestBase::testPrefetch(), PagingTestBase::testPrefetchBatch(), PagingTestBase::testRandomOp(), SnapshotSegmentTest::testRollback(), PagingTestBase::testSequentialOp(), and PagingTestBase::testSkipOp().
Configuration parameters.
The reason this is static is so that no constructor parameters (which burden virtual bases) are needed.
Definition at line 155 of file TestBase.h.
Referenced by TestBase::afterTestCase(), TestBase::beforeTestCase(), BTreeTxnTest::BTreeTxnTest(), CacheTestBase::CacheTestBase(), BackupRestoreTest::createSnapshotData(), DatabaseTest::DatabaseTest(), TestOptionsTest::extra(), DatabaseTest::loadDatabase(), SparseBitmapTest::openStorage(), PagingTestBase::PagingTestBase(), ParallelExecStreamSchedulerTest::ParallelExecStreamSchedulerTest(), RandomAccessFileDeviceTest::RandomAccessFileDeviceTest(), TestBase::readParams(), SegStorageTestBase::SegStorageTestBase(), TestOptionsTest::test1(), TestOptionsTest::test2(), BackupRestoreTest::testBackupCleanup(), TestBase::TestBase(), BTreeTxnTest::testCaseSetUp(), BTreeTxnTest::testCheckpoint(), DatabaseTest::testCreateEmpty(), DatabaseTest::testForceTxns(), BackupRestoreTest::testHeaderBackupRestore(), SegPageEntryIterTest::testIter(), SegStreamTest::testRead(), BTreeTxnTest::testTxns(), SegStreamTest::testWrite(), ThreadedTestBase::ThreadedTestBase(), and TestBase::~TestBase().
The documentation for this class was generated from the following file:
- /home/pub/open/dev/fennel/lucidera/test/LcsRowScanExecStreamTest.cpp
