Fennel: FlatFileExecStreamParams Struct Reference (original) (raw)
FlatFileExecStreamParams defines parameters for instantiating a FlatFileExecStream. More...
#include <[FlatFileExecStream.h](FlatFileExecStream%5F8h-source.html)>
Inheritance diagram for FlatFileExecStreamParams:
| Public Member Functions | |
|---|---|
| FlatFileExecStreamParams () | |
| Public Attributes | |
| std::string | dataFilePath |
| Path to the flat file containing tuples to be read. | |
| std::string | errorFilePath |
| Path to the error log used for writing errors encountered while processing tuples. | |
| char | fieldDelim |
| Delimiter used to separate fields in a row. | |
| char | rowDelim |
| Delimiter used to terminate a row. | |
| char | quoteChar |
| Character used to quote data values. | |
| char | escapeChar |
| Ignored outside of quoted values. | |
| bool | header |
| Specifies whether the flat file contains a header. | |
| int | numRowsScan |
| Specifies number of rows to scan when sampling data. | |
| std::string | calcProgram |
| Converts flat file text into typed data. | |
| FlatFileMode | mode |
| Mode in which to run the flatfile scan. | |
| int | errorMax |
| The maximum number of errors to allow before failing. | |
| int | errorLogMax |
| The maximum number of errors to log. | |
| bool | lenient |
| Whether to be lenient when reading flatfile columns. | |
| bool | trim |
| Whether to trim output columns. | |
| bool | mapped |
| Whether to map source columns to target columns by name. | |
| std::vector< std::string > | columnNames |
| Names of the target columns. | |
| TupleDescriptor | outputTupleDesc |
| TupleFormat | outputTupleFormat |
| SharedCacheAccessor | pCacheAccessor |
| CacheAccessor to use for any data access. | |
| SegmentAccessor | scratchAccessor |
| Accessor for segment to use for allocating scratch buffers. |
Detailed Description
FlatFileExecStreamParams defines parameters for instantiating a FlatFileExecStream.
Currently, ASCII data is supported. More parameters may be needed to support internationalization.
TODO: review whether it is ok to infer parsing and storage behavior from output tuple type. Probably should parse field and row delim on the Java side.
Definition at line 53 of file FlatFileExecStream.h.
Constructor & Destructor Documentation
| FlatFileExecStreamParams::FlatFileExecStreamParams | ( | | ) | [inline, explicit] | | -------------------------------------------------- | - | | - | -------------------- |
Member Data Documentation
The maximum number of errors to allow before failing.
Resets when the stream is reopened. A value of -1 indicates that there is no max.
Definition at line 125 of file FlatFileExecStream.h.
The maximum number of errors to log.
Resets when the stream is reopened. A value of -1 indicates that there is no max.
Definition at line 131 of file FlatFileExecStream.h.
Definition at line 37 of file SingleOutputExecStream.h.
Referenced by LcsClusterReplaceExecStreamTest::initClusterAppendParams(), LbmExecStreamTestBase::initValuesExecStream(), LcsClusterReplaceExecStreamTest::loadCluster(), LcsMultiClusterAppendTest::loadClusters(), LcsRowScanExecStreamTest::loadOneCluster(), LbmSearchTest::loadTableAndIndex(), BTreeSearchExecStream::prepare(), BTreeReadExecStream::prepare(), FlatFileExecStreamImpl::prepare(), SingleOutputExecStream::prepare(), MockProducerExecStream::prepare(), ExecStreamFactory::readTupleStreamParams(), LcsClusterReplaceExecStreamTest::replaceCluster(), LcsMultiClusterAppendTest::scanCols(), LcsClusterAppendExecStreamTest::setUpDelIndexScan(), LbmSplicerExecStreamTest::spliceInput(), ExecStreamTestSuite::testBTreeInsertExecStream(), CollectExecStreamTestSuite::testCollectCollectUncollectUncollect(), CollectExecStreamTestSuite::testCollectInts(), CollectExecStreamTestSuite::testCollectUncollect(), CalcExecStreamTestSuite::testConstant(), CorrelationJoinExecStreamTestSuite::testCorrelationJoin(), LhxAggExecStreamTest::testCountImpl(), LcsRowScanExecStreamTest::testFilterCols(), LhxAggExecStreamTest::testGroupCountImpl(), LhxJoinExecStreamTest::testImpl(), LbmLoadBitmapTest::testLoad(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), LcsClusterAppendExecStreamTest::testLoadSingleCol(), ExecStreamTestSuite::testMergeImplicitPullInputs(), ExecStreamTestSuite::testNestedLoopJoinExecStream(), LbmNormalizerExecStreamTest::testNormalizer(), LbmMinusExecStreamTest::testRestartingMinus(), LcsRowScanExecStreamTest::testSampleScanCols(), LcsRowScanExecStreamTest::testScanCols(), LbmSearchTest::testScanIdx(), ExecStreamTestSuite::testSingleValueAggExecStream(), LhxAggExecStreamTest::testSingleValueImpl(), LbmSortedAggExecStreamTest::testSortedAgg(), LhxAggExecStreamTest::testSumImpl(), and LcsClusterReplaceExecStreamTest::verifyCluster().
CacheAccessor to use for any data access.
This will be singular if the stream should not perform data access.
Definition at line 183 of file ExecStreamDefs.h.
Referenced by ExecStreamFactory::createPrivateScratchSegment(), LbmSearchTest::initBTreeExecStreamParam(), LbmLoadBitmapTest::initBTreeExecStreamParam(), LcsClusterReplaceExecStreamTest::initClusterAppendParams(), LbmSearchTest::initClusterScanDef(), LbmLoadBitmapTest::initClusterScanDef(), ExecStreamGraphEmbryo::initStreamParams(), FtrsTableWriterFactory::loadIndex(), LcsRowScanExecStreamTest::loadOneCluster(), BTreeExecStream::newWriter(), LcsRowScanBaseExecStream::prepare(), BTreeExecStream::prepare(), ExecStream::prepare(), ExecStreamFactory::readBTreeStreamParams(), ExecStreamTestSuite::testBTreeInsertExecStream(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), LcsClusterAppendExecStreamTest::testLoadSingleCol(), LcsClusterAppendExecStreamTest::testScanMultiCol(), LcsClusterAppendExecStreamTest::testScanSingleCol(), and LcsClusterReplaceExecStreamTest::verifyCluster().
Accessor for segment to use for allocating scratch buffers.
This will be singular if the stream should not use any scratch buffers.
Definition at line 189 of file ExecStreamDefs.h.
Referenced by ExecStreamFactory::createPrivateScratchSegment(), LbmSearchTest::initBTreeExecStreamParam(), LbmLoadBitmapTest::initBTreeExecStreamParam(), LcsClusterReplaceExecStreamTest::initClusterAppendParams(), ExecStreamGraphEmbryo::initStreamParams(), FtrsTableWriterFactory::loadIndex(), LcsRowScanExecStreamTest::loadOneCluster(), BTreeExecStream::newWriter(), LcsClusterAppendExecStream::prepare(), BTreeExecStream::prepare(), FlatFileExecStreamImpl::prepare(), MockResourceExecStream::prepare(), ExecStream::prepare(), ExecStreamTestSuite::testBTreeInsertExecStream(), LcsClusterAppendExecStreamTest::testLoadMultiCol(), and LcsClusterAppendExecStreamTest::testLoadSingleCol().
The documentation for this struct was generated from the following file:
- /home/pub/open/dev/fennel/flatfile/FlatFileExecStream.h
Generated on Mon Jun 22 04:00:32 2009 for Fennel by 
1.5.1