LLVM: llvm::cas::ondisk::OnDiskGraphDB Class Reference (original) (raw)

On-disk CAS nodes database, independent of a particular hashing algorithm. More...

#include "[llvm/CAS/OnDiskGraphDB.h](OnDiskGraphDB%5F8h%5Fsource.html)"

Public Types
enum class FaultInPolicy { SingleNode, FullTree }
How to fault-in nodes if an upstream database is used. More...
using HashingFuncT
Hashing function type for validation.
Public Member Functions
LLVM_ABI_FOR_TEST Error store (ObjectID ID, ArrayRef< ObjectID > Refs, ArrayRef< char > Data)
Associate data & references with a particular object ID.
LLVM_ABI_FOR_TEST Expected< std::optional< ObjectHandle > > load (ObjectID Ref)
ArrayRef< uint8_t > getDigest (ObjectID Ref) const
LLVM_ABI_FOR_TEST Expected< ObjectID > getReference (ArrayRef< uint8_t > Hash)
Form a reference for the provided hash.
LLVM_ABI_FOR_TEST std::optional< ObjectID > getExistingReference (ArrayRef< uint8_t > Digest)
Get an existing reference to the object Digest.
Expected< bool > isMaterialized (ObjectID Ref)
Check whether the object associated with Ref is stored in the CAS.
bool containsObject (ObjectID Ref) const
Check whether the object associated with Ref is stored in the CAS.
LLVM_ABI_FOR_TEST ArrayRef< char > getObjectData (ObjectHandle Node) const
object_refs_range getObjectRefs (ObjectHandle Node) const
LLVM_ABI_FOR_TEST size_t getStorageSize () const
unsigned getHardStorageLimitUtilization () const
void print (raw_ostream &OS) const
Error validate (bool Deep, HashingFuncT Hasher) const
Validate the OnDiskGraphDB.
LLVM_ABI_FOR_TEST ~OnDiskGraphDB ()

On-disk CAS nodes database, independent of a particular hashing algorithm.

Definition at line 256 of file OnDiskGraphDB.h.

◆ HashingFuncT

Initial value:

ArrayRef - Represent a constant reference to an array (0 or more elements consecutively in memory),...

This class consists of common code factored out of the SmallVector class to reduce code duplication b...

An efficient, type-erasing, non-owning reference to a callable.

Hashing function type for validation.

Definition at line 317 of file OnDiskGraphDB.h.

◆ FaultInPolicy

How to fault-in nodes if an upstream database is used.

Enumerator
SingleNode Copy only the requested node.
FullTree Copy the the entire graph of a node.

Definition at line 329 of file OnDiskGraphDB.h.

OnDiskGraphDB::~OnDiskGraphDB ( )

◆ containsObject()

bool llvm::cas::ondisk::OnDiskGraphDB::containsObject ( ObjectID Ref) const inline

◆ getDigest()

◆ getExistingReference()

◆ getHardStorageLimitUtilization()

unsigned OnDiskGraphDB::getHardStorageLimitUtilization ( ) const

Returns

The precentage of space utilization of hard space limits.

Return value is an integer between 0 and 100 for percentage.

Definition at line 1537 of file OnDiskGraphDB.cpp.

◆ getObjectData()

◆ getObjectRefs()

◆ getReference()

Form a reference for the provided hash.

The reference can be used as part of a CAS object even if it's not associated with an object yet.

Definition at line 1100 of file OnDiskGraphDB.cpp.

References I, and LLVM_UNLIKELY.

◆ getStorageSize()

size_t OnDiskGraphDB::getStorageSize ( ) const

Returns

Total size of stored objects.

NOTE: There's a possibility that the returned size is not including a large object if the process crashed right at the point of inserting it.

Definition at line 1533 of file OnDiskGraphDB.cpp.

◆ isMaterialized()

◆ load()

Returns

nullopt if the object associated with Ref does not exist.

Definition at line 1190 of file OnDiskGraphDB.cpp.

References llvm::sys::fs::closeFile(), createCorruptObjectError(), llvm::createFileError(), llvm::cas::ondisk::ObjectHandle::fromFileOffset(), llvm::cas::ondisk::ObjectHandle::fromMemory(), getDigest(), I, llvm_unreachable, llvm::make_scope_exit(), llvm::sys::fs::openNativeFileForRead(), llvm::sys::fs::mapped_file_region::readonly, llvm::Ref, and llvm::sys::fs::status().

◆ open()

Open the on-disk store from a directory.

Parameters

Path directory for the on-disk store. The directory will be created if it doesn't exist.
HashName Identifier name for the hashing algorithm that is going to be used.
HashByteSize Size for the object digest hash bytes.
UpstreamDB Optional on-disk store to be used for faulting-in nodes if they don't exist in the primary store. The upstream store is only used for reading nodes, new nodes are only written to the primary store. User need to make sure UpstreamDB outlives current instance of OnDiskGraphDB and the common usage is to have an UnifiedOnDiskCache to manage both.
Policy If UpstreamDB is provided, controls how nodes are copied to primary store. This is recorded at creation time and subsequent opens need to pass the same policy otherwise the open will fail.

Definition at line 1544 of file OnDiskGraphDB.cpp.

References llvm::sys::path::append(), llvm::argument_out_of_domain, llvm::cas::ondisk::CASFormatVersion, llvm::cas::OnDiskDataAllocator::create(), llvm::cas::OnDiskTrieRawHashMap::create(), llvm::sys::fs::create_directories(), llvm::createFileError(), llvm::createStringError(), DataPoolFilePrefix, DataPoolTableName, llvm::cas::ondisk::getOverriddenMaxMappingSize(), IndexFilePrefix, IndexTableName, SingleNode, and llvm::cas::ondisk::useSmallMappingSize().

Referenced by llvm::cas::ondisk::UnifiedOnDiskCache::open().

◆ print()

Definition at line 1017 of file OnDiskGraphDB.cpp.

References assert(), D(), llvm::Data, llvm::SmallVectorTemplateCommon< T, typename >::empty(), llvm::isAligned(), llvm::Align::Of(), llvm::Offset, llvm::SmallVectorTemplateBase< T, bool >::push_back(), llvm::sort(), and llvm::toString().

◆ store()

Associate data & references with a particular object ID.

If there is already a record for this object the operation is a no-op.

Parameters

ID the object ID to associate the data & references with.
Refs references
Data data buffer.

Definition at line 1398 of file OnDiskGraphDB.cpp.

References llvm::Alloc, assert(), createCorruptObjectError(), createTempFile(), llvm::Data, llvm::dbgs(), llvm::ArrayRef< T >::empty(), llvm::cas::FileOffset::get(), getDigest(), llvm::handleErrors(), I, LLVM_DEBUG, LLVM_UNLIKELY, P, llvm::Ref, Size, and llvm::Error::success().

◆ validate()

Error OnDiskGraphDB::validate ( bool Deep,
HashingFuncT Hasher ) const

Validate the OnDiskGraphDB.

Parameters

Deep if true, rehash all the objects to ensure no data corruption in stored objects, otherwise just validate the structure of CAS database.
Hasher is the hashing function used for objects inside CAS.

Definition at line 884 of file OnDiskGraphDB.cpp.

References llvm::arrayRefFromStringRef(), llvm::createStringError(), D(), llvm::ArrayRef< T >::drop_back(), llvm::sys::fs::exists(), llvm::MemoryBuffer::getFile(), llvm::cas::ondisk::InternalRef::getFromOffset(), I, llvm::illegal_byte_sequence, llvm::isAligned(), llvm_unreachable, llvm::Align::Of(), llvm::Offset, llvm::SmallVectorTemplateBase< T, bool >::push_back(), llvm::Ref, llvm::Error::success(), llvm::toHex(), llvm::toString(), and llvm::utohexstr().

The documentation for this class was generated from the following files: