Ensuring data integrity in the long term - iTernity (original) (raw)

Definition and protective measures

Scroll

Your Challenges
Archiving
Ensuring data integrity in the long term

Companies make important business decisions based on data. However, if this data contains errors or is even lost, this can have fatal economic and legal consequences. That's why it's crucial to safeguard the integrity of electronic data during its lifetime or retention period.

What is data integrity?

Data security and data integrity

Data security is an important component of data integrity. It ensures the protection of company data against misuse, and describes measures and techniques to shield data from unauthorized access.

Data integrity describes certain requirements for the protection and quality of digital data. To maintain data integrity, the consistency, completeness, accuracy, and validity of data must be ensured throughout the entire retention period. All data changes must be documented in a traceable manner so that data cannot be changed or manipulated unnoticed or without authorization. Data integrity thus has the overriding goal of protecting data from internal and external breaches.

The cornerstones of data integrity are data protection regulations, such as the GDPR and data security principles.

Loss of data integrity can lead to compromised data and have far-reaching consequences for companies. Serious data breaches and business decisions based on incorrect data can - in the worst case - threaten the very existence of companies.

Standard and definitions

Is there a standard for data integrity?

Despite the importance of the topic, there is no general consensus on the definition of data integrity. Nevertheless, it is crucial for companies to ensure the integrity of electronic data during its lifetime or retention period. In industries such as pharmaceuticals and healthcare, strict data integrity rules apply. Here, maintaining data integrity is an essential aspect of compliance requirements.

Thus, it becomes apparent that data integrity is dependent on legal requirements and industry-specific regulations, and its definition is tied to the context in which data is stored.

Legal definitions of data integrity

The General Data Protection Regulation (article 5/1f) in relation to the storage of personal data: "Personal data must be processed in a manner which ensures appropriate security of personal data, including protection against unauthorized or unlawful processing and against accidental loss, destruction, or damage by appropriate technical and organizational measures ("integrity and confidentiality")."

Part of the German Fiscal Code, the GoBD are "Principles for the proper management and storage of books, records and documents in electronic form and for data access".

Definition from the GoBD:

Definition of the GoBD1 (margin note 103) on the storage of tax-relevant data: "The taxpayer must secure its DP system against loss (e.g. untraceability, destruction, decay, and theft) and protect it against unauthorized entries and changes (e.g. through access controls)."

Industry-specific definitions of data integrity

Data integrity in pharma & life sciences:

US FDA definition (21 CFR Part 11, Electronic Records): "Data integrity refers to the completeness, consistency, and accuracy of data. Complete, consistent, and accurate data should be assignable, legible, original or copy recorded simultaneously, and accurate."

The FDA (Food and Drug Administration) uses the ALCOA principle to define a standard for data integrity. ALCOA Plus (ALCOA+) extends these conditions to include Complete, Consistent, Enduring, and Available.

As regulators become more stringent in their inspections, it is critical to be able to demonstrate compliance. The parameters of the ALCOA principle create a benchmark for holistic documentation here, which other industries also follow.

Data integrity in healthcare:

U.S. Department of Health and Human Services (HIPAA) definition: "Data integrity means that data or information is not altered or destroyed in an unauthorized manner."

Data integrity in the financial sector:

Payment Card Industry Data Security Standard (PCI DSS) definition: "Changes which disrupt data integrity are unauthorized, including alterations, additions, and deletions."

The PCI Security Standards Council uses the term "file integrity" instead of data integrity. The definition of file integrity can be found in the definition of "file integrity monitoring". It states that file integrity is a technique or technology under which specific files or logs are monitored to determine if they are modified. Further, PCI specifies that changes, additions, and deletions are classified as unauthorized changes which may interfere with file integrity.

Data integrity threats

The integrity of data can be impacted by a variety of threats. These include:

User error: People can accidentally or unintentionally delete data and make mistakes when manually entering or managing data. This increases the likelihood of errors, duplicates, or deletions. Errors due to manual entry can extend to the execution of processes, and thus distort results.
Transfer errors: Within retention periods, data often needs to be migrated multiple times. This involves transferring data from one system to another. This moving can result in transfer errors, which can lead to damage or loss of sensitive or critical data. For example, content-related and important meta-information such as retention period, timestamps, or IDs can be lost.
Cyber-attacks: Cyber-attacks can allow malware to enter IT systems and manipulate, delete, or encrypt data records. In addition, hardware can be compromised. This often happens unnoticed until malware is activated on the system by the attacker or a ransom demand is made.

Measures to ensure data integrity

Measures to ensure data integrity in the long term

So how can data integrity be protected? With the following points, you have the four most important measures at hand to maintain data integrity and minimize risks in your company:

Validate data: Use checksums to keep track of record consistency. So-called hash values, which are attached to files, can be used to identify modified or damaged data. An archiving system should be able to check and guarantee the integrity of the stored data. A Self-healing function automate the detection and repair of inconsistent data.
Back up data: Data backup is one of the most important security measures for successful data integrity, as it protects against permanent data loss. Backups protect your business from data loss and ransomware attacks. In doing so, backups should be performed as frequently as possible, especially for mission-critical data.
Access controls: Unauthorized access and unauthorized changes can be prevented by access controls. Here, it makes sense to proceed according to the "least privilege principle": Only authorized persons can access the data belonging to their area of responsibility.
Finding sources of errors: If an error or manipulation of data does occur, it is important to locate the source and be able to prove changes. This is where documenting your data handling processes plays a critical role. An audit trail can effectively minimize data integrity risks. An audit trail logs the various stages of the data lifecycle. All processes, from creation, through retention, maintenance, usage, to deletion are recorded here. It also records what change and work was done, when, and by which user. This information can ensure compliance with regulations and be used as a basis for legal evidence.

An audit trail should have the following characteristics:

Logs must be generated automatically.
Users should not have access to audit trails.
Each event (create, read, modify, delete) is logged with a timestamp.
In addition, each event is assigned to a single user.

Whitepaper: Compliance for business data

Data retention periods of ten, thirty or even more years apply to many types of documents and data. At the same time, the storage industry experiences rapid technological advances. Solutions without hardware dependency and with open industry standards are in demand.

Learn in this whitepaper how a software-based approach can help you protect the integrity of corporate data and meet compliance requirements in the long term.

Download whitepaper

Whitepaper: Software-Defined Archiving

Learn how a software-defined approach can help you protect business-critical data in a future-proof, flexible and cost-effective way.

Everything about archiving

What is archiving? What are the challenges and benefits? What is the deal with revision security, data integrity and WORM storage? Learn everything you need to know about digital archiving here.

Your Challenges
Archiving
Ensuring data integrity in the long term