Data management

Learning outcome statements

The learning outcome statements relevant for this section are:

1. define data management
2. demonstrate an understanding of data preprocessing and the steps to convert data for further analysis, including data consolidation, data cleaning (cleansing), data transformation, and data reduction
3. discuss the importance of having a documented record retention (or records management) policy

Definition of data management

Definitions

Data management

The set of processes, policies, and practices used to collect, store, organize, and protect an organization’s data assets throughout their life cycle. Data governance sets the rules and accountability for data; data management carries them out day to day.

Data preprocessing

To maximize the value of data, organizations engage in data preprocessing, a critical step in preparing raw data for analysis. This process is a key component of data management, ensuring that data is refined and structured before being used for decision-making. Raw data collected from various sources is often incomplete, inconsistent, and noisy, making it unsuitable for direct analysis. Proper preprocessing ensures that data is accurate, reliable, and structured for meaningful insights. Without effective preprocessing, organizations may struggle with inefficiencies, errors, and poor decision-making due to low-quality data.

The key steps in this process include:

1. Data consolidation

This step involves merging data from multiple sources into a centralized system to ensure consistency and accessibility. By integrating data across various departments or business locations, organizations can improve reporting accuracy and eliminate inefficiencies caused by data silos.

For example, in the retail industry, a company consolidates sales data from multiple store locations into a central database, enabling accurate performance tracking and streamlined inventory management across all outlets.

2. Data cleaning (cleansing)

This step involves identifying and correcting errors, inconsistencies, and redundancies in datasets to improve data quality and reliability. It ensures that the information stored is accurate, complete, and standardized for further processing and analysis.

For example, in healthcare, a hospital system removes duplicate patient records and corrects inconsistencies in patient demographic information to improve the accuracy of medical histories. This reduces errors in patient treatment plans and enhances the overall efficiency of medical services.

3. Data transformation

This step involves converting data from one format to another to ensure compatibility and usability across different systems. Transformation can include normalization, aggregation, and standardization of data, making it more meaningful for analysis and reporting.

For example, in banking, financial transactions from different global branches are standardized into a uniform format to comply with international financial regulations and enhance reporting accuracy.

4. Data reduction

This step involves simplifying and minimizing the volume of data while preserving its essential information. By removing redundant or non-essential data points, organizations can optimize storage, enhance processing speed, and improve the efficiency of data analysis.

Organizations use various methods to reduce data volume while retaining its value. Techniques include:

• sampling, which selects a representative subset of data for analysis,
• aggregation, where data is grouped to provide summaries rather than raw details, and
• compression, which encodes data more efficiently to save storage space.

For example, in manufacturing, IoT sensor data from production machines is filtered to retain only essential performance metrics, reducing data storage costs and improving analysis efficiency.

Watch for the difference: data consolidation merges data from multiple sources into one system, data transformation changes a dataset’s format or structure (normalization, standardization), and data reduction shrinks the volume of data while keeping its essential information. Aggregation can serve either goal - it supports transformation when it restates data in summary form, and reduction when it cuts volume by replacing detail with summaries.

Record retention policy

A record retention policy outlines the rules for storing, maintaining, and disposing of organizational records. It ensures that businesses retain essential records for operational, legal, and regulatory purposes while systematically eliminating outdated or redundant data. Effective record retention policies help organizations manage risks, improve efficiency, and ensure compliance with industry standards.

Why a documented policy matters

A retention policy only works if it’s written down and consistently applied. Documentation gives employees a clear, consistent schedule to follow instead of ad hoc decisions about what to keep or discard. It also gives auditors and regulators evidence that the organization is complying with applicable laws, and it supports defensible disposal - destroying records on a documented, routine schedule, rather than selectively, protects the organization against claims of improperly destroying evidence. A documented policy must also build in a litigation hold: when a lawsuit, investigation, or regulatory inquiry is pending or reasonably anticipated, normal disposal schedules are suspended for any records that might be relevant, and those records must be preserved until the hold is lifted.

Regulatory bodies influencing record retention policy

Record retention policies are often influenced by various regulatory bodies, industry standards, and internal governance requirements. Some of the key entities that mandate record retention include:

• Government regulations: Laws such as the Sarbanes-Oxley Act (SOX) and the Health Insurance Portability and Accountability Act (HIPAA) set minimum retention periods for financial and healthcare records. The General Data Protection Regulation (GDPR) works in the opposite direction: its storage-limitation principle requires personal data to be kept no longer than necessary for the purpose it was collected for. Together, these laws mean a retention policy must set both a minimum retention time and a point by which a record must be disposed of.
• Industry-specific guidelines: Certain industries have their own data retention requirements, such as financial institutions following SEC and IRS regulations or pharmaceutical companies complying with FDA documentation rules.
• Internal policies: Organizations may establish their own guidelines based on operational needs, legal requirements, and best practices.

Types of records to retain

Organizations must retain different types of records for varying periods depending on legal, operational, and business requirements.

Since record retention requirements vary by jurisdiction, it is unlikely that the CMA exam will ask for a specific number of retention years. However, having a general understanding of retention periods is useful, especially for case-based questions (CBQs).

The IRS generally requires supporting tax records to be kept for 3 years after filing, and longer in some cases: 6 years if income was substantially underreported, 7 years for bad-debt or worthless-securities claims, and indefinitely for fraudulent or unfiled returns. Many companies keep financial records for 7 years as an internal safe harbor.

Record type	Examples	Typical retention
Financial records	Accounting records, tax returns, invoices, audit reports	Often 7 years internally; general ledgers and annual financial statements are often kept permanently
Employee and HR records	Personnel files, payroll, benefits documentation, performance evaluations	3-7 years after termination
Legal documents	Contracts, intellectual property records, corporate governance documents, compliance filings	Often permanent or long-term
Customer and client data	Purchase records, customer communication, service agreements	3-10 years
Healthcare and patient records	Medical records, prescriptions, insurance claims	5-15 years
IT and security logs	Access logs, cybersecurity reports, system backup records	6 months-3 years

Expiration of retention period

When the retention period for records expires, organizations must ensure that data is disposed of securely and in compliance with legal and industry standards. Proper record disposal prevents unauthorized access, data breaches, and non-compliance risks. The following methods are commonly used for secure record disposal:

1. Shredding: Physical records, such as financial documents and personnel files, should be shredded to prevent reconstruction and unauthorized retrieval.
2. Data wiping: Digital files must be permanently erased using secure wiping software to prevent data recovery.
3. Degaussing: Magnetic storage media, such as hard drives and tapes, can be degaussed to erase stored data beyond retrieval.
4. Incineration: Highly sensitive physical records may be incinerated as an extra layer of security to ensure complete destruction.
5. Third-party disposal services: Certified disposal vendors can assist in securely destroying large volumes of records in compliance with industry regulations.

Organizations should maintain documentation of disposed records and ensure that record disposal processes align with internal policies and regulatory requirements.