Data management

Learning outcome statements

The learning outcome statements relevant for this section are:

1. define data management
2. demonstrate an understanding of data preprocessing and the steps to convert data for further analysis, including data consolidation, data cleaning (cleansing), data transformation, and data reduction
3. discuss the importance of having a documented record retention (or records management) policy

Definition of data management

Definitions

Data management

A broader discipline than data governance and refers to the set of processes, policies, and practices used to collect, store, organize, and protect an organization’s data assets. It ensures that data is accessible, accurate, and secure throughout its lifecycle, supporting business operations, decision-making, and regulatory compliance.

Data preprocessing

To maximize the value of data, organizations engage in data preprocessing, a critical step in preparing raw data for analysis. This process is a key component of data management, ensuring that data is refined and structured before being used for decision-making. Raw data collected from various sources is often incomplete, inconsistent, and noisy, making it unsuitable for direct analysis. Proper preprocessing ensures that data is accurate, reliable, and structured for meaningful insights. Without effective preprocessing, organizations may struggle with inefficiencies, errors, and poor decision-making due to low-quality data.

Additionally, data preprocessing directly supports the data life cycle by transforming raw captured data into meaningful and actionable information, facilitating better utilization in subsequent stages such as analytics, reporting, and archival.

The key steps in this process include:

1. Data consolidation

This step involves merging data from multiple sources into a centralized system to ensure consistency and accessibility. By integrating data across various departments or business locations, organizations can improve reporting accuracy and eliminate inefficiencies caused by data silos.

For example, in the retail industry, a company consolidates sales data from multiple store locations into a central database, enabling accurate performance tracking and streamlined inventory management across all outlets.

2. Data cleaning (cleansing)

This step involves identifying and correcting errors, inconsistencies, and redundancies in datasets to improve data quality and reliability. It ensures that the information stored is accurate, complete, and standardized for further processing and analysis.

For example, in healthcare, a hospital system removes duplicate patient records and corrects inconsistencies in patient demographic information to improve the accuracy of medical histories. This reduces errors in patient treatment plans and enhances the overall efficiency of medical services.

3. Data transformation

This step involves converting data from one format to another to ensure compatibility and usability across different systems. Transformation can include normalization, aggregation, and standardization of data, making it more meaningful for analysis and reporting.

For example, in banking, financial transactions from different global branches are standardized into a uniform format to comply with international financial regulations and enhance reporting accuracy.

4. Data reduction

This step involves simplifying and minimizing the volume of data while preserving its essential information. By removing redundant or non-essential data points, organizations can optimize storage, enhance processing speed, and improve the efficiency of data analysis. Simplifying data also ensures that only relevant and actionable insights are retained, making it easier for analysts and decision-makers to derive meaningful conclusions.

Organizations use various methods to reduce data volume while retaining its value. Techniques include:

• sampling, which selects a representative subset of data for analysis,
• aggregation, where data is grouped to provide summaries rather than raw details, and
• compression, which encodes data more efficiently to save storage space.

For example, in manufacturing, IoT sensor data from production machines is filtered to retain only essential performance metrics, reducing data storage costs and improving analysis efficiency. Additionally, companies may use machine learning models to identify redundant data points and remove them without losing critical insights.

Record retention policy

A record retention policy outlines the rules for storing, maintaining, and disposing of organizational records. It ensures that businesses retain essential records for operational, legal, and regulatory purposes while systematically eliminating outdated or redundant data. Effective record retention policies help organizations manage risks, improve efficiency, and ensure compliance with industry standards.

Regulatory bodies influencing record retention policy

Record retention policies are often influenced by various regulatory bodies, industry standards, and internal governance requirements. Some of the key entities that mandate record retention include:

• Government Regulations: Laws such as the Sarbanes-Oxley Act (SOX), the General Data Protection Regulation (GDPR), and the Health Insurance Portability and Accountability Act (HIPAA) dictate how long financial, healthcare, and personal records must be maintained.
• Industry-Specific Guidelines: Certain industries have their own data retention requirements, such as financial institutions following SEC and IRS regulations or pharmaceutical companies complying with FDA documentation rules.
• Internal Policies: Organizations may establish their own guidelines based on operational needs, legal requirements, and best practices.

Types of records to retain

Organizations must retain different types of records for varying periods depending on legal, operational, and business requirements. Below are common types of records and general guidelines for their retention.

Since record retention requirements vary by jurisdiction, it is unlikely that the CMA exam will ask for a specific number of retention years. However, having a general understanding of retention periods is useful, especially for essay questions.

Financial records

Includes accounting records, tax returns, invoices, and audit reports. Retention periods vary, but generally, these records should be kept for 7 to 10 years to comply with tax and financial reporting regulations. The Internal Revenue Service (IRS) recommends keeping tax-related documents for at least seven years to ensure compliance with audits and tax disputes. Additionally, some financial documents, such as general ledgers and annual financial statements, may require permanent retention to meet long-term regulatory and reporting obligations.

Employee and HR records

Covers personnel files, payroll records, benefits documentation, and performance evaluations. Retention requirements differ by jurisdiction but typically range from 3 to 7 years after an employee’s termination.

Legal documents

Includes contracts, intellectual property records, corporate governance documents, and compliance filings. These records often require permanent or long-term retention.

Customer and client data

Retains purchase records, customer communication, and service agreements. Depending on industry regulations, these records should be kept for 3 to 10 years.

Healthcare and patient records

Includes medical records, prescriptions, and insurance claims. Retention periods vary by country and regulatory framework, often requiring records to be maintained for 5 to 15 years.

IT and security logs

Covers access logs, cybersecurity reports, and system backup records. These records are generally retained for 6 months to 3 years, depending on industry security requirements.

Expiration of retention period

When the retention period for records expires, organizations must ensure that data is disposed of securely and in compliance with legal and industry standards. Proper record disposal prevents unauthorized access, data breaches, and non-compliance risks. The following methods are commonly used for secure record disposal:

1. Shredding: Physical records, such as financial documents and personnel files, should be shredded to prevent reconstruction and unauthorized retrieval.
2. Data wiping: Digital files must be permanently erased using secure wiping software to prevent data recovery.
3. Degaussing: Magnetic storage media, such as hard drives and tapes, can be degaussed to erase stored data beyond retrieval.
4. Incineration: Highly sensitive physical records may be incinerated as an extra layer of security to ensure complete destruction.
5. Third-party disposal services: Certified disposal vendors can assist in securely destroying large volumes of records in compliance with industry regulations.

Organizations should maintain documentation of disposed records and ensure that record disposal processes align with internal policies and regulatory requirements.