Understanding and Handling Data Integrity Errors in Python Data Models

When working with data models in Python, especially with databases or data validation libraries, maintaining data integrity is crucial. Data integrity errors occur when the data does not meet expected formats, uniqueness constraints, or relationships, which can lead to bugs or data loss.

In this article, we'll explore what data integrity errors are, common causes, and how to handle them effectively using Python. We'll focus on simple examples using custom classes and the popular `pydantic` library, which helps validate data models.

### What Are Data Integrity Errors? Data integrity errors happen when data violates the rules your application or database expects. For example, trying to insert a duplicate user ID when IDs must be unique, or a string being too long for a field.

### Example: Simple Python Class With Validation

class User:
    def __init__(self, user_id, name):
        if not isinstance(user_id, int):
            raise ValueError("user_id must be an integer")
        if not name:
            raise ValueError("name cannot be empty")
        self.user_id = user_id
        self.name = name

# Usage:
try:
    user = User("abc", "Alice")  # This will raise an error
except ValueError as e:
    print(f"Data integrity error: {e}")

In this example, the constructor checks if `user_id` is an integer and if the `name` is not empty. If these conditions fail, it raises a `ValueError`, which we catch and handle gracefully.

### Using Pydantic to Manage Data Integrity

Pydantic is a powerful library that automatically performs data validation using Python type hints, raising clear errors when data integrity rules fail.

from pydantic import BaseModel, ValidationError, constr

class UserModel(BaseModel):
    user_id: int
    name: constr(min_length=1)  # Name must not be empty

try:
    user = UserModel(user_id="abc", name="")  # Invalid data
except ValidationError as e:
    print("Data integrity errors:", e.errors())

Pydantic automatically checks that `user_id` is an integer and that the name is at least one character. When invalid data is passed, it throws a `ValidationError` with detailed info.

### Handling Unique Constraints (Simple Example)

Another common data integrity rule is uniqueness — for example, user IDs must be unique in a system. Here's a simple example that tracks existing IDs and raises an error if a duplicate is added.

class UserRegistry:
    def __init__(self):
        self.users = {}

    def add_user(self, user_id, name):
        if user_id in self.users:
            raise ValueError(f"User ID {user_id} already exists.")
        self.users[user_id] = name

registry = UserRegistry()
try:
    registry.add_user(1, "Alice")
    registry.add_user(1, "Bob")  # Duplicate ID
except ValueError as e:
    print(f"Data integrity error: {e}")

### Summary Handling data integrity errors early prevents broken logic and corrupted data. You can use simple validation within your classes or leverage libraries like Pydantic for automatic checking. Also, be mindful of application-wide rules like uniqueness, and enforce those with checks and clear exception handling.

By understanding why data might be invalid and coding defensive checks, you make your Python data models more robust and reliable — giving you confidence in your data's correctness.