Custom Validation with JSON Schemas#

Overview#

Pain001 v0.0.47 introduces externalized JSON Schemas for payment data validation. This enables:

  • Declarative validation rules without code changes

  • Multi-version support (pain.001.001.03 through 11)

  • Custom bank-specific validation with schema extensions

  • Clear error messages with JSON path notation

Quick Start#

Using the Default Schema#

from pain001.validation.schema_validator import SchemaValidator

# Load schema for ISO version
validator = SchemaValidator("pain.001.001.03")

# Validate payment data
data = {
    "id": "MSG001",
    "date": "2024-01-15",
    "debtor_name": "John Doe",
    # ... other required fields
}

errors = validator.validate_data(data)
if errors:
    for error in errors:
        print(f"{error.path}: {error.message}")
else:
    print("✓ Data is valid")

Batch Validation#

# Validate multiple rows at once
rows = [row1, row2, row3, ...]
total, valid, errors = validator.validate_batch(rows)

print(f"Valid: {valid}/{total}")
for row_idx, row_errors in errors:
    print(f"Row {row_idx}: {len(row_errors)} errors")

Schema Structure#

Each schema file defines:

  1. Required fields: Must be present in every transaction

  2. Field types: string, integer, number, boolean

  3. Validation rules:

    • Length constraints (minLength, maxLength)

    • Numeric ranges (minimum, maximum)

    • Pattern matching (regex for IBAN, BIC, currency codes)

    • Enum validation (allowed values)

Example schema excerpt:

{
  "required": ["id", "date", "debtor_name"],
  "properties": {
    "id": {
      "type": "string",
      "minLength": 1,
      "maxLength": 35,
      "pattern": "^[A-Za-z0-9...]*$"
    },
    "payment_amount": {
      "type": "number",
      "minimum": 0.01,
      "maximum": 999999999.99
    }
  }
}

Error Reporting#

Validation errors include:

  • path: JSON pointer to the invalid field (e.g., $.debtor_account_IBAN)

  • message: Human-readable error description

  • rule: Validation rule that failed (pattern, required, minimum, etc.)

  • value: The invalid value

Example error:

$.debtor_account_IBAN: 'INVALID123' does not match '^[A-Z]{2}[0-9]{2}[A-Z0-9]+$'

Supported ISO Versions#

Schemas are available for all pain.001 versions:

Version

File

Status

pain.001.001.03

pain.001.001.03.schema.json

✅

pain.001.001.04

pain.001.001.04.schema.json

✅

pain.001.001.05

pain.001.001.05.schema.json

✅

pain.001.001.06

pain.001.001.06.schema.json

✅

pain.001.001.07

pain.001.001.07.schema.json

✅

pain.001.001.08

pain.001.001.08.schema.json

✅

pain.001.001.09

pain.001.001.09.schema.json

✅

pain.001.001.10

pain.001.001.10.schema.json

✅

pain.001.001.11

pain.001.001.11.schema.json

✅

Creating Custom Schemas#

Bank-Specific Validation#

Create custom schemas in pain001/schemas/custom/:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Raiffeisen Austria - pain.001.001.03",
  "type": "object",
  "allOf": [
    { "$ref": "../pain.001.001.03.schema.json" }
  ],
  "properties": {
    "debtor_agent_BIC": {
      "pattern": "^RZBAATWW.*$",
      "description": "Must be Raiffeisen Austria BIC"
    },
    "debtor_name": {
      "minLength": 3,
      "maxLength": 60
    }
  }
}

Usage:

validator = SchemaValidator(
    "pain.001.001.03",
    schema_dir=Path("pain001/schemas/custom")
)

Regional Extensions#

{
  "title": "SEPA Core - pain.001.001.03",
  "type": "object",
  "allOf": [
    { "$ref": "../pain.001.001.03.schema.json" }
  ],
  "properties": {
    "service_level_code": {
      "const": "SEPA",
      "description": "SEPA payments only"
    },
    "currency": {
      "const": "EUR",
      "description": "EUR currency only"
    }
  }
}

Advanced Usage#

Extracting Required Fields#

validator = SchemaValidator("pain.001.001.03")
required_fields = validator.get_required_fields()
print(f"Required fields: {required_fields}")

Getting Field Information#

# Get field schema
field_schema = validator.get_field_schema("debtor_account_IBAN")
print(f"IBAN pattern: {field_schema['pattern']}")

# Get field description
description = validator.get_field_description("payment_amount")
print(f"Amount: {description}")

Row-Level Validation#

# Validate single row
is_valid, errors = validator.validate_row(row_data)

if is_valid:
    print("✓ Row is valid")
else:
    for error in errors:
        print(f"✗ {error}")

Migration Guide#

From Hardcoded Validation#

Old approach (hardcoded in Python):

required_columns = ["id", "date", "nb_of_txs", ...]  # Hardcoded

def validate_csv_data(data):
    for row in data:
        for col in required_columns:
            if col not in row:
                raise ValidationError(f"Missing {col}")

New approach (schema-based):

from pain001.validation.schema_validator import SchemaValidator

validator = SchemaValidator("pain.001.001.03")
errors = validator.validate_data(row)
if errors:
    raise ValidationError(errors)

Backward Compatibility#

The original validate_csv_data() function still works and, since 0.0.67, reads its required columns from the same schemas through pain001.validation.required_columns(). Pass the message type (validate_csv_data(rows, "pain.001.001.09")) to get that version’s list; with no message type every column all bundled schemas describe is required, which is the historical 22-column contract.

Performance Notes#

  • Schema files are loaded once during validator initialization

  • Validation is performed efficiently by jsonschema library

  • Batch validation optimised for large datasets

Future Enhancements#

Planned features:

  1. pain.008 (Direct Debit) - Drop in schema file

  2. pain.002 (Status Reports) - New schema set

  3. Conditional validation - Cross-field rules

  4. Custom error messages - Localization support

Troubleshooting#

Schema File Not Found#

FileNotFoundError: Schema file not found: pain001/schemas/invalid.schema.json

Solution: Check message type format is correct (e.g., pain.001.001.03)

Invalid Schema JSON#

json.JSONDecodeError: Invalid JSON in schema file

Solution: Validate schema file syntax:

python -m json.tool pain001/schemas/custom/my_schema.json

Unexpected Validation Failures#

Enable detailed error reporting:

for error in errors:
    print(f"Path: {error.path}")
    print(f"Rule: {error.rule}")
    print(f"Message: {error.message}")
    print(f"Value: {error.value}")

References#