This document provides an overview of Call Detail Records (CDRs), their role in our system, and the workflow we use to obtain, normalise, and store them.

​
1. What are CDRs?

Call Detail Records (CDRs) are detailed logs that capture information about each telecommunications event. They typically include:
  • Call information: Timestamps, call duration, and status of the call.
  • Participants: Caller details, including identifiers like phone numbers.
  • Billing data: Information used for billing purposes, such as rates, charges, and usage type.
  • Technical data: Additional metadata that can include call routing details, network identifiers, and error codes.
These records are essential for operational monitoring, billing, and analytics in the telecommunications industry.

​
2. Usage in our system

Our system utilises CDRs for the following:
  • Analytics dashboard: (Monitoring, Usage analytics)
  • Billing
  • Notification

​
3. Data acquisition from MVNE providers

We obtain CDRs from our MVNE (Mobile Virtual Network Enabler) providers through two primary methods:
  • API integration:
    • Providers offer an API endpoint for CDR retrieval.
  • SFTP transfer:
    • Providers offer secure file transfers via SFTP.

​
Frequency and timing of incoming data:

  • Frequency:
    • Our system makes scheduled requests to fetch CDR data every hour. (The frequency is configurable as most providers will update CDRs every 15 minutes)
  • Data delay:
    • Although the records are updated every 15 minutes, the data itself is typically about an hour late. This means that each incoming batch reflects events that occurred approximately one hour before the time of retrieval.

​
4. Data normalisation and storage

​
4.1 Data normalisation

Upon receiving the raw CDRs, the data undergoes a normalisation process to align with our internal data structure. This process includes:
  • Standardisation:
    • Converting provider-specific formats into a uniform schema.
  • Data enrichment:
    • Adding any necessary metadata and reconciling data points to ensure consistency across sources.

​
4.2 Storage architecture

The normalised data is stored in two separate databases:
  • Consumption record database:
    • Contains all detailed, normalised CDR data.
    • Designed for in-depth analytics, troubleshooting, and historical analysis.
  • Backend database:
    • Stores summary information derived from the CDRs.

​
5. Provider data variability and considerations

It is important to note that CDR data is not uniform across all providers. Some key points include:
  • Variability in data fields:
    • Different providers may include different fields in their CDRs. Some might not supply certain details that others do.
  • Data availability:
    • Specific details (e.g., enhanced technical metadata or additional billing information) may be unavailable or only partially available from certain providers.
  • Impact on feature development:
    • Our system’s features that depend on CDR data must account for these variations. It is crucial to understand these dependencies when designing new features.
  • Provider evaluation:
    • Prior to onboarding a new provider, thorough inspection and testing of their CDR data is essential. This evaluation helps ensure compatibility with our system and confirms that the required data is available for analytics and operational needs.
Understanding these differences is critical for maintaining data integrity and ensuring that our analytics and billing processes continue to operate effectively regardless of provider-specific limitations.