In the dynamic and high-stakes environment of oil trading, timely and accurate data is critical to effective decision-making. A major international trading entity was facing systemic delays in receiving and processing data from various International Oil Companies (IOCs). These delays, driven by inconsistent data formats—ranging from Excel files and emails to ERP extracts—necessitated manual collation and standardization by an offshore team.

Objectives

The primary objective of this project was to streamline the data flow process from multiple upstream suppliers to the trading desk. The aim was to:

• Minimise data processing lead times
• Enhance data quality and usability for the trading team
• Reduce reliance on manual verification
• Increase responsiveness to market conditions

Approach

Diagnostic Assessment and Process Mapping

The engagement began with an in-depth assessment of the current-state data flow using value stream mapping techniques. This included:

• Reviewing SAP ERP process maps
• Interviewing supply and trading teams
• Auditing data collection and verification practices
• Identifying process handovers, bottlenecks, and rework loops

Data Quality and System Review

We collaborated closely with the SAP IT team to understand the end-to-end data handling process. Key findings included:

• Data was being collected from disparate sources including Excel files, unreadable PDF documents, and OCR conversions
• Uploaded data often moved between multiple databases, adding verification delays of up to two days
• Quality assurance checks, while methodologically sound, were overly rigorous and added limited value from a trading perspective

Stakeholder Alignment and Decision Calibration

Recognizing that a 100% data accuracy threshold created bottlenecks, we facilitated workshops with trading and optimization teams to establish acceptable accuracy thresholds. It was agreed that:

• A 95% accuracy level was sufficient to guide volume and trading decisions

• Early access to preliminary data—even if partially verified—would significantly improve responsiveness and reduce decision latency

Process Redesign and Automation

The team implemented a series of optimizations using LEAN methodologies:

• Eliminated redundant verification steps, reducing quality checks to two
• Enabled early-stage access to data from the primary database
• Increased server synchronization frequency, replacing the single midnight sync with multiple daily updates
• Developed automated data ingestion workflows to reduce manual inputs