Maritime IoT Sensor Analytics on Databricks
Case Study
Vessel Sensor Analytics on Databricks for a Maritime Technology Group
The company collects sensor data from vessels at sea — engine performance, fuel consumption, navigation and equipment condition — across a fleet operating worldwide. The data arrived in irregular bursts over constrained satellite links, in inconsistent formats that varied by vessel generation, and was largely analyzed after the fact, one ship at a time.
Dateonic built a Databricks platform that ingests these bursts reliably, normalizes readings into a common vessel and equipment model, and makes fleet-wide time-series analysis possible — supporting condition monitoring, fuel efficiency work and emissions reporting from one governed dataset.
Turnover
16 Months
Industry
Logistics & Maritime
Technology:
Azure Databricks, Delta Lake, Unity Catalog
The challenge
- Intermittent connectivity. Vessels upload in windows, not continuously. Late, duplicated and out-of-order data is normal rather than exceptional.
- Heterogeneous sensors. Different vessel generations and equipment vendors produce different tag names, units and sampling rates for the same physical measurement.
- Volume without structure. High-frequency readings accumulated quickly but were not modelled for fleet-level querying.
- Gaps and noise. Sensor dropouts and faulty readings were indistinguishable from genuine anomalies.
- Reporting obligations. Fuel and emissions reporting requires figures that can be traced back to raw measurements.
Our approach
Signal inventory. We catalogued the sensors actually in use across vessel classes and mapped each vendor tag to a canonical signal with defined units. This mapping is the backbone of everything downstream — without it, fleet comparison is meaningless.
Design for late and duplicate data. Ingestion was designed around the assumption that data arrives out of order and more than once, using idempotent processing and event-time rather than arrival-time logic.
Data quality as a first-class layer. Rather than silently cleaning readings, quality checks flag and classify them, so a dropout is visibly a dropout and not mistaken for an engine anomaly.
The solution
Reliable ingestion. Auto Loader picks up uploaded files as they land, handling schema evolution across vessel generations and processing each file exactly once.
Canonical signal model. Silver tables hold normalized time series keyed by vessel, equipment and canonical signal, with units standardized and quality flags attached.
Fleet-level gold tables. Aggregated voyage, consumption and equipment-condition tables let analysts compare vessels, routes and time periods without touching raw sensor data.
Governance and lineage. Unity Catalog controls access by vessel and business function, and lineage traces every reported figure back to the raw readings behind it — which is what makes regulatory reporting defensible.
Terraform-provisioned environments. Dev, test and production are code-defined and reproducible.
Results
- Fleet-wide analysis is possible. Vessels can be compared on the same signals, in the same units, over the same time basis.
- Late and duplicate uploads handled automatically. No manual reprocessing after connectivity gaps.
- Sensor faults are distinguishable from real events. Quality flags prevent dropouts from being read as anomalies.
- Traceable reporting. Fuel and emissions figures can be traced to the underlying measurements.
- New vessel onboarding is a mapping exercise, not a new pipeline build.
- A foundation for AI. Clean, labelled time series is what predictive maintenance and fuel-optimization models require.