NATS, the UK’s major provider of air traffic services, has published a preliminary report investigating the events that led to widespread disruption to passengers earlier this month.
The report was requested by Heidi Alexander, Secretary of State for Transport. A full report is to follow. It is due within 60 days of the incident.
Martin Rolfe, Chief Executive Officer, NATS, said:This was a software issue in a specific part of our flight data system, that we traced to a small subsection of coding. The issue has been identified and mitigation is in place while a permanent fix is safety tested and deployed.
He said this outage was unrelated to a major system disruption in August 2023. He further dismissed theories that this incident had been caused by military intervention.
What Happened
Specifically, the preliminary report says that the Tuesday 8 September incident was caused by a software issue in the National Airspace System (NAS), on which UK airspace management is built. The software in question allocates ‘squawk’ codes to individual aircraft when manual requests are made. These codes are normally allocated automatically and are used by air traffic control to identify flights on radar when the aircraft are airborne.
While a manual request for an aircraft code was processing, the system received a message for a higher-priority activity, causing the aircraft code request to be paused. When the request to generate a code resumed, the software bug meant the output was corrupted. The situation was declared a major incident at 12:32. The London Area Control (LAC) system, which handles flights above 24,500ft over England and Wales, repeatedly timed out trying to process this corrupted data. It eventually dropped its link to the NAS permanently at 13:32 as designed.
Controllers switched to manual fallback procedures. LAC sector entry was capped at around 30 aircraft per hour. UK departures were stopped and arrivals at some airports were limited, which also held departures at foreign airports bound for the UK.
Ground congestion led to diversions of airborne inbound flights from 15:20. The NAS was restarted between 15:17 and 16:09. Data reconciliation (duplicate flight plans and code/callsign mismatches) ran until 18:50 and all restrictions were lifted at 19:30.
The departure stop totalled around four and a half hours across a six-hour period.
The trigger window that resulted in this incident only lasted around a millisecond.
In the end, more than 2,000 flights were delayed, cancelled or diverted but NATS acknowledges that this understates the full disruption. It took more than two days for the backlog of passenger disruption to be cleared. Hundreds of thousands of passengers’ travel plans were disrupted.
Throughout the incident, safety margins were maintained and controllers kept radio and radar contact with all aircraft.
Early Warning
There was an early warning two and a half hours before. The link dropped for 45 seconds at 10:02 and recovered. An engineering incident was logged as having no operational impact and investigated, but the major incident did not begin until 12:32. There must be questions as to whether this signal should have been escalated sooner.
No Permanent Solution Yet
The NATS report says that the supplier – unnamed – that maintains the software, was called in for specialist support at 12:53. According to the report, the supplier has already developed and delivered a permanent fix to the NAS software, which is now being tested and will be deployed once testing is complete.
Martin Rolfe said:I would like to apologise again, very sincerely, to everyone who was affected last week. It's our job to get people where they want to go, quickly and without delay and we are devastated when that goes wrong. However, our primary role is to keep our skies safe, and everyone who flies through them. At no point last week was safety in question.
