Dr Hani Ahmed
Department of Systems and Control Engineering, Faculty of Engineering

QARSA is a novel hybrid reinforcement learning algorithm that integrates off-policy and on-policy learning principles by combining Q-learning and SARSA for the dynamic control of nonlinear systems. The proposed approach is designed to exploit the high sample efficiency and fast convergence characteristics of off-policy learning while maintaining the stability and reduced variance typically associated with on-policy methods. By introducing a tunable blending factor, QARSA enables a controlled balance between optimistic and conservative value updates, making it well-suited for environments that exhibit both deterministic and stochastic dynamics.

The performance of QARSA is evaluated using the CartPole-v1 benchmark environment within the OpenAI Gym framework and compared against standard Q-learning and SARSA implementations. All algorithms are implemented under identical experimental conditions to ensure a fair and consistent comparison. Performance is assessed using three key metrics: average reward, learning stability, and sample efficiency. Experimental results demonstrate that QARSA achieves higher average rewards while exhibiting improved learning stability and superior sample efficiency relative to its constituent algorithms. In particular, QARSA consistently learns effective control policies using fewer interactions with the environment, indicating enhanced robustness and reduced sensitivity to variance during training.

These findings suggest that hybrid reinforcement learning strategies such as QARSA can offer significant advantages in control tasks where learning stability, efficient data utilisation, and long-term performance are critical. The results provide valuable insights into the design of combined on-policy and off-policy reinforcement learning algorithms and highlight the potential applicability of QARSA to more complex and high-dimensional dynamic control problems for nonlinear systems.

Ms Cynthia Koopman
Institute of Aerospace Technologies

One of the key functions of Air Traffic Management is to balance airspace capacity and demand. Despite existing measures, demand– capacity imbalances still occur, often creating localised regions of high traffic complexity known as hotspots. These dynamic hotspots leave air traffic controllers with limited anticipation time, increasing workload and operational risk. Addressing this challenge requires adaptive, automated strategies capable of resolving complex aircraft interactions in real time. In this EU-funded ASTRA project, the use of Reinforcement Learning (RL) to resolve predicted hotspots in European airspace was investigated. RL agents were trained with a novel multi-headed policy network to issue speed, flight-level, and direct routing clearances. Three single-clearance agents showed isolated effects of each clearance type, while a multi-clearance agent operated on the full action space, selecting the most effective clearance combination for each scenario. These agents learned based on detailed information on primary and surrounding aircraft, hotspot complexity, and temporal context, while a multi-objective reward function prioritised safe, efficient, and minimally disruptive solutions. An evaluation across thousands of historical hotspots demonstrates that RL agents can effectively resolve the majority of complex events. The multi-clearance agent achieved the highest resolution rate, particularly for moderate complexity hotspots, whereas the flight-level clearance agent produced the most efficient solutions. These results highlight RL as a viable tool for tactical hotspot resolution, providing pre-emptive strategies that can reduce controller workload and enhance airspace safety and capacity.

Mr Samuel Baldacchino
Institute of Aerospace Technologies

TADA is a SESAR project focused on developing an AI-driven tool to support approach air traffic controllers (ATCOs) in managing and sequencing arrival flights within terminal airspace (TMA). The system aims to reduce ATCO cognitive workload, enhance situational awareness, and improve flight efficiency and safety by dynamically generating and displaying 4D Proposed Trajectories (4DPT) aligned with Arrival Manager (AMAN) sequences. TADA aims to merge the field of AI and aviation through the use of reinforcement learning to provide ATCOs with suggestions such as heading, speed and direct to commands to generate optimised trajectories based on Time to Gain and Time to Lose (TTG/TTL) values generated by AMAN, and alerts ATCOs to deviations. Additionally, it enhances inter-sector situational awareness by sharing 4DPT data with all relevant sectors within the TMA. Key benefits include increased predictability and safety, improved aircraft energy management, and more efficient, environmentally sustainable flight trajectories. The solution will be tested in Milan airspace, through two airports; Bergamo and Malpensa, using three use cases incorporating Point Merge System (PMS), Trombone Arrival Routes, and a hybrid of both. TADA introduces significant improvements over current systems by automating 4DPT generation, including Distance to Go (DTG) calculations, thereby enhancing situational awareness and operational efficiency. The presentation will provide a brief overview of the project's scope and objectives, detailing the operational concept and the foundational design of the reinforcement learning (RL) agent. It will also explore some of the final validation results obtained through Human In The Loop (HITL) testing with ATCOs.

Dr Timothy Kayode Samson
Institute of Aerospace Technologies

The final phase of flight, particularly the transition from cruise to landing, is the most safety-critical stage of an aircraft’s journey. To help pilots anticipate the landing runway, this study proposes an ML approach using historical runway usage derived from ADS-B trajectory data from the OpenSky Network between January 1 and May 31, 2024, combined with meteorological observations from Aerodrome Reports (METARs). A Python-based automated procedure assigned landing runways using systematic logic. The features used were meteorological data for Zurich Airport, temporal features, and aircraft type, based on their Wake Turbulence Category (WTC). Numerical features were scaled with a robust scaler. Data were split into training (80%) and testing (20%) sets. To handle class imbalance from unequal runway usage, the Synthetic Minority Over-sampling Technique (SMOTE) was applied. Four models – Decision Tree, Random Forest, XGBoost, and Support Vector Machine (SVM) – were implemented. Hyperparameter tuning employed a randomised search with cross-validation to identify optimal settings and mitigate overfitting. Performance was evaluated using accuracy, precision, and F1-score on the test set. Results showed no overfitting, as training and test performances were similar. Random Forest achieved the highest accuracy, with 0.9952 on validation data and 0.9652 on test data, outperforming other models. This shows that Random Forest is most suitable for predicting landing runways for large aircraft at Zurich Airport. Permutation importance showed the landing hour (0.2533), wind speed (0.1057), and wind direction (0.0815) were the most influential features, with landing hour being the strongest predictor of the runway used in landing.

Dr Asma Fejjari
Department of Communications and Computer Engineering, Faculty of Information and Communication Technology

Safety is the first priority for civil aviation operations. Recently, there has been interest in extracting information from digital flight data recorders as a proactive approach to raising safety standards and improving management. The analysis of recorded data offers crucial evidence to identify the causes of accidents and enhance aircraft design.

Over the past few decades, several anomaly detection techniques have been employed to find unusual patterns and identify dangerous behaviours in flight data. Traditional flight data anomaly detection relies primarily on statistical methods and rule-based strategies to identify parameters that exceed predetermined thresholds or deviate from expected values. However, these approaches have several drawbacks due to the complexity and volume of modern aviation data. Some of the main limits include the curse of dimensionality, high false alarm rates, challenges in managing complex data, and the inability to identify new or uncommon anomalies.

Recently, machine and deep learning approaches have emerged as a critical direction for overseeing pilot activities and identifying potential hazards from flight data. Contrary to classical anomaly detection methods, machine learning, and deep learning techniques are able to handle complex, high-dimensional flight data more effectively. In this work, we perform the recently developed machine and deep learning neural network architectures to identify anomalies in flight data. Real-world flight datasets are then used to detect anomalies and assess the performance of these models.

Mr Musawar Ahmad
Department of Computing and IT, Junior College

The increase in Internet of Things (IoT) devices that collect sensitive information requires due importance and attention, as it can impact our daily lives. These devices simplify and automate everyday tasks in a Smart Home, yet they contain security weaknesses. Existing measures are vulnerable to IoT infrastructure, default settings, and loose configuration, which can attract intruders. This paper proposes a supervised algorithm-based Intrusion Detection System (IDS) to detect cyber-attacks on Smart Home IoT network devices. The system consists of three main functions:

1) normalisation of the dataset;

2) classification of the dataset; and

3) graphical representation of the implemented algorithm results and attacks.

The study is on a smart home test bed on IoT devices such as: Fridge, Garage Door, GPS Tracker, Modbus, Motion Light, Thermostat and Weather Sensors. The dataset is assessed for network-based attacks such as: Backdoor, Denial of Service (DoS), Injection, Password, Ransomware, Scanning and Cross Site Scripting (XSS). The supervised machine learning algorithms J48, PART, Naïve Bayes, and Bayes Network achieve accuracies of: 97.80%, 97.80%, 99.04%, and 99.83%, respectively. The proposed architecture can inevitably differentiate between types of attacks on a Smart Home IoT network. The detailed class-level algorithm accuracy shows average results for Precision (0.998), Recall (0.998), F-Measure (0.998), ROC and PRC Area (1.000). The confusion matrix of the proposed algorithm shows the least confused matrix for detecting anomalies.

Dr Tanel Aruväli
Department of Industrial and Manufacturing Engineering, Faculty of Engineering

Manufacturing companies are increasingly exposed to volatility stemming from multiple severe disruptions in the external environment, posing significant challenges to shop floor operational stability, strategic planning, and investment management. Current resilience assessment approaches are largely fragmented, generic, and retrospective, providing limited capability to anticipate future resilience behaviour. To overcome these shortcomings and improve resilience assessment under volatile industrial conditions, the Resistte project introduces a novel approach to assessing resilience on shop floors. The project objective is to develop and validate a scalable simulation-based stress-testing tool to assess shop floor resilience under plausible disruption scenarios. The study is inspired by well-established EU-wide stress-testing methodologies used in the financial sector. Such a stress test aims to evaluate shop floors’ potential disruption-based risks and to simulate resilience dynamics over the coming periods, using company-specific data and disruption foresights. A predictive dimension is introduced by contextualising a Large Language Model to interpret processed data from a news database and retrieve potential company-specific external disruptions, with calibration. By applying sensitivity analysis and converting retrieved next-period potential external disruptions into specific effects on shop floor operations, resilience can be assessed across several categories. The company-specific disruption effects will be incorporated with the manufacturing system logic and adaptive recovery actions to simulate the resilience dynamics. Such a stress-testing tool enables manufacturing managers and stakeholders to make more informed decisions about long-term investment strategies and strategic planning. In parallel, investors and financial institutions gain deeper insights into prospective resilience conditions ahead of investment decisions.