CogSIMA 2023: Papers with Abstracts

Abstract. Strengthening health systems and information dissemination, and Internet connectivity in under- resourced communities, have presented profound challenges throughout the COVID-19 pandemic. This article describes research on an emerging cognitive cloud to edge community solution, the Internet Backpack, which was utilized for COVID-19 disaster response in rural and remote Costa Rican communities. The Internet Backpack utilizes novel tools and systems for controlling narrow band data pathways through selective engagement with one or more narrow band platforms of an omni-grid system. These methods enable cognitive cloud services to function reliably beyond the edge due to enhanced data compression for stronger, faster and stable transmission to off-grid environments with less latency, jitter and packet loss. In this article, we describe implementation of the Internet Backpack in Las Delicias, Los Angeles and El Palmar, three remote communities in Costa Rica. We highlight how the project has contributed to the objectives of providing Internet connectivity and consequently, increased access to COVID-19 health information in these underserved communities.

Abstract. The realization of safe networked traffic is getting more and more important. The planning and prediction of possible driving behaviors and the detection of missing actions in advance contribute to avoid critical situations. A decision making system enables the support the human operator (driver) and to supervise the human-machine interaction by proposing possible actions predicted by the system, by warning him or her by detecting critical situations, and to take over the driving functionality if necessary.
The contribution of the work is the development of a monitoring decision making system allowing the planning and prediction of possible driving behaviors, detection of missing actions, and to support the human operator to reach desired situations. A Situation Operator Modeling method is used as event-discrete approach to describe changes from the real world as well as driving behaviors as a graph-based model considering the changes in the environment. The behaviors of traffic-vehicles are calculated based on predicted trajectories using a Long Short-Term Memory (LSTM) Encoder Decoder algorithm. The approach is applied to an ’overtaking maneuver on a highway’. Decision options can be continually generated depending on the changes in the environment, can be suggested to the driver, and can support him or her to lead desired situations.

Abstract. Driving behavior estimations play a significant role in the development of Advanced Driving Assistance Systems (ADASs). The estimations are often developed using ma- chine learning-based approaches, which are influenced by different factors, such as input variables and design of methods. However, developing a suitable configuration can be complicated. In this contribution, an improved Hidden Markov Model (HMM)-based state machine model is introduced for the recognition of lane changing behaviors. Adapting a previously developed HMM model, the model consists of different sub-HMMs which are fused to develop the HMM estimations. A prefilter is introduced in the HMM to quantize the input variables into segments of observed sequences that distinguish different driving situations. Hence, optimization of the prefilter is performed. Different from the previous work, a state machine model is incorporated to develop the final behavior estimation using the estimations of the HMM model. To evaluate the estimation effectiveness, different driving features (inputs) are evaluated by using different combinations of sub-HMMs. Ex- perimental driving data based on six drivers used for the application of the method show that the approach generates adequate accuracy (ACC), detection rates (DR), and false alarm rates (FAR).

Abstract. The prediction and recognition models of driving behaviors are often based on ma- chine learning approaches. These models are required for the growth of advanced driving assistance systems. The performance of the model depends on the optimal parameters, hy- perparameters, and model structure. In the present study, hyperparameters of a previously developed model (neural network-based state machine model) are optimized for the lane changing recognition. Two methods are considered for the hyperparameter optimization: Bayesian optimization and Genetic algorithm (GA). Three lane changing behaviors are estimated. Real human driving data generated using a driving simulator are used for the parameterization. The aim is to compare the model’s recognition performance based on the two methods. Furthermore, comparisons between the models with optimized hyper- parameters and the original model (without hyperparameter optimization) are performed. The results show that the performance based on the Bayesian optimization is better than GA, while the original model still outperforms others.

Abstract. Cybersecurity incident response presents significant challenges, exacerbated by a limited understanding of the cognitive processes employed by cybersecurity professionals. Cognitive task analysis (CTA) is a valuable tool to address this knowledge gap and inform evaluation, training, and design of cybersecurity systems. However, the required access and cost have limited the number and scope of CTAs in cybersecurity. Therefore, a need exists for CTA-derived insights about incident response and methodology of CTA to support data collection in this rapidly evolving domain. In this paper, we explore some of the challenges specific to CTA in the context of incident response, present an example demonstrating how CTA facilitates insights by examining results obtained from a single subject matter expert (SME), and describe the role of CTA in our ongoing mixed methods research program. The application of CTA in supporting quantitative research holds promise for advancing cyber defense strategies.

Abstract. We discuss the negative impact of high levels of stress on a person’s cognitive situation awareness and management capability. This is a critical aspect to consider when designing people-centered Cyber-Physical-Social Systems (CPSS). High levels of stress can negatively affect how people perceive and interpret information, resulting in an inability to understand the situation adequately and inadequate decision-making. The paper highlights that high levels of stress can also make it challenging for people to follow rules and regulations in crises situations. We discuss several aspects of how CPSS could help people in crisis situations to better follow rules and regulations.

Abstract. This paper investigates the influence of culture on public behavior patterns (PBP) in the context of cyber-physical-social systems (CPSS) during crisis situations. CPSS integrate technology and social dimensions, but the understanding and inclusion of social phenomena in CPSS development and operation are still limited. The concept of public behavior pattern (PBP) is proposed to describe recurrent behaviors of communities. The paper focuses on the cultural domain and explores how cultural dimensions, based on Hofstede’s theory, could affect behavioral patterns in crisis situations. The study examines the cultural dimensions of Power Distance, Uncertainty Avoidance, and Individualism versus Collectivism in three groups: native Estonians in Estonia, Russian minority group in Estonia, and native Russians in Russia. The findings highlight differences between the cultural groups, emphasizing the importance of cultural context in crisis behavior.

Abstract. AI systems, particularly Large Language Models (LLMs), have the potential to improve telemedicine. However, there is a need to further investigate the effectiveness of AI decision support and clinician-AI collaboration in this context. This study examines the impact of AI-only and clinician-AI support systems on trust, acceptance, usability, and cognitive load in telemedicine scenarios. In a randomized controlled study, twenty non-medical participants were randomly assigned to receive support from an AI-only or clinician-AI decision support system during cardiopulmonary resuscitation (CPR) scenarios simulated in an augmented reality (AR) headset. We used ChatGPT 3, a widely used LLM, as the AI system. Participants' responses were measured using trust, acceptance, and usability questionnaires, as well as a wearable wristband to collect physiological data. The results show that the clinician-AI scenario was perceived as more useful compared to the ChatGPT-only scenario. The collaborative approach also led to higher heart rate variability (HRV) and lower LF/HF ratio, indicating potentially lower mental effort compared to ChatGPT-only. No significant differences were found in system usability scale (SUS) and electrodermal activity (EDA) levels between the scenarios. These findings highlight the importance of involving clinicians in AI- supported telemedicine. Further research should explore real-world applications to validate the preliminary results.

Abstract. Point-of-care ultrasound (POCUS) is becoming an increasingly important tool for diagnostic evaluation, clinical decision-making, and procedural guidance in the emergency department (ED). POCUS image acquisition is cognitively demanding and operator-dependent, making rigorous competency assessment critically valuable. Traditional methods for assessing POCUS competency are limited by subjectivity, requiring a more objective approach. In this study, we aimed to investigate an objective method for assessing POCUS competency using arm motion data and employing machine learning (ML) classification methods. We utilized a motion-capturing system to extract motion data while ED clinicians performed POCUS tasks. We used logistic regression (LR) and random forest (RF) classifiers to predict the expertise level (expert versus novice) based on flexion, abduction, and pronation motion metrics from the right and left wrists and elbows. The mean accuracy of the LR model was 0.80 (95% CI [0.76, 0.84]) with an area under the curve (AUC) of 0.84. The mean accuracy of the RF model was 0.91 (95% CI [0.89, 0.93]) with an AUC of 0.95. These results suggest that both methods show promise for predicting the level of clinical expertise based on arm motion data during POCUS, with the RF model outperforming LR in terms of accuracy. Our finding highlights the potential of using motion capture data and ML approaches to objectively evaluate POCUS competency with high accuracy for distinguishing between novice and expert ED clinicians. Future studies should include larger sample sizes to further improve the accuracy of the models, as well as to investigate other ML techniques.

Abstract. Effective teamwork plays a crucial role in the dynamic field of cybersecurity. However, identifying the specific knowledge, skills, and attitudes (KSAs) required for successful cybersecurity teamwork poses challenges. One of these challenges is limited access to proprietary information, which hinders the study of cybersecurity teamwork KSAs. To address this issue, this research proposes the use of a cybersecurity-themed board game as an experimental testbed to observe and analyze teamwork dynamics. By studying teamwork within this simulated environment, valuable insights can be gained that have the potential to translate to real-world cybersecurity teams. This research aims to contribute to the understanding of cybersecurity team dynamics and inform the development of future experimental testbeds that can provide insights into effective cybersecurity teamwork. By acknowledging the challenges associated with limited accessibility and employing innovative methodologies, this study seeks to make meaningful contributions to knowledge of cybersecurity teamwork.

Abstract. Artificial Intelligence is set to encompass additional decision space that has traditionally been the purview of humans. However, this decision space remains contested. Incongruencies between artificial intelligence and human rationalization processes introduce uncertainties in human decision-making, which require new conceptualizations that capture these distinct types of uncertainties. Hence, developing new ways to model human and artificial intelligence interactions are necessary to account for such uncertainties and improve situation awareness and decision-making. In this paper, we outline current conceptualizations of human and machine rationalities. Next, we offer the concept of rational prediction deviations (via quantum probability theory) for capturing uncertainty in situational awareness. Lastly, we propose a human-in-the-loop construct to explicate how applications of quantum probability theory in decision science can ameliorate situation awareness models by providing a novel way to capture distinct dynamics of decision making.

Abstract. Human-robot teams operate in uncertain environments and need to accomplish a wide range of tasks. A dynamic understanding of the human’s workload can enable fluid inter- actions between team members. A system that seeks to adapt interactions for a human- robot team needs to quantify the distribution of workload across the different workload components. A workload assessment algorithm capable of estimating the demand placed on the human’s visual resources is required. Further, adaptive systems will benefit from measures of uncertainty, as these measures inform interaction adaptations. Two machine learning methods’ capacity to estimate visual workload for a human-robot team operat- ing in a non-sedentary supervisory environment are analyzed. A key finding is that the uncertainty-aware method outperforms the other approach.

Abstract. Artificial Intelligence (AI) can be easily integrated into virtual education to drive adaptive instruction and real-time constructive feedback to students, offering a possible conduit for fostering discovery curiosity in learners. This study examines and characterizes Human-AI-Teaming (HAT) coordination dynamics to monitor the inception of discovery curiosity in online laboratories of interactive molecular dynamics (IMD). We used molecular physics measures (kinetic/ potential energy and action) obtained from simple and complex examples of simulated mouse tracking datasets in IMD log files as a proxy for understanding the context of molecular sciences and developing novel interactions for inquiry. These measures are good features of our HAT context because kinetic energy reflects the system’s atoms’ overall motion regarding the individual atoms’ speed. While kinetic energy represents if a learner applies artificial forces to the task, potential energy can be AI’s response to these forces. The action is a systems-level reaction to the changes during the task. By applying nonlinear dynamical systems methods to the physics measures, we extracted the Largest Lyapunov Exponent and Determinism metrics as HATs’ coordination stability and predictability, respectively. The findings underline that while the more complex IMD task required less stable and predictable HAT coordination dynamics, the simple task is more. One explanation is that AI needs to anticipate the learner by providing feedback at the right time and place during the more complex IMD task to initiate and sustain the learner's discovery curiosity. In IMD, future HAT design should consider coordination dynamics for fostering ‘discovery curiosity’ and practical learning.

Abstract. Abstraction levels can be explicitly studied, changed and analyzed with metrics on their impact for a given problem. Also different methods for analyzing abstraction levels and their metrics can lead to different conclusions. Hence researchers usually iteratively experiment with these different methods to find the right abstraction level and metric for specific problems. To illustrate these points, we first study the use of Quad-Trees to characterize swarms, and then compare different methods using the metrics efficacy and efficiency. The goal of this work is to create an architecture and processes that will enable a self-aware system to conduct these types of experiments, and use these methods and metrics for analyzing the appropriateness and the impact of abstraction levels in order to improve its own performance.

Abstract. If we are going to have computing systems that can be trusted to help us with difficult situations in difficult environments, then those systems are going to need much more capability, both for actions that conform to our goals for the systems, and for appropriate adaptations to unexpected or difficult conditions in their operational environment.
We may not be able to communicate with them in any timely or even useful way, so they will need to have strong autonomy in their action and adaptation decision processes. This paper extends earlier work with further implications and expectations, along with
some design notes for experiments that we are in the process of developing.
The first key finding of this investigation is that systematic language and the expressive and analytic properties of symbol systems are extremely important: abstractions cannot be computed without symbol systems; analogies cannot be discovered without symbol systems; models cannot be analyzed without symbol systems; and there are many other processes that we think are necessary that are greatly facilitated by explicit symbol systems. The difficulty is that symbol systems cannot be indefinitely elaborated without a cor- responding reduction process (by the “Get Stuck” Theorems in the field of Computational Semiotics, which studies the use of symbol systems by computing systems), so some kind of balance must be kept between what the system needs to know and how much that
knowledge requires of its resources.

Abstract. We present the PINPOINT project aimed at designing a semi-automated risk assess- ment framework to support decision-making for military operations and civilian missions. These missions typically take place in critical international areas of remote countries with little infrastructure and require specific technical and operational support. To effectively plan and execute risk mitigation strategies, decision-makers require approaches for sys- tematic, comprehensive risk analysis and proposals for measures. During the planning and execution phases, risk analysis must be updated regularly based on available informa- tion derived from Open Source data in combination with reliable and accurate Position-, Navigation- and Timing-Data (PNT) monitoring. The proposed framework focuses on in- novations in the areas of a systematic risk analysis with derived mitigation actions, Open Source Intelligence (OSINT) related to these areas, Artificial Intelligence (AI) technologies for multiple types of sources (sensors, news, social media) and PNT monitoring based on GNSS-data analysis.

Abstract. Context Space Theory (CST) is a geometrical approach used to represent contexts and situations in situation-aware computing applications. In this theory, situations are repre- sented in a multidimensional space, where each dimension corresponds to an interesting feature of the context. The primary advantage of CST lies in its capacity to effortlessly integrate multiple factors, creating a meaningful representation of situations that can be observed and manipulated by experts. Moreover, it empowers experts to customize the sit- uation space to align with their knowledge and understanding of the situation. However, when applied to real-world scenarios, modeling complex situation spaces can be time- consuming and labor-intensive. This is due to the manual effort required in defining con- tribution functions for each context feature, as well as determining weights and thresholds to identify the situation space.
To address this challenge, the paper proposes a hybrid approach that combines decision trees with the CST, thereby expediting the definition of situation spaces. Decision trees are employed to automatically identify an initial definition of the contribution functions and weights, reducing the workload on human experts. To demonstrate the efficacy of this approach, the paper showcases a case study focused on the management of the Covid-19 pandemic situation in Italy.

Abstract. In order to design a system to support a users’ situation awareness as they navigate and execute in a complex domain, the information landscape for the domain must be defined. Such a landscape becomes critical for recognizing and evaluating deficiencies in a user’s cognitive processing of the information landscape, and as such, the user’s state of situation awareness. This paper leverages previous work in defining the information landscape for the domain of rotorcraft pilotage, and explores an approach to identifying common deficiencies in a user’s awareness of the information landscape. This paper takes into account prominent frameworks on situation awareness, and presents a methodology leveraging Large Language Models (LLMs) to identify common information deficiencies occurring at various phases of rotorcraft flight through text mining aviation incident and accident reports. Once these deficiencies are identified, the system can provide holistic situation awareness considering a larger data space than the user, and also provide human-centered awareness of prioritized concerns of situation awareness factors and possible mitigations.

Abstract. The rapidly advancing technological development allows for increased automation of traffic behaviour, including nautical systems. Innovative technologies for cognitive situa- tion modelling form the basis for an autonomous assessment of environmental conditions, which can then be used to realise autonomous navigation behaviour. Cutting-edge vehi- cles designed for autonomous operations are currently in development, demonstrating the capability to operate seamlessly even in busily frequented harbour areas and waterways. In this article, we present a novel framework for analysing the spatio-temporal nautical behaviour of maritime surface vessels. In a scenario, we assess and analyse navigation data gathered from the Kiel Fjord as part of a demonstration use case. The presented frame- work integrates a variety of different approaches and thus represents the basic technology for assessing and modelling ship behaviour prior to operational use, as well as detecting anomalous events during utilisation.