Commonsense Visual Sensemaking for Autonomous Driving: on Generalised Neurosymbolic Online Abduction Integrating Vision and Semantics

EasyChair Preprint 4845

Abstract

We  demonstrate the need and potential of systematically integrated vision and semantics solutions for visual sensemaking in the backdrop of autonomous driving. A general neurosymbolic method for online visual sensemaking using answer set programming (ASP) is systematically formalised and fully implemented. The method integrates state of the art in visual computing, and is developed as a modular framework that is generally usable within hybrid architectures for realtime perception and control. We evaluate and demonstrate with community established benchmarks KITTIMOD, MOT-2017, and MOT-2020. As use-case, we focus on the significance of human-centred visual sensemaking ---e.g., involving semantic representation and explainability, question-answering, commonsense interpolation--- in safety-critical autonomous driving situations. The developed neurosymbolic framework is domain-independent, with the case of autonomous driving designed to serve as an exemplar for online visual sensemaking in diverse cognitive interaction settings in the backdrop of select human-centred AI technology design considerations.

PUBLICATION NOTE.

This is a preprint / review version of an accepted contribution to be published as part of the Artificial Intelligence Journal (AIJ).∗ The article is an extended version of an IJCAI 2019 publication [74]. The overall scientific agenda (pertaining to Cognitive Vision and Deep Semantics [12]) driving this research is available at:

CoDesign Lab (EU) > Cognitive Vision  /  https://codesign-lab.org/cognitive-vision/ 

Related select publications: https://codesign-lab.org/select-papers/#cognitive vision

∗ The AIJ published version is final; it also fully incorporates all reviewer feedback.

Keyphrases: Answer Set Programming, Cognitive Vision, Deep Semantics, Human-Centred Computing and Design, Knowledge Representation and Reasoning, Spatial Cognition and AI, Standardisation in Driving Technology, Visual Abduction, autonomous driving, commonsense reasoning, declarative spatial reasoning

@booklet{EasyChair:4845,
  author    = {Jakob Suchan and Mehul Bhatt and Srikrishna Varadarajan},
  title     = {Commonsense Visual Sensemaking for Autonomous Driving: on Generalised Neurosymbolic Online Abduction Integrating Vision and Semantics},
  howpublished = {EasyChair Preprint 4845},
  year      = {EasyChair, 2020}}