Evaluating Optimization Solvers and Robust Semantics for Simulation-Based Falsification

8 pages•Published: September 25, 2020

Abstract

Temporal-logic based falsification of Cyber-Physical Systems is a testing technique used to verify certain behaviours in simulation models, however the problem statement typically requires some model-specific tuning of parameters to achieve optimal results. In this experience report, we investigate how different optimization solvers and objective functions affect the falsification outcome for a benchmark set of models and specifications. With data from the four different solvers and three different objective functions for the falsification problem, we see that choice of solver and objective function depends both on the model and the specification that are to be falsified. We also note that using a robust semantics of Signal Temporal Logic typically increases falsification performance compared to using Boolean semantics.

Keyphrases: Cyber-Physical Systems, falsification, testing

In: Goran Frehse and Matthias Althoff (editors). ARCH20. 7th International Workshop on Applied Verification of Continuous and Hybrid Systems (ARCH20), vol 74, pages 259--266

Links:	https://easychair.org/publications/paper/XFJH
	https://doi.org/10.29007/f4vs