poster

Sim-ATAV: Simulation-Based Adversarial Testing Framework for Autonomous Vehicles

Authors:

Cumhur Erkan Tuncali,

Georgios Fainekos,

Hisahiro Ito,

James KapinskiAuthors Info & Claims

HSCC '18: Proceedings of the 21st International Conference on Hybrid Systems: Computation and Control (part of CPS Week)

Pages 283 - 284

https://doi.org/10.1145/3178126.3187004

Published: 11 April 2018 Publication History

Get Access

Abstract

One of the main challenges in testing autonomous driving systems is the presence of machine learning components, such as neural networks, for which formal properties are difficult to establish. We present a simulation-based testing framework that supports methods used to evaluate cyber-physical systems, such as test case generation and automatic falsification. We demonstrate how the framework can be used to evaluate closed-loop properties of autonomous driving system models that include machine learning components.

References

[1]

T. Dreossi, A. Donze, and S. A. Seshia. 2017. Compositional Falsication of Cyber-Physical Systems with Machine Learning Components. In NASA Formal Methods (NFM) (LNCS), Vol. 10227. Springer, 357--372.

Google Scholar

[2]

T. Dreossi, S. Ghosh, A. Sangiovanni-Vincentelli, and S. A. Seshia. 2017. Systematic Testing of Convolutional Neural Networks for Autonomous Driving. In Reliable Machine Learning in the Wild (RMLW).

Google Scholar

[3]

Georgios Fainekos, Sriram Sankaranarayanan, Koichi Ueda, and Hakan Yazarel. 2012. Verification of Automotive Control Applications using S-TaLiRo. In Proceedings of the American Control Conference.

Crossref

Google Scholar

[4]

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In 3rd International Conference on Learning Representations (ICLR).

Google Scholar

[5]

Alan Hartman. 2005. Software and hardware testing using combinatorial covering suites. Graph theory, combinatorics and algorithms 34 (2005), 237--266.

Google Scholar

[6]

BaekGyu Kim, Akshay Jarandikar, Jonathan Shum, Shinichi Shiraishi, and Masahiro Yamaura. 2016. The SMT-based automatic road network generation in vehicle simulation environment. In International Conference on Embedded Software (EMSOFT). ACM, 18:1--18:10.

Digital Library

Google Scholar

[7]

BaekGyu Kim, Yusuke Kashiba, Siyuan Dai, and Shinichi Shiraishi. 2017. Testing Autonomous Vehicle Software in the Virtual Prototyping Environment. Embedded Systems Letters 9, 1 (2017), 5--8.

Digital Library

Google Scholar

[8]

Olivier Michel. 2004. Cyberbotics Ltd. Webots: professional mobile robot simulation. International Journal of Advanced Robotic Systems 1, 1 (2004), 5.

Crossref

Google Scholar

[9]

M. O'Kelly, H. Abbas, and R. Mangharam. 2017. Computer-aided design for safe autonomous vehicles. In 2017 Resilience Week (RWS). 90--96.

Google Scholar

[10]

Cumhur Erkan Tuncali, Theodore P. Pavlic, and Georgios Fainekos. 2016. Utilizing S-TaLiRo as an Automatic Test Generation Framework for Autonomous Vehicles. In IEEE Intelligent Transportation Systems Conference.

Crossref

Google Scholar

Cited By

View all

Crespo-Rodriguez VNeelofar Aleti ASaadatmand MLonetti FBudnik CLi JGuerriero A(2024)PAFOT: A Position-Based Approach for Finding Optimal Tests of Autonomous VehiclesProceedings of the 5th ACM/IEEE International Conference on Automation of Software Test (AST 2024)10.1145/3644032.3644457(159-170)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3644032.3644457
Habeeb PD’Souza DLodaya KPrabhakar P(2024)Interval Image Abstraction for Verification of Camera-Based Autonomous SystemsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2024.344830643:11(4310-4321)Online publication date: Nov-2024
https://doi.org/10.1109/TCAD.2024.3448306
Goyal SGriggio ATonetta S(2024)System-level Simulation-based Verification of Autonomous Driving Systems with the VIVAS Framework and CARLA SimulatorScience of Computer Programming10.1016/j.scico.2024.103253(103253)Online publication date: Dec-2024
https://doi.org/10.1016/j.scico.2024.103253
Show More Cited By

Recommendations

Sim-Tree: indexing moving objects in large-scale parallel microscopic traffic simulation
SIGSIM PADS '14: Proceedings of the 2nd ACM SIGSIM Conference on Principles of Advanced Discrete Simulation

Performance is one of the major concerns in large-scale parallel microscopic traffic simulations. This paper focuses on one of the most time-costly data structures: the two-dimensional spatial index. A drawback of using popular two-dimensional tree-...
J-Sim: A Simulation Environment for Wireless Sensor Networks
ANSS '05: Proceedings of the 38th annual Symposium on Simulation

Wireless Sensor Networks (WSNs) have gained considerable attention in the past few years. As such, there has been an increasing need for defining and developing simulation frameworks for carrying out high-fidelity WSN simulation. In this paper, we ...
Human-Guided Reinforcement Learning With Sim-to-Real Transfer for Autonomous Navigation
Reinforcement learning (RL) is a promising approach in unmanned ground vehicles (UGVs) applications, but limited computing resource makes it challenging to deploy a well-behaved RL strategy with sophisticated neural networks. Meanwhile, the training of RL ...

Comments

Information & Contributors

Information

Published In

HSCC '18: Proceedings of the 21st International Conference on Hybrid Systems: Computation and Control (part of CPS Week)

April 2018

296 pages

ISBN:9781450356428

DOI:10.1145/3178126

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 April 2018

Check for updates

Qualifiers

Poster
Research
Refereed limited

Conference

HSCC '18

Sponsor:

SIGBED

HSCC '18: 21st International Conference on Hybrid Systems: Computation and Control

April 11 - 13, 2018

Porto, Portugal

Acceptance Rates

Overall Acceptance Rate 153 of 373 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
473
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Crespo-Rodriguez VNeelofar Aleti ASaadatmand MLonetti FBudnik CLi JGuerriero A(2024)PAFOT: A Position-Based Approach for Finding Optimal Tests of Autonomous VehiclesProceedings of the 5th ACM/IEEE International Conference on Automation of Software Test (AST 2024)10.1145/3644032.3644457(159-170)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3644032.3644457
Habeeb PD’Souza DLodaya KPrabhakar P(2024)Interval Image Abstraction for Verification of Camera-Based Autonomous SystemsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2024.344830643:11(4310-4321)Online publication date: Nov-2024
https://doi.org/10.1109/TCAD.2024.3448306
Goyal SGriggio ATonetta S(2024)System-level Simulation-based Verification of Autonomous Driving Systems with the VIVAS Framework and CARLA SimulatorScience of Computer Programming10.1016/j.scico.2024.103253(103253)Online publication date: Dec-2024
https://doi.org/10.1016/j.scico.2024.103253
Bozzano MCimatti ACristoforetti MGriggio ASvaizer PTonetta S(2024)Towards Formal Design of FDIR Components with AILeveraging Applications of Formal Methods, Verification and Validation. Software Engineering Methodologies10.1007/978-3-031-75387-9_19(311-328)Online publication date: 26-Oct-2024
https://doi.org/10.1007/978-3-031-75387-9_19
Goyal SGriggio AKimblad JTonetta S(2023)Automatic Generation of Scenarios for System-level Simulation-based Verification of Autonomous Driving SystemsElectronic Proceedings in Theoretical Computer Science10.4204/EPTCS.395.8395(113-129)Online publication date: 15-Nov-2023
https://doi.org/10.4204/EPTCS.395.8
Gao LZhou RZhang K(2023)Scenario Generation for Autonomous Vehicles with Deep-Learning-Based Heterogeneous Driver Models: Implementation and VerificationSensors10.3390/s2309457023:9(4570)Online publication date: 8-May-2023
https://doi.org/10.3390/s23094570
Habeeb PDeka ND’Souza DLodaya KPrabhakar P(2023)Verification of Camera-Based Autonomous SystemsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2023.324013142:10(3450-3463)Online publication date: Oct-2023
https://doi.org/10.1109/TCAD.2023.3240131
Simbeck K(2023)They shall be fair, transparent, and robust: auditing learning analytics systemsAI and Ethics10.1007/s43681-023-00292-74:2(555-571)Online publication date: 11-May-2023
https://doi.org/10.1007/s43681-023-00292-7
Tian HWu GYan JJiang YWei JChen WLi SYe D(2022)Generating Critical Test Scenarios for Autonomous Driving Systems via Influential Behavior PatternsProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering10.1145/3551349.3560430(1-12)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3551349.3560430
Tian HJiang YWu GYan JWei JChen WLi SYe DRoychoudhury ACadar CKim M(2022)MOSAT: finding safety violations of autonomous driving systems using multi-objective genetic algorithmProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3540250.3549100(94-106)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3540250.3549100
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Recommendations

Sim-Tree: indexing moving objects in large-scale parallel microscopic traffic simulation

J-Sim: A Simulation Environment for Wireless Sensor Networks

Human-Guided Reinforcement Learning With Sim-to-Real Transfer for Autonomous Navigation

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations