Managing Uncertainty in Unmanned Aircraft System Safety Performance Requirements Compliance Process

Authors: Achim Washington, Reece Clothier, Jose Silva

Abstract:

System Safety Regulations (SSR) are a central component to the airworthiness certification of Unmanned Aircraft Systems (UAS). There is significant debate on the setting of appropriate SSR for UAS. Putting this debate aside, the challenge lies in how to apply the system safety process to UAS, which lacks the data and operational heritage of conventionally piloted aircraft. The limited knowledge and lack of operational data result in uncertainty in the system safety assessment of UAS. This uncertainty can lead to incorrect compliance findings and the potential certification and operation of UAS that do not meet minimum safety performance requirements. The existing system safety assessment and compliance processes, as used for conventional piloted aviation, do not adequately account for the uncertainty, limiting the suitability of its application to UAS. This paper discusses the challenges of undertaking system safety assessments for UAS and presents current and envisaged research towards addressing these challenges. It aims to highlight the main advantages associated with adopting a risk based framework to the System Safety Performance Requirement (SSPR) compliance process that is capable of taking the uncertainty associated with each of the outputs of the system safety assessment process into consideration. Based on this study, it is made clear that developing a framework tailored to UAS, would allow for a more rational, transparent and systematic approach to decision making. This would reduce the need for conservative assumptions and take the risk posed by each UAS into consideration while determining its state of compliance to the SSR.

Keywords: Part 1309 regulations, unmanned aircraft systems, system safety, uncertainty.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1316620

References:

[1] ATSB, “A safety analysis of remotely piloted aircraft systems,” 2017.
[2] CAA, “CAP-722, Unmanned Aircraft System Operations in UK Airspace - Guidance,” London UK Civil Aviation Authority (CAA), Department of Transport (DfT), London, UK, 2015.
[3] R. A. Clothier, B. P. Williams, and K. J. Hayhurst, “Modelling the Risks Remotely Piloted Aircraft Pose to People on the Ground,” Saf. Sci., vol. 101, pp. 33–47, 2018.
[4] SAE ARP 4761, “Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment.” SAE International, 1996.
[5] NATO Standardization Agency (NSA), “STANAG 4671 - Unmanned Aerial Vehicles Systems Airworthiness Requirments (USAR),” Brussels, Belgium, 2009.
[6] EASA, “‘Prototype’ Commission Regulation on Unmanned Aircraft Operations - Explanatory Note,” 2016.
[7] JARUS Working Group 6, “Safety Assessment of Remotely Piloted Aircraft Systems,” AMC RPAS.1309, no. 2, 2015.
[8] EUROCAE, “UAS / RPAS Airworthiness Certification ‘1309’ System Safety Objectives and Assessment Criteria,” MALAKOFF, France, 2013.
[9] A. Washington, R. A. Clothier, and B. P. Williams, “A Bayesian Approach to System Safety Assessment and Compliance Assessment for Unmanned Aircraft Systems,” J. Air Transp. Manag., vol. 62, pp. 18–33, 2017.
[10] A. Washington, R. A. Clothier, B. P. Williams, and J. Silva, “Managing Uncertainty in the System Safety Assessment of Unmanned Aircraft Systems,” in 17th Australian International Aerospace Congress: AIAC 17, Melbourne, Vic, Australia, 2017, pp. 611–618.
[11] FAA, “Advisory Circular 23.1309-1E, System Safety Analysis and Assessment for Part 23 Airplanes.,” 2011.
[12] FAA, “Advisory Circular 25.1309-1A, System Design and Analysis,” US Department of Transportation, Federal Aviation Administration, 1988.
[13] R. A. Clothier and P. P. Wu, “A Review of System Safety Failure Probability Objectives for Unmanned Aircraft Systems,” in 11th International Probabilistic Safety Assessment and Management (PSAM11) Conference and the Annual European Safety and Reliability (ESREL 2012) Conference, Helsinki, 2012.
[14] SAE ARP 4754A, “Guidelines for Development of Civil Aircraft and Systems.” SAE International, 2010.
[15] NATO Standardization Agency, “AEP-83, Light Unmanned Aircraft Systems Airworthiness Requirements,” 2014.
[16] EASA, “E.Y013-01 Policy Statement Airworthiness Certification of Unmanned Aircraft Systems (UAS),” 2009. (Online). Available: https://easa.europa.eu/system/files/dfu/E.Y013-01_ UAS_ Policy.pdf. (Accessed: 23-Oct-2015).
[17] JAA/EUROCONTROL, “UAV Task-Force Final Report: A concept for European regulations for civil unmanned aerial vehicles (UAVs),” 2004.
[18] R. Clothier, B. P. Williams, J. Coyne, M. Wade, and A. Washington, “Challenges to the Development of an Airworthiness Regulatory Framework for Unmanned Aircraft Systems,” in 16th Australian International Aerospace Congress (AIAC 16), 2015, pp. 87–98.
[19] R. A. Clothier, J. L. Palmer, R. A. Walker, and N. L. Fulton, “Definition of an airworthiness certification framework for civil unmanned aircraft systems,” Saf. Sci., vol. 49, no. 6, pp. 871–885, 2011.
[20] R. A. Clothier, N. L. Fulton, and R. A. Walker, “Pilotless aircraft: the horseless carriage of the twenty-first century?,” Journal of Risk Research, vol. 11, no. 8. pp. 999–1023, 2008.
[21] M. Elbanhawi, A. Mohamed, R. Clothier, J. L. Palmer, M. Simic, and S. Watkins, “Enabling technologies for autonomous MAV operations,” Prog. Aerosp. Sci., vol. 91, pp. 27–52, 2017.
[22] R. Clothier, “Turning Hype into Reality: Unmanned Aircraft Systems and the Challenges Ahead.” AAUS, 2016.
[23] Department of Defense, “Unmanned Aerial Vehicle Reliability Study,” United States of America, 2003.
[24] SAE ARP 5150, “Safety Assessment of Transport Airplanes in Commercial Service,” 2013.
[25] E. T. Jaynes, Probability Theory: The Logic of Science. Cambridge University Press, 2003.
[26] H. Dezfuli, D. Kelly, C. Smith, K. Vedros, and W. Galyean, “Bayesian Inference for NASA Probabilistic Risk and Reliability Analysis,” NASA/SP-2009-569, 2009.
[27] T. Perez, “Ship seakeeping operability, motion control, and Autonomy - A Bayesian Perspective,” IFAC -PapersOnline, pp. 217–222, 2015.
[28] T. Perez, R. A. Clothier, and B. Williams, “Risk-management of UAS Robust Autonomy for Integration into Civil Aviation Safety Frameworks,” in Australian System Safety Conference (ASSC 2013), 2013, pp. 37–45.
[29] S. Guarro, “Risk assessment of new space launch and supply vehicles,” in 11th International Probabilistic Safety Assessment and Management Conference and the Annual European Safety and Reliability Conference 2012, PSAM11 ESREL 2012, 2012, pp. 5157–5164.
[30] S. D. Guikema and M. E. Pate-Cornell, “Bayesian Analysis of Launch Vehicle Success Rates,” J. Spacecr. Rockets, vol. 41, no. 1, pp. 93–102, 2004.
[31] United States Nuclear Regulatory Commission, “Reactor safety Study. An Assessment of Accident Risks in U.S. Commercial Nuclear Power Plants,” 1975.
[32] E. Ancel, F. M. Capristan, J. V. Foster, and R. C. Condotta, “Real-time Risk Assessment Framework for Unmanned Aircraft System (UAS) Traffic Management (UTM),” in 17th AIAA Aviation Technology, Integration, and Operations Conference,Denver, Colarado, 2017.
[33] L. C. Barr, R. L. Newman, E. Ancel, C. M. Belcastro, J. V. Foster, J. K. Evans, and D. H. Klyde, “Preliminary Risk Assessment for Small Unmanned Aircraft Systems,” in 17th AIAA Aviation Technology, Integration, and Operations Conference, Denver, Colarado, 2017.
[34] B. J. M. Ale, L. J. Bellamy, R. van der Boom, J. Cooper, R. M. Cooke, L. H. J. Goossens, A. R. Hale, D. Kurowicka, O. Morales, A. L. C. Roelen, and J. Spouge, “Further development of a Causal model for Air Transport Safety (CATS): Building the mathematical heart,” Reliab. Eng. Syst. Saf., vol. 94, no. 9, pp. 1433–1441, 2009.