Navigation
View Diagram

AMLAS outline

Example 1 - Pedestrian identification
Note 1 - Consider architectural features
Note 2 - Consider human contribution
System safety requirements [A]
Description of the system environment [B]
System description [C]
ML component description [D]

ML safety assurance scoping argument pattern [F]

Example 5 - Defining acceptable risk level
Example 6 - Translating safety requirements
Example 7 - Performance and robustness
Note 5 - Semantic gap
Note 6 - Under‐specificity
Note 7 - Safety criticality
Note 8 - Performance and robustness
Note 9 - Measurable features
Note 10 - Overall performance
Note 11 - Dimensions of variation
Note 12 - Tolerable risk

ML safety requirements argument pattern [I]

Data requirements [L]
Data requirements justification report [M]

Example 14 - Vehicle gathering video data
Example 15 - Handling missing data
Example 16 - Retinal scans
Example 17 - Augmenting the dataset
Example 18 - ML classifier for cancer
Note 16 - Justifying collected data
Note 17 - Rationale for existing data sources
Note 18 - Using simulators
Note 19 - Preprocessing of data
Note 20 - Labelling data
Note 21 - Combinations of variation
Data generation log [Q]

Example 19 - JAAD open dataset
Example 20 - Using US data in the UK
Example 21 - Data collection using controlled trials
Example 22 - Completeness of datasets
Example 23 - Occlusion
Note 22 - Financial and practical concerns
Note 23 - Simulation for data augmentation
Note 24 - Context-specific features
Note 25 - Dimensions of variability
Note 26 - Quantisation
Note 27 - Balance of datasets
Note 28 - Imbalanced dataset

ML data argument pattern [R]

Example 24 - Medical prognosis problems
Example 25 - Deep Neural Networks
Example 26 - Data samples from a photo simulator
Note 29 - Overfitting to training data
Note 30 - Trade-offs
Model development log [U]

Example 30 - Image analysis
Example 31 - Perception pipeline
Internal test results [X]

Model learning argument pattern [W]

Example 33 - Vehicle stopping distance
Example 34 - Contextual mutators
Example 35 - DeepMind retinal diagnosis system
Example 36 - Contextually meaningful values
Example 37 - Airborne collision avoidance system
Note 31 - Development and verification independence
Note 32 - Independence from verification data
Note 33 - Test completeness
Note 34 - Human evaluation
Note 35 - Point-wise robustness
ML verification evidence [Z]
Verification log [AA]

ML verification argument pattern [BB]

Example 42 - ML component for object classification
Example 43 - Pedestrian detection
Example 44 - Pedestrian detection - occlusion
Example 45 - Computational power
Example 46 - Pedestrian detection performance requirement
Note 37 - Violations of assumptions
Note 38 - Monitoring the system
Note 39 - Latency
Erroneous behaviour log [DD]

Operational scenarios [EE]

ML deployment argument pattern [GG]

Afterword - authors and thanks

AMLAS was developed and written by Dr Richard Hawkins, Dr Colin Paterson, Dr Chiara Picardi, Dr Yan Jia, Professor Radu Calinescu and Dr Ibrahim Habli, University of York.

It would not have been possible to produce this guidance without the numerous insightful interactions of the authors with a wide range of experts across industry and academia. We cannot acknowledge them all personally here, but their contributions are very much appreciated. In particular, we would like to thank the following AAIP Programme Fellows and colleagues who kindly reviewed and provided feedback on an initial draft of the guidance:

  • Rob Ashmore (DSTL)
  • Alec Banks (DSTL)
  • Simon Burton (Fraunhofer IKS)
  • Jelena Frtunikj (ArgoAI)
  • Lydia Gauerhof (Robert Bosch GmbH)
  • Simos Gerasimou (University of York)
  • Farah Magrabi (Macquarie University)
  • Mike Parsons (University of York)
  • Roger Rivett (Jaguar Land Rover (retired))
  • Simon Smith (CACI)
  • Mark Sujan (Human Factors Everywhere)
  • Sean White (NHS Digital)

We would very much value feedback on the guidance provided in AMLAS. We would in particular encourage the reader where appropriate to apply this guidance to the development of systems and share those experiences with the authors.

This work has been funded by Lloyd's Register Foundation and the University of York through the Assuring Autonomy International Programme.

  • Assuring Autonomy logo
  • Lloyd's Register Foundation Logo
  • University of York logo

Our site depends on cookies to provide our service to you. If you continue to use this site we will assume that you are happy with that. View our privacy policy.