By Ferhan Bugay, Executive Director, Aero Support Group · 16 November 2026
An SMS can be complete on paper and still not manage risk. The SMS Maturity Assessment is our tool for finding out which one you have. It scores each feature of a safety management system on four stages, from documented to effective, against the SMS framework in ICAO Annex 19, 3rd Edition, which is applicable from 26 November 2026. An assessment team scores independently, the tool flags where they disagree, and it will not accept a high score without evidence beyond the manual. You can download it free at the end of this article.
Why maturity, and not compliance alone
A compliance checklist asks whether a requirement is met. That works for a document review, but it hides the difference between an SMS that exists and one that works. A hazard log can be in place and never updated from the hangar floor. Safety performance indicators can be defined and never lead to a decision. The 3rd Edition puts more weight on that difference: States are to assess the SMS of their service providers periodically and monitor their safety performance. A maturity scale makes the difference visible feature by feature, before the authority does.
What the tool assesses
The assessment covers 28 features in five pillars. The first four follow the components and elements of the SMS framework in Annex 19, Appendix 2, together with the SMS provisions of Chapter 4. The fifth pillar, safety culture and human performance, is ASG’s addition, because many SMS problems we find in audits start there. Features added or changed by the 3rd Edition are marked in the workbook.
| Pillar | Features | Weight (default) |
|---|---|---|
| 1. Safety policy, objectives and resources | 10, including safety objectives, accountable executive and resources, safety manager, emergency response coordination, SMS documentation, and system description and interfaces | 15% |
| 2. Safety risk management | 5: hazard identification, interface hazards, safety reporting, risk assessment and mitigation, safety investigation | 20% |
| 3. Safety assurance | 5: safety performance measurement, progress towards safety objectives, effectiveness of risk controls, management of change, continual improvement | 30% |
| 4. Safety promotion | 3: training and competence, safety communication, feedback to reporters | 17.5% |
| 5. Safety culture and human performance | 5: just culture in practice, reporting culture, human performance, operational pressure and authority to stop, safety culture measurement | 17.5% |
The weights can be changed on the setup sheet to fit the organisation and the purpose of the assessment.
The scoring scale
| Score | Stage | What must be true |
|---|---|---|
| 0 | Not present | No evidence that the feature exists. |
| 1 | Present | The feature is documented in the SMS manual or management system. |
| 2 | Suitable | The documented feature suits the size, nature, complexity and risks of the organisation, with owners, methods and records defined. |
| 3 | Operating | The feature is in use and produces output consistently over time. |
| 4 | Effective | The feature achieves its intended outcome and has a positive effect on safety. |
The scale follows the Present, Suitable, Operating and Effective approach used in the EASA Management System Assessment Tool, with 0 added for “not present”. A feature counts as achieved at 3 by default; for a stricter assessment the threshold can be set to 4.
How the team scoring works
Up to five assessors score every feature on their own. The tool takes the average as the consensus score (median or lowest can be chosen instead) and flags any feature where the assessors are two points or more apart, so the team talks about it before agreeing a result. Each assessor also ticks the evidence seen: document, record, interview or observation. A score of 3 or 4 with no evidence logged triggers a warning, and so does a high score that rests on documents only.
This matters because a single assessor tends to score what they expect to find. Independent scoring followed by a short consensus session gives a result the accountable executive can trust, and it shows where the assessors themselves need calibration.
What you get
- An SMS score out of 100, weighted by pillar.
- Attainment for each pillar and the weakest-link stage, the stage of the lowest-scoring feature.
- Gap points for each feature, showing how much of the missing score sits where.
- An action plan with priorities, so the team starts with the gaps that hold the system back.
How to run an assessment
- Fill in the setup sheet: organisation, type, scope, date, assessors and scoring parameters.
- Collect evidence across the review period, including the most recent month. Select record samples yourself where possible.
- Let each assessor score every feature on what they saw, not on what the manual says.
- Hold a consensus session on the flagged features and the evidence warnings.
- Agree actions on the action plan, starting with high priority, and repeat the assessment after implementation.
Which ASG tool to use
Our Annex 19 SMS Assessment is built for a quick start on Amendment 2: it comes in versions for airlines and MROs, aerodromes and heliports, and other organisations, and includes a differences checklist. The SMS Maturity Assessment is for a fuller, team-based review: several assessors, calibration, adjustable weights and the fifth pillar on safety culture and human performance. Use it for internal self-assessment, pre-audit preparation, or as a structure for the periodic SMS assessment your authority will carry out.
How ASG can help
ASG runs facilitated SMS maturity assessments with an evidence review for each feature, and supports SMS implementation through ASG RADAR. See our SMS development consulting or write to info@aerosupport360.com.
References
- ICAO Annex 19, Safety Management, 3rd Edition, November 2025 (Chapter 4 and Appendix 2)
- ICAO Doc 9859, Safety Management Manual
- ICAO Doc 10159, Safety Intelligence Manual
- EASA Management System Assessment Tool
Free download: SMS Maturity Assessment
An Excel workbook with 28 SMS features in five pillars, scoring for up to five assessors, evidence checks, a dashboard and an action plan. Leave your details and the download link opens straight away; we also send it by email.
We use your details to send you the download link and to follow up on your request. If you tick the box above, we will also send you occasional ASG updates. We do not share them with third parties. See our Privacy Policy.
Frequently asked questions
Whether each of 28 SMS features is documented, suitable, operating and effective, against the SMS framework in ICAO Annex 19, 3rd Edition, plus a fifth pillar on safety culture and human performance.
Up to five. Each scores independently, and the tool flags features where they differ by two points or more.
No. Operating and effective need evidence beyond the manual, such as records, interviews or observation. The tool warns when a high score rests on documents only.
The Annex 19 SMS Assessment is a sector-specific quick start with an Amendment 2 differences checklist. The SMS Maturity Assessment is a team-based review with calibration, adjustable weights and a safety culture pillar.
Yes. Leave your details in the form and the download link opens straight away. We also send it by email.
Subscribe to Global Aviation Safety Radar
Weekly aviation safety and compliance intelligence covering ICAO, IATA, EASA, and UK CAA developments — translated into practical implications for airlines, MROs, airports, ground handlers, and aviation leaders.