Quality Attributes and Non-Functional Requirements
By the end of this module you will be able to:
- Distinguish quality attributes from functional requirements and explain why both are necessary
- Name the eight ISO/IEC 25010:2023 quality characteristics and their primary sub-characteristics
- Write a specific, measurable quality attribute scenario using the SEI six-part template
- Explain the trade-offs between competing quality attributes and apply ATAM thinking to architectural decisions
Real-world failure · 1 August 2012
Knight Capital Group: 45 minutes, $440 million, and three quality attributes that failed simultaneously.
On 1 August 2012, Knight Capital Group deployed a software update to its equity trading system. The update accidentally reactivated obsolete trading code that had been dormant for years. Within seconds of market open at 09:30, the system began executing a pattern of rapid buy and sell orders that acquired large positions in hundreds of stocks at above-market prices.
The trading system was functionally correct: it executed the orders it was instructed to execute. Every transaction settled accurately. No functional requirement was violated. What failed were quality attributes. Reliability failed: deprecated code had not been removed or disabled, leaving a latent fault that the next deployment could activate. Safety failed: there was no circuit breaker to halt runaway trading behaviour when position sizes or loss rates exceeded defined thresholds. Observability failed: alerts took 45 minutes to trigger by which point Knight Capital had lost $440 million.
The company was acquired by a competitor within weeks. The SEC's post-incident report, published in October 2013, identified the absence of these quality attribute controls as the root cause. Not a bug in business logic. Not a hardware failure. The absence of measurable, designed-for non-functional requirements.
A system can be functionally correct and still destroy a $440 million company in 45 minutes. What does that tell us about non-functional requirements?
With the learning outcomes established, this module begins by examining what quality attributes are in depth.
3.1 What quality attributes are
Every system has quality attributes: how fast it responds, how often it is available, how easy it is to change, how well it resists attack. The difference between systems that work well under pressure and systems that collapse is whether these attributes were made explicit, specified precisely, and used to drive architectural decisions before a line of code was written.
A quality attribute, also called a non-functional requirement (NFR) or quality of service requirement, describes how well a system performs a function rather than what function it performs. Quality attributes constrain the design space and determine which architectural approaches are even possible.
The distinction between functional requirements and quality attributes is precise. "The user can log in" is a functional requirement: it tells you what to build. "Login should succeed in under 200ms for 99% of requests" is a quality attribute: it tells you how well the login feature must work. A system can satisfy every functional requirement and still fail to meet any of its quality attributes. Knight Capital on 1 August 2012 is the proof.
Further examples: "The system sends email notifications" is functional. "Notifications should be delivered within 30 seconds of the triggering event under peak load" is a quality attribute. "Users can search the product catalogue" is functional. "Search should return results in under 500ms at the 95th percentile for catalogues up to 10 million items" is a quality attribute.
With an understanding of what quality attributes are in place, the discussion can now turn to the iso/iec 25010:2023 quality model, which builds directly on these foundations.
3.2 The ISO/IEC 25010:2023 quality model
ISO/IEC 25010:2023, the international standard for software quality, defines eight top-level quality characteristics. Understanding this taxonomy prevents teams from treating quality as a single undifferentiated concern. Each characteristic has distinct sub-characteristics and distinct architectural implications.
Functional suitability: does the system do what it should? Sub-characteristics: completeness, correctness, appropriateness. Knight Capital satisfied this characteristic. Functional correctness alone was not enough.
Performance efficiency: how fast and resource-efficient is it? Sub-characteristics: time behaviour (response time, throughput), resource utilisation, capacity. A p99 of 2,340ms is a measurable violation of this characteristic.
Compatibility: does it work with other systems? Sub-characteristics: co-existence, interoperability. Often the least specified in enterprise projects despite being frequently the source of integration failures.
Usability: can users accomplish their goals? Sub-characteristics: learnability, operability, user error protection. Frequently conflated with functional requirements but distinct: a feature can work correctly and still be unusable.
Reliability: does it work without failure? Sub-characteristics: availability, fault tolerance, recoverability. The Knight Capital circuit breaker failure is a reliability failure under the fault tolerance sub-characteristic.
Security: does it protect against threats? Sub-characteristics: confidentiality, integrity, non-repudiation, authenticity. Zero-trust architecture is a direct response to the security characteristic.
Maintainability: can it be changed efficiently? Sub-characteristics: modularity, reusability, analysability, testability. Code with no component boundaries has near-zero analysability.
Portability: can it run in other environments? Sub-characteristics: adaptability, installability, replaceability. Container-based deployment is a direct architectural response to this characteristic.
“The quality model determines which quality characteristics will be taken into account when evaluating the properties of a software product.”
ISO/IEC 25010:2023 - Section 4.2, Product Quality
The significance of this standard is that it provides a shared vocabulary for specifying what quality means in a specific context. Without this vocabulary, teams argue about whether a system is good enough with no common reference. With it, they can specify which characteristics matter, at what levels, and verify whether the system meets them.
With an understanding of the iso/iec 25010:2023 quality model in place, the discussion can now turn to writing measurable quality attribute scenarios, which builds directly on these foundations.
3.3 Writing measurable quality attribute scenarios
Vague quality attributes cannot drive architectural decisions. "The system should be fast" could mean 10ms or 10 seconds. "The system should be available" could mean 99% uptime (87 hours of downtime per year) or 99.999% (5 minutes per year). These differences have entirely different architectural implications.
The Software Engineering Institute (SEI) at Carnegie Mellon University provides a scenario template that forces specificity. A quality attribute scenario has six elements: source, stimulus, environment, artefact, response, and response measure. All six must be specified for a scenario to be useful.
Source: who or what generates the stimulus? A user, an external system, an attacker, a monitoring service.
Stimulus: the specific event that occurs. A search query, a login attempt, a database failure, a configuration change.
Environment: the system state at the time. Normal load, peak load, post-failure mode, during a deployment.
Artefact: the specific component or system affected. The search service, the authentication module, the entire system.
Response: what the system does in response to the stimulus. Returns results, rejects the request, fails over to a replica.
Response measure: how you know the response was adequate. In under 400ms, at the 95th percentile; with no customer-visible data loss; within 30 seconds.
A complete performance scenario: "When 500 customers simultaneously request account balances during normal business hours, the balance API returns correct data in under 150ms at the 99th percentile."
An availability scenario: "When any single database instance fails during business hours, the system continues serving customer read requests within 30 seconds, with no customer-visible data loss."
A security scenario: "When an unauthenticated request is made to any customer account endpoint, the system rejects the request with HTTP 401 and logs the attempt, within one network round-trip."
A Knight Capital-style safety scenario that was not written: "When any trading service accumulates a loss exceeding 1% of daily capital in any 60-second window, all order submission is halted within 5 seconds and an incident is raised automatically."
Common misconception
“NFRs are optional extras to add after the core features are built.”
Knight Capital's $440 million loss was not caused by a missing feature. The system was functionally complete. What was missing were the reliability (circuit breaker), safety (trading halt), and observability (real-time alerts) quality attributes. NFRs are system survival requirements. They must be specified before architecture is designed, not retrofitted after a production incident reveals their absence.
With an understanding of writing measurable quality attribute scenarios in place, the discussion can now turn to architecture trade-off analysis (atam), which builds directly on these foundations.
3.4 Architecture trade-off analysis (ATAM)
Quality attributes are not free and they conflict. Improving one attribute often makes another worse. Architecture is, in large part, the practice of making these trade-offs deliberately and recording the reasoning so that future decisions do not inadvertently reverse them.
The Architecture Trade-off Analysis Method (ATAM), developed at the SEI at Carnegie Mellon University, is a structured evaluation process for assessing how architectural decisions affect competing quality attributes. ATAM was designed to be run before finalising an architecture, not after. It identifies sensitivity points (decisions that strongly affect a single quality attribute), trade-off points (decisions that affect two or more quality attributes in conflicting ways), and risks (architectural decisions that may not support a quality attribute requirement under certain conditions).
The most common trade-offs in practice: strong consistency versus high availability (the CAP theorem, published by Eric Brewer in 2000, shows that a distributed system cannot have both during a network partition); performance versus security (every authentication check and input validation step adds latency); simplicity versus scalability (a single server is simple to operate but cannot scale horizontally); time to market versus maintainability (a shortcut today is a structural cost tomorrow, and architectural debt is the highest-interest form).
Adding a caching layer improves performance (cache hits reduce latency from 100ms to 5ms) but hurts consistency (cached data may be stale), increases operational complexity (another component to monitor and configure), and raises infrastructure costs. These trade-offs are not good or bad in the abstract. They are appropriate or inappropriate for the specific use case. The job of the architect is to make the trade-off explicitly, based on measured quality attribute requirements.
“There are no silver bullets. Every architectural decision is a trade-off.”
Bass, Clements, Kazman (2021) - Software Architecture in Practice, 4th ed., Chapter 3: Understanding Quality Attributes
The formal expression of this in the SEI methodology is ATAM, which provides a structured process for evaluating how architectural decisions affect competing quality attributes. You cannot optimise for all quality attributes simultaneously. You choose which ones matter most for the specific context and design accordingly.
With an understanding of architecture trade-off analysis (atam) in place, the discussion can now turn to quality attribute utility trees, which builds directly on these foundations.
3.5 Quality attribute trade-off workshop
Use this workshop to explore the trade-off landscape for a hypothetical system. Raise sliders to express the priority of each quality attribute. The tool detects known tensions (combinations that create architectural conflicts) and synergies (combinations that reinforce each other). This mirrors the ATAM sensitivity and trade-off point analysis.
With an understanding of quality attribute trade-off workshop in place, the discussion can now turn to quality attribute utility trees, which builds directly on these foundations.
3.6 Quality attribute utility trees
A quality attribute utility tree is a structured way to prioritise quality attributes by business impact. Developed as part of the ATAM methodology, it forces stakeholders to move from vague aspirations to specific scenarios ranked by importance and difficulty.
The tree has three levels. The root is "utility": the overall quality of the system. The first level of branches are the quality attributes that matter for this system (performance, availability, security, maintainability). The second level are the specific scenarios for each attribute, derived from real business requirements. Each scenario is rated on two dimensions: its importance to the business (high, medium, low) and its expected difficulty to achieve architecturally (high, medium, low). High-importance, high-difficulty scenarios are the architectural risks that ATAM was designed to surface.
For Knight Capital, a proper utility tree analysis would have placed "trading halt on runaway loss" as a high-importance, high-difficulty scenario under the safety quality attribute. That placement would have made it a first-class architectural concern that required a concrete design response, not a missing feature.
"The system should be fast" is not a quality attribute. "95% of checkout requests should respond in under 300ms at peak Black Friday load, measured at the client" is one. The difference is not pedantry. It is the difference between an architecture you can design deliberately and one you can only hope works.
Knight Capital lost $440 million on 1 August 2012. The trading system executed orders correctly. Which quality attributes failed, and what does this demonstrate about NFRs?
A specification says: 'The checkout service should handle peak load.' Using the SEI scenario template, what is missing from this statement?
According to ISO/IEC 25010:2023, which quality characteristic specifically covers fault tolerance and recoverability?
Key takeaways
- Quality attributes describe how well a system performs a function, not what function it performs. Knight Capital's $440 million loss proves that functional correctness is not sufficient.
- Vague quality attributes cannot drive architectural decisions. Use the SEI six-part scenario template: source, stimulus, environment, artefact, response, and response measure.
- ISO/IEC 25010:2023 defines eight quality characteristics: functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability, and portability.
- Quality attributes trade off against each other. Performance versus security, availability versus consistency, simplicity versus scalability. The Architecture Tradeoff Analysis Method (ATAM) provides a structured process for making these trade-offs explicit.
- Quality attribute scenarios must be defined before the architecture is designed, not after. Designing first and specifying quality attributes second produces Knight Capital-style failures.
- A quality attribute utility tree forces prioritisation by business impact, surfacing high-importance, high-difficulty scenarios as architectural risks before any code is written.
Standards and sources cited in this module
ISO/IEC 25010:2023, Systems and Software Quality Requirements and Evaluation (SQuaRE)
International standard for software quality. The primary taxonomy reference for Section 3.2. Defines the eight quality characteristics and their sub-characteristics.
Bass, L., Clements, P., Kazman, R. (2021). Software Architecture in Practice, 4th ed. Addison-Wesley
Chapter 3: Understanding Quality Attributes; Chapter 4: Availability; Chapter 9: Performance
Defines quality attribute scenarios and the ATAM evaluation method. The SEI scenario template in Section 3.3 and the utility tree in Section 3.6 both come from this source.
Sections II-III: Findings of Fact
The regulatory findings that document the Knight Capital failure. Primary source for the opening case study. Identifies the absence of quality attribute controls as the root cause.
The original SEI technical report defining ATAM. Referenced in Section 3.4 for the Architecture Trade-off Analysis Method, sensitivity points, trade-off points, and risks.
The original presentation of the CAP theorem. Referenced in Section 3.4 as the formal basis for the consistency versus availability trade-off in distributed systems.
Chapter 3: The Architecture Tradeoff Analysis Method
Detailed case studies of ATAM in practice. Useful for understanding how quality attribute utility trees are constructed and how sensitivity and trade-off points are identified in real systems.
What comes next: Quality attributes drive architectural decisions. But decisions made without records become institutional folklore that disappears when people leave. Module 4 introduces Architecture Decision Records: the practice of capturing why a decision was made, what alternatives were rejected, and what consequences it creates.
Module 3 of 22 in Foundations