A framework for testing non-functional properties in software product lines
Loading...
Date
2025-08
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of New Brunswick
Abstract
In software product line engineering (SPLE), delivering high-quality end products is essential for marketplace success. SPLE is a systematic approach to developing a family of software products from a set of reusable software artifacts. The quality of a software system depends on both its functional properties (FPs)—the functionalities it provides— and its non-functional properties (NFPs)—the quality attributes it possesses. Non-functional properties are particularly critical because they directly impact user satisfaction, determine project success, and significantly influence market acceptance.
While research in software product line testing has expanded over the past decade, the focus has predominantly been on testing functional properties. Despite their significance, non-functional properties testing has received limited research attention, representing a substantial gap in the field. The primary objective of this thesis was to investigate a framework for testing the non-functional properties of final software applications within software product lines, particularly those developed using the feature and goal model approach.
Testing non-functional properties in finalized applications presents significant challenges for application test engineers within established software product line practices. These challenges include the lack of clear and testable non-functional requirements, the absence of systematic management of such requirements from the early stages, and the lack of reuse strategies in traditional testing approaches. These issues contribute to ambiguity, hinder traceability and validation, and result in redundant testing efforts, complicating the overall testing process for non-functional properties.
Goal models have proven useful in single software engineering for capturing both functional and non-functional requirements of a system in terms of goals based on stakeholders’ preferences. In SPLE, research has used goal models to address non functional aspects in traditional feature models during domain analysis and as a requirements tool for guiding feature selection. However, the application of goal models in testing of product line core components or final applications is still unexplored. Scholars argue that goal models are most effective when their concepts and contents are considered throughout all phases of development, as this facilitates requirements traceability.
To this end, this thesis contributes to existing research by presenting a comprehensive testing framework utilizing goal models to address the testing of non-functional properties of final applications in the context of software product lines developed with the feature and goal model approach. The framework integrates two complementary methodologies: (1) a methodology to initiate non-functional property testing during domain engineering phase by creating reusable test case templates for core features, and (2) a methodology to create executable test cases for final applications, leveraging the reusable domain test case templates. Together, these methodologies provide practical solutions to existing challenges, enabling systematic reuse, improved efficiency, and more effective testing of non-functional properties in SPL applications.