TECHNICAL REPORT 9547 fso Published 1988-04-01 Programming language processors - Test methods Guidelines for their development and acceptability Processeurs de langage de programmation - Methodes d'essai - Lignes directrices pour leur élaboration et acceptabilite ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies (IsO member bodies). The work of preparing International Standards is normally carried out through IsO technical committees. Each member body interested in a subject for which a technical committee has been established has the right to be represented on that committee. may propose the publication of a technicai report of one of the following types : type 1, when the necessary support within the technical committee cannot be obtained for the publication of an international Standard, despite repeated efforts; type 2, when the subject is still under technical development requiring wider exposure; type 3, when a technical committee has collected data of a different kind from that which is normally published as an International Standard ("'state of the art", for example). Technical reports are accepted for publication directly by Iso Council. Technical reports types 1 and 2 are subject to review within necessarily have to be reviewed until the data they provide is considered no longer valid or useful. developing the programming languages themselves. It should facilitate the preparation of specific test methods which will help implementors to produce standard-conforming language processors, thereby benefitting end users. 8861 UDC 681.3.068 : 620.1 Ref. No. 1SO/TR 9547 : 1988 (E) 9547 ISO/TR S @ International Organization for Standardization, 1988 Printed in Switzeriand Price based on 6 pages Not for Resale ISO/TR 9547 : 1988 (E) CONTENTS 3 0. INTRODUCTION 3 1. SCOPE AND FIELD OF APPLICATION 2. DEFINITIONS 3 3 2.1 CONFIGURATION 2.2 4 EXTENSION 2.3 4 IMPLEMENTATION DEFINED 2.4 4 PROCESSOR 2.5 TEST 4 4 2.6 4 2.7 TEST PROGRAM 4 2.8 TEST REPORT 2.9 TEST $ SUITE 4 4 2.10 TEST TOOLS 4 2.11 REQUIRED DOCUMENTS 4 2.12 SUBSET 3. OBJECT TO BE TESTED 4 4. DESCRIPTION OF TEST METHODS 5 4.1 5 TEST SUITE 4.2 TEST TOOLS 6 4.3 TECHNICAL PROCEDURES 6 OVERALL SCHEME FOR THE TESTING 6 4.3.1 4.3.2 RELATED DOCUMENTATION 9 5. 7 GUIDELINES FOR THE DEVELOPMENT OF THE TEST METHODS 5.1 CONSIDERATION OF EXISTING TEST METHODS 7 5.2 CRITERIA FOR DESIGN OF TEST SUITES 7 5.3 8 TESTABILITY 5.4 8 5.5 9 SUBSET TESTING 5.6 6 PORTABILITY REQUIREMENTS 9 5.7 TEST TOOLS 6. AND REVISION OF TEST SUITES 9 MAINTENANCE 7. AVAILABILITY 9 8. 9 ACCEPTABILITYOFTESTMETHODS 8.1 6 CRITERIA FOR ACCEPTABILITY OF THE TEST METHOD 8.2 REVISION OF ' 10 THE TEST METHOD 2 Copyright International Organization for Standardization ed without license from IHS Not for Resale ISO/TR 9547 : 1988 (E) 0. INTRODUCTION programming language topic: area isa living new languages are developed, old ones are revised. In those two cases, conformity requirements should notbe forgotten; therefore, test considerations should be included in the development of language standards and their revisions. The aim of this document is to provide guidelines for the development of a test method based on a set of test programs and precise steps that shouid be taken in order for a test method to get formal approval from Iso. These guidelines should be read in conjunction with a given ianguage standard so as to produce These guideiines are written for test method developers and for those who will approve these test methods in Iso. The producing standard conforming language thereby processors. benefiting end users. When reading these guideiines. it should be borne in mind that programming language standards have not yet reached a level of precision and completeness to allow conformity tests produced for every aspect and feature of a language described in a progamming language standard. Testing issues are rarely the primary objective of standardiza