© 1979-2003 by Ludwig Benner, Jr. .All rights reserved.

Guide 6

TASK GUIDANCE FOR PREPARING
INVESTIGATION TEST PLANS

For Use During MES-Based Investigations


Table of Contents

    Part 1: OVERVIEW

    Part 2. TEST PLAN DEVELOPMENT PROCESS

    Part 3. QUALITY MANAGEMENT. Part 4. ATTACHMENTS
    Go to Guide: 0 1 2 3 4 5 6 7 8 9 10

    FINDING EBs WITH TESTS

    Incident investigators sometimes turn to examinations, tests, teardowns or simulations (referred to as tests here) to find data to transform into Event Blocks. Their challenge is to assure that any such efforts produce needed and acceptable EBs for their Matrixes.

    The general task is to

    1. identify what gap in Matrix to address with tests
    2. prepare, with expert help, a plan for performing the tests
    3. implement the plan
    4. convert the results into EBs

    Procedures for this task are contained in this Guide.

    Part 1. OVERVIEW

    INTRODUCTION

    The Multilinear Events Sequencing technology-based investigation process is a "research defining" methodology. It defines unknown areas of events flows as gaps on the MES Matrix. Gaps indicate where the process being investigated is not understood. To bridge Matrix gaps, MES-tree analyses techniques, among others, can generate hypothetical EBs to describe the missing events. Confirming those events usually requires acquisition of new data, or reexamination of previously acquired data. To acquire new data from people, talk to them, as discussed in EB Guide 1. To acquire data from things, and create things EBs, investigators use other techniques such as examinations, tests, teardowns and simulations. This Guide describes test planning tasks needed to ensure efficient effective data acquisition.

    Aggressive or conscientious Investigators intuitively try to eliminate all unknowns during investigations. That requires "research" time and money, among other demands. Investigators use the fact that the accident occurred and the need to understand it to justify this research. A test planning process helps investigators and investigation managers distinguish data actually needed from what would be nice to have, by forcing decisions about objectives, methods, outputs and funding, among other tasks. Thus the test planning process can serve as an investigation management tool, as well as a tool to ensure sound, efficient technical results.

    OBJECTIVES

    The objective of any tests should be to extract needed data from objects or people to create gap-filling EBs; to prevent the unintended destruction of data; and to promote testing or examination efficiency and value. The objective of the Guide is to present the main tasks required to accomplish this efficiently and effectively, and support development of a complete description and explanation of what happened.

    APPLICABILITY

    This Guide is applicable to tests conducted for investigators. The "Do no harm" rule addresses the handing of people, objects and data by the investigator. During the investigation, try to prevent unintended changes to people or objects as they are handled during the initial stages of an investigation. Plans and procedures to implement this rule should be a part of every investigator's training.

    This test planning process is applicable to the full range of investigation testing tasks performed to support MES investigations and Matrix development. Its main focus is testing and simulations or reenactments to support verification possible scenarios to fill gaps remaining after the investigator's initial examination of objects is completed. That kind of task is usually performed by "experts" rather than the investigator.

    DATA REQUIRED

    The input data required for the test planning process includes data about the:

    1. Postulated events during the gaps in the MES Matrix that the tests are to demonstrate or validate.

    2. Availability of facilities, personnel and methods needed to produce the needed EBs.

    3. Physical entities that are available for testing.

    4. People who have an interest in the investigation, and their interests and capabilities.

    5. Information about funding for the test or examination.

    The data should be verified before it is used in a test plan. Verification of the data used for test planning may require new inquiries and additional investigative effort, particularly with respect to data items 1 and 4.

    DATA SOURCES

    The test planning data sources vary with the type of data required. Data about the interests and capabilities of people interested in the investigation can be acquired by observation and recording of comments or inquiries addressed to the investigator during the investigation, and by identifying the persons whose actions - especially "programmers"- affected events on the Matrix. Data about the hypothesized events during gaps and the physical entities available for testing generally are acquired by observation of the Matrixes and objects surviving the phenomenon during the course of the investigation. Information about the availability of the resources needed to do the testing have to be acquired by an investigator much like any other "purchasing" process.

    GUIDING PRINCIPLES FOR TEST PLAN DEVELOPMENT.

    Development of test and simulation plans should be governed by some essential principles. Every investigator who handles objects during an investigation should know and observe these principles.

    1. No Plan, No Tests!

    The GOLDEN RULE of INVESTIGATION TESTING. Never allow or initiate any destructive testing or examination that could introduce changes into any one-of-a-kind physical object without a test plan. Do not begin test work on an object without a test plan agreeable to all affected parties having a legitimate interest in what happened, the test results and data produced.

    2. Whoever Owns the Ball Names the Game.

    Respect the property rights of the owner of the physical object to be tested. Never do any testing on an object without the concurrence of the owner in the test plan. If you are the owner, do not let others change or destroy your object until you are satisfied the planned tests will provide ALL the data you need - especially if destructive testing will occur. This can get complicated when government and private sector investigators are involved, and you may need legal advice.

    3. Respect Management-by-Objectives Principles.

    Tests on physical objects should be performed only to achieve some explicitly stated objective. Any test should contributes to better understanding of the part of process indicated by a gap on the MES Matrix. If you don't recognize how it will, delay the testing until you find out - preferably on paper.

    A indispensable application of this principle is to the funding plan for proposed tests and simulations. This is the step that develops the value of the data to be acquired through testing. It also introduces the consideration of optional ways to get the desired data, or devising a way to use the gap in the investigation. For example, if the investigation defines a gap which requires research to determine what happened, it might be appropriate to propose that a party which should be able to provide the answers do the research, rather than the investigating party.

    4. Keep Test(s) Relevant.

    The mental movie framework should control hypothesis development, and debris testing should be limited to identifying, describing or confirming changes of state during the mishap. Test outcomes should provide specific EBs for the process model you are constructing or reconstructing. Testers love to give you data their way. They also like to propose "just one more test" after the results are in; don't fall into the trap of using your tests (and money) for their research purposes. Do your hypothesizing and conjecturing on paper. That is faster, cheaper and easier. Don't use tests to go fishing.

    5. Scale the Plan to the Value of the Data it Will Produce.

    Tests aren't free. Balance the results (certainty, credibility, legal needs, etc.) against the cost (dollars, delays, diversion of manpower, etc.) of the testing to be sure the test(s) planned is properly tailored to the need.

    6. Progressive destruction demands priorities

    Don't destroy it before you get the data it holds.Any actions that change the state of an object before you complete an examination or test series jeopardizes needed data. Mentally walk through the test sequence to ensure that when you want to examine an object, it has not been changed by prior work that could have been reschedule or resequenced - unless the decision is made after due deliberation and consensus of the interested parties. For example, determine if the consequences of trying to operate a system or component before tearing it down would do more harm than tearing it down and then reassembling it to test its operation.

    7. Remember the 6 Ps.

    Systematically consider extracting data from physical objects in all major ways, including non-destructive examination of their PROPERTIES, appearance or attributes, analysis of discernible PATTERNS, observing POSITION relationships, studying PAPERS with specifications or procedures, or asking PEOPLE about observations or habits, as well as inspecting or testing PARTS. Investigators can not predict where useful data will be found in a specific investigation, so they should be prepared to use any search tools available. to help them find it before they explore testing options.

    Simulations are treated as "tests" in this Guide if they are conducted for the same purposes. Planning simulations requires diligence to ensure that the actions and interactions just before the gap in the events flow are faithfully reproduced. If this is not possible, the results of the simulations should be recognized as approximations, and assigned the confidence warranted by the circumstances and their predictive validity. These considerations must be addressed during the simulation planning tasks.

    Teardowns are also treated as "tests" in this Guide if they are conducted for the same purposes. Teardowns risk changing states of components of the object during the teardown, destroying the record of the states during the occurrence. Planning teardowns requires diligence to ensure that this does not happen which, depending on the object, may require judicious sequencing of the disassembly tasks, tearing down identical objects to establish benchmarks, photographic documentation of observations during the teardown, or other precautions. A thorough dialogue about the plan is better than an aggressive and hurried teardown.


    Part 2. TEST PLAN DEVELOPMENT PROCESS

    PLANNING PROCESS TASKS

    The main tasks in the MES test planning process are:

    Determine what data you need, using MES Matrix or Matrix gaps.

    Determine if and how you can get the data by testing the physical object(s) available. Some common tests are listed in the last section of this Guide.

    Write a Test Plan that will produce the data to support the MES-based event building blocks you are trying to verify or discover.

    Get everyone interested in the test outcomes to concur in the plan.

    Do a quality check on the plan to assure that the planned test will address all the required data elements, and that it will actually produce the data to support the MES-based event building blocks you need to verify.

    BASIC TEST PLAN ELEMENTS.

    1. Test/examination objectives.

    Which MES Matrix gap is being addressed, and for which hypothesized events in that gap are validating data being sought? If more than one party is involved, objectives desired by each of the parties may have to be stated. Focus on gaps - it helps keep testing efforts concentrated on what needs to be known. Never lose sight of the end point - EBs to add to the MES Matrix to make it more complete.

    2. Physical objects to be examined.

    Describe the object(s) being tested or examined and document them to assure tests are performed on parts everyone expects to be tested, and so their original state can be replicated if necessary. Indicate here any protective measures for the objects, required to preserve them prior to the test, or after the test. A good practice is to use chain-of-custody identifiers on the object and all documentation about the object.

    3. General test approaches.

    Use this section to record any general principles to be followed by the testers, any assumptions that need to be documented before the test begins, and how the objects and tests will be documented. This is where any differences in the approach can be reconciled. For example, should a device be operated before it is dismantled, or should the dismantling be done before it is operated?

    4. Test/examination procedures to be followed.

    State the name of the test protocol and equipment and the citation, if it has been formalized in the literature or elsewhere. Alternatively, record the procedure and equipment in detail if it has not been formalized elsewhere. Define and document the measurements to be produced. If unique, make sure to define the procedures carefully to avoid complaints later. Use an MES Matrix to lay out the test procedure so you can check it for oversights and omissions before you start. Specify chain-of-custody requirements, precautions and responsibilities, points of contact, and any security tasks.

    This section should state the specifications for the deliverables that will be produced and quality control criteria that will be used to verify the results.

    5. Interpretation of results.

    Hypothesize potential test outcomes and determine among the parties how each potential outcome will be stated in EB formats on MES Matrix. If this task is done properly, the specific outcome may be uncertain, but there should be no surprises at the end of the test. The place for discovery of this kind is in the MES-tree analysis procedure for the Matrix gaps.

    6. Schedule of testing.

    State what work will be accomplished, when it will start and where, the schedule for any drafts to be circulated if applicable, and when the deliverables will be delivered.

    7. Distribution of deliverables.

    Depending on the nature of the investigation, deliverables from the testing efforts my require controls that should be described in this section. State who "owns" rights in the deliverables, and who can use or allocate them and for what purposes in the future. Any confidentiality or security precautions should be specified in this section.

    8. Disposition of tested objects.

    State who will specify disposition of the tested object(s) and the time limit for disposition. Usually the owner is given control over the disposition. Anticipated litigation may influence this section.

    9. Funding of test work.

    Specify who pays what to whom. Who will pay for the test(s), or if more than one party, who will pay for what part of the test(s) and who will spend and who will get what monies? Be aware that this requirement can be used very effectively to dissuade proponents of unsound hypotheses to focus on hypothesis that MES-Trees have shown to have some credibility. But if somebody wants to do a special test the investigator has good reason to believe is not needed (based on the MES Matrix), is willing to pay for it and live with the results, and it can be done without interfering with other tests, work it into the test plan.

    OPTIONAL TEST PLAN ELEMENTS.

    10. Media inquiries.

    If the investigation has attracted public interest, specify who will respond to the media, and how inquiries to the individuals and organizations actually performing the test(s) or others who might be called should be referred to that spokesperson.

    11. Safety precautions.

    Where risk of injury or property damage is associated with the test procedures, any required risk control precautions should be specified in the Test Plan. This will be determined by the nature of the test(s) or the test object(s) OSHA or other regulatory compliance instructions are appropriate for this section.

    12. Concurrence.

    When more than one party is involved, get every interested party to affix a signature to the test plan signifying concurrence in the plan. Be sure to get concurrences in any changes to the Test Plan that may be necessitated by staged or phased outputs.

    Part 3. QUALITY MANAGEMENT

    TEST PLAN QUALITY CONTROL

    The quality control process begins with checking the adequacy of the hypothesized building blocks created during the investigation. If they are flawed, then any test planning work will create problems. Each EB for which supporting data will be sought should be checked for content and form, and be reconsidered iteratively as the work progresses.

    Difficulty designing a test plan to produce the supporting data is usually an indicator that the event being supported may not be adequately defined, or that the event may have to be broken down further to get supporting data. Sometimes, you find you are looking at the wrong object to test for the data.

    After the plan is completed, check the Plan against the description of the elements listed above. Flow chart the planned procedures on an MES Matrix, especially if any controversy about test sequencing or priorities arises during the planning process, or is expected during the test work or after the results are received.

    The concurrence process and funding plans will disclose points of difference that may reflect quality problems, as well as differences in opinions among investigators. (MES Matrixes will help dispel the differences.)

    Make sure the testers are familiar with the MES Matrix and your EB needs for the Matrix before they begin their testing or examination. If feasible, show them gaps or event scenario they will be addressing, to help them focus on the needed outputs.

    The Table 6-1 Check List can be used for quality assurance reviews, as a memory jogger. For many tests, it may be desirable to specifiy conformance with standards available from organizations like ASTM, for example, in the plan.

    Table 6-1. Test Plan Review Check List

    (attach to or make part of Test Plan where feasible.)

    Test/examination objectives support description or explanation of hypothesized scenario
    Physical objects to be examined are concretely defined.
    General test approaches documented - including priorities.
    Test/examination procedures to be followed are accurately defined and documented.
    Interpretation of results documented.
    Schedule of testing tasks specified.
    Distribution of deliverables specified.
    Disposition of tested objects resolved.
    Funding of test work negotiated and documented.
    Responsibility for and handling of Media inquiries defined.
    Safety precautions identified and documented.
    Acceptance procedure defined and staffed.
    Concurrence confirmed by signatures of participants.

    Part 4. Attachment I

    Examples Of Methods Used To
    Extract Data From Things During Investigations

    Note: Do not confuse these data extraction methods with data analysis methods. Extraction methods only give you raw data to create event building blocks; analysis methods use the extracted data to get to an understanding of the process being investigated..

    NON-DESTRUCTIVE TEST EXAMPLES

    A useful source of testing protocols is the ASTM. The ASTM offers a set of standards for various kinds of general test methods (ASTM Volume 14.02, General Test Methods; Chromotography; Forensic Sciences; Terminology; Conformity Assessment; Statistical Methods, July 2003) that is quite extensive.

    The follow list is intended to provide a sense of the variety of test types that might be considered candidates to specify to acquire needed data for EBs.

    (Add your own as you learn of new methods.)
    1. Visual inspection for residues, deviations from blueprints, deformations, etc.

    2. Radiographic inspection techniques, including x-ray, fluoroscopy, and other radiographic scanning techniques

    3. Reconstruction of surviving parts in mock-ups

    4. Microphotography of parts, residues, etc.

    5. Electron microscopic examination of parts, residues, surfaces, etc.

    6. Char pattern inspections for burned parts

    7. Scratch mark inspections for material displacement or removal

    8. Weighing techniques for deposition or removal of material

    9. X-ray diffraction examination of crystalline structures

    10. Thermal gradient inspection techniques for examination of evidence of temperature gradients experienced by materials

    11. Stress pattern inspection techniques for evidence of directions of stressors, deformation of stressees

    12. Ultra-violet or infra-red photographic examination for evidence of changes of state, blood, tissue

    13. Electrical conductivity/resistance tests (if arcing not involved)

    14. Flow tests for viscosity, angle of repose, air-entrainment data

    15. Dimensional analysis documentation to identify changes from as-new condition

    16. Dye-penetrant or magnetic inspection for cracks, defects

    17. Lubricity tests to identify lubricant service performance data

    18. Operating test of subsystem to identify operating parameter data, performance data

    19. High-speed photography of operating subsystem to observe timing, nature of process

    20. Forensic fingerprinting analyses

    21. Computer-generated graphical models from event data recorders

    22. Add your own here)



    Destructive test examples

    1. Wet chemical, gas chromatographic, infra-red spectrum, and other chemical test methods to identify chemical constituents

    2. Chemical tests to identify properties of chemicals, such as pH (acidity/alkalinity) tests, miscibility, reactivity with other materials, etc.

    3. Physical tests for melting points, boiling points, freezing points; gas, liquid or solid densities; etc.

    4. Metallurgical tests such as charpy, bend, tensile test methods

    5. Micrometallurgical, x-ray and microphotographic inspections of specimens cut from parts

    6. Thermal testing of materials for char pattern data, effluent gas data, rate-of-burn data, residue confirmation data, extinguishability data, BTU content, heat of combustion, decomposition rates

    7. Simulation or reenactment of collision, fire, explosion, etc., to reproduce debris, residues

    8. Flash point and auto-ignition tests to determine ignition temperature data for liquids, gases, dusts, powders, etc.

    9. Toxicological tests for toxicity data; asphyxiation tests for concentration/effects data

    10. Chemical liquid and gaseous exposure corrosion tests for material degradation data

    11. Polymerization/decompositiion tests for material reaction data

    12. Crystal growth tests to identify environment at time of crystal formation

    13. Incubation tests for etiologic and infectious agents

    14. Electrical flow, conductivity, resistance, arcing, and related tests for electrical behavior

    15. Air buoyancy tests for mixing, fall-out and dispersion data

    16. Proof pressure testing of components to destruction to validate design assumptions

    17. DNA, type and other blood tests

    18. Destructive char and thermal residue analyses





    Go to Guide: 0 1 2 3 4 5 6 7 8 9 10