Incident investigators sometimes turn to examinations, tests, teardowns or simulations (referred to as tests here) to find data to transform into Event Blocks. Their challenge is to assure that any such efforts produce needed and acceptable EBs for their Matrixes.

The general task is to

1. identify what gap in Matrix to address with tests
2. prepare, with expert help, a plan for performing the tests
3. implement the plan
4. convert the results into EBs

Procedures for this task are contained in this Guide.

The Multilinear Events Sequencing technology-based investigation process is a "research defining" methodology. It defines unknown areas of events flows as gaps on the MES Matrix. Gaps indicate where the process being investigated is not understood. To bridge Matrix gaps, MES-tree analyses techniques, among others, can generate hypothetical EBs to describe the missing events. Confirming those events usually requires acquisition of new data, or reexamination of previously acquired data. To acquire new data from people, talk to them, as discussed in EB Guide 1. To acquire data from things, and create things EBs, investigators use other techniques such as examinations, tests, teardowns and simulations. This Guide describes test planning tasks needed to ensure efficient effective data acquisition.

Aggressive or conscientious Investigators intuitively try to eliminate all unknowns during investigations. That requires "research" time and money, among other demands. Investigators use the fact that the accident occurred and the need to understand it to justify this research. A test planning process helps investigators and investigation managers distinguish data actually needed from what would be nice to have, by forcing decisions about objectives, methods, outputs and funding, among other tasks. Thus the test planning process can serve as an investigation management tool, as well as a tool to ensure sound, efficient technical results.

OBJECTIVES

The objective of any tests should be to extract needed data from objects or people to create gap-filling EBs; to prevent the unintended destruction of data; and to promote testing or examination efficiency and value. The objective of the Guide is to present the main tasks required to accomplish this efficiently and effectively, and support development of a complete description and explanation of what happened.

APPLICABILITY

This Guide is applicable to tests conducted for investigators. The "Do no harm" rule addresses the handing of people, objects and data by the investigator. During the investigation, try to prevent unintended changes to people or objects as they are handled during the initial stages of an investigation. Plans and procedures to implement this rule should be a part of every investigator's training.

This test planning process is applicable to the full range of investigation testing tasks performed to support MES investigations and Matrix development. Its main focus is testing and simulations or reenactments to support verification possible scenarios to fill gaps remaining after the investigator's initial examination of objects is completed. That kind of task is usually performed by "experts" rather than the investigator.

DATA REQUIRED

1. Postulated events during the gaps in the MES Matrix that the tests are to demonstrate or validate.
   
   
2. Availability of facilities, personnel and methods needed to produce the needed EBs.
   
   
3. Physical entities that are available for testing.
   
   
4. People who have an interest in the investigation, and their interests and capabilities.
   
   
5. Information about funding for the test or examination.

The data should be verified before it is used in a test plan. Verification of the data used for test planning may require new inquiries and additional investigative effort, particularly with respect to data items 1 and 4.

DATA SOURCES

The test planning data sources vary with the type of data required. Data about the interests and capabilities of people interested in the investigation can be acquired by observation and recording of comments or inquiries addressed to the investigator during the investigation, and by identifying the persons whose actions - especially "programmers"- affected events on the Matrix. Data about the hypothesized events during gaps and the physical entities available for testing generally are acquired by observation of the Matrixes and objects surviving the phenomenon during the course of the investigation. Information about the availability of the resources needed to do the testing have to be acquired by an investigator much like any other "purchasing" process.

GUIDING PRINCIPLES FOR TEST PLAN DEVELOPMENT.

Development of test and simulation plans should be governed by some essential principles. Every investigator who handles objects during an investigation should know and observe these principles.

1. No Plan, No Tests!

The GOLDEN RULE of INVESTIGATION TESTING. Never allow or initiate any destructive testing or examination that could introduce changes into any one-of-a-kind physical object without a test plan. Do not begin test work on an object without a test plan agreeable to all affected parties having a legitimate interest in what happened, the test results and data produced.

2. Whoever Owns the Ball Names the Game.

Respect the property rights of the owner of the physical object to be tested. Never do any testing on an object without the concurrence of the owner in the test plan. If you are the owner, do not let others change or destroy your object until you are satisfied the planned tests will provide ALL the data you need - especially if destructive testing will occur. This can get complicated when government and private sector investigators are involved, and you may need legal advice.

3. Respect Management-by-Objectives Principles.

Tests on physical objects should be performed only to achieve some explicitly stated objective. Any test should contributes to better understanding of the part of process indicated by a gap on the MES Matrix. If you don't recognize how it will, delay the testing until you find out - preferably on paper.

A indispensable application of this principle is to the funding plan for proposed tests and simulations. This is the step that develops the value of the data to be acquired through testing. It also introduces the consideration of optional ways to get the desired data, or devising a way to use the gap in the investigation. For example, if the investigation defines a gap which requires research to determine what happened, it might be appropriate to propose that a party which should be able to provide the answers do the research, rather than the investigating party.

4. Keep Test(s) Relevant.

The mental movie framework should control hypothesis development, and debris testing should be limited to identifying, describing or confirming changes of state during the mishap. Test outcomes should provide specific EBs for the process model you are constructing or reconstructing. Testers love to give you data their way. They also like to propose "just one more test" after the results are in; don't fall into the trap of using your tests (and money) for their research purposes. Do your hypothesizing and conjecturing on paper. That is faster, cheaper and easier. Don't use tests to go fishing.

5. Scale the Plan to the Value of the Data it Will Produce.

Tests aren't free. Balance the results (certainty, credibility, legal needs, etc.) against the cost (dollars, delays, diversion of manpower, etc.) of the testing to be sure the test(s) planned is properly tailored to the need.

6. Progressive destruction demands priorities

Don't destroy it before you get the data it holds.Any actions that change the state of an object before you complete an examination or test series jeopardizes needed data. Mentally walk through the test sequence to ensure that when you want to examine an object, it has not been changed by prior work that could have been reschedule or resequenced - unless the decision is made after due deliberation and consensus of the interested parties. For example, determine if the consequences of trying to operate a system or component before tearing it down would do more harm than tearing it down and then reassembling it to test its operation.

7. Remember the 6 Ps.

Systematically consider extracting data from physical objects in all major ways, including non-destructive examination of their PROPERTIES, appearance or attributes, analysis of discernible PATTERNS, observing POSITION relationships, studying PAPERS with specifications or procedures, or asking PEOPLE about observations or habits, as well as inspecting or testing PARTS. Investigators can not predict where useful data will be found in a specific investigation, so they should be prepared to use any search tools available. to help them find it before they explore testing options.

PLANNING PROCESS TASKS

The main tasks in the MES test planning process are:

BASIC TEST PLAN ELEMENTS.

1. Test/examination objectives.

Which MES Matrix gap is being addressed, and for which hypothesized events in that gap are validating data being sought? If more than one party is involved, objectives desired by each of the parties may have to be stated. Focus on gaps - it helps keep testing efforts concentrated on what needs to be known. Never lose sight of the end point - EBs to add to the MES Matrix to make it more complete.

2. Physical objects to be examined.

Describe the object(s) being tested or examined and document them to assure tests are performed on parts everyone expects to be tested, and so their original state can be replicated if necessary. Indicate here any protective measures for the objects, required to preserve them prior to the test, or after the test. A good practice is to use chain-of-custody identifiers on the object and all documentation about the object.

3. General test approaches.

Use this section to record any general principles to be followed by the testers, any assumptions that need to be documented before the test begins, and how the objects and tests will be documented. This is where any differences in the approach can be reconciled. For example, should a device be operated before it is dismantled, or should the dismantling be done before it is operated?

4. Test/examination procedures to be followed.

State the name of the test protocol and equipment and the citation, if it has been formalized in the literature or elsewhere. Alternatively, record the procedure and equipment in detail if it has not been formalized elsewhere. Define and document the measurements to be produced. If unique, make sure to define the procedures carefully to avoid complaints later. Use an MES Matrix to lay out the test procedure so you can check it for oversights and omissions before you start. Specify chain-of-custody requirements, precautions and responsibilities, points of contact, and any security tasks.

This section should state the specifications for the deliverables that will be produced and quality control criteria that will be used to verify the results.

5. Interpretation of results.

Hypothesize potential test outcomes and determine among the parties how each potential outcome will be stated in EB formats on MES Matrix. If this task is done properly, the specific outcome may be uncertain, but there should be no surprises at the end of the test. The place for discovery of this kind is in the MES-tree analysis procedure for the Matrix gaps.

6. Schedule of testing.

State what work will be accomplished, when it will start and where, the schedule for any drafts to be circulated if applicable, and when the deliverables will be delivered.

7. Distribution of deliverables.

Depending on the nature of the investigation, deliverables from the testing efforts my require controls that should be described in this section. State who "owns" rights in the deliverables, and who can use or allocate them and for what purposes in the future. Any confidentiality or security precautions should be specified in this section.

8. Disposition of tested objects.

State who will specify disposition of the tested object(s) and the time limit for disposition. Usually the owner is given control over the disposition. Anticipated litigation may influence this section.

9. Funding of test work.

Specify who pays what to whom. Who will pay for the test(s), or if more than one party, who will pay for what part of the test(s) and who will spend and who will get what monies? Be aware that this requirement can be used very effectively to dissuade proponents of unsound hypotheses to focus on hypothesis that MES-Trees have shown to have some credibility. But if somebody wants to do a special test the investigator has good reason to believe is not needed (based on the MES Matrix), is willing to pay for it and live with the results, and it can be done without interfering with other tests, work it into the test plan.

OPTIONAL TEST PLAN ELEMENTS.

10. Media inquiries.

If the investigation has attracted public interest, specify who will respond to the media, and how inquiries to the individuals and organizations actually performing the test(s) or others who might be called should be referred to that spokesperson.

11. Safety precautions.

Where risk of injury or property damage is associated with the test procedures, any required risk control precautions should be specified in the Test Plan. This will be determined by the nature of the test(s) or the test object(s) OSHA or other regulatory compliance instructions are appropriate for this section.

12. Concurrence.

When more than one party is involved, get every interested party to affix a signature to the test plan signifying concurrence in the plan. Be sure to get concurrences in any changes to the Test Plan that may be necessitated by staged or phased outputs.

TEST PLAN QUALITY CONTROL

The quality control process begins with checking the adequacy of the hypothesized building blocks created during the investigation. If they are flawed, then any test planning work will create problems. Each EB for which supporting data will be sought should be checked for content and form, and be reconsidered iteratively as the work progresses.

Difficulty designing a test plan to produce the supporting data is usually an indicator that the event being supported may not be adequately defined, or that the event may have to be broken down further to get supporting data. Sometimes, you find you are looking at the wrong object to test for the data.

After the plan is completed, check the Plan against the description of the elements listed above. Flow chart the planned procedures on an MES Matrix, especially if any controversy about test sequencing or priorities arises during the planning process, or is expected during the test work or after the results are received.

The concurrence process and funding plans will disclose points of difference that may reflect quality problems, as well as differences in opinions among investigators. (MES Matrixes will help dispel the differences.)

Make sure the testers are familiar with the MES Matrix and your EB needs for the Matrix before they begin their testing or examination. If feasible, show them gaps or event scenario they will be addressing, to help them focus on the needed outputs.

The Table 6-1 Check List can be used for quality assurance reviews, as a memory jogger. For many tests, it may be desirable to specifiy conformance with standards available from organizations like ASTM, for example, in the plan.

Table 6-1. Test Plan Review Check List (attach to or make part of Test Plan where feasible.)

Test/examination objectives support description or explanation of hypothesized scenario

Physical objects to be examined are concretely defined.

General test approaches documented - including priorities.

Test/examination procedures to be followed are accurately defined and documented.

Interpretation of results documented.

Schedule of testing tasks specified.

Distribution of deliverables specified.

Disposition of tested objects resolved.

Funding of test work negotiated and documented.

Responsibility for and handling of Media inquiries defined.

Safety precautions identified and documented.

Acceptance procedure defined and staffed.

Concurrence confirmed by signatures of participants.