The Federal Aviation Administration Acquisition System Toolset

"HOW TO"

Since it is difficult to enforce compliance after a contract is awarded if vendor capabilities are inadequate, offerors must demonstrate the ability to incorporate human factors design criteria and guidelines into their system design and engineering before contract award. The Government first plans the approach and then includes human factors requirements in the Screening Information Request (SIR), which includes the proposal evaluation criteria. Offerors show they understand the requirements by making human factors commitments in their proposals. The offerors must demonstrate comprehension of and the ability to comply with the total system performance concept as well as their ability to integrate human considerations into system design and development. The human factors practitioner, having provided input to the source selection plan, helps determine how well offerors have met the human factors selection criteria.

Step 1:

Provide Input to the

Screening Information Request

The Integrated Product Team (IPT) Human Factors Coordinator assists in developing the documentation the offeror must submit and the proposal evaluation criteria. The criteria must define the quantity and quality of the effort required. The human factors portion of the criteria should contain two primary requirements.

. Require offerors to define how they will organize and manage their human factors program for the system.

. Require offerors to describe how they will execute the technical human factors program and integrate human factors throughout their design and engineering efforts.

For nondevelopmental items (NDI) or commercial-off-the-shelf (COTS) procurements, the hardware and/or software has already been developed, so the criteria will focus on the existing product as opposed to a product to be developed. Human factors criteria must still be met.

The SIR (usually in Section L) describes the information an offeror must provide to the Government against which the proposal will be evaluated. The human factors criteria (to be included in Section L) can be stated as a separate criterion or be embedded with other criteria such as system engineering or training.

Table 6-1 lists some potential human factors inputs to the proposal evaluation criteria.

Step 2:

Provide Input to the Source Selection Plan

After human factors criteria have been developed and are included in the proposal evaluation criteria portion of the SIR (Section L), the IPT Human Factors Coordinator should help develop the Source Selection Plan.

The weight human factors will have in rating and ranking the proposals must be determined. This will vary greatly from system to system with the greatest influence being the degree of human involvement as part of the total system. The total weight is 100% and there are legitimate competing interests for priority. If human factors is considered a separate criterion, it is assigned a weight, as are other criteria such as technical and cost (Figure 6-1.). If human factors criteria are embedded within other criteria, it is assigned a weight as a sub-element of the main criterion (criteria).

Regardless of the approach taken, the human factors criteria must be visible and given sufficient weight, consistent with the nature of the program including the degree of human involvement, performance risks, consequence of error, and the like.

Finally, the human factors practitioner determines how each human factors criterion will be evaluated. The scoring will normally be based on quantitative and qualitative factors. The following figure demonstrates a conceptual breakout of human factors elements in a Source Selection Plan where human factors is a separate criterion.

Step 3:

Participate on

the Source Selection

Team

Representation of human factors expertise on source selection team or panel(s) will provide the capability to adequately assess the human factors aspects of proposals. The human factors representative must be technically qualified in human factors and adequately trained in the source selection process.

Figure 6-1. Sample weighting of human factors criteria.

Minimal qualifications and training for the team representative include knowledge of:

The overall system and its intended purpose in the field.

- The human interface required to achieve optimum system performance.

- The human performance concerns and issues applicable to the system.

- The requirements, specifications, special instructions, deliverables, and evaluation criteria as set forth in the SIR as well as what evidence is sufficient to demonstrate compliance with the criteria.

- The procedures for rating and ranking the proposals.

CHECKLIST

QUESTIONS

Evaluation Criteria

- Have human performance criteria or standards been identified for the system and quantified in the SIR?

- Does human factors (as a separate criterion or as embedded criteria in other primary factors) adequately represent the user performance, risks, complexity, consequence, and exposure?

- Are offerors required to develop a human factors program management plan?

- Are offerors required to demonstrate technical competence in human factors?

Source Selection Plan

- Have human factors criteria been adequately and clearly identified in the source selection plan?

- Are human factors criteria adequately weighted for this system (considering degree of human interface with hardware and/or software)?

Source Selection Teams

- Is there a human factors member on the source selection team or supporting panel(s)?

- Is the human factors member technically qualified to evaluate human factors aspects of the proposals?

- Where human factors criteria are embedded with other criteria, is human factors represented in those other criteria evaluations?

- Is the source selection team adequately appraised on the evidence necessary to demonstrate vendor capability and compliance?

TABLE 6-1 POTENTIAL HUMAN FACTORS INPUTS TO THE PROPOSAL EVALUATION CRITERIA
Management Planning	- Adequacy of offeror's human factors organization, level of effort, lines of authority, visibility to top management, and potential impact on design decisions. - Adequacy of offeror's concept for contributing to and helping to execute the human factors program.
Execution	- Coordination of human factors activities with the total management system and work breakdown structure. - Coherence of offeror's plan for tracking and reporting human factors task performance and for assuring quality.
Technical Qualifications	- Quality of offeror's and subcontractor's previous experience in human factors-related tasks. - Capability of offeror's personnel, including key subcontractor personnel, to perform required human factors tasks.

Evaluation	- Adequacy of offeror's methodology for validating human factors requirements as part of the test and evaluation requirements identified in SIR. - Adequacy of test and evaluation facilities to perform human factors assessments and analyses.
Human Factors Understanding	- Offeror's understanding of human factors concepts as a means for enhancing total system performance. - Adequacy of offeror's concept for assuring that the system design will reflect human factors goals and constraints.
Training	- Indicates how the training developer will serve as a resource for design ideas and for assessing the training impact on design. - Understanding of the impact of design on training devices and other aids. - Recognizes the impact of skill decay on sustainment training and demonstrates capability for reducing skill decay through cost-effective changes in the design. - Recognizes the influence of human aptitude on success in training and consequently, on system performance. - Recognizes the value of positive transfer of current skills on new training.
Human Engineering	- Staffing level and quality of offeror's human factors engineers, including subcontractors, available for this system. - Adequacy of plan for functional and/or task analysis and critical task identification to determine appropriate task burden on humans. - Shows approach for tracking the functions, information flow, and processing steps that the operator must monitor. - Adequacy of plans for estimating physical and cognitive workloads of operators and maintainers, by group and individually, with reference to staffing and training constraints. - Adequacy of approach for allocating functions to the human, hardware, or software for optimum system performance. - Addresses the design of the work environment, including space claims and other workstation variables, as the work environment influences system performance. - Ensures that human engineering data collection, testing, and evaluation plans use appropriate and valid equipment and techniques such as mockups, simulations, models, and prototypes. - Adequacy of plans to conduct failure analysis and documentation of redesigns made in response to human-system performance problems and failures.
Staffing	- Adequacy of approach to reduce staffing needs while maintaining desired system performance. - Adequacy of plans for analyzing trade-offs among design options that could produce lifecycle personnel savings and costs, informing the Government of results and making appropriate design changes. - Addresses the impact of varying staffing levels on total system performance.
Human Resource Skills	- Demonstrates an understanding of the projected operators and maintainers and the human factors goals and constraints that are imposed by that target population. - Ability to recognize the use of skill specialties that present staffing difficulties or are low in density and would be difficult to expand quickly. - Adequacy of plans for identifying the human resource-intensive aspects of the system and explaining how alternative designs will be pursued. - Adequacy of plans to identify and clarify personnel workload issues during design work. - Addresses the impact of varying skill and experience levels on total system performance. - Identifies skills that are critical to successful mission performance and explains how these skills relate to the capabilities of the operators, maintainers, and supporters.
System Safety and Health Hazards	- Adequacy of plans to identify potential safety hazards in all environments over system lifecycle and documentation of acceptable residual risks. - Estimates severity, frequency, and scope of exposure of risks, incidents, and accidents. - Demonstrates a plan for tracking changes in design and for continuously evaluating safety impacts. - Adequacy of plans to establish pre-defined levels of acceptable risk and estimates the influence of these risks on operator and maintainer performance. - Demonstrates an understanding of health hazards, including secondary impacts on staffing decisions. - Adequacy of plans to identify psychological influences on human performance that can be controlled favorably through system design. - Evaluates hazards in the intended operating environments and determines priorities for control through initial design and retrofit. - Identifies alternative technical concepts to control, reduce, or avoid health hazard risks. - Demonstrates ability to prepare test and evaluation plans using state-of-the-art practices, criteria, standards, and lessons learned data bases.



Systems Integration	- Assures integration of human and machine within a system (for example, engineering decisions should be made with continual reference to human performance and system functions should be matched to human attributes during task allocation). - Adequacy of plans to coordinate and efficiently conduct the collection, analysis and interpretation of human performance data. - Assures that performance of the system is consistent with the performance and goals of larger enclosing systems. - Shows that trade-off and sensitivity analyses are used to evaluate design alternatives with appropriate emphasis on human impacts. - Presents valid human performance tests of the system in realistic and anticipated environments and combinations of environments. - Shows that system design and human factors analysis will be performed, so that problems are fed back and eliminated early in the design phase.
Operations and Support Cost Evaluation	- Adequacy of offeror's analysis of system costs and projections in relation to human factors topics. - Adequacy of offeror's cost trade-off analysis in meeting human factors-related requirements.