PMI-PBA : Evaluation (Domain 5)
PMI – PMI-PBA : Certified Professional in Business Analysis - Domain 5 - Evaluation
The Evaluation domain represents the final, critical phase of the business analysis lifecycle within the PMI Professional in Business Analysis (PMI-PBA) framework. While previous domains focus on identifying needs, planning activities, and eliciting or monitoring requirements, Domain 5 is dedicated to confirming that the delivered solution actually fulfills those requirements and realizes the intended business value. Accounting for 10% of the PMI-PBA examination, this domain evaluates a practitioner’s ability to assess solution performance after implementation, gather stakeholder feedback, and document the lessons learned that will inform future organizational initiatives.
This study guide provides a comprehensive breakdown of the tasks, processes, and techniques essential for mastering the Evaluation domain. It emphasizes the strategic intersection of project delivery and organizational value, ensuring that practitioners can validate that a solution addresses a validated problem and aligns with the strategic business architecture.
The Strategic Significance of Solution Evaluation
Evaluation is not merely a post-project formality; it is a strategic function that operates at the intersection of delivery and value. Historically, organizational failures are often linked to flawed front-end analysis or inaccurate requirements gathering. Domain 5 serves as the corrective and confirmatory mechanism that ensures the project has steered away from scope creep and toward value-driven delivery.
In many organizational environments—whether Agile, Waterfall, or Hybrid—the focus often remains on project management metrics like schedule compliance and cost variances. Business analysis, however, looks toward the business case and the value proposition. The evaluation process determines if the solution meets the documented benefits and solves the original business problem identified during the Needs Assessment. This domain requires practitioners to look beyond “is it finished?” to ask “did it work?” and “does it provide the value we promised?”
Validating Solution Test Results against Acceptance Criteria
The first primary task in the Evaluation domain is the validation of test results, reports, and other evidence against established acceptance criteria. This process ensures that the developed solution satisfies the requirements defined during the Analysis phase.
Business analysts do not necessarily perform all the testing themselves, but they are responsible for ensuring that the results demonstrate compliance with stakeholder needs. This involves:
- Reviewing Test Evidence: Analyzing reports and results to ensure they align with the requirements baseline.
- Applying Acceptance Criteria: Using the metrics and quality standards defined during the Planning and Analysis domains to judge whether a requirement has been successfully met.
- Consistency Check: Ensuring that the evidence provided is accurate, complete, and directly traceable back to the business objectives.
By validating results against the acceptance criteria, the business analyst provides the necessary assurance that the solution is technically and functionally ready for use by the stakeholders.
Conducting Gap Analysis and Communicating Deltas
Gap analysis is a core technique within the Evaluation domain used to identify discrepancies between the intended solution scope and the actual developed solution. When a solution is tested or deployed, it may not perfectly align with the original requirements. These discrepancies are often referred to as “deltas.”
To effectively manage these gaps, the business analyst must:
- Identify Discrepancies: Use quality assurance tools and methods to pinpoint where the developed solution falls short of or deviates from the requirements.
- Analyze Impact: Determine how these gaps affect the overall business case and value proposition.
- Communicate Findings: Facilitate clear communication between technical teams and stakeholders regarding identified issues.
- Resolve Discrepancies: Enable stakeholders to make informed decisions on whether to resolve the discrepancies, accept them as is, or defer them to a future release.
Clear communication of gaps is vital to maintaining stakeholder trust and ensuring that the final deployment is based on a realistic understanding of the solution’s capabilities.
Obtaining Formal Stakeholder Sign-off for Deployment
Before a solution can be fully deployed, the business analyst must obtain formal sign-off from the appropriate stakeholders. This task utilizes decision-making techniques to facilitate consensus and achieve a formal agreement that the solution is acceptable for use.
Sign-off is a milestone that indicates the organization is ready to proceed with deployment. The business analyst’s role in this task includes:
- Securing Consensus: Using techniques such as multi-voting or the Delphi technique to align diverse stakeholder views.
- Documenting Approval: Ensuring that the sign-off is captured formally, providing a clear audit trail for the transition from the development phase to the operational phase.
- Facilitating the Decision: Presenting test results, gap analysis reports, and validation evidence to give stakeholders the confidence they need to authorize deployment.
This process acts as a final gateway, ensuring that the organization does not deploy a solution that fails to meet the fundamental business needs or lacks the support of key influencers.
Evaluating Deployed Solutions against the Business Case
Evaluation continues even after the solution has been deployed into the live environment. Task 4 of the domain focuses on determining how well the deployed solution meets the original business case and value proposition. This is the ultimate test of the business analysis effort.
Post-deployment evaluation involves:
- Measuring Value Realization: Determining if the expected benefits (e.g., increased revenue, reduced costs, improved efficiency) are actually being achieved.
- Comparing Results to Baselines: Looking back at the goals and objectives defined in the Needs Assessment to see if the implementation has successfully addressed the business problem.
- Valuation Techniques: Employing methods like Cost-Benefit Analysis or Return on Investment (ROI) assessments to quantify the success of the initiative.
This long-term view of evaluation helps organizations understand the true impact of their investments and provides data-driven insights into whether the project was a strategic success.
Key Performance Indicators (KPIs) and Metrics for Evaluation
Key Performance Indicators (KPIs) are essential measurement tools used throughout the Evaluation domain. They provide the quantitative basis for determining if a solution is meeting its requirements and delivering value.
KPIs are typically established during the Planning phase but are rigorously applied during Evaluation. Effective use of KPIs includes:
- Alignment with Objectives: Ensuring that every KPI measured during evaluation directly relates to a project goal or business objective.
- Measurability: Using metrics that are explicit and actionable, allowing for objective assessments of solution performance.
- Strategic Reporting: Presenting KPI data to stakeholders to demonstrate the progress toward benefits realization.
Without well-defined KPIs, evaluation becomes subjective and prone to bias, making it difficult to prove the ROI of a solution to organizational leadership.
Day-in-the-Life (DITL) and User Acceptance Testing (UAT)
To ensure a solution works in the real world, business analysts often employ specific validation techniques like Day-in-the-Life (DITL) testing and User Acceptance Testing (UAT). These techniques focus on the end-user’s experience and the solution’s operational fitness.
- Day-in-the-Life (DITL) Validation: This involves observing or simulating a user’s typical daily workflow using the new solution. It helps identify if the solution integrates smoothly into existing processes or if it creates new, unforeseen bottlenecks.
- User Acceptance Testing (UAT): This is the final stage of functional testing where actual users verify that the solution works as expected in a production-like environment. UAT metrics are critical for identifying if the solution is “fit for purpose” and “fit for use.”
Both DITL and UAT provide qualitative and quantitative data that is often more insightful than technical system testing alone, as they highlight the human and process elements of the solution.
Post-Implementation Reviews and Lessons Learned
A critical component of the Evaluation domain is the documentation of lessons learned through Post-Implementation Reviews (PIR). This process ensures that the organization captures knowledge that can improve future projects and business analysis activities.
The PIR process includes:
- Gathering Stakeholder Feedback: Interviewing or surveying stakeholders to understand their perspectives on the solution and the project process.
- Documenting Successes and Failures: Recording what went well and what challenges were encountered during elicitation, analysis, and implementation.
- Knowledge Transfer: Sharing these insights with the broader organization to avoid repeating past mistakes and to replicate successful strategies.
By treating every evaluation as a learning opportunity, business analysts contribute to the continuous improvement of the organization’s project delivery capabilities.
Quality Assurance Tools and Methods in Business Analysis
Quality Assurance (QA) tools are not exclusive to the technical testing team; they are vital for the business analyst during the Evaluation domain. These tools help in resolving discrepancies between the solution scope and the developed product.
Common QA-related activities for the business analyst include:
- Auditing Requirements: Ensuring that every requirement in the baseline has a corresponding test case and a verified result.
- Delta Analysis: Using systematic methods to categorize and prioritize gaps found during testing.
- Root Cause Analysis: If a solution fails to meet a requirement, using techniques like the “5 Whys” or Ishikawa diagrams to understand if the failure is due to a technical defect, a misunderstanding of the requirement, or a change in the business environment.
Using a structured QA approach allows the business analyst to provide an objective, evidence-based assessment of the solution’s quality.
Valuation Techniques for Assessing Solution Success
To determine if a solution meets the business case, business analysts use various valuation techniques. These methods allow for the comparison of different solution options and the measurement of the implemented solution’s final impact.
Key valuation techniques include:
- Cost-Benefit Analysis: Comparing the total expected cost of the solution against the total expected benefits.
- Kano Model: Assessing stakeholder satisfaction and the perceived value of different product features.
- SWOT Analysis: Evaluating the strengths, weaknesses, opportunities, and threats associated with the implemented solution to determine its long-term viability.
- Purpose Alignment Matrix: Ensuring the solution’s features align with the organization’s strategic goals.
These techniques help the business analyst move beyond functional validation and into the realm of strategic business value, proving that the solution was a worthwhile investment for the organization.
Domain 5 Summary Table: Tasks and Execution
The following table summarizes the core process architecture for Domain 5, mapping the specific tasks to the necessary analytical focus.
| Task Number | Task Description | Analytical Focus |
|---|---|---|
| Task 1 | Validate test results against acceptance criteria. | Compliance and technical readiness. |
| Task 2 | Analyze and communicate identified gaps/deltas. | Discrepancy resolution and scope alignment. |
| Task 3 | Obtain formal stakeholder sign-off. | Consensus building and deployment authorization. |
| Task 4 | Evaluate deployed solution against the business case. | Benefits realization and strategic value. |
Domain 5 Study and Preparation Strategies
Preparing for the Evaluation domain of the PMI-PBA exam requires a shift from “how to build” to “how to measure.” Because the exam is heavily scenario-based, you must be able to apply these concepts to real-world project situations across Agile, Waterfall, and Hybrid environments.
Understanding Methodology Variations
The approach to evaluation differs significantly depending on the project methodology:
- Agile Environments: Evaluation is continuous. Feedback is gathered in every iteration or sprint review, and validation occurs through frequent demos and user story acceptance.
- Waterfall Environments: Evaluation is often a formal phase at the end of the project lifecycle, involving structured UAT and a single formal sign-off.
- Hybrid Environments: The business analyst must balance the flexibility of iterative feedback with the formal requirements of structured deployment and sign-off.
Active Study Techniques
- Mini-Case Studies: Create short scenarios where a solution has been deployed but a KPI is not being met. Ask yourself: What is the first thing a business analyst should do? (Often, it is root cause analysis or gap communication).
- Process Walkthroughs: Mentally walk through a sample project from the initial business case to the post-implementation review. Identify how the business case is used as the “north star” for final evaluation.
- Technique Matching: Practice matching tools like the Kano model or DITL validation to specific evaluation challenges.
Exam Day Considerations
Remember that for the PMI-PBA, questions often ask for the “best” or “next” step in a scenario. In Evaluation scenarios, the answer is frequently tied to:
- Communicating gaps to stakeholders immediately.
- Using established acceptance criteria to make objective judgments.
- Verifying value realization against the original business case.
- Ensuring formal sign-off is achieved before moving to the next stage.
Short-Answer Questions
- What is the primary purpose of the Evaluation domain in business analysis?
- How do acceptance criteria influence the validation of test results?
- In the context of Domain 5, what is a “delta”?
- Why is stakeholder sign-off considered a critical “gateway” task?
- Which document serves as the primary baseline for evaluating a deployed solution’s success?
- What is the difference between User Acceptance Testing (UAT) and Day-in-the-Life (DITL) validation?
- How does a business analyst use the Kano model during the evaluation process?
- What is the objective of a Post-Implementation Review (PIR)?
- Which analytical technique helps identify why a deployed solution is failing to meet a specific KPI?
- How does evaluation differ in an Agile environment compared to a Waterfall environment?
Answer Key
- Answer: The primary purpose is to confirm that the delivered solution fulfills the requirements and meets the established business need and value proposition.
- Answer: Acceptance criteria provide the specific metrics and quality standards used to determine objectively whether a solution satisfies its requirements.
- Answer: A delta is a gap or discrepancy between the original solution scope/requirements and the actual developed solution.
- Answer: It ensures formal stakeholder consensus and authorization, confirming that the solution is acceptable for deployment and meets their values.
- Answer: The business case, as it contains the original value proposition, goals, and objectives the solution was intended to achieve.
- Answer: UAT focuses on users verifying that the solution works as expected, while DITL simulates typical daily workflows to ensure operational integration and identify bottlenecks.
- Answer: The Kano model is used as a valuation technique to determine stakeholder satisfaction and the perceived value of different features in the delivered solution.
- Answer: The objective is to gather stakeholder feedback and document lessons learned to improve future business analysis and project activities.
- Answer: Root cause analysis techniques, such as the “5 Whys” or Ishikawa (fishbone) diagrams, are used to identify the underlying cause of performance gaps.
- Answer: In Agile, evaluation and feedback are continuous and iterative (e.g., sprint reviews), whereas in Waterfall, evaluation is typically a formal, sequential phase at the end of the project.
Advanced Design and Scenario Questions
- Scenario Analysis: A solution has been deployed that meets 100% of the documented functional requirements, yet the Post-Implementation Review reveals that stakeholders are highly dissatisfied because the solution has added 20% more time to their daily tasks. Design a root cause analysis strategy to determine where the business analysis process failed and suggest which Evaluation technique should have been used earlier to prevent this outcome.
- Strategic Alignment: You are evaluating a deployed solution for a global organization. The technical KPIs show the system is performing optimally, but the financial ROI is significantly lower than predicted in the business case due to a sudden shift in market conditions. How should you frame your evaluation report to the executive sponsors, and what valuation techniques would you use to support your findings?
- Agile vs. Waterfall Evaluation: Compare the formal sign-off process in a highly regulated Waterfall project with the “Definition of Done” and user story acceptance in an Agile project. Explain how the business analyst ensures that “value realization” is not lost in either approach.
- Gap Resolution Strategy: During UAT, a critical gap is identified: the solution cannot handle the required data volume for peak season, a requirement that was deferred in the Analysis phase but is now deemed essential. Propose a communication and decision-making plan to obtain stakeholder consensus on whether to halt deployment or proceed with a workaround.
- Metric Design: Develop a set of three KPIs for a new customer service portal. One KPI must measure functional compliance, one must measure user experience/fitness for use, and one must measure strategic business value alignment. Explain how you would validate each during the Evaluation phase.
Glossary of Key Domain Terms
- Acceptance Criteria: The specific requirements, metrics, and quality standards that a solution must meet to be accepted by stakeholders.
- Business Case: A document providing the justification for an initiative, detailing the business problem, proposed solution, and expected value.
- Day-in-the-Life (DITL) Validation: A technique used to simulate or observe a user’s typical daily workflow to ensure a solution integrates effectively.
- Delta: A discrepancy or gap identified between the requirements and the developed solution.
- Evaluation: The domain focused on assessing solution performance and confirming the realization of business value.
- Gap Analysis: The process of identifying the difference between the current state (developed solution) and the desired state (requirements).
- Ishikawa Diagram: A root cause analysis tool (also known as a fishbone diagram) used to identify potential causes for a problem or gap.
- Kano Model: A valuation technique used to categorize and prioritize stakeholder needs based on their impact on satisfaction.
- Key Performance Indicator (KPI): A measurable value that demonstrates how effectively a solution is achieving its key business objectives.
- Lessons Learned: Knowledge gained from the process of performing a project, documented to improve future performance.
- Post-Implementation Review (PIR): A formal evaluation of a solution conducted after deployment to gather feedback and assess success.
- Quality Assurance (QA): The systematic process of determining whether a product or service meets specified requirements.
- Requirements Traceability Matrix (RTM): A grid that links requirements from their origin to the deliverables that satisfy them, used during evaluation to ensure compliance.
- Sign-off: The formal acceptance and approval of a deliverable or solution by stakeholders.
- Solution Assessment: The process of evaluating whether a solution is effective in solving the original business problem.
- Stakeholder Engagement: The process of effectively involving stakeholders to ensure their needs are met throughout the evaluation.
- User Acceptance Testing (UAT): The final stage of validation where end-users test the solution to ensure it is fit for purpose.
- Validation: The process of ensuring that a solution meets the defined requirements and delivers the intended value.
- Value Proposition: The promise of value to be delivered by the solution, serving as a primary benchmark for evaluation.
- Valuation Techniques: Methods used to determine the worth or success of a solution, such as ROI or Cost-Benefit Analysis.
Leaderboard
No scores saved yet. Be the first!
25 Questions — PMI – PMI-PBA : Certified Professional in Business Analysis - Domain 5 - Evaluation
Expand any question to reveal the correct answer and explanation.
-
1 A business analyst (BA) is evaluating a newly implemented automated inventory system. The system meets all documented requirements, but warehouse staff find the interface so complex that processing time has increased by $15\%$. Which aspect of evaluation is the BA primarily addressing by identifying this issue?
Consider the difference between a solution being built correctly and a solution working effectively for its users.
Solution performance
This focuses on the actual effectiveness and efficiency of the solution in the real-world environment rather than just technical compliance.
-
✗ Technical verification
Verification confirms the system was built according to specifications, which was already achieved in this scenario.
-
✗ Requirement acceptance
The scenario states the requirements were met, suggesting acceptance was likely already granted based on the initial documentation.
-
✗ Traceability maintenance
Traceability involves linking requirements to objectives, but does not measure the actual efficiency of the resulting solution.
-
-
2 During Task 1 of the Evaluation domain, a BA discovers that the test evidence for a high-priority requirement is inconclusive due to an unstable test environment. What is the most appropriate next step?
Think about the level of certainty required to confirm that a business need has been technically fulfilled.
Validate the solution results only after the test environment is stabilized and tests are re-run
Validation requires credible evidence against acceptance criteria; inconclusive data prevents a valid determination of requirement satisfaction.
-
✗ Mark the requirement as satisfied if the development team provides a verbal walkthrough
Verbal confirmations do not constitute formal test evidence or objective validation against documented acceptance criteria.
-
✗ Perform an impact analysis to see if the project can go live without this requirement
While impact analysis is useful, the primary task in evaluation is to determine if the solution satisfies the stated requirements based on evidence.
-
✗ Initiate a change request to modify the acceptance criteria to fit the current results
Lowering standards to match poor performance undermines the integrity of the solution and the business case.
-
-
3 In a Hybrid project environment, the BA is analyzing gaps between the developed solution and the solution scope. They find that several 'must-have' features from the requirements baseline are missing, yet the Product Owner wants to sign off. What is the BA's primary responsibility?
Reflect on the BA's role as a bridge between technical delivery and stakeholder expectations.
Communicate the discrepancies to all relevant stakeholders to resolve the discrepancy before deployment
Task 2 of Evaluation specifically requires enabling stakeholders to resolve discrepancies between scope, requirements, and the developed solution.
-
✗ Update the requirements baseline to remove the missing features to match the developed solution
Modifying the baseline to cover up omissions ignores the business need and bypasses proper governance and evaluation.
-
✗ Quietly add the missing features to the next iteration's backlog without notifying the project manager
Transparency is critical in evaluation; skipping the communication of gaps prevents stakeholders from making informed deployment decisions.
-
✗ Defer to the Product Owner's authority and process the sign-off immediately
The BA's role is to ensure the solution meets requirements and business needs; they must first highlight the risks associated with missing scope.
-
-
4 A company implements a new CRM tool to increase customer retention by $10\%$. Six months post-deployment, retention has only increased by $2\%$. The BA is performing a post-implementation review. Which tool or technique is most appropriate to determine the root cause of this variance?
Identify the technique focused on digging beneath the surface of a performance gap.
Root cause analysis
Techniques like the Fishbone diagram or 5 Whys are specifically used in evaluation to identify why realized benefits differ from expected outcomes.
-
✗ Requirement traceability matrix
The matrix ensures requirements were developed and tested but does not analyze the underlying business reasons for poor performance.
-
✗ Delphi technique
This is a consensus-building tool used for forecasting or decision-making, not for diagnosing technical or organizational variances.
-
✗ Documentation management
This focuses on the storage and versioning of artifacts rather than analytical problem-solving of post-deployment performance.
-
-
5 When obtaining stakeholder sign-off on a developed solution (Evaluation Task 3), the BA encounters a group of stakeholders who cannot reach a consensus. Which technique should the BA use to facilitate a final decision?
Think about tools specifically designed to resolve group deadlock by weighing and selecting options.
Multi-voting
Multi-voting is a recognized decision-making technique used to narrow down options and reach an agreement among diverse stakeholders.
-
✗ Brainstorming
Brainstorming is used for idea generation during elicitation or analysis, not for formalizing a final decision or sign-off.
-
✗ SWOT analysis
SWOT is a valuation tool used to assess strategic position rather than a tool for reaching group consensus on a specific solution release.
-
✗ Decomposition
Decomposition breaks down requirements into smaller parts but does not facilitate the interpersonal agreement needed for sign-off.
-
-
6 Which of the following best describes the difference between 'Verification' and 'Validation' within the context of PMI-PBA Evaluation?
Distinguish between meeting a technical specification and meeting an overall organizational objective.
Verification ensures the product was built correctly, while validation ensures the correct product was built to meet the business need
Verification focuses on technical specifications (the 'how'), whereas validation focuses on the fulfillment of business goals (the 'why').
-
✗ Verification is done by stakeholders, while validation is done exclusively by the technical team
Stakeholders are typically heavily involved in validation (UAT), while technical teams handle internal verification (testing).
-
✗ Verification happens post-deployment, while validation occurs only during the design phase
Validation is a key part of the Evaluation domain occurring before and after release to ensure value delivery.
-
✗ There is no difference; they are interchangeable terms for testing the solution results
In professional business analysis, these are distinct activities with different goals and target outcomes.
-
-
7 The BA is evaluating a deployed solution and notices that while the solution meets the 'Value Proposition' defined in the business case, it has introduced unexpected manual work for the finance department. This is an example of:
Look for the term that describes the 'unfinished' or 'problematic' parts of a solution's transition to the future state.
A residual capability gap
This identifies a discrepancy between the desired future state and the actual results of the implemented solution.
-
✗ Successful benefits realization
Benefits realization focuses on the positive value achieved, not the negative side effects or new manual burdens created.
-
✗ Baseline misalignment
While it might indicate a planning error, the term for the missing functionality or efficiency in the actual solution is a capability gap.
-
✗ Technical debt
Technical debt usually refers to suboptimal code or design choices, whereas this is a functional or process efficiency gap.
-
-
8 To evaluate a deployed solution (Task 4), a BA compares the current outcomes against the 'Business Case'. If the BA finds that the market conditions have changed significantly since the project was initiated, what should they do?
Think about how a BA must account for the environment in which the solution exists.
Document the external variances and assess if the solution still provides strategic value
Evaluation requires identifying the reasons behind variances, including external factors like market shifts, to recommend further action.
-
✗ Ignore the market changes and report only on the internal requirement fulfillment
Ignoring external context prevents an accurate assessment of whether the solution actually solves the business problem.
-
✗ Automatically declare the solution a failure because it no longer matches the business case
A solution may still provide value even if the original business case assumptions have shifted; a thorough evaluation is required first.
-
✗ Force the stakeholders to adopt the original KPIs to maintain consistency
Applying irrelevant KPIs to a changed business environment leads to inaccurate data and poor strategic decision-making.
-
-
9 A BA is working on Task 2 (Analyze and communicate gaps). They identify that the solution scope statement included a requirement for 'real-time' data, but the developed solution provides 'near real-time' ($5$-minute delay) data. What is the most appropriate quality assurance tool to use here?
Consider the specific technique used to measure the distance between 'what was asked for' and 'what was delivered'.
Gap analysis
Gap analysis specifically compares the current state (near real-time) with the desired state (real-time) to identify discrepancies.
-
✗ MoSCoW prioritization
Prioritization helps rank requirements before development; it is not a primary tool for comparing developed results against scope.
-
✗ Brainstorming
Brainstorming generates ideas but lacks the structured analytical framework required to document technical gaps between scope and solution.
-
✗ Data flow diagram
While a DFD shows data movement, it is a modeling tool, not a tool for analyzing gaps between performance and requirements.
-
-
10 During a post-deployment evaluation, the BA uses a 'Kano Model' to survey users. What is the BA likely trying to achieve?
Think about a technique that categorizes features based on their impact on user happiness.
Understanding which solution features are perceived as basic needs versus 'delighters' to prioritize enhancements
The Kano Model is a valuation tool used in evaluation to assess how features contribute to customer satisfaction and value.
-
✗ Measuring the technical uptime and response time of the system servers
Kano focuses on user perception and satisfaction, not on technical infrastructure performance metrics.
-
✗ Ensuring that the requirements were properly traced back to the project charter
Traceability is a separate domain; the Kano model is used to assess the qualitative value delivered by the solution.
-
✗ Establishing a communication plan for the deployment phase
The Kano model is an analytical tool for value assessment, not a management plan for communication logistics.
-
-
11 Task 3 of Evaluation involves obtaining stakeholder sign-off to proceed with deployment. If a critical defect is found, but the BA determines the business risk of not deploying is higher than the risk of the defect, what should the BA do?
Consider the role of the BA in facilitating transparent, value-based decisions during the release process.
Present the risk-benefit analysis to stakeholders and seek approval for a 'conditional sign-off' or release with a known defect
The BA facilitates decision-making; providing clear evidence on risks allows stakeholders to make informed choices about deployment.
-
✗ Conceal the defect until after deployment to ensure the sign-off is obtained without delay
Transparency and ethical behavior are core to the BA role; concealing defects violates the integrity of the evaluation process.
-
✗ Automatically reject the solution and halt the project until the defect is resolved
BAs should support business value; if the cost of delay exceeds the cost of the defect, a total halt may not be the best recommendation.
-
✗ Update the acceptance criteria to exclude the failing requirement so the defect disappears
Manipulating criteria to avoid reporting defects is unprofessional and fails to meet the actual business need.
-
-
12 The BA is performing 'Solution Validation' (Task 1). Which of the following sources is LEAST likely to be used as 'test evidence'?
Differentiate between project management artifacts (how the project is run) and business analysis artifacts (what the product does).
The project budget report
Budget reports track spending and financial health, but they do not contain evidence that technical requirements have been satisfied.
-
✗ User Acceptance Testing (UAT) results
UAT is a primary source of evidence showing that stakeholders have validated the solution against their needs.
-
✗ Quality Control (QC) inspection reports
Technical inspections provide data on whether the solution meets defined quality and technical standards.
-
✗ Automated test logs
Logs provide objective, system-generated evidence that specific functions perform as expected under defined conditions.
-
-
13 In an Agile environment, which of the following Evaluation activities is most likely to occur during a 'Sprint Review'?
Focus on the Agile event where the 'potentially shippable product' is inspected by the business.
Demonstrating the solution increment to stakeholders to obtain validation and feedback
Sprint Reviews are specifically designed for validating the developed work against acceptance criteria and gathering stakeholder input.
-
✗ Conducting a root cause analysis for a technical server failure that occurred mid-sprint
Technical root cause analysis is a diagnostic activity, not the primary focus of the collaborative Sprint Review meeting.
-
✗ Updating the requirements traceability matrix for the entire project lifecycle
RTM maintenance is an ongoing administrative task, not a collaborative validation session with stakeholders.
-
✗ Defining the solution scope for the next project phase
Scope definition is a Needs Assessment activity, whereas Sprint Reviews focus on evaluating what has already been built.
-
-
14 A BA is evaluating a solution and notices that it meets all 'functional' requirements but fails several 'quality' (non-functional) requirements, such as page load time. What is the BA's best course of action during Task 2 (Analyze gaps)?
Consider the importance of 'how' a system works alongside 'what' it does.
Analyze the impact of the failed quality requirements on the overall business case and communicate the risk to stakeholders
Quality requirements are essential for solution success; failing them represents a gap that must be analyzed for its business impact.
-
✗ Ignore the non-functional failure since the 'main' features are working correctly
Failing non-functional requirements can lead to poor adoption or system failure, even if functional features are present.
-
✗ Relocate the non-functional requirements to the next project phase without stakeholder review
Moving requirements without consultation bypasses the evaluation process and ignores potential immediate operational risks.
-
✗ Focus solely on obtaining sign-off, as quality requirements are usually considered optional
Quality requirements (performance, security, etc.) are rarely optional and are critical to fulfilling the business case.
-
-
15 Which valuation technique is most helpful during Task 4 (Evaluate deployed solution) to determine if the solution aligns with the organization's strategic objectives?
Look for a strategic tool used to analyze the business environment and competitive position.
SWOT Analysis
SWOT helps assess how the implemented solution affects the organization's strengths, weaknesses, opportunities, and threats in relation to its goals.
-
✗ Requirement Walk-through
Walk-throughs are verification techniques used to find errors in documentation, not for assessing high-level strategic value realization.
-
✗ Decision Tree
Decision trees are used for modeling logic and choices during the analysis phase, not for evaluating post-deployment value.
-
✗ State Diagram
State diagrams are modeling tools that show the lifecycle of an object, which is unrelated to strategic valuation.
-
-
16 A BA has completed the evaluation of a solution and identified that the solution has $80\%$ adoption among target users. The business case expected $95\%$ adoption. Which activity should the BA perform NEXT?
Recall the steps necessary to move from observing a problem to understanding it.
Investigate the reasons behind the $15\%$ adoption variance through surveys or focus groups
Task 4 of Evaluation involves comparing actual results to expected benefits and identifying reasons behind variances.
-
✗ Close the project immediately since the majority of users are using the solution
Project closure happens after evaluation is complete; failing to investigate an adoption gap leaves the business case unfulfilled.
-
✗ Declare the adoption metrics as 'out-of-scope' and focus on technical uptime
Adoption is often a critical KPI for benefits realization; ignoring it undermines the value of the business analysis work.
-
✗ Request a budget increase for more training without analyzing the root cause
Requests for resources should be based on data-driven evidence from an evaluation, not on assumptions about the cause of a gap.
-
-
17 What is the primary objective of Task 2 in the Evaluation domain: 'Analyze and communicate the solution's identified gaps and deltas'?
Focus on the ultimate goal of providing stakeholders with high-quality information for decision-making.
To enable stakeholders to resolve discrepancies between the desired and developed solution
Communication of gaps is meant to facilitate decision-making, whether it involves fixing the gap, accepting it, or modifying requirements.
-
✗ To assign blame to the development team for missing requirements
Evaluation is a professional process for solution improvement and value realization, not for performance punishment.
-
✗ To create new requirements that extend the project timeline and budget
While new requirements might emerge, the primary task is analyzing existing gaps relative to the current project scope.
-
✗ To ensure that all documentation is version-controlled in the repository
Version control is a management task; analyzing gaps is an analytical task focused on solution quality and fit.
-
-
18 A BA is assessing 'Operational Readiness' during the Evaluation phase. Which of the following would be a key indicator that the organization is ready to transition to the new solution?
Identify the factors that demonstrate the business can actually function using the new product.
The help desk staff has been trained and access permissions have been configured
Operational readiness involves ensuring the support structures and technical environment are prepared for the transition to the future state.
-
✗ The project charter has been signed by the sponsor
Charter signing occurs during project initiation (Needs Assessment), long before operational readiness becomes relevant.
-
✗ The requirements management plan has been finalized
This plan is a deliverable of the Planning domain, defining how BA work will be managed, not how the solution will be used.
-
✗ A list of potential vendors has been identified
Vendor identification typically happens during Needs Assessment or Analysis when exploring product options.
-
-
19 Post-deployment, a BA determines that a solution failed to meet a regulatory compliance requirement. Which of the following is the most likely cause identified during the evaluation analysis?
Think about the foundational mistakes that occur at the start of the requirements lifecycle.
Erroneous assumptions during the requirement gathering phase
PMI notes that variances often stem from incorrect assumptions made early in the project during elicitation and analysis.
-
✗ The use of Agile rather than Waterfall methodology
Methodology itself does not cause compliance failure; poor requirement management or elicitation is the typical root cause.
-
✗ Performing too many workshops with stakeholders
Collaboration generally improves requirement quality; it is unlikely to be the cause of missing a critical compliance standard.
-
✗ A high number of correctly identified project risks
Identifying risks helps prevent failures; it does not lead to missing requirements.
-
-
20 A BA is using 'Value Stream Mapping' during the Evaluation of a deployed process. What is the intended outcome of this valuation technique?
Consider a technique focused on the flow of work and the elimination of unnecessary activity.
Identifying non-value-added steps (waste) in the new process to measure efficiency gains
Value stream mapping is a technique used in evaluation to see how the solution has improved the flow of value through the organization.
-
✗ Mapping the hierarchy of stakeholders by their level of influence
Stakeholder mapping identifies relationships and power, not the process flow or value delivery of the solution.
-
✗ Estimating the total cost of ownership for a software license
Cost estimation is a planning or analysis task; value stream mapping specifically focuses on process steps and waste.
-
✗ Tracking the version history of the requirements document
Versioning is a configuration management activity, whereas value stream mapping is an analytical efficiency tool.
-
-
21 During the evaluation of a solution's test results (Task 1), the BA finds that $2\%$ of the requirements failed testing, but these requirements were labeled as 'Optional' (Could-have). What is the BA's best recommendation for the sign-off meeting?
Think about a balanced approach that acknowledges minor issues while still moving the business forward.
Recommend deployment while documenting the failed requirements as 'residual capability gaps' for future resolution
If critical requirements are met, 'optional' failures may be accepted to preserve deployment timelines, provided they are tracked.
-
✗ Refuse to allow the meeting to occur until $100\%$ of all requirements pass
Waiting for perfection in optional features can delay business value and violate the 'equilibrium between expenses and advantages'.
-
✗ Suggest that the testing team falsify the results since the requirements were only optional
Falsifying results is unethical and misrepresents the state of the solution to stakeholders.
-
✗ Tell the stakeholders the solution is perfect so they sign off without questions
Honesty in evaluation is required for stakeholders to make informed, risk-based decisions.
-
-
22 Task 4: Evaluate the deployed solution includes 'measuring realized benefits against the original business case'. Who is the most appropriate person to provide input on benefit realization?
Identify the individual who 'owns' the problem and the expected reward of the solution.
The benefit owner or key business stakeholder
The benefit owner is accountable for the results post-implementation and is best positioned to confirm if the business intent was met.
-
✗ The junior developer who wrote the code
Developers focus on technical output, while benefit realization is a business-centric measurement of value.
-
✗ The project scheduler
Schedulers track time and milestones, which are project performance metrics, not business value metrics.
-
✗ The procurement officer
Procurement handles vendor contracts and purchasing, not the internal realization of project benefits.
-
-
23 Which Evaluation deliverable serves as the basis for stakeholders to give formal or verbal approval to move forward with the product launch?
Focus on the specific document that summarizes whether the product is 'ready'.
Solution evaluation report or acceptance results
This report summarizes the findings of the evaluation, including test results and gap analysis, justifying the release decision.
-
✗ The requirements traceability matrix (RTM)
The RTM is a tracking tool, but the summarized evaluation of findings is what drives the actual launch decision.
-
✗ The project charter
The charter starts the project; it does not provide the evidence needed to end a phase and launch a solution.
-
✗ The elicitation plan
The elicitation plan is a planning document used at the start of requirements discovery, not at the end of solution evaluation.
-
-
24 A BA is performing a 'Solution Assessment' post-deployment. They discover that although the system works, the organization's culture is resisting the change, leading to low adoption. What should the BA include in their evaluation report?
Consider how the BA should respond to 'human factors' that prevent a solution from succeeding.
Recommendations for organizational change management or further training to address adoption barriers
Evaluation should identify adoption barriers and suggest organizational changes to ensure the intended value is realized.
-
✗ A request to uninstall the system immediately because it is not working as intended
Uninstalling is an extreme measure; the BA should first recommend corrective actions to resolve the human/process variances.
-
✗ A statement that business analysis work is complete since the technical code is defect-free
BA work continues into evaluation to ensure the business problem is solved, not just that code is delivered.
-
✗ A list of stakeholders to be fired for resisting the new technology
The BA role is focused on process and solution improvement, not on personnel disciplinary actions.
-
-
25 In the Evaluation domain, what is the purpose of 'setting up key performance metrics and data collection mechanisms' BEFORE implementation?
Think about the importance of having the right yardstick ready before the project ends.
To ensure that accurate data is available for a fair and fair post-implementation assessment
Planning for evaluation early prevents costly changes and ensures the BA can objectively measure success against the business case.
-
✗ To keep the project manager busy during the design phase
Evaluation planning is a critical BA task focused on value, not an administrative task for the project manager.
-
✗ To allow the development team to hard-code the success results into the system
Metrics must be objective and reflect real-world data, not fabricated or hard-coded results.
-
✗ To ensure that all stakeholders are automatically signed off on the results
Metrics provide the data for sign-off, but sign-off itself is a separate decision-making process involving judgment.
-