Project Management5 April 2025

A data-driven use case planning and assessment approach for AI portfolio management.

Introduction

The advent of Artificial Intelligence (AI) presents seemingly unlimited possibilities for companies across various business areas. However, despite this potential, AI technology has not yet been universally adopted. Research indicates that by 2022, only 50% of companies worldwide had deployed AI in at least one business function. This lag in adoption can be attributed to several persistent barriers and challenges. These include a lack of a defined AI strategy, insufficient management support, resistance from employees and customers, and a deficiency in the necessary expertise and skills. From a technical standpoint, issues related to the availability and quality of data, inadequate infrastructure, and the complexities of integrating AI solutions into existing systems and processes also impede adoption. Financial concerns, such as high research and development costs and uncertainty regarding the return on investment (ROI) of AI solutions, further contribute to hesitancy. Moreover, legal and ethical considerations, as well as the inherent risks associated with AI, play a significant role. A fundamental challenge lies in the difficulty companies face in identifying suitable AI use cases.

To effectively navigate these challenges, extensive planning is crucial before embarking on the resource-intensive implementation phase of AI projects. This planning should primarily focus on the identification and rigorous evaluation of AI use cases that are both valuable and feasible. Recognizing this need, prior research has developed various approaches and methodologies for AI use case planning, which often differ in their underlying concepts and the evaluation criteria they employ. Additionally, the emergence of AI use case databases, such as AWS Use Case Explorer and DataRobot Pathfinder, offers a source of inspiration and a knowledge base, albeit with limitations in supporting comprehensive evaluation. The research presented in the focal paper seeks to address these limitations by proposing a data-driven framework that integrates a standardized use case description with a systematic planning method to support the identification, evaluation, and prioritization of suitable and feasible AI use cases. This literature review aims to summarize, synthesize, and critically evaluate the existing research landscape on data-driven AI use case planning and portfolio management, primarily based on the insights provided by this novel approach. The structure of this review will first explore the main themes identified in the research, followed by an examination of points of agreement, debate, and existing gaps. Subsequently, the methodologies employed in the research will be discussed, along with the development of the field over time as portrayed in the source. Finally, critical insights into the alignment and divergence of this body of work will be offered, culminating in a comprehensive conclusion.

Main Themes in Data-Driven AI Use Case Planning and Portfolio Management

Several overarching themes emerge from the research presented in the source regarding data-driven AI use case planning and portfolio management.

The Critical Need for Structured Planning and Assessment: The paper emphasizes that due to the inherent complexities and potential costs associated with AI initiatives, a systematic and well-structured approach to planning and assessment is paramount. It highlights that identifying and evaluating valuable and feasible AI use cases before committing significant resources is essential to mitigate risks and maximize the chances of success. The research notes that wrong decisions made early in the AI lifecycle can lead to substantial future costs. The proposed framework addresses this by outlining a three-step iterative planning method (ideation, scoping, and assessment) that guides organizations through the process of defining and evaluating AI use cases.
Value and Feasibility as Central Evaluation Dimensions: A consistent theme across the research is the centrality of business value and feasibility as key dimensions for evaluating and prioritizing AI use cases. Business value is often broken down into sub-dimensions such as strategic value (e.g., increased customer satisfaction, improved image) and financial value (e.g., cost savings, revenue increase). Feasibility, on the other hand, encompasses technical, organizational, and data-related aspects, including model complexity, data complexity, integration complexity, required expertise, and risk classification. The paper’s AI use case data model is structured around these two core dimensions, providing a framework for systematically collecting and analyzing relevant data to inform evaluation and prioritization decisions.
The Foundational Role of Data: The research underscores the critical role of data in the success of AI use cases. Issues related to data availability, accessibility, and quality are frequently cited as technical barriers to AI adoption. The proposed data-driven approach explicitly focuses on the systematic collection and storage of specific use case data to foster transparency and build a knowledge base. The feasibility assessment within the framework heavily considers data complexity, encompassing factors such as data availability, accessibility, and quality. Exploratory data analysis and the identification of metadata such as format, size, and storage location are also highlighted as important aspects of feasibility assessment.
Standardization and Knowledge Management through AI Use Case Libraries: The paper advocates for the standardization of AI use case descriptions and the creation of AI use case libraries to gather ideas, document and compare solutions, assess feasibility, and plan implementation. The proposed AI use case data model aims to provide this standardized description through a comprehensive set of metadata. The benefits of such standardization include ensuring consistency and reuse across projects, enhancing collective understanding and assessment of AI initiatives, and facilitating the creation of a knowledge base for data-driven decisions. Existing AI use case databases are acknowledged as initial steps in this direction but are deemed insufficient for thorough evaluation.
Enabling Data-Driven Decision Making for AI Portfolio Management: The overarching goal of the proposed framework is to enable data-driven decision making for AI use case portfolio management. By systematically collecting, storing, and analyzing data related to the value and feasibility of AI use cases, organizations can make more informed and traceable decisions regarding which initiatives to prioritize and invest in. The framework introduces quality gates at each stage of the planning process to ensure data completeness and plausibility, further supporting robust decision-making. The prioritization matrix, based on the dimensions of value and feasibility, provides a visual tool for making these decisions.

III. Points of Agreement, Debate, and Gaps in the Research (Based on the Source)

The research presented in the source aligns with and diverges from existing literature on AI use case planning in several key aspects.

Points of Agreement:
There is a general agreement on the necessity of having structured methodologies for identifying, evaluating, and prioritizing AI use cases. The paper acknowledges the existence of several such methods in prior research.
The consensus that business value and feasibility are fundamental criteria for assessing AI initiatives is evident. Various existing methodologies also focus on these two dimensions during the evaluation phase.
The importance of data-related factors in determining the feasibility of AI use cases is a shared understanding. The availability, quality, and accessibility of data are consistently highlighted as crucial considerations.
The recognition of existing AI use case databases as valuable resources for inspiration and knowledge sharing represents another point of agreement. These databases serve as a starting point for organizations looking to explore potential AI applications.
Points of Debate and Differences:
A key area of difference lies in the variety of existing AI use case planning methods, which differ in their procedural steps, evaluation criteria, and the extent to which they cover the entire AI lifecycle. The proposed method distinguishes itself by its explicit focus on the systematic collection and utilization of data throughout the planning process to drive decision-making.
Approaches to ideation also vary, with some methods being business-driven (starting from problems or needs) and others being technology-driven or data-driven (exploring new possibilities based on available technologies or data). The paper acknowledges both evolutionary and revolutionary approaches to ideation.
While AI use case databases exist, they utilize different metadata to describe use cases. The proposed AI use case data model aims to provide a more standardized and comprehensive set of metadata relevant for evaluation and prioritization.
Gaps in the Research:
The paper identifies a need for cross-domain planning methodologies that can facilitate the transfer of successful AI use cases across different industries or domains, a flexibility often lacking in existing approaches.
A significant gap highlighted is the insufficient attention paid to ethical considerations, bias mitigation, and fairness in many existing AI use case planning approaches. The paper acknowledges the increasing emphasis on responsible AI and the need to incorporate these ethical dimensions.
The research notes a lack of robust planning approaches that can adapt in real-time to rapidly changing environments and provide continuous decision support. Traditional AI planning often struggles with dynamic situations.
Efficient and adaptive resource allocation methods for AI projects represent another area where further research is needed, as traditional planning often faces challenges in this regard.
Current AI use case databases, while useful for inspiration, are limited in their ability to support comprehensive evaluation and feasibility assessment necessary for data-driven decision-making. They often lack the detailed metadata required for thorough analysis.

Relevant Methodologies Used (Based on the Source)

The development and evaluation of the proposed data-driven AI use case planning framework rely on several key research methodologies.

Action Design Research (ADR): The overarching methodology employed in the research is Action Design Research (ADR). ADR is a method that combines Design Research (DR) and Action Research (AR), allowing for the iterative building, intervening, and evaluating of artifacts (in this case, the AI use case planning method and the AI use case data model) in a real-world organizational context. The ADR process involves problem formulation, several building, intervention, and evaluation (BIE) cycles, and a final phase of reflection and learning followed by formalization of learning. This iterative approach allows for continuous refinement of the artifacts based on feedback and practical application.
Structured Literature Review: To establish the research gap and understand the existing body of knowledge, the authors conducted a structured literature review. This review identified and analyzed existing methods for identifying, evaluating, and prioritizing AI use cases, as well as existing AI use case databases. The findings from the literature review informed the initial design of the proposed framework.
Case Study Research: The intervention and evaluation phases of the ADR process are primarily conducted through case studies. Both internal case studies (within the OEM partner) and external case studies (with other partner companies) were carried out to test the applicability, comprehensibility, and added value of the developed data model and planning method. These case studies involved applying the framework to real-world AI use case examples and gathering feedback from practitioners.
Mention of other relevant methodologies: The paper also makes reference to other methodologies relevant to AI and data management, such as CRISP-DM (Cross-Industry Standard Process for Data Mining) and TDSP (Team Data Science Process), in the context of AI and ML lifecycle models that include problem identification and data understanding phases. Additionally, design thinking is mentioned as a user-centric approach that can be used during the ideation phase of AI use case planning. The Technology-Organization-Environment (TOE) framework is also noted as a tool used in the preparation phase of some existing methodologies for identifying environmental factors.

Development in the Field Over Time (Based on the Source)

The source provides insights into the development of AI adoption and related research over time. Initially, despite the vast potential of AI, its adoption was hindered by various technical, organizational, and financial barriers. By 2022, a significant portion of companies had started deploying AI, indicating a growing but still incomplete integration of the technology.

Research in the field has progressed from developing various methods for identifying and evaluating AI use cases to the emergence of practical tools like AI use case databases. Early research focused on structured approaches to planning and assessment, often emphasizing business value and feasibility. The development of AI use case databases represented a step towards knowledge sharing and inspiration for new AI applications.

More recently, research efforts have started to focus on more advanced and nuanced aspects of AI use case planning, such as the need for cross-domain applicability, the integration of ethical considerations, the development of real-time and adaptive planning approaches, and the optimization of resource allocation for AI projects. The research presented in the source contributes to this evolution by proposing a more integrated and data-driven framework that builds upon earlier work while addressing some of its limitations, particularly in the systematic use of data for evaluation and decision-making. The focus on a comprehensive AI use case data model also signifies a progression beyond simple descriptive databases towards more analytical tools for AI portfolio management.

Critical Insights and Alignment/Divergence

The research presented in the source offers valuable insights into the field of data-driven AI use case planning and portfolio management, demonstrating both alignment with and divergence from existing literature.

Alignment with Existing Research:
The paper aligns with the broad recognition in the literature of the importance of structured planning processes for AI initiatives. The proposed three-step method echoes the sentiment that a systematic approach is crucial for success.
The central role given to business value and feasibility as evaluation criteria is consistent with numerous existing methodologies. The paper's breakdown of these dimensions into sub-dimensions further elaborates on this common ground.
The acknowledgment and initial building upon the concept of AI use case databases for knowledge sharing demonstrates an alignment with practical efforts in the field. The aim to enhance these databases with more comprehensive evaluation capabilities is a logical progression.
The iterative nature of the proposed scoping and assessment steps aligns with other methodologies that also advocate for an iterative approach to AI use case planning.
Divergence and Novel Contributions:
A key divergence and significant contribution of this research lies in its explicit and central focus on a data-driven approach. Unlike many existing methodologies that may mention data as a factor, this framework systematically integrates data collection, storage, and retrieval into each step of the planning process to inform decision-making. This addresses a recognized gap in leveraging data more effectively for AI use case planning.
The development and detailed articulation of a specific and comprehensive AI use case data model with clearly defined entities, attributes, and quality gates represent another novel contribution. This goes beyond the more general descriptions found in some literature and the limited metadata in existing AI use case databases. The data model provides a structured foundation for collecting and analyzing relevant information.
The proposed framework integrates the planning method directly with the data model, emphasizing their interaction for data collection, storage, and retrieval throughout the planning lifecycle. This tight coupling to enable data-driven prioritization distinguishes it from methodologies that might treat planning and data considerations more separately.
The explicit introduction of quality gates at each stage to ensure data completeness and plausibility before moving forward in the planning process adds a layer of rigor and control that may not be as formally emphasized in other approaches.
Overall Contribution to the Understanding of the Topic: This research significantly contributes to the understanding of data-driven AI use case planning and portfolio management by providing a tangible and structured framework for leveraging data to enhance decision-making. It moves beyond conceptual discussions of the importance of planning and data to offer a concrete methodology and a detailed data model that can be applied in practice. By iteratively developing and evaluating this framework through real-world case studies, the research provides valuable insights into the practical challenges and benefits of a data-driven approach. The identification of limitations and future research opportunities further contributes to the ongoing development of this critical field.

VII. Conclusion

In conclusion, the research on data-driven AI use case planning and portfolio management, as exemplified by the presented framework, highlights the critical need for structured and data-informed approaches to navigate the complexities of AI adoption. The central themes of structured planning, the importance of value and feasibility, the foundational role of data, the benefits of standardization through AI use case libraries, and the goal of enabling data-driven decision making are consistently emphasized. While aligning with existing research on the necessity of planning and the importance of core evaluation dimensions, this work diverges by offering a specific and integrated data-driven framework comprising a detailed planning method and a comprehensive AI use case data model. The use of Action Design Research and real-world case studies provides a robust methodology for developing and evaluating these artifacts. The identified gaps in existing research, such as the need for cross-domain planning, greater attention to ethical considerations, and more dynamic planning approaches, underscore the ongoing evolution of the field. The limitations of the presented research, including potential biases and the dependency on technical expertise, point towards important avenues for future work, such as longitudinal studies, the development of training for non-technical users, and the incorporation of emerging concerns like sustainability and AI legislation. Ultimately, this body of work contributes significantly to a more nuanced understanding of how organizations can effectively leverage data to improve the identification, evaluation, and management of their AI initiatives, fostering more informed and strategic investments in AI technologies.

Frequently Asked Questions about Data-Driven AI Use Case Planning

Why is a data-driven approach important for AI portfolio management?

A data-driven approach to AI portfolio management addresses several key challenges hindering successful AI adoption. Many companies struggle to identify suitable AI use cases and lack a systematic way to evaluate their potential value and feasibility. Traditional methods often lack the rigor and transparency needed for making informed investment decisions in AI. By systematically collecting and analyzing data related to potential AI applications, organizations can better understand the strategic and financial value, technical feasibility (including data availability and complexity), required resources, and potential risks associated with each use case. This allows for more informed prioritization and resource allocation, ultimately increasing the likelihood of successful AI implementation and a positive return on investment.

What are the main components of the proposed data-driven AI use case planning framework?

The framework consists of two interconnected main components: an AI use case data model and an AI use case planning method. The data model serves as a standardized structure for describing AI use cases using relevant metadata and assessment criteria. It includes 18 entities with numerous attributes categorized as text, numeric value, and category (single or multiple choice), and organized into metadata and assessment data (further divided into value and feasibility dimensions). The planning method is a three-step iterative process (ideation, scoping, and assessment) that guides users through the collection, storage, and utilization of data based on the data model. It emphasizes quality gates at each step to ensure data completeness and plausibility before progressing.

How does the AI use case data model help in managing AI initiatives?

The AI use case data model provides a structured and standardized way to document and compare AI use cases. By capturing a wide range of metadata and assessment data, it enables the creation of a central AI use case library or database. This library facilitates several key functions:

Transparency: Provides a clear overview of all AI use cases and ideas within the organization.
Knowledge Base: Stores valuable information about past, present, and potential AI initiatives, enabling learning and reuse.
Search and Filtering: Allows users to find specific or similar AI use cases based on various criteria like business unit, AI capability, or data source.
Synergy Identification: Helps identify potential overlaps or complementarities between different AI use cases.
Data-Driven Decision Making: Provides the necessary data for evaluating and prioritizing AI use cases based on value and feasibility dimensions.
Standardized Communication: Ensures consistent understanding and assessment of AI initiatives across different stakeholders.

What are the three main steps in the AI use case planning method, and what happens in each?

The AI use case planning method involves three main steps:

Ideation: This initial step focuses on generating and describing initial AI use case ideas based on existing problems, needs, or new opportunities. Each idea is documented with basic information (Quality Gate 1 attributes) and then reviewed for potential prioritization.
Scoping: Prioritized ideas move to the scoping phase, where a cross-functional team gathers more detailed information, particularly regarding technical and organizational feasibility, as well as initial estimates of value and costs (Quality Gate 2 attributes). This step can be iterative, with increasing levels of detail in each cycle.
Assessment: In the final step, the assessment team evaluates the scoped AI use cases based on a comprehensive set of criteria, including calculated numeric values for value and costs, and a thorough risk assessment (Quality Gate 3 attributes). This data is used to categorize the use case in a prioritization matrix, leading to a final decision on whether to proceed with implementation, put the project on hold, or reject it.

How are value and feasibility assessed within this framework?

Value is assessed by considering three sub-dimensions: strategic value (non-monetary benefits like increased customer satisfaction), financial value (quantifiable monetary gains such as cost savings or revenue increase), and costs (including implementation, deployment, and maintenance). The overall value is determined by comparing the strategic and financial values with the associated costs. Feasibility is evaluated based on five dimensions: model complexity (related to the AI solution itself), data complexity (availability, quality, and accessibility of data), integration complexity (extent of changes needed in existing systems and processes), required know-how (availability of necessary skills and expertise), and risk classification (potential risks and their consequences). Each of these dimensions is further broken down into specific attributes within the AI use case data model.

What role do "quality gates" play in the AI use case planning method?

Quality gates are checkpoints implemented after each of the three planning steps (ideation, scoping, and assessment). At each quality gate, the collected AI use case data is reviewed for plausibility and completeness. If the data does not meet the required standards, the process reverts to the previous step for further refinement and data collection. This iterative process ensures that only well-defined and sufficiently documented AI use cases move forward to the next stage, minimizing wasted resources on poorly conceived or inadequately understood initiatives. The quality gates help maintain data integrity and support more informed decision-making at each prioritization point.

How does this data-driven approach facilitate better decision-making in AI portfolio management?

By systematically collecting, storing, and analyzing data related to AI use cases, the proposed framework enables traceable and data-based prioritization decisions. The standardized AI use case descriptions and the defined value and feasibility dimensions allow for objective comparisons between different AI initiatives. The prioritization matrix, populated with data-driven assessments, provides a visual tool for portfolio managers to identify high-value and high-feasibility projects. Furthermore, understanding the specific factors contributing to value or complexity (as captured in the data model) allows for targeted actions to improve the potential of AI use cases, such as reducing complexity by breaking down projects or increasing value by identifying new drivers.

What are some limitations of this approach and potential areas for future research?

One limitation is the potential for bias introduced by the composition of the development team and the participants in the case studies. Future research could involve a more diverse group of stakeholders. The reliance on technical expertise for certain aspects of the framework can also be a barrier for non-technical participants, suggesting a need for more accessible training and instructions. While the case studies indicated the added value of the approach, further longitudinal studies with companies from diverse industries are needed to validate its robustness and adaptability over time. Future research could also focus on extending the data model to incorporate aspects like sustainable AI and AI legislation (e.g., the EU AI Act) or to capture data across the entire AI use case lifecycle, enabling continuous learning and improvement. Additionally, exploring the potential for partial automation of the assessment process based on historical data is a promising avenue for future work.

(Bodendorf, 2025)

Bodendorf, F. (2025). A data-driven use case planning and assessment approach for AI portfolio management. Electronic Markets, 35(1). https://doi.org/10.1007/s12525-025-00759-x