Exploring application portfolio management in Indonesia: a case study of the Indonesia agency for the assessment and application of technology

ABSTRACT


INTRODUCTION
In July 2022, Indonesia's Minister of Finance highlighted an enormous number of applications owned by Indonesia's government.A total of 400,000 applications were registered.This staggering number was believed to have become a burden on national expenditures.The increasing number of digital services and rapid transformation have made it urgent to manage application portfolios and implement strategies to ensure that information and communication technology (ICT) investments are practical and efficient [1].Investment decision-making is usually based on several aspects, including planning, redundancy, risk, and resources [2].If not done correctly, issues such as poor ICT investment, redundant applications within institutions [3], and bad ICT-related decisions can occur.Application portfolio management (APM) is a strategy that can be implemented to reduce these issues.APM is a strategy that an organization can use to Comput Sci Inf Technol ISSN: 2722-3221  Exploring application portfolio management in Indonesia: a case study of … (Riri Kusumarani) 77 classify existing or planned applications that it owns into groups to align applications with the business core.APM can further coordinate ICT investment planning [4], lower ICT costs [5], manage risk [6], and manage ICT resources so that organizations can align business processes and ICT investment [7].Recently, the practice of APM has grown rapidly to help reduce costs for applications that do not need it [8].
Many frameworks are commonly used to implement APM.We focus on the two most used frameworks, McFarlan's strategic grid and gartner's TIME model.As a national research institution, the agency for assessing and applying technology's (BPPT) duty is to enact government roles in research.To do so, BPPT is supported by various applications spread across departments.According to our independent observation, BPPT has not developed an application portfolio management strategy for existing and planned applications.With the digital transformation program adding 41 new applications to support management, BPPT is vulnerable to poor application management strategy issues.
Surprisingly, there is little information available about how Indonesian governments manage their existing applications.As a result, this study aims to fill the gap by revealing how BPPT, Indonesia's research institution, manages its application.APM is also applied to the proposed 41 digital applications.We contend that BPPT necessitates APM in order to ensure that ICT investments are made effectively and efficiently.
The following questions will be addressed by this research: − How does BPPT implement the APM strategy for the 41 digital applications?− How are the 41 digital applications mapped using McFarlan's strategic grid?− How are the 41 digital applications mapped using GARTNER's TIME Model?− What kind of recommendations towards the 41 applications based on the two frameworks?
The following sections are a literature review in which a discussion on APM methodology will be introduced along with the two frameworks that will form the basis of our research.Still, in the same section, a brief introduction to BPPT and its 41 digital applications will be discussed.Section three is the research methodology section, followed by the results and discussion sections.Our last section is the conclusion.

LITERATURE REVIEW 2.1. Application portfolio management
Portfolio management is a step related to a product, including the evaluation process and priority selection to determine the priority of a product so that it can reallocate available resources [9].Portfolio management is often used to make appropriate, effective, and efficient decisions.This method has been widely used in various sectors, such as water supply [10], project management [11], tourism management [12], and information technology [13].
Application portfolio management (APM) is part of information technology portfolio management that focuses on software that directly supports business processes.APM is a process carried out by stakeholders as the basis for making decisions regarding existing organizational applications [3].Previous research [14] stated that APM successfully simplifies the process of creating application portfolios to manage complex application portfolios.The first stage of APM is carried out by collecting a list of all applications that have been developed or are still in the development process.The data owner and users of each application are then identified.Furthermore, the application is assessed based on its level of use by rationalizing it as a whole.
The list of collected applications is then carried out in an inventory to determine each application's total cost and business value.At this stage, business capabilities determine what the business is doing today and what needs to be done to meet current and future challenges.Based on the results of the previous stages, an application architecture framework is created by defining a set of business, information, and application concepts that the organization wants to see in the long term.These results are then re-aligned with the information technology (IT) roadmap at the institution.

McFarlan's strategic grid
McFarlan's strategic grid is a strategy for mapping applications into a grid based on their contribution to the organization [15].In a previous study [16], [17], a grid was divided into four categories: support, key operational, high potential, and strategic, as shown in Figure 1.Each category describes each application's contribution to the organization.Bintang et al. [18] mapped all the applications to McFarlan's strategic grid based on identification by the general good of the infrastructure, human resources, and portfolio of applications.Research conducted [19] uses McFarlan's strategic grid to create a future application portfolio for nonfranchise pharmacy companies.An analysis of the functionality of each application to the organization is carried out to map each application.Another study [20] used McFarlan's strategic grid to analyze the alignment between business strategy and information system planning at the API Polytechnic Yogyakarta.The support category is for applications that support the ease of business processes but do not affect the organization's business success.Usually, those applications are used for administration and the back office.The key operational category is for applications that manage business processes and provide decision support but do not directly affect corporate profits.The high potential category is for applications that have potential and value in the future but do not currently have a significant impact on the organization.And the last is the strategic category for applications that facilitate business processes and significantly affect the organization's continuity.

Gartner's TIME model
Like Mc'Farlan's strategic grid, the gartner TIME model also maps applications into a four-quadrant grid.Gartner's TIME model is used to evaluate ICT investments in a company.T (tolerate), I (Invest), M (Migrate), and E (Eliminate) represent recommendations for an ICT application/investment.TIME has been known for its ease of application and understanding in executing the APM process.
The mapping application of gartner's TIME Model method is based on its business value and technical quality [21].Business value is obtained from the user's functionality, efficiency, reliability, and frequency of use assessment, while IT quality is obtained from compatibility, security, maintenance, and portability.After each application gets these two values, a mapping is carried out using 2-dimensional coordinates with business values as the x-axis and IT quality (technical efficiency and safety) as the y-axis, as shown in Figure 2.

Figure 2. Two-dimensional coordinates of gartner's TIME model [21]
The first quadrant is "tolerate," in which applications with low support for business value but a working function and high IT quality will be put.The Invest category is for applications with high business value and IT quality.Applications fall into this quadrant relatively easily, experiencing minor disruptions, serving their purpose, providing source code availability, and generating cost savings or income.
The third quadrant is "eliminate," in which applications with low IT quality and less support for business value will reside.The last quadrant is "migrate," a category for applications with high business value but low IT quality.The business is using the application and can articulate its value.Still, there is a high cost of support, a lack of knowledge, a lack of source code, or a variety of other factors that make it an expensive application to maintain.

BPPT digital transformation
The agency for the assessment and application of technology (BPPT) is a non-ministerial government institution charged with assessing and applying technology on behalf of the government.BPPT is assisted in carrying out its management duties and functions by a number of applications distributed across several units.In order to implement e-government, digital transformation is required.The goal of digital transformation is to improve public services and governance while also preparing the government for Industry 4.0 and Society 5.0 [22].In 2019, BPPT began digital transformation activities, one of which was the development of 41 management support applications in response to the implementation of Presidential Regulation Number 95 of 2018 of the Republic of Indonesia, concerning the Electronic-Based Government System.

METHOD
This study used a mixed methodology to assess the existing 41 digital applications under BPPT's Digital Transformation program.This study used the McFarlan strategic grid and gartner's TIME Model as an analytical framework for application portfolio management.A survey targeting users and developers of each application was done in August 2021.Experts with a more profound knowledge of 41 applications were invited for an interview.The list of questions was adopted from ISO 25010.

Data collection
Data collection regarding the application is required to find out the current state of the application.Information related to the application is necessary to make the application portfolio, so we strive to balance the value of the application and the cost caused.The required information includes the application's name, the version used, the function, the number of users, the operating system, the application owner, system security, and so on.
APM data can be collected in three ways: automatically, semi-automatically, and manually [23].Manual data collection can be accomplished through a survey-collection through a survey on several units related to the work in charge of the application.The data collection was done in two phases.The data collection phase was conducted from July 2021 to November 2021.In the first phase, we collected technical information on applications to build an application catalog.The information required includes the name of the application, the version used, the function of the application, the user, the operating system, the owner of the application, the security system, and others.
In the second phase, we collected user and developer data through an online questionnaire.The survey was conducted to determine the IT quality and business value, which were then used to map the application into the gartner TIME model.In this study, business value measurement was determined by the ability to solve a business need and provide operational efficiencies, a high utilization degree, and a positive user experience.In contrast, the support aspects chose IT quality measurement, data accuracy, source code availability and quality, reliability and security, response time and ease of change, and technology.Those aspects were adapted from FPT-intellinet [24], as shown in Table 1 and Table 2 Provides Operational Efficiencies Applications are easy to use and easy to learn [25] User Experience Probability of the application working correctly in a specific environment and time Data Accuracy Frequency (average) usage of the app in one week/month/year Utilization The level of user satisfaction after using the application [26] User Experience Interoperability Data can only be accessed by authorized users (Confidentiality), changes to data and application components are accessed by authorized users (integrity), guarantees that users cannot deny the operations they perform (non-repudiation), and all actions on the application can be traced (accountability), the ability to identify legitimate users (authenticity)

Security
The ability to maintain without changing or reducing the original application's quality.
Source code availability and quality The ability of applications to operate with other infrastructure without requiring high modifications.

Ease of Change
The cost that may incur from maintenance Cost Saving A questionnaire adapted from ISO 25010 was made to get the value of these aspects.ISO standards are often used to test application quality, such as in research [27].The survey used a measurement scale of 0 to 5, where zero indicates no assessment aspect of the application, and five indicates that the application shows a very high assessment aspect.
Our survey respondents came from different working units, such as the Bureau of Organizational Human Resources, the Legal Bureau for Cooperation and Public Relations, the Financial Planning Bureau, the Education and Training Center, the Inspectorate, and the General Bureau.All respondents were briefed on how to fill out the questionnaire and validated as employees working from designated units.The chosen experts for the interview were BPPT employees with at least 20 years of working experience (System managers?) in the IT field and had a deep knowledge of the 41 applications.These experts agreed to a recorded online interview.

Survey and expert result
There were 102 data collected for 25 applications.All data were manually validated.The developer survey collects information to get the value of IT quality.Aspects that are assessed include compatibility, security, maintenance, and portability.Previous research stated that interviewing with experts is one of the most effective ways to gather information about the application [28].Information was obtained by combining survey data collection and expert interviews to prevent bias [29].Our experts put weight on each IT quality and business value criterion, as shown in Table 3.Furthermore, each of our experts classifies 25 applications into four quadrants of McFarlan's strategic grid.The results will be discussed in section 4.4.

RESULT AND DISCUSSION
Among the 41 applications used in digital transformation activities at BPPT, 16 were not included in the analysis because the developer could not be contacted or the application was not used when data collection was carried out.Only 25 applications will be used for analysis.An application survey was carried out on 25 existing applications to obtain technical details.

Incomplete documentation
Good software documentation provides important information regarding the description of the software, what the program does, and how it should run [30].In the end, incomplete documents will complicate software development and maintenance in the future.In fact, developers often override this issue.Throughout the data collection phase, we experienced difficulties collecting complete documentation.We asked our application owner to submit at least six documents.These documents are business process, SRS, user acceptance test documentation, user manual, system documentation, and user role documentation.Only 12 out of 27 applications are complete with a business process document.While 10 out of 27 is assisted with SRS document, 13 out of 27 have the source code documented, and only 3 applications have database documentation.
This incomplete documentation showed that the software development process at BPPT put aside documentation as its slightest concern.Documentation for software development needs to be considered an essential factor.In his 11 -page research, [31] argued the importance of user documentation by elaborating on a misconception.His study showed that even the most well-designed software needs documentation.In alignment with this issue, we found many vendors that were not contactable for completing our survey, and most of the submitted documentation was incomplete.This raises the issue of software sustainability because the vendor is supposed to supply the application owner with complete documentation.

Application catalog
An application catalog was created using a simple spreadsheet to record the data.Catalogs are considered documentation suitable for recapitulating software products simply [32].The collected information was stored in an online storage application with limited access.The catalog consisted of 25 applications with detailed information.The application catalog can then be used by the person in charge of the application for application monitoring to find resources and simplify maintenance and repair.In addition, the purpose of creating an applications catalog is so that development will not depend on employee information, which may be lost but depends on documents.An example of an application catalog that has been created can be seen in Table 4.

McFarlan's strategic grid
The mapping of 41 applications into the McFarlan strategic grid was based on our experts' assessment conducted at the interview and portfolio of applications.The number of applications that fell into the support category was 21.It was the most significant number compared to other categories.While the key operation category has had seven applications, the strategic category has had 10 applications, and the high potential category has had three applications.The result of this mapping, as shown in Table 4 reflects that the application function in one of the largest research institutions in Indonesia has not yet played an important role.In this case, the applications in BPPT are seen as tools to support the operational implementation of research activities.
The results of this mapping could be considered for setting priorities for ICT investment in BPPT.Applications that fall into the "high potential" and "strategic" categoriesy should be a top priority for investment.By utilizing the results of this mapping, ICT investment in organizations is expected to be more optimal.Many institutions are experiencing increased ICT investments cost and are stuck with poorly managed applications.Application portfolio management is carried out to ensure that investments are made following the institution's ICT roadmap.

Gartner's TIME model mapping
Due to the limited information, only 16 applications can be analyzed and mapped into gartner's TIME model.Gartner's TIME Model mapping was done by looking at the results of the IT Quality and Business Value assessment results from the survey that has been done previously.As shown in Table 5, among 16 applications, only 4 applications fall into the eliminated category, while the rest are categorized as invest.From this, we can conclude that digital application in BPPT focuses on the longer-term application that can be developed further rather than a one-time-use application.These applications were also highly supporting BPPT's business value.
As shown in Table 6, four applications were within the eliminated quadrant: e-Public Relations, E-KI, E-Arsip, and Kerjasama Online.These four applications show a lack of IT quality and low to average support for BPPT's business value.To date, these applications have been merged into one big application four main functions, as inherited from their predecessors.The next research phase can do further analysis to elaborate more on the new application value of the two frameworks.Surprisingly, no applications fit the "tolerate" and "migrate" categories.The earlier category is filled with applications that have a high quality of ICT but offer little support for the institution's business value.Our results revealed that BPPT focused on developing an application that supports business value.

Combine McFarlan strategic grid and gartner's TIME model
Our analysis combined McFarlan strategic grid and gartner's TIME Model to propose a powerful recommendation.By far, this research is the first attempt to combine the two frameworks.We first create a combination of results from both frameworks.As gartner's TIME Model only resulted in two categories for an existing application, the recommendation resulted in 4 types.The recommendation can be seen in Figure 3.
Based on the gartners TIME mapping, 4 applications fell into the "Eliminate" category, and the rest fell into the "Invest" category.Of these 4 applications, only 1 was an application with a strategic function, while the others were support functions.The BMN Plus application is an application that is considered to have a strategic function, but after mapping gartner's TIME, this application was recommended to be eliminated, this was because this application had not been optimized enough by the user, so its function was considered insignificant to the organization's success; the expected strategic functions were not supported by good IT quality.
Another elimination recommendation falls on applications with a support function: e-KI, Kerjasama Online (Online Collaboration), and e-Arsip (e-archive).Aside from the fact that these applications do not have substantial business support for the organization, they are also built with poor IT quality.To confirm these results, we consulted experts.When this research was conducted, there was a process of significant changes to research institutions in Indonesia.The merging of research institutions in Indonesia into one organization called BRIN resulted in a difference in the entire management process.Therefore, even though the recommendations are considered appropriate, the results of the recommendations cannot be implemented.However, the output of the APM can still be used for decision-makers to make a more objective comparison of which applications can still be used on a larger scale.So, organizations can save costs more than building new applications even though they have functioned that old applications can fulfill.

CONCLUSION
The staggering fact that Indonesia's government registered as many as 400,000 digital applications raises our concern about how Indonesia's government institutions manage their application portfolio.Surprisingly, no research has been done to expose how government institutes manage their existing and future applications.As a case study, this study looked at how BPPT, a government research institute, manages its application portfolio.Our findings are important in different aspects.First, we found that most of the digital applications registered on BPPT fall into the operational/management support category.This is contradictory, as BPPT is a research institute, where research is the main business value of an institution.Does this pattern apply to most institutions in Indonesia, where applications are seen as a managementsupporting tool and not for core business support?Future research should use the different institutions as a research subject and compare the results with our findings.Second, we tested and confirmed the robustness of McFarlan's strategic grid and gartner's TIME model as a framework for the APM process.Our work will be among the first to apply the two frameworks in Indonesia's context.IT decision-makers can utilize our findings to create a strategy that fits BPPT's business value.However, in October 2021, through Presidential Decree No. 78, about the National Research and Innovation Agency, BPPT was officially merged into one new agency.Nevertheless, our findings can help align existing applications that BPPT has with the new agency's planned applications.APM can be at its best when all parties collaborate.The lack of application information will slow down the process, affecting the recommendation quality later.We strongly recommend conducting APM on the new institution by collaborating.

Figure 1 .
Figure 1.Four categories of McFarlan's strategic grid

ComputFigure 2 .
Figure 2. The result of combining McFarlan and gartner's model

Table 1 .
. Application IT quality aspects IT Quality Aspect The function of the application is to complete tasks following the organization's function Solves a Business Need The time and resources required to use the application

Table 2 .
Application business value aspects

Table 3 .
IT quality and business value aspects weight

Table 4 .
An example of an application catalog

Table 5 .
The result of Mcfarlan's strategic grid mapping

Table 6 .
The result of gartner's TIME model mapping