The impact of usability in information technology projects

ABSTRACT


INTRODUCTION
Extensive research has been conducted on information technology (IT) projects' success and failure factors, revealing the inherent risk involved in such endeavors [1]- [3].It is well established that achieving success in IT projects is a complex and challenging process that requires careful planning, execution, and management.Despite the significant efforts invested in IT project management, the failure rates still need to be lowered.Therefore, understanding the critical factors that impact project success, such as usability, is essential for achieving favorable outcomes.
However, it is important to note that successful information system and technology (IS/IT) projects are not a rare phenomenon [4], [5].While failures in IS/IT projects have been widely studied, critical success factors that contribute to favorable outcomes have also been extensively researched.It is imperative to identify and prioritize these success factors to mitigate risks and ensure the success of IS/IT projects.
Within the triad of cost, time, and quality, quantifying quality is often the most challenging, as it encompasses a range of factors that vary across projects.While cost and time can be more easily measured, the measurement of quality is multifaceted and can include aspects such as maintainability and usability.The present study will focus on usability as a critical quality factor in the success of IS/IT projects.Despite being an integral aspect of computing since its inception, the contribution of usability to project success has yet to

Usability
Usability is defined in ISO/IEC 25010 as "the degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use" [10], [11].While another definition describes usability as being "easy to learn, useful (provide necessary functionality), easy to use and pleasant to use" [12]- [14].Despite the various definitions, usability is widely acknowledged as being context-dependent.Another scholar emphasizes that usability needs to be referenced to a particular context [15].
Usability is a component of user experience (UX), which is part of practical acceptability according to Nielsen's model [16].The components of usability are shown in Figure 1.The figure also shows the position of usability related to the system and practical acceptability.
Measuring usability presents challenges due to its multidimensional nature, influenced by various factors such as the user's task, the product itself, and the specific environment [17], [18].To address this, researchers often adopt an approach that involves engaging participants to perform tasks using the system, and then assessing key factors like the time taken to complete tasks, the rate of successful scenario completions, and the time required for error recovery [19].This task-oriented evaluation allows for a more comprehensive understanding of usability and its impact on the overall user experience, making it a valuable method in usability research.
In the context of project management, usability is considered a non-functional requirement (NFR) and part of the quality factor [20], which is one of the three common criteria used to measure project success [21], [22].Previous research has demonstrated that usability can improve satisfaction, productivity, and technology acceptance, ultimately contributing to the success of IS/IT projects [8].Therefore, understanding and prioritizing usability as a critical quality factor is essential for achieving favorable outcomes in IT projects.
To strengthen the argument for how usability success contributes to the overall success of an IS/IT project, prior research has demonstrated that improved usability leads to enhanced user satisfaction, increased productivity, and better technology acceptance [9].Furthermore, assessing usability poses unique challenges Comput Sci Inf Technol ISSN: 2722-3221  The impact of usability in information technology projects (Freddy Richard Putra Hulu) due to its multidimensional nature, which is influenced by various factors specific to the user's task, the product, and the environment in which it is used, making it more complex than traditional one-dimensional product or user characteristics [10], [11].This nuanced understanding of usability's impact on the user experience underscores its crucial role in the success of IS/IT projects and highlights the need for tailored approaches in evaluating and enhancing usability.
In conclusion, usability is a critical quality factor in IS/IT projects that can impact project success.Usability is challenging to measure due to its multidimensional nature and its dependence on context.However, assessing usability can be achieved by asking participants to perform tasks using the system and measuring factors such as task completion time, success rate, and error recovery time.Prioritizing usability as an NFR can improve satisfaction, productivity, and technology acceptance, ultimately contributing to the success of IS/IT projects.
Figure 1.Model of the system acceptability attributes by nielsen (recreated)

IS/IT project
A project is an endeavor undertaken to create a unique product, service, or other result [23].This endeavor has to be temporary.Hence, we could easily distinguish the project from operational activities.IS/IT Project is not defined far from the project definition as well.Bannerman presents a definition for "IT Project", "Discrete and unique activities that serve as vehicles of multidimensional IT-based change" [24]- [26].
The word "multidimensional" serves well in today's dynamics of business and other sectors of the world.Businesses are not only the ones who benefit from IS/IT development but every other organization, from the government to the private sector, from small organizations to large enterprises.Nowadays, IT Projects become a vehicle of transformation and growth for the business.The global IT Investment in 2022 was estimated by Gartner to grow and reaches 4,4 trillion dollars.It only grows from previous years amid the pandemic.
Nevertheless, there is complexity in every IS/IT Project that has been kicked off.This is the one that makes an IS/IT Project unique.Moreover, one project had a different complexity from another, and this not only makes it unique but also makes it prone to failure [3], [27].But as it was mentioned before, the failure and success of IT Projects have been widely researched in empirical studies and systematic reviews.This paper is not research on IT Projects and failure but is more focused on the usability in IS/IT Projects.
Although IT project success is important for the overall IT success [28], there are no formal criteria for project success.However there is a criterion that is stated very well by Iriarte and Bayona [25].The criteria are divided into two groups which are project management success and project product success.Each group has three criteria, and there are system quality criteria that fall into both groups.The list of project success criteria is shown in Figure 2.
A systematic literature review comprising 39 studies conducted by Iriarte and Bayona identified several critical success factors in IS/IT projects [25].These factors include the capability and skills of team members, user involvement, top management support, consultant knowledge and technical expertise, organizational maturity and culture, internal communication, project organization structure and responsibilities, effective planning, and the ability to handle changes and deviations.Lamprou identified additional critical success factors, including project finance/funding and economics, team competency and effectiveness, and project managers' capability and experience [5].
However, research has also shown that inadequate attention to these critical success factors can lead to project failure.Inadequate management support, lack of technical know-how among team members, and lack of project managers' capability and experience have been identified as factors contributing to project Recent research has highlighted the need for customized approaches to understanding project failure in IS/IT projects.Gupta has suggested the importance of researching project failure prediction under dynamic scenarios [2], while Money has proposed a forensic analysis framework to comprehensively understand the causes of project failure in IS/IT projects [30].These approaches are based on the principle that a comprehensive understanding of each project is necessary to understand why, where, when, how, and what causes project failure.Another research shows that IT project failure comes from the neglect of Non-Functional Requirements in the Development phase [31].The findings of these studies emphasize the importance of identifying and addressing critical success factors and potential failure points to improve project outcomes in IS/IT projects.
According to Figure 2, which outlines the criteria for project success, system quality is an essential component of both project management success and project product success.System quality is a measure of users' perceptions of adaptability, availability, reliability, response time, and usability [32], [33].Specific usability characteristics, such as appropriateness, recognizability, learnability, and operability, are also associated with improved system quality across four factors, including custom software quality, documentation, time and budget, and new package functionalities [33].
Given these findings, it is evident that usability is a crucial factor in achieving success in IS/IT projects.By improving system quality, usability can positively impact project management success and project product success.Therefore, it is crucial to prioritize usability as an essential component of IS/IT projects to improve the overall success of the project.

METHOD
This paper employs the preferred reporting items for systematic reviews and meta-analyses (PRISMA) methodology to conduct a literature review.PRISMA is a widely-used methodology in the academic world for conducting systematic reviews or meta-analyses on a topic.The PRISMA methodology comprises four key steps: identification, screening, eligibility, and inclusion [34].Table 1 outlines the inclusion and exclusion criteria employed in this study.The straightforward description of the method depicted in Figure 3. Using this methodology, this paper aims to provide a comprehensive review of the existing literature on usability and its impact on IS/IT project success.

Table 1. Inclusion and exclusion criteria Inclusion Criteria
Exclusion Criteria 1.Either title, abstract, or keywords related to this paper's topic 1. Paper in a language other than English 2. Articles about IS/IT Projects and Usability
In searching for the related studies, these main keywords were used, "Information System Projects" AND "usability".Next, the term "Information System Projects" expanded into similar words and terms such as Information Technology as well as the abbreviation.Thus, the produced keywords are as follows, "IT Project", "IT Projects", "Information Technology Project", "Information Technology Projects", "IS Project", "IS Projects", "Information System Project", "Information System Projects".On the other hand, the usability keyword is not modified or added with other related terms such as User Experience or User Interface.These two other keywords will skew the query results since it has different meaning and academic topics.

Screening
The screening process is an important step in the PRISMA methodology as it aims to remove duplicate results from various databases.In some cases, articles may be indexed in two or more databases, resulting in duplication.To identify and remove these duplicates, researchers must manually read and compare the title, abstract, and keywords of each article.This ensures that only unique articles are included in the study and eliminates any potential bias or error resulting from the inclusion of duplicate results.
During the screening process, any literature that cannot be retrieved in full text is excluded.This step is necessary as it allows researchers to determine the quality, details, and methodology of the articles.Only literature that can be retrieved in full text can be properly evaluated and, therefore, cannot be included in the study.By excluding these articles, researchers can ensure that the study is based on a thorough and reliable analysis of the existing literature.

Eligibility
The third step in the PRISMA methodology is the eligibility check, which involves reading the entire article that passed the screening step.During this step, researchers evaluate the article based on the inclusion criteria, which typically involves assessing whether the article can answer at least one of the research questions.If an article meets the inclusion criteria, it will be selected and included as a reference for the research.By conducting a thorough eligibility check, researchers can ensure that the selected articles are relevant and reliable and can provide valuable insights into the topic being studied.

Included
After passing the first three steps, the last step in PRISMA is to determine how the articles act as a source for this current systematic review.In the PRISMA model in Figure 3, there are two articles that are included from other sources, whilst both obtained from the same database source as the database searching.These two articles were selected from dl.acm.org and scopus.comsince it has the same literature relation and connection to this current research.The final results of the method finally included 10 literatures, which shown in the Table 2 from identification process up to the included.

RESULTS AND DISCUSSION
In this section, the selected studies will be analyzed to facilitate comparisons between articles.The analysis will begin by examining the project management methodology, including whether the traditional waterfall or Agile methodology was used.The next focus will be the product developed in the project, with a brief analysis of the project results (whether it failed or succeeded) if applicable.However, the most critical aspect of the analysis will be the positioning of usability in the project, including whether the project management plan explicitly mentions usability as a key factor.Furthermore, Table 3, which contains the titles of the articles to be reviewed, will provide a comprehensive overview of the literature sources considered in this analysis.
Given the crucial role of usability in IS/IT project success, it is essential to understand how it is positioned in each study.By examining how usability is incorporated into the project management plan, researchers can gain insights into the best practices for integrating usability into IS/IT projects.This analysis will contribute to a more comprehensive understanding of how usability impacts project success and provide valuable insights for future research in this area.To simplify the presentation of the selected articles, they will be summarized in Table 4 based on the main problem of project management related to usability.The table will also include information on the type of research conducted to show how the research was carried out.Most of the selected articles are based on empirical research, with two articles focusing on specific case studies and one article using a qualitative comparative analysis approach.By presenting the information in a clear and concise format, the results of the literature review can be easily understood and used to inform future research in this area.It is not easy to reveal the factors that impact software product usability and the success of an entire project.
The aim is to determine factors with a positive or negative impact on delivering usability in software products and the success or failure of an entire project.7 Case Study A small company does not have enough resources for HCI, whilst a software engineer is more concerned with the performance and capabilities of the developed software.This problem increases if the software development method is Agile.8 Empirical Research There are five failed projects that have been investigated, and one of the problems is usability which needs to be addressed.Need to perform the usability test better, e.g., only asked about the user interface with a few users.
Did not check usability with real users because the assumption was that since the expert had designed it, it was good.9 Case Study The availability of patient demographic data and poor quality of clinical referrals.Healthcare quality improvement projects often use a narrow range of metrics that provide just one measure of systems.10 Case Study IS/IT Implementation projects come into a fail because of a lot of factors, and User Experience is one of them.
The quality system is the most influential factor in information system project success based on User Experience, while the quality of the system itself is influenced by the constructs of usability, affect, and user value.
Once the main problems identified in each selected article have been analyzed, Table 5 will be used to summarize the research findings and recommendations.This summary will serve as a basis for the analysis related to the research questions in this paper.The column numbering of The study of user participation in IT projects demonstrated its significance in ensuring product usability.
Collected data shows that product usability increased in projects where end users were involved.
The findings of an examination of the factors influencing (high) product usability.This analysis reveals that the following factors influenced the user participation variable: group size, project participation stage, and group size, with group size having the greatest impact on product usability.
Improving the success rate of an IS/IT Project (particularly in developing an Information system for an organization), it is important to involve real end-user.

2
Involving users in the classic waterfall method does not suitable for the Agile method.
Since Lean attempts to incorporate user testing into the scrum sprint or other Agile Timebox, it is important to involve users in carrying out regular testing of the deliverables or the increment of the sprint.3 The survey of projects also revealed that formalization of project documentation ensures high usability.
The author proposes a set of rules that could help the IT Project deliver better usability.
From the ten rules developed, rules number 1,2,3,4,7,9, and 10 contain direct guidelines to improve the quality of usability.For this paper, rules number 1 and number 2 will be summarized as "the better the formalized documentation, and the higher the involvement of the enduser, then the higher the usability quality." 4 There is a real problem regarding IT project usability, especially in developing countries.Lack of IT skill and experience, combined with the organization's failure to state the quality attribute such as usability on the procurement document, lead to failure in IT Project.
There is a need to develop formal standards and guidelines for the government to improve the IT Investment of the government itself.In this case, these are standards and guidelines for usability or other quality attributes for Information System Development.While the author involves in the project as a Human-Computer Interaction researcher, joined the project team, and made usability tests and recommendations, the usability quality of the software was improved.5 User involvement is important, but only some phases of the project need the end user.This paper concludes that the best phase for involving the user is the requirement phase and the testing phase.
During the user requirements analysis phase, project managers should ensure that they focus their attention on user participation.They should, in particular, ensure that actual end-users of the IS participate.This should help to improve the software's perceived usability.

6
There are several factors that contribute to the project's failure, and usability is one of the factors.Usability issues have happened in all types of project management methods, such as waterfall or Agile.
We should bring software developers and UI designers closer together to ensure that a reported and approved change is both possible to implement and does not degrade usability.
We believe that software architecture should include not only the technical aspects required to develop and implement a product (namely functional requirements) but also usability interaction with users as part of the non-functional requirements.7 To verify the impact of changes on usability and user experience, we conducted an interaction flow review.This analysis focused on user interface and interaction elements, including validation elements such as multiple features, UI style, overall usability, and user ratings.
At the end of the case study, the end-users said that there is a better dialogue and understanding between clients and developers (IT vendors).Moreover, the improvement is also measured in the design quality, usability, and user experience.In addition to that, the feeling of security and bravery in the development while there is an HCI expert joining the development team.

8
IT Project failure is preventable, whether in cost, time, and quality.One of the most reported key fail factors is the failure to predict the system integration and false estimation of the human factor (the developers and the end-users) The author develops a causes and cures model to map any kind of IT Project fail factor to researched-based cures.Organizations could apply the cures throughout the project to avoid or mitigate the risks.9 The improvement of the healthcare IT Project could be reached through usability improvement of the Information System used.
The involvement of the HCI Experts in the hospital-based improvement project has led to better product usability.The observed improvement is in the quality of clinical referrals and the availability of patient data, particularly demographic data.10 Inexperienced users in rural areas could be benefited from Information System in a better way if the usability of the system is good.
The authors developed a model that combines the User Experience model with Information System Project Success.
The most influential factor of IS/IT Projects based on User Experience is the quality system, and this quality is heavily influenced by Usability as well as the effect and User Values.Government or other policymakers in rural areas regarding Information technology should pay a better understanding and attention to the aspect of User Experience for every IS/IT Project kicked off.

RQ1: How the usability positioned in IT project?
The usability factor is already well-understood as a critical component of product quality in IS/IT project development.However, project managers often find themselves in a position where they must balance Comput Sci Inf Technol ISSN: 2722-3221  The impact of usability in information technology projects (Freddy Richard Putra Hulu) 15 the achievement of project goals in terms of time, cost, and quality.In other words, quality must compete with the other two factors, as achieving better quality often requires additional time and resources, including those needed to improve usability.Previous research suggests that some projects need to conduct usability tests on their products, despite recommendations to invest up to 10% of the budget in usability [44].Unfortunately, this study was unable to determine whether this recommendation was applied in the projects reviewed.Moreover, it remains to be seen why many projects do not include usability requirements or allocate sufficient time and resources for usability testing and user involvement.

RQ2: How to improve the quality of usability factor in the IT project success?
The involvement of end-users in the project is widely recognized as a key factor in achieving project success, as stated in the literature review.However, the involvement must be done in the correct manner, as involving end-user managers is not suitable.End-user managers are not the actual end-users who will regularly use the system.Moreover, the involvement of end-users must be timed appropriately and taken into consideration during the construction phase.
Furthermore, in addition to involving end-users, improving the development team's HCI knowledge or skill is also important.This can be achieved by providing additional resources to the team, such as hiring HCI experts or training software engineers in the basics of user experience.Such efforts will lead to the of better products that meet end-users needs and expectations.In addition to project management and organizational efforts, it is also necessary for governments to develop and establish formalized standards and guidelines.These standards and guidelines can help small organizations that lack the resources for usability testing to improve the quality of their products.Ultimately, this can lead to the delivery of better products to clients and end-users.

Discussion
This subsection aims to contextualize the present study within the available body of evidence by providing a discussion that compares and contrasts the findings with those of similar studies.In doing so, we seek to elucidate the contribution of this research to the field of usability in IS/IT project management.Furthermore, the limitations of this study will be presented, along with suggestions for future research that could help fill the gaps in the current body of knowledge regarding usability in IS/IT projects.

Interpretation of results in the context of other available evidence
The primary discovery of our study is that involving end-users and usability experts is crucial in enhancing the usability of the project's end product.Improved usability implies better system quality, ultimately increasing the likelihood of success in IS/IT projects.In addition to these findings, we have identified several other factors that should be considered by project members, such as formalized documentation, standards and guidelines, and communication and collaboration among designers, developers, and end-users.
Our findings are consistent with a previous study conducted by Vince Bruno in 2011, which investigated the importance of usability in IS/IT projects.One of the primary outcomes of Bruno's study was the significance of creating a "usability mindset" in key project stakeholders through a collaborative approach that highlights involvement in usability activities to attain a positive usability outcome [45].In the research, he suggested creating a shared usability vision for the project or promoting a usability mindset that extends beyond the project and across the organizational culture to other projects.The concept of a usability vision is connected with usability standards and guidelines, as they foster shared understanding and expectation of the desired usability outcome of a project.Usability goals promote a usability mindset, extending beyond the project context mindset [46].
Four major themes emerged from Bruno's research: usability mindset, collaborative approach, usability practice, and project constraints [45].The most significant relationships found in the study are the two robust connections between the themes of a collaborative approach and a usability mindset.The research emphasized the importance of developing a symbiotic relationship between involvement and a usability mindset to achieve a beneficial usability outcome.Another noteworthy finding is the consistency of our results concerning the roles of usability in overall project success.Bruno's research conducted in 2013 reveals that usability is an iterative process that facilitates stakeholder involvement, articulates usability goals and requirements, and avoids technological constraints.These critical aspects are instrumental in achieving a successful usability outcome in a project [47].
Another study by Bruno proposed role differentiation between Usability consultants or experts and usability practitioners in IS/IT projects.The research revealed that there should be usability managers who focus on strategic usability issues and improving stakeholder collaboration, usability consultants who concentrate more on complying with usability activities within a process, and organizational practitioners who focus more on nurturing and educating usability understanding within the organization and stakeholders involved in an IS/IT project.Furthermore, the study showed that an iterative usability process, stakeholder involvement, articulating usability goals and requirements, and avoiding technological constraints are critical to achieving a successful usability outcome in a project.
While our research and the previous study both recommend the participation of end-users, usability experts, and practitioners, there are differences in emphasis.The previous research emphasizes the involvement of usability practitioners rather than end-users.In contrast, our selected research shows that five studies suggest end-user' involvement and two studies suggest the involvement of usability experts.
In alignment with the broader objectives identified within the realm of IT project management, recent studies corroborate the significance of user engagement and satisfaction in determining project success [48].Husam AlFahl's research, particularly relevant in this context, identifies three pivotal constructsapplication design, application utility, and user interface structure -as the strongest predictors of user's continued intention to utilize a mobile application and their loyalty towards it.This finding underscores a parallel in our research, reinforcing the notion that the sustained engagement of end users is a crucial, albeit often underemphasized, objective in the lifecycle of IT projects.Emphasizing these aspects can significantly contribute to the holistic success and long-term viability of IT initiatives.

Limitation and future research
The research in this paper has certain limitations, such as a relatively small number of studies included.To address this, it may be necessary to expand the search to include other databases and sources of information.Additionally, the research may not have been conducted in a thorough or rigorous manner, and the authors acknowledge that there may be more information to be gained from the selected previous research studies.In future research, it would be beneficial to extend the search for academic literature beyond the sources used in this paper.

CONCLUSION
In conclusion, this paper makes a substantial contribution by highlighting the crucial role of quality enhancement through usability in the realm of IS/IT projects.The success of an IS/IT project should not be measured solely in terms of time, financial resources, and technical excellence.Instead, it requires in-depth consideration of user experience.Even if a project is executed within its allocated resources and schedule, its long-term viability can still be compromised if end-users are not provided with a satisfying experience.This research proposes three strategic approaches for improving usability: user engagement, the involvement of HCI experts, and the adoption of formalized usability standards and guidelines.Organizations have the option of implementing incremental improvements in usability or can seek the expertise of HCI professionals for a more standardized approach.It is essential to understand that IS/IT projects involve significant investments and carry inherent risks of failure.Therefore, comprehensive measures are imperative to prevent yet another project failure.


ISSN: 2722-3221 Comput Sci Inf Technol, Vol. 5, No. 1, March 2024: 7-18 10 failure.Other factors identified in the literature include poor supplier selection, data-related technical problems, and insufficient user involvement and commitment [29].The causes of project failure can vary depending on the project lifecycle or phase, geography, and type of measured failure.

Table 2 .
Comparison of number of researches by sources

Table 3 .
Included articles to be reviewed

Table 4 .
Articles main problem 13e impact of usability in information technology projects (Freddy Richard Putra Hulu)13

Table 5 .
Table 5 corresponds to Table 3, which lists the articles to be reviewed.By presenting the research findings and recommendations in a concise and organized manner, this table will facilitate the comparison and synthesis of the results, providing insights into the best practices for incorporating usability into IS/IT projects.Research recommendations