APPLAUSE：2023數字質量狀況報告（英文版）（43頁）.pdf

《APPLAUSE：2023數字質量狀況報告（英文版）（43頁）.pdf》由會員分享，可在線閱讀，更多相關《APPLAUSE：2023數字質量狀況報告（英文版）（43頁）.pdf（43頁珍藏版）》請在三個皮匠報告上搜索。

1、2023State of Digital QualityState of Digital Quality 20232Contents 03 Executive Summary04 Methodology05 Functional Testing11 Accessibility Testing17 Localization Testing22 Customer Journey Testing25 Payment Testing29 ConclusionState of Digital Quality 20233Executive summaryFor the second year,Applau

2、se analyzed a representative sample of our testing data to identify and analyze the most common flaws in digital experiences and map out how companies can stop them from reaching end users.We evaluated experiences across more than 16,000 individual mobile devices,1,000 unique desktops and 500 OS ver

3、sions to assess endless combinations of networks,browsers,payment instruments and integrations worldwide.We hold unique insight into the state of digital quality based on our work with companies in diverse industries and regions,from varying technological maturity levels,who are creating a wide rang

4、e of application types.This year we included additional data about the most frequently tested devices,configurations and payment instruments in different locales and industries,as well as the average number of configurations tested.We also shared observations on the core capabilities,practices and p

5、rocesses in place at organizations at different levels of commitment to digital quality.Some highlights from this years report:Digital quality is an intersectional discipline.Functionality,localization,accessibility,payments,customer experience and UX bleed into one another.Assess different componen

6、ts of an app or digital experience holistically,not just in isolation,to truly understand customer encounters with your brand.With Applause testers uncovering more than 200,000 defects last year,its clear that development organizations have a long way to go in the pursuit of excellence.Applauses 202

7、3 accessibility and inclusive design survey found that 43%of respondents rated accessibility as a top priority for their organization.Nearly three quarters 72%reported that their company has a dedicated accessibility expert on staff.Product designers and engineers are building accessibility into pro

8、ducts and code sooner and more often than in the past.As more personalized experiences become the norm,organizations must assess functionality and UX across more device combinations,configurations,workflows and dimensions.Test matrices and applications themselves are becoming more complex and harder

9、 to manage.Training and testing the AI and machine learning(ML)engines that support greater personalization requires specialized data and knowledge that many businesses dont have and struggle to find.Applauses AI and Voice survey found that 86%of respondents have concerns about bias in AI that compa

10、nies must address to maintain customers trust and loyalty.Though companies take many different approaches to digital quality and progress along different paths,weve observed some commonalities that correspond to different stages of commitment to and development of digital excellence.Rather than form

11、al maturity models,we offer some simple frameworks that organizations may use to benchmark the structure,efficiency,effectiveness and scalability of their digital quality efforts across varying dimensions.While cultures,processes and maturity levels are dynamic and theres not always clear demarcatio

12、n where one stage ends and another begins,an honest look at the status quo can spotlight which efforts will deliver the greatest impact to digital quality.State of Digital Quality 20234MethodologyWe analyzed results from a representative sample of closed test cycles executed between January 1,2022 a

13、nd December 31,2022.A test cycle is how Applause defines each unique set of tests:a client sends us testing parameters builds,scope,coverage,etc.and we create a test cycle that includes the specific test cases and scenarios to be tested.We analyzed data across industries,testing categories and regio

14、ns spanning more than 200,000 bugs,tens of thousands of devices,and thousands of device/OS/browser combinations.Testing included websites,apps,IoT devices,mobile web and mobile apps in real-world scenarios.Device coverageThe figures in this report reflect tests across the following scope worldwide:M

15、obile DevicesMobile makes167Mobile models2,296Mobile OSes21Mobile OS versions396Mobile web browsers36Mobile carriers573MediaSet-top/streaming devices181Gaming consoles35Smart TVs2,270TV providers65ISPs164DesktopDesktop web browsers35Desktop OSes10Desktop OS versions202Payment MethodsCredit/debit car

16、ds3,224E-Wallets48Mobile wallets87Alternate payments83HomeSmart home devices84State of Digital Quality 20235Functional testingAsk a developer,a QA engineer and user whether an app works and you may get three very different answers.Ultimately,however,individual users determine whether each app works

17、well enough for them to adopt it.Starting with the basics from making sure users can download and install an app,through validating more complex workflows and integrations,including APIs,IoT and Bluetooth connections organizations must ensure basic functionality before releasing an app to customers

18、which doesnt always begin to address UX concerns.While functional tests themselves are often not difficult,conducting the number of tests required in the necessary timeframe,across the spectrum of devices and OS configurations needed,poses a challenge for many QA and development teams.In addition,te

19、ams often prioritize testing over developing testing strategy or doing the routine work that makes testing more efficient and effective in the long run.Raise your hand if youve ever put off test suite maintenance,released while testing was in progress,pulled in help from other departments to complet

20、e testing,or released with risk because there was no time.The data setA representative sample of functional tests.38,920test cycles70+industries153countries173,725bugsState of Digital Quality 202364.8%3.9%17.8%49%24.5%Bug type definitionsBug TypeUser ExperienceContent“This doesnt read right”Typos,gr

21、ammatical issuesCrash“The app shut down”App closes or quits unexpectedlyFunctional(workflow errors)“This doesnt work right”Buttons dont respond when clicked,searches return incorrect resultsLags and latency“This is taking too long”Sluggish performance,freezesVisual“This doesnt look right”Misaligned

22、content or page elements,content doesnt fit an areaBug type breakdownSee the prevalence of each type of bug across the data we analyzed:17.8%Content4.8%Crash49%Functional3.9%Lags&latency24.5%VisualThe composition of bug types identified remains the same as last year.Functional,visual and content def

23、ects continue to comprise over 90%of all bugs found.While crashes are the most critical flaw,they continue to plague apps with no sign of year-over-year improvement.For functional testing device coverage comparisons and the most popular devices tested,please see Appendix A.State of Digital Quality 2

24、0237Functional testing framework:core capabilities and practicesIn our interactions with clients around the globe,Applause sees a wide range of approaches to digital quality.Our new frameworks detail the stages we commonly observe as companies progress to digital excellence,complete with examples of

25、 the types of organizational capabilities and activities that characterize each stage.While most companies and teams mix practices from different stages in the following table,a single stage typically defines the businesss prevailing approach to quality.Though it may seem like theres no clear demarc

26、ation between levels,those who are closest to the process will at some point recognize that the shift from one point to the next has occurred.The framework can serve as a valuable starting point for understanding your organizations progress along its digital quality journey and how to level up to re

27、ap the competitive advantages associated with improved quality and usability.If your organization is in the process of establishing or enhancing systems to ensure quality,use this report as a guide to develop plans that cover the most common devices,emerging forms of payment,and key considerations a

28、round accessibility and localization,as these are trends Applause sees organizations embrace as they improve quality and UX.How one fintech ensures innovation and excellenceAn Applause global fintech customer with a leading financial trading platform was challenged to maintain quality,innovation and

29、 personnel management.Hundreds of apps needed to function seamlessly within the high-stakes platform,yet the firm was struggling to release new features while maintaining massive regression testing suites.In addition,all testers had to be highly trained in financial trading.The fintech teamed with A

30、pplause to oversee comprehensive delivery and engagement management,recruiting a highly skilled team of over 50 testers from over 70 countries with financial services experience to work across 18 engineering teams.Applause owns the QA function at the fintech partner and provides best practices and g

31、uidance on test strategies.The fintech has prevented lost revenue by minimizing bugs escaping to production,has deeper insight into common functional issues and has a QA team at the ready that can scale as needed.Functional testing frameworkDigital quality emergence:Lack of formal systems,processes

32、and documentation while individuals may have their own methods and documentation,the organization has no consistent methodology or approach to quality.Examples of typical testing activities and processes:Conducting reactive testing after development Running tests without documenting test cases or te

33、st run results;or documenting poorly Running test cases inconsistently Skipping test case documentation or writing test cases in ways that make them difficult to follow or reproduce Practicing some exploratory testing and dogfooding Using disparate environments for development,testing and production

34、Digital quality essentials:Early stages of defining and documenting processes and procedures;establishing some consistency and structure around test efforts.Teams may have their own unique processes,efforts may still be siloed.Examples of typical testing activities and processes:Documenting test cas

35、es for feature-based tests Ensuring test cases are written clearly Defining a device coverage matrix Maintaining a known issues/bug backlog list Testing releases pre-production Conducting unit,smoke and regression testing for major app components or workflows Performing exploratory testing for new f

36、eatures/app changes Recording test run results Automating frequently executed/rarely changed testsDigital quality expansion:The organization has a clear process in place and uses a broad range of testing types.Some reporting is in place.Focus on coverage,scalability and efficiency across the organiz

37、ation.Examples of typical testing activities and processes:Maintaining a defined device coverage matrix based on data about website/app usage Conducting regression testing for all workflows Testing user acceptance and UX for new features/app changes Leveraging test automation for repetitive tests;re

38、viewing and updating automation scripts regularly Documenting test cases/suites for all features Measuring quality KPIs with data and reportingDigital quality excellence:Quality is embedded in the companys DNA and built into all products and experiences from end to end with testing and feedback thro

39、ughout the SDLC.Examples of typical testing activities and processes:Testing throughout the SDLC,including in-sprint testing in pre-production Incorporating the voice of the customer into product design Delivering exceptional UX and experiences across all touchpoints Maintaining a strong test case m

40、anagement process Automating all repetitive tests that humans cant do better Reviewing and refining testing processes regularly Proactively balancing manual functional,exploratory and automated testing;documenting when to use each test type Exploring new testing processes to boost quality,efficiency

41、 and coverage Driving innovation throughout the SDLC Using reports to analyze trends and identify areas for improvement8State of Digital Quality 2023State of Digital Quality 20239Functional testing recommendations Proactively examine how well your testing practices serve your organization,and most i

42、mportantly,your end users and clients.To truly improve,you need to understand your current state,which may be siloed or inconsistent across different teams,products,groups and even individual team members.Find whats working and replicate that as well as whats not working and needs to be addressed.If

43、 your organization has achieved excellence,think about what you need to do to maintain that position and how you can serve as an exemplar for other teams and businesses.Exceptional quality doesnt just require time for testing;it demands time for managing testing strategy as well as capturing and eva

44、luating data.Build in time for developers and testers to document processes,review with teams,and regularly update to ensure they still work and that everyones following those processes.Organizations that dont invest in the foundational elements of a quality program,including documenting processes,m

45、anaging test cases and recording test run results,will spend unnecessary time bringing new team members up to speed,trying to establish what tests were run and how,and hopelessly backtracking to find when and how a defect slipped through,with no preventative measures in place to keep the breakdown i

46、n systems from recurring.Recognize that achieving excellence doesnt mean the work is done.Ideally,as your QA practices improve,youll uncover problems earlier in the development cycle,when theyre easier and less expensive to fix,rather than when they reach production.If youre not finding defects duri

47、ng testing,thats great but that doesnt mean you should stop testing.It only takes one defect to reach production to damage the customer experience,and,depending on severity,impact revenue.Excellence is a constantly moving target as new technologies and innovations come to market.While you may be hig

48、hly automated and mature with some apps,to maintain competitive advantage,youll likely have to develop new features that your customers demand.Its crucial to maintain your investment in quality.The ability to quickly adapt and scale depends on maintaining the capabilities youve built.Guard them fier

49、cely to reduce risk and maintain your competitive advantage.State of Digital Quality 202310The role of test automation Its impossible to automate all aspects of QA.An effective test automation strategy,however,is a crucial component on the journey to DevOps and exceptional digital quality.Unfortunat

50、ely,too many organizations struggle to establish,maintain and expand test automation to take full advantage of this method of testing.When a team understands the purpose,objectives and limitations of automation,its possible to develop a strategy that scales easily and frees up human testers for task

51、s automation simply cant handle.Teams that dedicate the time and effort necessary to defining a strategy,selecting and implementing tools,and developing a library of automation scripts give themselves improved coverage,as well as relief from monotony.Frequently run tests with repeatable steps and an

52、 expected outcome offer the greatest return on the investment in automation.Tests that require complicated setup are also strong candidates for automation,as an error setting up a test can be costly in terms of tester time and can cause releases to slip.However,automated tests that create false posi

53、tives and break regularly cause problems for teams,rather than reducing burden.In addition,the approach your team takes to automation,such as scripted tests versus low code/no-code solutions,influences which tests are the best candidates for automation.Choose carefully before automating.Top 5 tests

54、to automateOur recommendations for the top 5 tests to automate fall into the categories below and are frequently repeated and easily scripted:Unit Tests1.Regression Tests2.Smoke Tests3.Integration Tests4.Performance Tests5.State of Digital Quality 202311Accessibility testingThe World Health Organiza

55、tions latest estimate is that one in six people,or 16%of the global population,lives with some form of disability.1 In addition,many people experience temporary disabilities due to injury,illness and aging at some point in their lives.Further estimates suggest that friends and family of persons with

56、 disabilities(PWD)account for an approximate 3.3 billion potential consumers who act based on their connection to PWD.2 Leading global enterprises across all industries have realized that digital experiences that people with disabilities can navigate easily often create a better UX for all of their

57、customers.In addition,some accessibility fixes,like addressing missing or unclear labels,enable automation engineers to build more automation and move faster.Its clear that a focus on accessibility helps everyone,and as organizations commit to more inclusive design and better digital accessibility,t

58、he bar gets higher for businesses everywhere,from all sectors.While innovation helps enable an expanded total addressable market,the fact remains that organizations must meet global accessibility compliance in accordance with national accessibility laws around the world,as not doing so creates legal

59、 exposure.In fact,in 2022,in the U.S.alone,there were 2,387 web accessibility lawsuits filed,which includes an increase of 143%in companies that received multiple lawsuits year over year.3The data setA representative sample of accessibility tests.1,273test cycles23industries17countries30,541bugs1“Di

60、sability and health.”World Health Organization,March 7,2023.2“2020 Global Economics of Disability.”Return on Disability Group,September 1,2020.3“2022 Website Accessibility Lawsuit Recap.”A,2023.State of Digital Quality 202312Bug type definitionsBug TypeUser ExperienceAnimation“I dont know whats happ

61、ening on the screen”A user cant see content that moves,such as animation telling users to perform an actionColor“This doesnt make sense”Users cant identify information communicated only through colorColor contrast“This all looks the same”Insufficient color contrast ratioHTML validator“This page seem

62、s like something is missing”Issues in HTML code that do not impact the keyboard navigation and screen reader behaviorKeyboard navigation“I cant use my keyboard to navigate”People using alternative keyboards or speech input devices as keyboard emulators cannot navigate a page Bug TypeUser ExperienceS

63、creen readers“My screen reader isnt working”Readable text for screen readers is missingOther A11y“This isnt working for me”Poor user experience for PWDVideo“I cant tell whats going on in this video”Missing closed captions or audio descriptionsZoom“I cant see what I need to”Text gets cut off when a u

64、ser zooms in beyond a certain pointState of Digital Quality 202313Bug type breakdownSee the prevalence of each type of bug across the data we analyzed:Screen readers64.8%Color contrast12.6%Keyboard navigation9.2%Other A11y6.9%Zoom2.8%Color1.6%HTML validator1.5%Video.38%Animation.16%Screen reader def

65、ects continue to make up the vast majority of all accessibility errors.Bug type distributions remained consistent,typically fluctuating less than 2%+/-from last year.For accessibility device coverage comparisons and the most popular devices tested,please see Appendix B.Creating a culture of inclusiv

66、ityA leading video conferencing solution provider made a significant commitment to make its video conferencing application collaborative,accessible and inclusive of all people with varying abilities and backgrounds.It embraced a shift left to bring inclusive design and innovation earlier in the SDLC

67、.Its focus went beyond simply meeting WCAG standards:It wanted to stimulate innovation in its organization that would derive from involving the most diverse population it could,emblematic of its users around the world.As it developed its empathy-based approach to inclusive design,it rallied team mem

68、bers to become ambassadors and help spread the word throughout the organization.The company got management buy-in,and as momentum continued,the company partnered with Applause to provide education sessions including PWD demonstrating their interactions with the conferencing platform to broaden aware

69、ness of the overall empathy-based inclusive design program.To keep up momentum,it built a repository,an accessiwiki,that serves as a resource employees can use to get a wide range of information around accessibility from WCAG standards explained to internal initiatives,inclusive design basics and mo

70、re.State of Digital Quality 202313State of Digital Quality 202314Accessibility framework:core capabilities and practicesAssumes that many/most of the elements from the corresponding stage of the functional testing framework are in place.While accessibility typically starts with conformance and compl

71、iance,once organizations embrace empathy-based design and development,the user experience improves dramatically.Our customers who have adapted to focus on inclusive design report that this shift creates a cycle of ongoing innovation that benefits all users,not just PWD.Accessibility testing framewor

72、kDigital quality emergence:Limited understanding of applicable laws and regulatory requirements.Lack of formal systems,processes and documentation,no consistent methodology or approach to accessibility or inclusivity.Examples of typical accessibility testing activities and processes:Identifying some

73、 accessibility issues;resolving or remediating high-priority A11y issues that are identified Assessing conformance to WCAG and/or locally applicable regulations once or infrequently Emphasizing risk mitigation;making changes in response to customer complaints or threats of legal action Relying solel

74、y on automated toolsDigital quality essentials:Early stages of defining and documenting processes and procedures;establishing some consistency and structure around accessibility.Examples of typical accessibility testing activities,resources and processes:Designating an owner of the accessibility pro

75、gram and identifying organizational champions Performing periodic assessments and automated checks to ensure conformance to WCAG and compliance with locally applicable regulations Offering training on accessibility best practices and inclusive design Providing developers and/or product owners access

76、 to PWD Conducting design reviews and in-sprint testing to identify problems earlier in the SDLC Engaging PWD to perform usability tests and provide feedbackDigital quality expansion:The organization has a clear process for ensuring accessibility in place and uses various testing types.Some reportin

77、g is in place.Focus on coverage,scalability and efficiency across the organization.Examples of typical accessibility testing activities,resources and processes:Documenting best practices and checking them during development Maintaining a knowledge base and offering training on best practices Holding

78、 empathy-based design and development workshops with PWD Incorporating input from PWD into the design and development process;benchmarking to understand the validity of customer criticisms around accessibility and usability Providing attestation(VPATS)Digital quality excellence:The organization prio

79、ritizes inclusivity at all levels;testing and feedback from PWD occur throughout the SDLC Examples of typical accessibility testing activities,inclusive design resources and processes:Creating a Center of Excellence with guidance on design and development best practices Requiring ongoing,mandatory t

80、raining on accessibility best practices and inclusive design Following inclusive hiring practices and developing a diverse workforce Going beyond attestation and demonstrating thought leadership in inclusive design and product development Incorporating accessibility into the companys contracting and

81、 procurement process Using analytics to drive priorities and continuous improvement15State of Digital Quality 2023State of Digital Quality 202316Accessibility testing recommendations Know which local and regional laws govern your business and make sure youre in compliance.It happens:We meet with cus

82、tomers who simply dont know that theyre violating the law.Even if youre located outside of Europe,for example,the European Accessibility Act will have strict penalties for companies doing business in the EU that fail to comply.In addition,certain industries may be subject to additional regulatory re

83、quirements,like the Twenty-First Century Communications and Video Accessibility Act for media and telecommunications providers.Find allies in the organization who are passionate about accessibility to serve as ambassadors and drive awareness;get an executive sponsor.Time and time again,customers des

84、cribe this scenario to their account teams at Applause:One person in the organization champions accessibility.They seek out like-minded colleagues and start sharing ideas.They start making some changes that pay off.They get an executive sponsor on board.Leadership starts understanding and embracing

85、the value of inclusive design.The grass-roots,bottom-up efforts combine with top-down support and then:Awareness.Training.Input from PWD into the design process.Empathy.Inclusion.A better UX for all.Prioritize screen readers.Websites must be accessibly designed in order for screen readers to provide

86、 the full experience of the web page to the user.This complex and widely-used assistive technology does much more than simply read a wide variety of text combinations.Screen readers can be used to spell and find specific words,announce cursor location,locate specifically colored text,indicate an act

87、ive choice on a website menu and much more.Theres a lot that can go wrong,but getting this one element right would solve nearly two-thirds of the accessibility issues our testers uncover.In addition,correctly building page titles,headings,anchor tags and site maps improve accessibility and,at the sa

88、me time,improve search engine optimization.“When people talk about quality code,its much more than just speed and availability and all of the things that developers would think.If your code is not accessible,how can someone describe it as high quality?”Heather Hepburn,Accessibility LeadState of Digi

89、tal Quality 202317Localization testingEntering new markets requires much more than simple language translations to get new customers on board,your apps need to reflect an understanding of the local language,values,preferences and processes.Seemingly small errors,like missing translations,incorrect c

90、ountry and region names or workflows that dont align with local customs can cause customers to abandon transactions and put the business at risk.Almost half of the worlds people identify one of 10 languages as their native tongue.but that number doesnt factor in regional differences and dialects.4Th

91、e data setA representative sample of tests evaluating localization.345test cycles11industries87countries8,120bugs4“The 10 Most Spoken Languages In The World.”James Lane,Babbel Magazine.February 9,2023.State of Digital Quality 202318Bug type definitionsBug TypeUser ExperienceCorrupted characters“This

92、 doesnt look right,where are the accent marks?”Currency and number format“That date seems wrong for the holiday theyre promoting”Missing translation“Im not sure what this says,theres no translation”Other/general“That photo is offensive,this brand doesnt understand me”Poor translation“I dont think th

93、ats the right word here”Truncation&overlap“The text on this button is cut off”Bug type breakdownSee the prevalence of each type of bug across the data we analyzed:4%Corrupted characters1.7%Currency&numbers36.4%Missing translation14.4%Other/general36.1%Poor translation7.5%Truncation&overlapFor locali

94、zation device coverage comparisons,please see Appendix C.4%1.7%36.4%14.4%36.1%7.5%State of Digital Quality 202319Localization framework:core capabilities and practicesAssumes that many/most of the elements from the corresponding stage of the functional testing framework are in place.Localization enc

95、ompasses far more than just translation.When done right,it builds credibility(and can improve search engine optimization).When done poorly,it can alienate potential customers,spark regional site blockages and boycotts and trigger other ugly consequences.Speaking your customers language is a necessit

96、y for global brands:In one study,nearly 90%of non-English speaking consumers stated they are more likely to purchase a product from brands with multilingual support systems and knowledge bases and 75%said they would become repeat customers if a brand offers multilingual customer care.590%of non-Engl

97、ish speaking consumers stated they are more likely to purchase a product from brands with multilingual support systems and knowledge bases.5“Survey of 8,709 Consumers in 29 Countries Finds that 76%Prefer Purchasing Products with Information in their Own Language.”CSA Research,July 7,2020.State of Di

98、gital Quality 2023Localization testing frameworkDigital quality emergence:Lack of formal systems,processes and documentation,no consistent methodology or approach to translation or localization.Examples of typical localization activities and processes:Translating some high-priority content without v

99、alidating that translations are contextually correctDigital quality essentials:Early stages of defining and documenting processes and procedures;establishing some consistency and structure around localization.Examples of typical localization activities and processes:Ensuring all content that should

100、be localized is accurately translated Verifying that all currencies,dates,symbols and measurements are converted to the appropriate units and formats Validating that forms work correctly Localizing content for the businesss top marketsDigital quality expansion:The organization has a clear process fo

101、r ensuring accurate localization and uses various testing types to validate.Some reporting is in place.Focus on coverage,scalability and efficiency.Examples of typical localization activities and processes:Using native speakers in market to validate translations and idioms Ensuring that visual eleme

102、nts fit translations Validating that imagery and colors are culturally appropriate Aligning workflows with expected processes Putting a process in place to resolve conflicts and update assets Validating strings pre-production Maintaining a glossary of preferred translations for key words and phrases

103、 Using a minimum of double-blind support to validate translations,not including the content creator Localizing applications for all markets where the business operatesDigital quality excellence:Awareness of cultural differences and commitment to respecting the norms in different markets serves as a

104、competitive differentiator.Examples of typical localization activities and processes:Leveraging cultural values and norms to create relevant offerings Differentiating between distinct dialects,such as Portuguese vs.Brazilian Portuguese Providing a channel for customers to report any issues Having a

105、process to manage issues Assessing how customer journeys vary across markets Factoring in accessibility concerns,such as whether screen readers work for right to left languages Demonstrating geopolitical awareness;correcting errors that can damage earnings or reputation Localizing in native language

106、s;factoring in regional variations to find solutions that works for all stakeholders20State of Digital Quality 202321Localization testing recommendations Maintain a translation glossary for frequently used terms and company-specific language.Save time with a resource that developers can refer to as

107、they localize apps;update and expand as you enter new markets or localize additional features,products and websites.Be consistent.For example,if the chatbot on your website offers service options in French,make sure the language youre using to describe products and processes there aligns with the te

108、rminology in the French version of your app and document that preferred phrasing in your glossary.Double-check accuracy and nuance with double-blind validation.Translators and testers each bring their own background and linguistic preferences to the tasks at hand.Those can influence how they interpr

109、et various translations.If theres disagreement on a particular meaning or translation,its best to understand and resolve it before releasing your website or app to customers.Include customer support and post-purchase experiences in your localization efforts.Dont let your quickstart guide or installa

110、tion instructions become a foreign language fail.Commit to supporting your customers in their local languages throughout the full experience with your brand.Navigating local preferencesOne GPS firm was having difficulty providing a consistent in-car experience.To find out how its product worked in a

111、 car in Paris,two employees from the companys German office connected with an Applause tester,a Paris native who knew the language and the city very well.The Parisian drove the team around for several hours to test the navigation,carefully listening to the voice giving the directions.The tester foun

112、d significant issues in translations for the most common routes to local points of interest.The GPS company was able to correct the problems before launching in Paris.State of Digital Quality 202321State of Digital Quality 202322Customer journey testingOur customers use many dimensions to assess dig

113、ital quality,including a range of metrics directly from customers.Customer sentiment and feedback,customer satisfaction research,retention and churn rates,the number of customer support requests its impossible to evaluate the customer experience without input from real or potential customers.You nee

114、d to understand and evaluate all the interactions customers have with your organization:myriad complex sets of transactions and tasks that often meander across varying channels,both online and in store.Not all organizations have the specialized skills and resources necessary to conduct thorough UX r

115、esearch covering all customer journeys,both physical and digital,from end to end.And as they expand into new markets,even those organizations that do have dedicated UX research teams may have difficulty quickly sourcing research participants.While solid functional testing and payment testing efforts

116、 can uncover a wide range of problems,they dont get to the heart of UX issues or problems with operational readiness,such as lack of instruction on how to proceed with curbside pickup once an order is ready.Customer journey testing is essential to root out and rectify these types of problems.Types o

117、f customer journey frictionCustomer journey friction typically falls into one of four categories(or combines some of these elements):Functional errors1.Payment issues2.Poor UX3.Insufficient operational readiness4.State of Digital Quality 202323Using AI,voice and chatbots to enhance the customer expe

118、rience As AI and voice take on ever-expanding roles in personalization and customer support,the quest to hire staff with more specialized and in-demand skill sets becomes even more intense.Applauses 2023 AI and Voice survey found that of more than 5,000 respondents worldwide,92%typically expect comp

119、anies to have chatbots on their apps or websites so they do not need to call for support.If customers do have to call,86%always or sometimes expect to be greeted by an IVR system when they call.In addition,consumers are concerned about bias in AI due to poor or insufficient training data:86%of surve

120、y respondents indicated some level of concern.While only 19%stated they were very concerned about bias,41%indicated that their level of concern depended on the situation and use case.Respondents expressed similar sentiment around whether AI technology should be regulated,with the majority(53%)saying

121、 regulation should depend on its use.With customer concerns and expectations in mind,organizations must have a strategy in place to incorporate chatbots,IVR and AI into customer touchpoints in ways that streamline and enhance the overall experience.Thorough testing and adequate ML training are cruci

122、al parts of that equation.State of Digital Quality 202324Customer journey testing recommendationsIf you dont have the bandwidth or resources to test end-to-end journeys,use data to pinpoint where youre losing customers,then focus there.If users are abandoning a loan application on your website,you k

123、now to look at the application form,flow and information youre requesting.If customers are downloading your app for a free trial but they arent converting to paid subscribers,youve got a different problem than if theyre subscribing,then canceling.Look at the data you have available to determine whic

124、h elements of the customer journey demand attention,then test to understand how you can improve and optimize.If you lack data,you need to focus effort on deliberate journey mapping to understand the various paths your customers take so you can then research and refine their journeys.Recognize that U

125、X and customer journey testing require ongoing effort.These arent one-time assessments theyre ongoing evaluations to understand customer perceptions,inform the product development process and guide market strategy as consumer preferences shift and evolve.Revisit as you add new features,make substant

126、ial changes to workflows and enter new markets.Find the cadence that makes the most sense for each of your digital products,properties and omnichannel experiences.Whether your customer journeys are strictly physical,entirely digital or a hybrid of both,you need real-world insight into your customers

127、 experiences and obstacles.State of Digital Quality 202325Payment testing Without thorough payment testing processes in place,organizations cant confidently ensure customer transactions work as intended.As digital wallets,alternative payment methods,and options like buy now,pay later(BNPL)become mor

128、e popular,payment testing must adapt and evolve to cover these new payment instruments and paths to purchase.Testing with dummy cards just wont cut it real-world testing with live payment instruments is the only way to guarantee transactions work properly for every customer,every time.Popularity of

129、different payment methods in 2022BNPL users:360 million6Digital wallet users:3.4 billion76“Buy now pay later users to reach over 900 million globally by 2027,driven by low cost of credit.”Juniper Research,August 8,2022.7“Digital wallet users to exceed 5.2 billion globally by 2026,as digitisation acc

130、elerates cashless transition.”Juniper Research,August 2,2022.The data setA representative sample of payment testing data for digital payment methods.1,582test cycles103countries5,847bugsState of Digital Quality 202326Bug type breakdownSee the prevalence of each type of bug across the data we analyze

131、d:8.4%Content1.6%Crash82.7%Functional2%Lags&latency5.3%VisualThe prevalence of each type of payment bug remained remarkably consistent with last year,varying less than 1%from year to year.With a whopping 80%of all payment bugs classified as functional bugs or workflow errors,without corrective actio

132、n,transactions will fail,delivering a serious hit to revenue.For payment device coverage comparisons,please see Appendix D.Payment testing framework:core capabilities and practicesAssumes that many/most of the elements from the corresponding stage of the functional testing framework are in place.Eve

133、ry payment transaction incorporates a series of handoffs.Connections between hardware,software,payment processors,banks,card networks and digital wallets all hold the potential for things to go wrong.And then there are the transactions that never happen because customers cant use their preferred pay

134、ment methods,loyalty points,coupons,or BNPL.Without a comprehensive and consistent approach to payment testing in place,businesses leave money on the table.5.3%2%1.6%8.4%82.7%State of Digital Quality 2023Payment testing frameworkDigital quality emergence:Lack of formal systems,processes and document

135、ation,no consistent methodology or approach to payment testing.Examples of typical payment testing activities and processes:Testing with dummy cards and accounts Dogfooding;testing with friends and familyDigital quality essentials:Early stages of defining and documenting processes and procedures;est

136、ablishing some consistency and structure around payment testing.Examples of typical payment testing activities and processes:Validating that transactions work using live payment instruments:testing both purchases and returns Defining a payment instrument coverage matrix,including at least the 5-10 m

137、ost popular payment instruments in market Testing when changing payment gateways;validating code for new gateways or payment processors Verifying code for launches in new countriesDigital quality expansion:The organization has a clear process in place to ensure the full range of payment activities w

138、ork properly and uses various testing types to assess functionality and UX.Some reporting is in place.Focus on coverage,scalability and efficiency across the organization.Examples of typical payment testing activities and processes:Creating a payment instrument coverage matrix based on data about we

139、bsite/app usage,local user preferences Validating new payment methods prior to launch Collecting and acting on UX feedback Conducting payment instrument regression testing for all features,including product returns and customer service interactions Testing payments using specific devices,banks and B

140、IN combinationsDigital quality excellence:The organization thoroughly tests end-to-end payment workflows and UX,including a wide variety of payment instruments,and understands the variations in preferred payment instruments and workflows across markets.Examples of typical payment testing activities

141、and processes:Conducting ongoing testing for loyalty programs,promotional offers,and options like BNPL Testing negative/unavailable payment instruments(soft or hard decline,blocked,expired)Performing wallet cycling testing with multiple payments in different balance states Live testing one-of-a-kind

142、 payments which require holder presenceFor card issuers,networks&fintechs:Verifying purchase states live directly with the bankFor merchants:Verifying purchase results live directly with the bank27State of Digital Quality 202328Payment testing recommendationsTest with real payment instruments to cap

143、ture intelligence you simply cant get from dummy cards.Without real payment instruments,you cant validate information in more complex flows like multifactor authentication,how a charge appears on a credit card statement or whether certain payment methods or BINs arent working properly.Use data to dr

144、ive decisions about which payment methods and instruments to offer or accept in different markets.As new digital wallets,cryptocurrencies,BNPL partners and alternate payment providers emerge,deepen your understanding of your customers preferred payment methods in each market where you operate to det

145、ermine whether you should add them into the mix.Build in the appropriate amount of time to test certain types of transactions.BNPL,recurring transfers,subscription renewals,loyalty programs and scheduled payouts are just a few categories of transactions that can take more time to test.For example,if

146、 a BNPL transaction requires multiple payments over a month-long period,you need to ensure all the payments process correctly.What happens if an auto transfer is scheduled for the 31st day of each month and a month only has 30 days?Or the credit card on file expires before an automatic renewal payme

147、nt goes through?Give your teams the time needed to thoroughly evaluate these scenarios.State of Digital Quality 202329Whats next in digital quality?Use this report as a benchmark to better understand the gaps in your organizations capabilities and processes and map out a plan to improve your digital

148、 quality,regardless of your companys current state.With the current market volatility,importance of global reach and need to maximize ROI on digital investments,theres no getting around the facts:digital quality is an intersectional discipline that calls not just for testing,but for clear testing st

149、rategy,documentation,test suite maintenance,research and analysis.With the right lens and focus on digital,youll see the incremental investment in the form of ROI as you look at customer retention,increased revenue,reduced time to market and other KPIs.State of Digital Quality 202330How do your comp

150、anys digital experiences compare?What role does quality play throughout the SDLC at your organization?Please share your own stories of creating digital quality for your customers,and tell us what youd like to see covered in next years report.We look forward to hearing from you.View our companion rep

151、orts at StateOfDigitalQ.Contact us at .AcknowledgementsThank you to the business leaders,community members and subject matter experts who shared their time,knowledge and insight to shape this report.Report Authors:Jenn Waltner,David Carty,Paul Hoffman,Annabel van DaalenCreative Design:Karley Searles

152、,Joe Stella,Megan GawlikState of Digital Quality 202331Appendix A:Functional testing data Average device coverage comparison across industriesIn this table,a configuration refers to a unique browser and OS combination(desktop)or device and OS combination(mobile&tablets)IndustryTest cyclesAverage num

153、ber of desktop configurations tested per cycleMobile&tablet configurations tested per cycleCredit&debit cards tested per cycleE-wallets tested per cycleMobile wallets tested per cycleALL37,1179.214.84.81.82.6Retail23.10%9.313.84.42.54.5Media&Telco37%8.816.17.81.61.9Finance16.40%8.712.431.32.2Health&

154、Wellness1.80%12.316.38.8N/AN/ATravel&Hospitality9.70%9.614.33.11.51.9B2B Software11.90%11.914.411N/AState of Digital Quality 202332Average device coverage comparison across regions Some test cycles span multiple regions.The data below includes only tests conducted in the stated region.RegionTest cyc

155、lesAverage number of desktop configurations testedMobile&tablet configurationsCredit&debit cardsE-walletsMobile walletsAfrica5,4043.25.34.31.21.5Asia18,3034.77.85.51.41.7Europe23,8746.19.94.51.31.9Oceania4,9822.22.82.61.11.2North America22,4225.38.14.31.41.8South America&LATAM1,5776.111.37.71.32.4St

156、ate of Digital Quality 202333Most popular device configurations testedWhile these were the configurations Applause tested most in each region,its vital to review and prioritize your customer data and preferences to develop your test plan per build or release.Also,consider OS adoption rates in your p

157、lan for example,iOS users upgrade OSes far more frequently than Android users.Determine what portion of testing resources you want to allocate to current versus older OS versions.RegionDesktopsMobile&tabletAfrica1.Windows 10 64-bit,Chrome2.Windows 10,Chrome3.Windows 11,Chrome4.Windows 10 Professiona

158、l 64 bit,Chrome5.Windows 10 64-bit,Firefox1.Huawei Mate 20 Lite,Android 102.Samsung Galaxy J7,Android 9.0(Pie)3.Tecno Spark 4,Android 104.Xiaomi Redmi 7A,Android 105.Samsung Galaxy S22 5G,Android 12Asia1.Windows 10,Chrome2.Windows 10 64-bit,Chrome3.Windows 11,Chrome4.Windows 10,Firefox5.Windows 11 H

159、ome,Chrome1.Samsung Galaxy S10,Android 122.Samsung Galaxy S9,Android 103.Xiaomi Redmi Note 7,Android 104.OnePlus 6T,Android 115.Apple iPhone 8,iOS 15.5Europe1.Windows 10,Chrome2.Windows 10 64-bit,Chrome3.Windows 10,Firefox4.Windows 11,Chrome5.Windows 10 64-bit,Firefox1.Samsung Galaxy S9,Android 102.

160、Samsung Galaxy S10 Plus,Android 123.Samsung Galaxy S21 5G,Android 124.Samsung Galaxy S8,Android 9.0(Pie)5.Apple iPhone 11,iOS 15.5State of Digital Quality 202334Most popular device configurations tested(continued)RegionDesktopsMobile&tabletOceania1.Windows 10,Chrome2.Windows 10 64-bit,Chrome3.Window

161、s 10 Home,Chrome4.Windows 10 64-bit,Firefox5.Windows 11,Chrome1.Samsung Galaxy A20,Android 112.Huawei Mate 20 Pro,Android 103.Samsung Galaxy Note 9,Android 104.Apple iPhone 8,iOS 15.25.Apple iPhone 11,iOS 15.0.2North America1.Windows 10 64-bit,Chrome2.Windows 10,Chrome3.Windows 11,Chrome4.Windows 11

162、 Home,Chrome5.Windows 10,Firefox1.Samsung Galaxy S9,Android 102.Samsung Galaxy S9+,Android 103.Samsung Galaxy S10+,Android 104.Samsung Galaxy S21 Ultra 5G,Android 12 4.Samsung Galaxy S20 FE,Android 12South America&LATAM1.Windows 10 64-bit,Chrome2.Windows 10,Chrome3.Windows 11,Chrome4.Windows 10,Fire

163、fox5.Windows 10 Professional 64 bit,Chrome1.Samsung Galaxy S9+,Android 102.Samsung Galaxy S20 FE,Android 123.Samsung,Galaxy S9,Android 104.Xiaomi Poco X3 NFC,Android 115.Google Pixel 3 XL,Android 12State of Digital Quality 202335Appendix B:Accessibility testing dataAs companies work to comply with l

164、ocal regulations and conform to industry-and region-specific requirements,in-market testing on the most common devices,browsers and networks is a crucial component in delivering fully accessible experiences for all users.Average device coverage comparison across industriesIn this table,a configurati

165、on refers to a unique combination of browser,OS and screen reader.IndustryTest cyclesAverage number of desktop configurations testedAverage number of mobile&tablet configurations testedALL1,2733.43.1Retail39.40%3.62.4Media&Telco12%2.53.4Finance12.90%3.94.1Health&Wellness6.50%3.75.9Travel&Hospitality

166、15.60%2.72.9B2B Software13.60%2.83.1State of Digital Quality 202336Average device coverage comparison across regionsIn this table,a configuration refers to a unique browser and OS combination or device and OS combination.RegionTest cyclesAverage number of desktop configurations testedAverage number

167、of mobile&tablet configurations testedAsia5862.32.7Europe60921.7Oceania1021.51.4North America5382.42.2South America&LATAM442.92.3State of Digital Quality 202337Most popular device configurations testedRegionDesktopsMobile&tabletAsia1.Windows 10,Chrome2.Windows 10,Firefox3.Windows 10 64-bit,Chrome4.M

168、ac OS,Big Sur 11.6,Safari5.Mac OS,Big Sur 11.5.2,Safari1.OnePlus 7,Android 112.Apple iPhone SE(2020),iOS 15.3.13.Apple iPhone 11,iOS 15.44.Motorola One Fusion Plus,Android 114.Apple iPhone 11,iOS 15.4,Safari4.Apple iPhone 13 Pro Max,iOS 15.0.2Europe1.Windows 10,Chrome2.Windows 10,Firefox3.Windows 10

169、 64-bit,Chrome4.Windows 7,Firefox5.Windows 11,Chrome1.Apple iPhone SE(2020),iOS 15.12.Google Pixel 2,Android 9.0(Pie)3.Apple iPhone SE(2020),iOS 14.5.14.Apple,iPhone X,iOS 15.0.25.Samsung Galaxy Note 9,Android 10State of Digital Quality 202338Most popular device configurations tested(continued)Regio

170、nDesktopsMobile&tabletNorth America1.Windows 10,Chrome2.Windows 10,Firefox 3.Windows 10 64-bit,Chrome4.Windows 11,Chrome5.Windows 11 Home,Chrome1.Samsung Galaxy S10,Android 112.Apple iPhone SE(2020),iOS 15.0.23.Samsung Galaxy A32,Android 114.Apple iPhone 8 Plus,iOS 145.Apple iPhone XR,iOS 14.8Oceani

171、a1.Windows 10 64-bit,Chrome2.Windows 10 64-bit,Firefox3.Windows 10,Firefox4.macOS Catalina 10.15.4,Safari5.macOS Monterey 12.2.1,Safari1.Xiaomi Mi A1(5X),Android 9.0(Pie)2.Xiaomi,Mi A1(5X),Android 9.0(Pie),Chrome3.Apple iPhone 6S,iOS 15.4.14.Oppo Find X3 Lite5.Apple iPhone 6S,iOS 15.4.1,SafariSouth

172、America&LATAM1.Windows 10 64-bit,Firefox2.Windows 10,Chrome3.Windows 10 64-bit,Internet Explorer 114.Windows 10 64-bit,Chrome4.Windows 10,Firefox1.Huawei Y5 lite,Android 8.12.Nokia 6,Android 9.0(Pie)3.Apple iPhone 11 Pro,iOS 14.4.2,Safari4.Apple iPhone SE(2016),iOS 13.1.2State of Digital Quality 202

173、339Appendix C:Localization testing dataAverage device coverage comparison across industriesIn this table,a configuration refers to a unique browser and OS combination or device and OS combination.IndustryTest cyclesAverage number of desktop configurations testedAverage number of mobile&tablet config

174、urations testedAll2731127.2Retail43.80%13.617.2Media&Telco25.20%11.414.3Finance4.80%4.88Betting&Gaming23.30%14.980.5Travel&Hospitality2.90%014.1State of Digital Quality 202340Average device coverage comparison across regions Some test cycles span multiple regions.The data below includes only tests c

175、onducted in the stated region.RegionTest cyclesAverage number of desktop configurations testedAverage number of mobile&tablet configurations testedAfrica5.20%1.810.7Asia25.80%5.210Europe41.40%9.618.4North America14.40%7.611.1Oceania5.80%1.72.7South America&LATAM7.40%5.115.9State of Digital Quality 2

176、02341Appendix D:Payment testing dataAverage number of payment methods tested across regionsRegionTest cyclesAverage number of alternate payments testedAverage number of mobile wallets testedAverage number of e-wallets testedAfrica4.10%1.31.61.2Asia21.60%2.31.81.5Europe36.50%1.71.41.7North America21.

177、00%1.51.81.4Oceania5.10%11.21.1South America&LATAM11.70%1.52.11.4State of Digital Quality 202342Most popular payment methods testedRegionAlternate paymentsMobile walletsE-walletsAfrica1.Payoneer2.M-Pesa3.Orange Money 4.MTN1.M-Pesa2.Orange Money 2.MTN2.Payoneer3.PayPal1.M-Pesa2.Orange Money 2.MTN2.Pa

178、yonee3.PayPalAsia1.Netbanking2.Suica3.Internet Bank Payment(IBP)4.Payoneer1.PayPal 2.Google Pay3.Apple Pay4.Paytm1.PayPal2.Alipay3.Osaifu-Keitai4.Visa Checkout 4.Amazon PayEurope1.Klarna2.Revolut2.Curve3.Sofort4.Internet Bank Payment(IBP)1.PayPal 2.Google Pay3.Apple Pay4.Revolut5.Payoneer1.PayPal 2.

179、Skrill3.Neteller3.MasterPass5.Visa CheckoutRegionAlternate paymentsMobile walletsE-walletsNorth America1.Klarna2.Netbanking3.Interac4.Payoneer1.PayPal 2.Apple Pay3.Google Pay4.Apple Wallet1.PayPal 2.Visa Checkout3.MasterPass/SRC4.Amazon Pay 4.Google Pay SendSouth America&LATAM1.Payoneer2.OXXO Pay3.Boleto4.Internet Bank Payment(IBP)1.PayPal 2.Visa Checkout3.Skrill4.Google Pay Send1.PayPal 2.Google Pay3.Any4.Apple PayState of Digital Quality The State of Digital Quality 2023