Defect Management

Definitions

Defect/bug

  • A human error or mistake in the code, causing an observed failure
  • Causes undesired effects on the application or system
  • Applicable to production or pre-production environments

Incident

  • Unplanned outages
  • Degradations or issues with application or system performance
  • Applicable to production environment only

Defect categorization

These guidelines outline how we prioritize defects during the QA cycle.

Priority: 1: Critical

Severe impact on the application, efficiency, and/or customer experience.

  • Defects that cause great deviation from business requirements, user stories and/or acceptance criteria.
  • Defects that obstruct continuance of the testing process, such as environment issues caused by hardware failures or back-end services
  • Defects that obstruct application functionality, such as non-functional button/links where we are seeing more than 10% fallout
  • Defects that are detrimental to revenue, such as incorrect pricing on products, inability to make bill payments, etc
  • Defects that are detrimental to reputation/brand, such as logo not displaying properly
  • Regulatory French language issues, such as incorrect spelling and grammar
  • Accessibility defects that completely block users to complete critical actions / flows (login, register, pay bills, etc.):
    • Keyboard traps , especially on critical areas (login, registration, forms, navigation menus, etc.) & revenue-related flows
    • Broken or no keyboard functionality, especially on critical areas (login, registration, forms, navigation menus, etc.) & revenue-related flows
    • Color contrast is below 4.5:1 for regular text and below 3:1 for large text (24px/1.5em)
    • Missing labels and/or text on form fields, buttons, links, images, etc.
    • Error messaging: labels for form fields don’t get updated when there’s an error
    • Error messaging: messages have low color contrast (red on white)

Priority: 2: High

Major impact on the application, efficiency, and/or customer experience

  • Defects that do not critically obstruct the continuance of the testing process, obstruct application functionality, or degrade system performance, but cause great deviation from business requirements and/or acceptance criteria. Some examples include:
    • Superscripts malfunctioning
    • Spelling mistakes/typos
    • Banner images not displaying
    • CSS not loading
    • Javascript not loading
    • Non-functional buttons (CTAs) / links, with 7-9% transaction fallout seen
    • Accessibility defects
      • Keyboard focus is reset / not managed when interacting with UI elements (and not contained within that UI); e.g. pressing ENTER to open a menu moves the focus to the top of the page; or dynamic page scrolls not moving the focus with them
      • Error messages don’t take keyboard focus when displayed
      • Keyboard navigation doesn’t work with NVDA & Firefox
      • Links or form controls don’t show a visible focus indicator when the user tabs to them

Priority: 3: Medium

Moderate impact on the application, efficiency, and/or customer experience

  • Defects that do not highly obstruct the continuance of the testing process, obstruct application functionality, or degrade system performance, but do result in marginal deviation from business requirements and/or acceptance criteria. Some examples include:
    • Inconsistency in global elements, such as drop down menus and radio
    • Expand/collapse page/module components not functioning
    • CSS not consistent between breakpoints (mobile, tablet, desktop)
    • Text cut off and breaking onto two lines
    • Light overlapping
    • Functionality exists but the description is missing
    • Non-functional buttons (CTAs) / links, with 4-6% transaction fallout being seen
    • Accessibility defects
      • Meaningful sequence - tabbing through a page should go from left to right, top-down; same idea when cursoring with a screen reader

Priority 4: Low

Minor impact on the application, efficiency, and/or customer experience.

  • Defects that do not moderately obstruct the continuance of the testing process, obstruct application functionality, or degrade system performance, but do result in minimal deviation from business requirements and/or acceptance criteria. Some examples include:
    • Non-functional buttons (CTAs) / links, with 1-3% transaction fallout being seen
    • Defects that are negligible and have inconsequential impact on the customer experience, such as overlapping images and incorrect punctuation