Skip to content

Perform Repository-Wide Documentation Accessibility Scoring #898

Closed

Description

@dessyordanova

Summary

Conduct a comprehensive accessibility assessment of the documentation repository by evaluating all published Markdown articles against a standardized documentation accessibility rubric. The goal is to establish an overall accessibility score for the documentation set, identify priority accessibility issues, and generate a detailed remediation report.

The assessment must analyze the Markdown source content only. It must not evaluate rendered site behavior, product functionality, UI components, code correctness, image pixel content, or claim WCAG conformance.

Objectives

  • Generate a single overall accessibility score for the documentation repository.
  • Score every eligible documentation article using a consistent evaluation framework.
  • Calculate average scores across all accessibility dimensions.
  • Identify all Critical and High severity accessibility issues.
  • Detect recurring Medium and Low severity patterns.
  • Produce a report that can serve as a baseline for future accessibility improvement efforts.

Repository Scope

Include

Recursively analyze all .md and .markdown files.

Exclude

  • .git
  • .github
  • node_modules
  • _site
  • bin
  • obj
  • Generated API reference content excluded by site configuration.
  • Audit output folders.
  • Unpublished or excluded articles.
  • Files configured with publish: false.

The report must provide inclusion and exclusion counts as well as exclusion reasons.

Assessment Methodology

Dimension Weight
Heading Structure 20%
Alternative Text 20%
Link Text Clarity 15%
Plain Language 15%
Table Accessibility 10%
Formatting Independence 10%
Callout Structure 5%
Code Block Accessibility 5%

Scoring Scale

  • 1 = Poor
  • 2 = Significant Issues
  • 3 = Needs Improvement
  • 4 = Good
  • 5 = Excellent

Overall Score Formula

Calculate the overall article score using the weighted average of all dimension scores.

Repository-wide score = Mean of all per-article overall scores.

Round displayed values to one decimal place.

Content Interpretation Rules

  • Assess only Markdown source.
  • Correctly recognize frontmatter, fenced code blocks, snippet/include directives, HTML tables, images, and callouts.
  • Do not treat technical syntax as prose.
  • Do not evaluate content outside the article source.

Absent Content Rule

If an article contains none of a particular content type, assign a score of 5 for that dimension.

Severity Classification

Critical

Blocks access to essential content.

Examples:

  • Missing meaningful heading hierarchy.
  • Informative images without alt text.
  • Entire article relies on non-descriptive link text.

High

Significantly reduces accessibility.

Examples:

  • Repeated heading-level skips.
  • Data tables without headers.
  • Multiple code blocks without language identifiers.

Medium

Readability or comprehension obstacles.

Low

Minor accessibility improvements.

Human Review Cases

When image purpose cannot be determined from source, record Human Review Required instead of assigning severity.

Deliverables

accessibility-report.md

1. Average Scores

Include:

  • Total articles scored.
  • Total articles excluded.
  • Average score for each accessibility dimension.
  • Overall documentation accessibility score.

2. Summary

Cover:

  • Overall accessibility state.
  • Strongest dimensions.
  • Weakest dimensions.
  • Common Medium and Low issue patterns.
  • Counts of articles containing Critical and/or High issues.
  • Limitations of source-only analysis.

3. Critical and High Issues

Severity Article Line Dimension Issue Recommended Fix

Requirements:

  • Group findings by article.
  • Critical findings listed before High findings.
  • Use repository-relative paths.
  • Provide remediation recommendations.
  • Do not include per-article scores.

scores.jsonl

Generate one JSON record per article containing:

  • Dimension scores.
  • Overall weighted score.
  • Critical and High findings.
  • Recurring issue patterns.

Validation Requirements

  • Verify all eligible articles were processed.
  • Verify scored count equals included count.
  • Verify score calculations.
  • Mark the report as Provisional if any article could not be scored.

Success Criteria

  • Every eligible article is assessed.
  • Repository-wide accessibility score is calculated.
  • Accessibility trends are identified.
  • Critical and High issues are documented with remediation guidance.
  • accessibility-report.md and scores.jsonl are generated.
  • Documentation source files remain unmodified.

Expected Outcome

Establish a repository-wide accessibility baseline, identify priority remediation work, and create a measurable benchmark for future accessibility improvements.

Activity

  1. dessyordanova commented on Oct 30, 2026

    @dessyordanova
    CollaboratorAuthor

    After analysing the audit report, several critical issues have been addressed: #900

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions