Skip to content

Change Alternative Sources UI#1148

Merged
pooleycodes merged 9 commits intomainfrom
accessible-alternativity
Jan 28, 2026
Merged

Change Alternative Sources UI#1148
pooleycodes merged 9 commits intomainfrom
accessible-alternativity

Conversation

@pooleycodes
Copy link
Contributor

@pooleycodes pooleycodes commented Jan 22, 2026

Description

  1. Change the UI for alternative source notice to be in line with User Research. Added the Alternate Sources display with queries to the lookup table as the source of where the entities are coming from when there quality is notes as 'some'.
  2. Hard Coded a exception where missing value is reference it becomes 'MUST' fix.
  3. Slight Improvements on display of errors - all error detail messaging now stored in the submit service.
  4. Download URL for alternate data uses the new field query parameter in download lambda.
  5. Get fallback when head request fails - signed urls often only allow get requests through, so a head request appears to show 403 blocked, When a HEAD request returns 403 (appears blocked), the frontend performs a minimal GET request with [Range: bytes=0-0] to check the actual status code as an alternate validation method. This prevents false positives where legitimate signed URLs are incorrectly flagged as inaccessible.

What type of PR is this? (check all applicable)

  • Refactor
  • Feature
  • Bug Fix
  • Optimization
  • Documentation Update

Related Tickets & Documents

QA Instructions, Screenshots, Recordings

New Alternate Content Display:
image

Added/updated tests?

We encourage you to keep the code coverage percentage at 80% and above.

  • Yes
  • No, and this is why: Only need to update current tests with the new expected content.
  • I need help with writing tests

Summary by CodeRabbit

  • Documentation

    • Expanded local Jira Service Desk testing instructions with step‑by‑step guidance and a caution about communities accounts.
  • New Features

    • Alternate data sources now displayed in dataset overviews and data views; UI updated with a “Become the authoritative source” flow and CSV-labelled download button.
    • Download links can include per-field selections for finer exports.
  • Bug Fixes

    • Improved error messaging for inaccessible URLs and more robust URL accessibility checks (HEAD→GET fallback).
  • Tests

    • Updated unit tests to match new alternate source behaviour and accessibility checks.

✏️ Tip: You can customize this high-level summary in your review settings.

@pooleycodes pooleycodes linked an issue Jan 22, 2026 that may be closed by this pull request
4 tasks
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 22, 2026

Walkthrough

This PR surfaces alternate data sources: it extends authority middleware to populate alternateSources when authority is "some", filters system fields from specifications, threads alternateSources/uniqueDatasetFields through middleware to views and schemas, updates templates and download URL handling, adds a HEAD→GET fallback for URL checks, and expands the README Jira local-setup steps.

Changes

Cohort / File(s) Summary
Documentation
readme.md
Expanded Jira Service Desk local-development instructions (4 → 9 steps) and added a caution about communities accounts; added steps to create project/service desk, API key, capture JIRA_URL and build JIRA_BASIC_AUTH.
Controllers
src/controllers/resultsController.js, src/controllers/submitUrlController.js
resultsController: compute quality_criteria_level via qualityLevel with a field-specific override for missing reference. submitUrlController: HEAD uses configured axios; on 403 fallback to GET bytes=0-0; adjust Content-Length from Content-Range; add fallback logging.
Common Middleware
src/middleware/common.middleware.js
Removed limit: 1 from prepareAuthority’s second platform call; set req.alternateEntityList for quality 'some'; added fetchAlternateSources (exports results as alternateSources), filterOutSystemFields; added processAuthoritativeMiddlewares and included filterOutSystemFields into processSpecificationMiddlewares.
Middleware Integration
src/middleware/datasetOverview.middleware.js, src/middleware/dataview.middleware.js
Replace direct prepareAuthority/pullOutDatasetSpecification with ...processAuthoritativeMiddlewares and ...processSpecificationMiddlewares; thread alternateSources and uniqueDatasetFields through request; download URL now supports per-field query params.
Schemas
src/routes/schemas.js
Added optional alternateSources to OrgDatasetOverview and OrgDataView as v.optional(v.array(v.strictObject({ name: NonEmptyString }))).
Views / Templates
src/views/components/alternativeSourceNotice.html, src/views/organisations/dataset-overview.html, src/views/organisations/dataview.html
alternativeSourceNotice macro signature extended to accept alternateSources; templates render single/multiple alternate sources, special-case a named source, update download button text to "Download alternative source data (CSV)" and add preventDoubleClick.
Error / Minor Views
src/views/check/error-redirect.html, src/views/check/results/results.html
Added branch for "The URL must be accessible" with access/permissions guidance and 403 reference; dataset banner now always uses requestParams.dataset (removed fallback).
Tests
test/unit/middleware/common.middleware.test.js, test/unit/views/organisations/dataset-overview.test.js
Updated expectations: removed limit: 1 in prepareAuthority test; adjusted alternative-source text check and changed download button label expectation to include "(CSV)".

Sequence Diagram(s)

sequenceDiagram
    participant Req as Request Handler
    participant Auth as prepareAuthority
    participant Platform as Platform API
    participant FetchAlt as fetchAlternateSources
    participant DB as Database
    participant Filter as filterOutSystemFields
    participant Render as Template Renderer

    Req->>Auth: determine authority for dataset
    Auth->>Platform: fetch entities (quality: 'some')
    Platform-->>Auth: return entities
    Auth->>Req: set req.authority and req.alternateEntityList
    alt req.authority == 'some'
        Req->>FetchAlt: fetch alternateSources (entity list)
        FetchAlt->>DB: query distinct organisation names (exclude current)
        DB-->>FetchAlt: organisation names
        FetchAlt->>Req: set req.alternateSources
    end
    Req->>Filter: remove system fields from req.specification
    Filter-->>Req: spec without system fields
    Req->>Render: render dataset/dataview with alternateSources and fields
    Render-->>Req: response HTML
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

Suggested labels

enhancement

Suggested reviewers

  • eveleighoj
  • rosado

Poem

🐰 I hopped through middleware, found alternate trails so bright,

Trimmed out system fields and set the sources right.
HEAD then tried a GET when the door was shut,
CSVs ready, downloads quick — a carrot for the cut.
Hooray for tidy flows and a rabbit's quiet delight!

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Change Alternative Sources UI' accurately reflects the primary focus of this changeset, which centres on updating the alternative sources notice UI component and supporting infrastructure.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link

github-actions bot commented Jan 22, 2026

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 66.09% 6738 / 10194
🔵 Statements 66.09% 6738 / 10194
🔵 Functions 63.78% 273 / 428
🔵 Branches 79.41% 903 / 1137
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
src/controllers/resultsController.js 58.22% 65% 50% 58.22% 20-39, 66-70, 74-83, 91-92, 94-95, 99-100, 139-141, 170-178, 190-191, 228-229, 238-246, 249-252, 255-274, 287-311, 321-322, 349-351, 367-372, 380-395, 402-423, 435-445, 448-450, 459-472, 519-538
src/controllers/submitUrlController.js 76.44% 73.91% 94.44% 76.44% 41-43, 133-176, 199-201, 211-212, 218-220, 236-238, 253-255
src/middleware/common.middleware.js 64.91% 92.56% 37.7% 64.91% 31-40, 44, 58-78, 91-92, 94-95, 107-109, 113, 129-136, 150-154, 177-192, 240-254, 312-325, 331-353, 436-444, 460-473, 538, 547-549, 579-581, 629-670, 710-714, 719-722, 734-738, 766-780, 787-811, 817-825, 829-845, 914-916, 989-996, 1011-1066, 1097-1098, 1101-1104, 1107-1120, 1145-1150, 1160-1163
src/middleware/datasetOverview.middleware.js 84.74% 56.25% 33.33% 84.74% 17-37, 79-83, 94-99, 136-138, 220
src/middleware/dataview.middleware.js 100% 66.66% 60% 100%
src/routes/schemas.js 100% 100% 100% 100%
Generated in workflow #1322 for commit c34245d by the Vitest Coverage Report Action

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
readme.md (1)

182-189: Clarify redundant project creation steps.

Steps 2 and 5 both mention creating a project:

  • Step 2: "Create a 'Customer Desk' Project"
  • Step 5: "create a new project"

If these refer to the same project, remove the duplication. If they're different projects, please clarify the distinction and purpose of each.

🤖 Fix all issues with AI agents
In `@readme.md`:
- Line 181: Update the single-line caution "Do not do this under a communities
account" to briefly define what a "communities account" is and state the risk:
e.g., explain it refers to free/shared/community-tier Atlassian accounts with
restricted permissions and potentially no support or billing separation, and
instruct readers to use a paid or org-managed account instead; modify the
sentence near the step text "Go to [JIRA Service Desk]..." so it reads something
like "Do not use a Communities (free/shared) account — use an org-managed or
paid Atlassian account to ensure proper permissions, support, and billing
isolation."

In `@src/middleware/common.middleware.js`:
- Around line 237-257: The generated SQL in fetchAlternateSources can produce an
invalid IN () when entityIds is empty; update the query builder in
fetchAlternateSources (used by fetchMany) to short-circuit: detect when
entityIds.length === 0 and return a safe no-row SQL (e.g., a SELECT that always
yields no rows such as SELECT DISTINCT o.name FROM lookup l LEFT JOIN
organisation o ON l.organisation = o.organisation WHERE 1=0) instead of
constructing WHERE l.entity IN (); keep the existing filtering on
req.orgInfo.organisation when returning the no-row query so the result key
alternateSources remains consistent.
- Around line 208-218: In prepareAuthority middleware update the
platformApi.fetchEntities call (where quality: 'some') to include an explicit
limit parameter (e.g., limit: 1000 or another appropriate cap) so
req.alternateEntityList (populated from someResult.formattedData) contains a
complete set; then in fetchAlternateSources guard against empty entityIds before
building the SQL IN clause (return early or skip the query when entityIds.length
=== 0) to avoid generating invalid SQL IN () syntax.

In `@src/views/check/error-redirect.html`:
- Around line 55-57: Two consecutive <p> elements in the error branch for "The
URL must be accessible" both use the same id "bad-upload", violating unique ID
rules and breaking the error-summary focus; update the template so only the
first paragraph retains id="bad-upload" (and its tabindex="-1") and remove the
id attribute from the second paragraph text about contacting IT, leaving the
copy unchanged otherwise to match the other error branches.

In `@src/views/check/results/results.html`:
- Around line 46-49: The datasetBanner call in results.html currently passes
options.requestParams.dataset without a fallback; update the call to use the
same fallback pattern used elsewhere (options.datasetName or
options.requestParams.dataset) so it becomes
datasetBanner(options.requestParams.organisationName, options.datasetName or
options.requestParams.dataset) — locate the datasetBanner invocation in
results.html and modify its second argument accordingly.

In `@src/views/components/alternativeSourceNotice.html`:
- Around line 3-21: The template macro alternativeSourceNotice currently
compares source.name to a hard-coded string with "&" which doesn't match the
canonical "and" used elsewhere; update the equality check on source.name inside
the loop to use the canonical spelling "Ministry of Housing, Communities and
Local Government" or (preferably) normalise the comparison (e.g., lower-case and
replace '&' with 'and' or trim punctuation) so that the condition in the {% if
source.name == ... %} reliably matches both variants and the notice renders
correctly.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
In `@src/controllers/submitUrlController.js`:
- Around line 139-156: When falling back to GET with Range: bytes=0-0 in
SubmitUrlController, the GET response's content-length may reflect only the
requested byte range; before returning getResponse in the GET-fallback branch in
submitUrlController.js, parse the Content-Range header (e.g. "bytes 0-0/12345")
to extract the total file size and overwrite
getResponse.headers['content-length'] with that total size so
urlResponseIsNotTooLarge() sees the real file size; if Content-Range is missing
or unparsable, fall back to the original headers but prefer the parsed total
when available.
- Around line 148-169: The GET fallback success condition in
SubmitUrlController.headRequest() currently treats only statuses <400 and 416 as
acceptable, so a GET 404 falls through and returns the original HEAD 403; update
the condition that checks getResponse.status to also consider 404 as a valid
returned response (i.e., include getResponse.status === 404 alongside the
existing checks) so the function returns getResponse (the GET response) when GET
yields 404 and the isUrlAccessible validator receives the correct 404 status.
🧹 Nitpick comments (1)
src/controllers/submitUrlController.js (1)

66-67: Track this TODO as a work item.

Leaving a TODO here risks it lingering indefinitely; please convert it into a tracked issue or remove the dead code path when possible.

@pooleycodes pooleycodes merged commit d955802 into main Jan 28, 2026
7 checks passed
@pooleycodes pooleycodes deleted the accessible-alternativity branch January 28, 2026 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Iterate content around alternative sources

2 participants