diff --git a/CHANGELOG.md b/CHANGELOG.md index b39a17ff..16b80375 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -30,6 +30,7 @@ Unreleased ## Enhancements * `simplecov uncovered` gained `--criterion line|branch|method` (default `line`) so the lowest-coverage listing can rank by branch or method coverage, not just line. * Added the criterion-first `coverage` configuration method — a uniform way to configure each coverage criterion (`:line`, `:branch`, `:method`) in one place. `coverage :line do minimum 90; minimum_per_file 80; maximum_drop 5 end` (or the one-liner `coverage :branch, minimum: 80`) enables the criterion and declares its thresholds with identical syntax regardless of criterion, because the criterion is fixed by the enclosing call rather than smuggled into the argument as the historical "a bare number means line coverage, every other criterion needs a Hash" special case. Verbs: `minimum`, `maximum`, `exact`, `maximum_drop`, `minimum_per_file` (with `only:` String-path / Regexp overrides), and `minimum_per_group`. Options: `primary:` (the report's leading criterion), `oneshot:` (oneshot-lines mode for `:line`), and `:eval`. The flat `minimum_coverage` family remains as suite-wide sugar. Thresholds feed the same internal stores, so exit-code enforcement is unchanged. See the "Per-criterion thresholds with `coverage`" section in the README. +* JSON formatter: `coverage.json` now carries a top-level `$schema` field holding the URL of the versioned canonical JSON Schema the document conforms to, plus a human-readable `meta.schema_version` (`"major.minor"`, currently `"1.0"`). The versioned canonical lives at `schemas/coverage-v1.0.schema.json` and is immutable per version, an unversioned convenience alias at `schemas/coverage.schema.json` always tracks the latest. Downstream tools can validate inputs, generate types, or pin to a known shape, and the document-level `$schema` makes each payload self-describing. The schema version is independent of the gem version: additive changes bump minor, removals or shape changes bump major and ship as a new `schemas/coverage-vX.0.schema.json` file so prior-version consumers stay valid. `meta.commit` carries the git commit SHA the report was generated against (or null outside a git checkout), so tools can recover the exact source from history even when `source_in_json false` omits the per-file source arrays. * Added `SimpleCov::ParallelAdapters` — a pluggable adapter interface for parallel test runners. SimpleCov's coordination with parallel test runners (deciding which worker does final-result work, waiting for siblings, knowing how many resultsets to expect) now routes through an adapter chain rather than hard-coding the `parallel_tests` gem's API. Two adapters ship: `ParallelTestsAdapter` wraps the historical grosser/parallel_tests gem (precise, gem-API-based); `GenericAdapter` handles any runner that follows the `TEST_ENV_NUMBER` / `PARALLEL_TEST_GROUPS` env-var convention without shipping a Ruby API. The practical impact: **parallel_rspec (and any similar env-var-only runner) now works out of the box** — previously every worker thought it was the "final" one and they clobbered each other's resultsets. Custom runners can register their own adapter via `SimpleCov::ParallelAdapters.register MyAdapter`, where `MyAdapter` subclasses `SimpleCov::ParallelAdapters::Base` and overrides the four contract methods (`active?`, `first_worker?`, `wait_for_siblings`, `expected_worker_count`). See #1065. * Added `SimpleCov.ignore_branches` for opting out of synthetic `:else` branches that Ruby's `Coverage` library reports for constructs with no literal `else` keyword — exhaustive `case/in` pattern matches, `case/when` without `else`, `||=` / `&&=`, and `if` / `unless` without `else`. Variadic; only `:implicit_else` is supported today, with room for future synthetic branch types. Calling it without (or before) `enable_coverage :branch` is harmless — the setting is stored and applies once branch coverage is enabled. Explicit `else` arms still count. See #1033. * Added `SimpleCov.cover` for declaring a positive coverage scope (the long-requested allowlist counterpart to `add_filter`). Accepts string globs, Regexps, blocks, or arrays of those; multiple calls union. When any `cover` matcher is configured the report drops every source file that doesn't match at least one of them, and string-glob matchers also expand on disk so files that exist but were never required during the run still appear in the report (at 0% coverage). Resolves the long-standing requests in #696 and #869. The companion `SimpleCov.no_default_skips` opts out of the filters that `SimpleCov.start` installs (hidden files, `vendor/bundle/`, test directories) so users who want to opt out wholesale don't have to call `clear_filters` themselves. @@ -45,6 +46,7 @@ Unreleased * Terminal output is now colorized when stderr is a TTY: coverage percentages in the formatter summary line and threshold-violation messages are rendered green (>= 90%), yellow (>= 75%), or red (< 75%) — matching the HTML report's thresholds. The "SimpleCov failed with exit N…" line is red and the "Stopped processing SimpleCov…" line is yellow. Respects `NO_COLOR` (force off, per no-color.org) and `FORCE_COLOR` (force on); `NO_COLOR` wins if both are set. See #1157. * CLI subcommands `coverage`, `report`, `uncovered`, and `diff` now colorize their printed percentages by the same threshold (and `diff` colors regressions red, improvements green). Auto-detect based on whether stdout is a TTY; the same `NO_COLOR` / `FORCE_COLOR` env vars apply. Each subcommand also accepts a `--no-color` flag as a per-invocation override. * Added `# simplecov:disable` / `# simplecov:enable` directive comments for selectively skipping `line`, `branch`, and `method` coverage. Block form (own line) opens a region until the matching `# simplecov:enable`; inline form (trailing a code line) skips just that line. Categories may be combined (`# simplecov:disable line, branch`); omitting categories targets all three. Any trailing text is treated as a free-form reason and discarded (e.g. `# simplecov:disable line legacy adapter`). Directive markers inside string literals or heredocs are ignored. +* Added `SimpleCov.source_in_json` (default true) to make the per-file `source` array in `coverage.json` opt-out. Tools that read the project's source files from disk don't need the embedded copy, and on larger projects it dominates the JSON payload. The HTML report's `coverage_data.js` still embeds source unconditionally because the client-side viewer renders source from there. See #1143. * JSON formatter: `meta.timestamp` is now emitted with millisecond precision (`iso8601(3)`) so the concurrent-overwrite warning can distinguish writes within the same wall-clock second * JSON formatter: added `total` section with aggregate coverage statistics (covered, missed, total, percent, strength) for line, branch, and method coverage. Line stats additionally include `omitted` (count of blank/comment lines, i.e. lines that cannot be covered) * JSON formatter: per-file output now includes `total_lines`, `lines_covered_percent`, and when enabled: `branches_covered_percent`, `methods` array, and `methods_covered_percent` diff --git a/Gemfile b/Gemfile index 8fa145b8..42f49c16 100644 --- a/Gemfile +++ b/Gemfile @@ -7,6 +7,7 @@ group :development do gem "capybara" gem "cucumber" gem "cuprite" + gem "json_schemer" gem "nokogiri" gem "rackup" gem "rake" diff --git a/Gemfile.lock b/Gemfile.lock index 95f5d75c..1d97ef1a 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -73,6 +73,7 @@ GEM websocket-driver (~> 0.7) ffi (1.17.4) ffi (1.17.4-java) + hana (1.3.7) io-console (0.8.2) io-console (0.8.2-java) irb (1.18.0) @@ -83,6 +84,11 @@ GEM jar-dependencies (0.5.7) json (2.19.5) json (2.19.5-java) + json_schemer (2.5.0) + bigdecimal + hana (~> 1.3) + regexp_parser (~> 2.0) + simpleidn (~> 0.2) language_server-protocol (3.17.0.5) lint_roller (1.1.0) logger (1.7.0) @@ -169,6 +175,7 @@ GEM lint_roller (~> 1.1) rubocop (~> 1.81) ruby-progressbar (1.13.0) + simpleidn (0.2.3) stringio (3.2.0) sys-uname (1.5.1) ffi (~> 1.1) @@ -203,6 +210,7 @@ DEPENDENCIES capybara cucumber cuprite + json_schemer nokogiri rackup rake diff --git a/README.md b/README.md index 76057055..3fbed761 100644 --- a/README.md +++ b/README.md @@ -1014,10 +1014,60 @@ SimpleCov.formatters = [ SimpleCov.formatter = SimpleCov::Formatter::JSONFormatter ``` +By default `coverage.json` carries the full source-text array for every file, which makes the payload self-contained +but dominates the file size on larger projects. Tools that read the project's source files directly from disk can opt +out of that field with: + +```ruby +SimpleCov.start do + source_in_json false +end +``` + +The HTML report's `coverage_data.js` always retains the source array — the client-side viewer renders source from +there. The setting only affects the side-file `coverage.json`. When the source is omitted, `meta.commit` (the git +commit SHA the report was generated against) lets tools recover the exact source lines from repository history. + > The JSON formatter was originally a separate gem, > [simplecov_json_formatter](https://github.com/codeclimate-community/simplecov_json_formatter). It is now built in and > loaded by default; existing code that does `require "simplecov_json_formatter"` will continue to work. +### JSON Schema for `coverage.json` + +`coverage.json` is a public contract, described by a JSON Schema (2020-12) so downstream tools can validate it, +generate types, or pin to a known shape. Every emitted document carries a top-level `$schema` URL pointing at the +versioned canonical, plus a human-readable `meta.schema_version` (`"major.minor"`). + +The **versioned canonical** lives at [`schemas/coverage-v1.0.schema.json`](schemas/coverage-v1.0.schema.json) and +long-lived integrations should pin to it. Once a SimpleCov release ships with a given versioned schema file, that file +is immutable: bug fixes, additions, or shape changes ship as a new versioned file (a minor or major bump), never as a +silent rewrite of an already-released one. Schemas may still be corrected in-place between gem releases — i.e., the +schema file as it currently exists on `main` may change before the next gem release, but the schema for any published +gem version stays frozen. A convenience alias at [`schemas/coverage.schema.json`](schemas/coverage.schema.json) always +tracks the latest and may shift when a new SimpleCov release bumps the schema. + +The schema version is independent of the gem version: + +- Additive changes (new fields) bump the **minor** segment. Existing consumers keep working. +- Removals or shape changes bump the **major** segment, and ship as a new `schemas/coverage-vX.0.schema.json` file so + v1.x consumers stay valid. + +The current version is **1.0**. Top-level structure: + +```jsonc +{ + "$schema": "https://raw.githubusercontent.com/simplecov-ruby/simplecov/main/schemas/coverage-v1.0.schema.json", + "meta": { /* schema_version, simplecov_version, command_name, project_name, timestamp, root, commit, line_coverage, branch_coverage, method_coverage */ }, + "total": { /* aggregate stats for lines (and branches / methods when enabled) */ }, + "coverage": { "": { /* per-file lines, source, branches, methods, etc. */ } }, + "groups": { "": { /* per-group stats + files */ } }, + "errors": { /* minimum_coverage, minimum_coverage_by_file, minimum_coverage_by_group, maximum_coverage, maximum_coverage_drop violations */ } +} +``` + +The `.resultset.json` file is **not** schema'd — it's SimpleCov-internal and may change shape across releases. Build +integrations on top of `coverage.json`. + ### More formatters, editor integrations, and hosted services * [Open Source formatter and integration plugins for SimpleCov](doc/alternate-formatters.md) diff --git a/lib/simplecov/configuration/formatting.rb b/lib/simplecov/configuration/formatting.rb index eae15fbd..ae83e5e1 100644 --- a/lib/simplecov/configuration/formatting.rb +++ b/lib/simplecov/configuration/formatting.rb @@ -65,6 +65,28 @@ def print_errors(value = :__no_arg__) @print_error_status = value end + # + # Get or set whether `coverage.json` includes the full source-text + # array for every file. Defaults to true. Set to false when a + # downstream tool reads the project's source files directly and + # only needs the coverage metrics, so `coverage.json` doesn't carry + # a copy of the source tree (which dominates the payload on larger + # projects). + # + # The HTML viewer's `coverage_data.js` always includes source — + # the client-side renderer needs it. Only `coverage.json` honors + # this setting. + # + # SimpleCov.start do + # source_in_json false + # end + # + def source_in_json(value = :__no_arg__) + return defined?(@source_in_json) ? @source_in_json : true if value == :__no_arg__ + + @source_in_json = value + end + # DEPRECATED: alias for `print_errors`. Same value, same behavior. def print_error_status warn "#{Kernel.caller.first}: [DEPRECATION] `SimpleCov.print_error_status` is deprecated. " \ diff --git a/lib/simplecov/formatter/html_formatter.rb b/lib/simplecov/formatter/html_formatter.rb index e1a7c55d..e106949d 100644 --- a/lib/simplecov/formatter/html_formatter.rb +++ b/lib/simplecov/formatter/html_formatter.rb @@ -15,11 +15,19 @@ class HTMLFormatter < Base DATA_FILENAME = "coverage_data.js" def format(result) - json = JSON.pretty_generate(JSONFormatter.build_hash(result)) - + # `coverage_data.js` feeds the client-side viewer, which renders + # source from the embedded array — it always needs `source`, + # regardless of `SimpleCov.source_in_json`. The side-file + # `coverage.json` honors the setting so downstream tools that + # read source from disk can opt into a smaller payload. When + # the setting is at its default (true), the two files share a + # single serialization. FileUtils.mkdir_p(output_path) - atomic_write(File.join(output_path, JSONFormatter::FILENAME), json) - atomic_write(File.join(output_path, DATA_FILENAME), "window.SIMPLECOV_DATA = #{json};\n") + viewer_json = JSON.pretty_generate(JSONFormatter.build_hash(result, include_source: true)) + coverage_json = SimpleCov.source_in_json ? viewer_json : JSON.pretty_generate(JSONFormatter.build_hash(result)) + + atomic_write(File.join(output_path, JSONFormatter::FILENAME), coverage_json) + atomic_write(File.join(output_path, DATA_FILENAME), "window.SIMPLECOV_DATA = #{viewer_json};\n") copy_static_assets # stderr, not stdout: this is a status message, not the program's diff --git a/lib/simplecov/formatter/json_formatter.rb b/lib/simplecov/formatter/json_formatter.rb index 6a639d31..db1478d2 100644 --- a/lib/simplecov/formatter/json_formatter.rb +++ b/lib/simplecov/formatter/json_formatter.rb @@ -13,8 +13,13 @@ module Formatter class JSONFormatter < Base FILENAME = "coverage.json" - def self.build_hash(result) - ResultHashFormatter.new(result).format + # `include_source:` defaults to `SimpleCov.source_in_json` (true + # by default) so the historical payload shape is unchanged. + # Callers that need the source array regardless of the global + # setting (the HTML formatter, which feeds the client-side + # viewer) pass `include_source: true` explicitly. + def self.build_hash(result, include_source: SimpleCov.source_in_json) + ResultHashFormatter.new(result, include_source: include_source).format end def format(result) diff --git a/lib/simplecov/formatter/json_formatter/errors_formatter.rb b/lib/simplecov/formatter/json_formatter/errors_formatter.rb index 29ece356..8e8f44cc 100644 --- a/lib/simplecov/formatter/json_formatter/errors_formatter.rb +++ b/lib/simplecov/formatter/json_formatter/errors_formatter.rb @@ -42,8 +42,8 @@ def format_minimum_by_file end def record_by_file(violation) - criterion_bucket = bucket(:minimum_coverage_by_file)[key_for(violation)] ||= {} - criterion_bucket[violation.fetch(:project_filename)] = expected_actual(violation) + file_bucket = bucket(:minimum_coverage_by_file)[violation.fetch(:project_filename)] ||= {} + file_bucket[key_for(violation)] = expected_actual(violation) end def format_minimum_by_group diff --git a/lib/simplecov/formatter/json_formatter/result_hash_formatter.rb b/lib/simplecov/formatter/json_formatter/result_hash_formatter.rb index e1f9f51f..ca59aa5d 100644 --- a/lib/simplecov/formatter/json_formatter/result_hash_formatter.rb +++ b/lib/simplecov/formatter/json_formatter/result_hash_formatter.rb @@ -1,5 +1,6 @@ # frozen_string_literal: true +require "open3" require "time" require_relative "errors_formatter" require_relative "source_file_formatter" @@ -10,24 +11,39 @@ class JSONFormatter # Builds the hash that JSONFormatter serializes to coverage.json: # meta, per-file coverage data, group totals, and aggregate stats. class ResultHashFormatter - def initialize(result) + # Bump SCHEMA_VERSION (and SCHEMA_URL) when the JSON shape + # changes. Additive changes bump the minor segment, removals or + # shape changes bump the major segment. The versioned file at + # schemas/coverage-vX.Y.schema.json is the canonical artifact + # consumers should pin to, schemas/coverage.schema.json is a + # convenience alias that always tracks the latest. See the + # `coverage.json` schema section of the README for the rationale. + SCHEMA_VERSION = "1.0" + SCHEMA_URL = "https://raw.githubusercontent.com/simplecov-ruby/simplecov/main/schemas/coverage-v#{SCHEMA_VERSION}.schema.json".freeze + private_constant :SCHEMA_VERSION, :SCHEMA_URL + + def initialize(result, include_source: true) @result = result + @include_source = include_source end def format { - meta: format_meta, - total: format_coverage_statistics(@result.coverage_statistics), - coverage: format_files, - groups: format_groups, - errors: ErrorsFormatter.new(@result).call + :$schema => SCHEMA_URL, + :meta => format_meta, + :total => format_coverage_statistics(@result.coverage_statistics), + :coverage => format_files, + :groups => format_groups, + :errors => ErrorsFormatter.new(@result).call } end private def format_files - @result.files.to_h { |source_file| [source_file.project_filename, SourceFileFormatter.new(source_file).call] } + @result.files.to_h do |source_file| + [source_file.project_filename, SourceFileFormatter.new(source_file, include_source: @include_source).call] + end end def format_groups @@ -39,16 +55,44 @@ def format_groups def format_meta { + schema_version: SCHEMA_VERSION, simplecov_version: SimpleCov::VERSION, command_name: @result.command_name, project_name: SimpleCov.project_name, timestamp: @result.created_at.iso8601(3), root: SimpleCov.root, + commit: git_commit + }.merge!(coverage_flags) + end + + # Full git commit SHA of `SimpleCov.root`'s HEAD, or nil when the + # project isn't a git checkout or git isn't on PATH. Recorded so tools + # can recover the exact source a report was generated against, which + # matters most when `source_in_json false` drops the source text from + # coverage.json. stderr is captured (not forwarded) so a non-git project + # doesn't print git's diagnostics to the build. + def git_commit + output, status = Open3.capture2e("git", "-C", SimpleCov.root.to_s, "rev-parse", "HEAD") + status.success? ? output.strip : nil + rescue StandardError + nil + end + + def coverage_flags + { + line_coverage: line_coverage_enabled?, branch_coverage: SimpleCov.branch_coverage?, method_coverage: SimpleCov.method_coverage? } end + # Mirrors SourceFileFormatter's predicate so meta.line_coverage + # tracks exactly which configurations cause the formatter to + # emit line stats. + def line_coverage_enabled? + SimpleCov.coverage_criterion_enabled?(:line) || SimpleCov.coverage_criterion_enabled?(:oneshot_line) + end + def format_coverage_statistics(statistics) result = {} result[:lines] = format_line_statistic(statistics[:line]) if statistics[:line] diff --git a/lib/simplecov/formatter/json_formatter/source_file_formatter.rb b/lib/simplecov/formatter/json_formatter/source_file_formatter.rb index 0c38eafd..d1ed758c 100644 --- a/lib/simplecov/formatter/json_formatter/source_file_formatter.rb +++ b/lib/simplecov/formatter/json_formatter/source_file_formatter.rb @@ -7,12 +7,13 @@ class JSONFormatter # in coverage.json: source code plus per-enabled-criterion arrays # and totals. class SourceFileFormatter - def initialize(source_file) + def initialize(source_file, include_source: true) @source_file = source_file + @include_source = include_source end def call - result = format_source_code + result = @include_source ? format_source_code : {} result.merge!(line_coverage_section) if line_coverage_enabled? result.merge!(branch_coverage_section) if SimpleCov.branch_coverage? result.merge!(method_coverage_section) if SimpleCov.method_coverage? @@ -44,6 +45,7 @@ def line_coverage_section lines_covered_percent: @source_file.covered_percent, covered_lines: covered, missed_lines: missed, + omitted_lines: @source_file.never_lines.size, total_lines: covered + missed } end diff --git a/schemas/coverage-v1.0.schema.json b/schemas/coverage-v1.0.schema.json new file mode 100644 index 00000000..ff8d68e8 --- /dev/null +++ b/schemas/coverage-v1.0.schema.json @@ -0,0 +1,300 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://raw.githubusercontent.com/simplecov-ruby/simplecov/main/schemas/coverage-v1.0.schema.json", + "title": "SimpleCov coverage.json", + "description": "Schema for the coverage.json file emitted by SimpleCov's JSONFormatter. Versioned independently of the gem. Non-breaking additions bump the minor segment of schema_version, breaking changes bump the major segment. The versioned file at schemas/coverage-vX.Y.schema.json is the canonical artifact for that version, schemas/coverage.schema.json is a convenience alias for the latest.", + "type": "object", + "required": ["$schema", "meta", "total", "coverage", "groups", "errors"], + "additionalProperties": false, + "properties": { + "$schema": { + "type": "string", + "format": "uri", + "description": "URL of the JSON Schema this document conforms to. Pinned to the versioned canonical URL so documents stay validatable against the exact contract they were emitted under, even after the schema evolves." + }, + "meta": { + "type": "object", + "required": [ + "schema_version", + "simplecov_version", + "command_name", + "project_name", + "timestamp", + "root", + "commit", + "line_coverage", + "branch_coverage", + "method_coverage" + ], + "additionalProperties": false, + "properties": { + "schema_version": { + "type": "string", + "const": "1.0", + "description": "Schema major.minor. Additive changes bump minor; breaking changes bump major. Update this `const` whenever the schema version is bumped so documents claiming a different version don't quietly validate against this contract." + }, + "simplecov_version": { + "type": "string", + "description": "The version of the SimpleCov gem that produced this file." + }, + "command_name": {"type": "string"}, + "project_name": {"type": "string"}, + "timestamp": { + "type": "string", + "format": "date-time", + "description": "ISO 8601 timestamp with millisecond precision." + }, + "root": { + "type": "string", + "description": "Absolute path to SimpleCov.root at the time of write." + }, + "commit": { + "type": ["string", "null"], + "description": "Full git commit SHA of `root`'s HEAD when the report was generated, or null when the project isn't a git checkout (or git isn't available). Lets tools recover the exact source a report reflects, which matters most when `source_in_json` is false and the per-file `source` arrays are omitted." + }, + "line_coverage": {"type": "boolean"}, + "branch_coverage": {"type": "boolean"}, + "method_coverage": {"type": "boolean"} + } + }, + "total": {"$ref": "#/definitions/totals"}, + "coverage": { + "type": "object", + "description": "Map of project-relative file paths to per-file coverage data.", + "additionalProperties": {"$ref": "#/definitions/source_file"} + }, + "groups": { + "type": "object", + "description": "Map of group names to per-group totals plus the list of files in the group.", + "additionalProperties": {"$ref": "#/definitions/group"} + }, + "errors": { + "type": "object", + "description": "Threshold violations from minimum_coverage, minimum_coverage_by_file, minimum_coverage_by_group, maximum_coverage, and maximum_coverage_drop. Empty object when no thresholds were violated.", + "additionalProperties": false, + "properties": { + "minimum_coverage": { + "type": "object", + "description": "Keyed by criterion: lines, branches, methods.", + "additionalProperties": {"$ref": "#/definitions/expected_actual"} + }, + "minimum_coverage_by_file": { + "type": "object", + "description": "Keyed by project-relative filename, then by criterion (lines, branches, methods).", + "additionalProperties": { + "type": "object", + "additionalProperties": {"$ref": "#/definitions/expected_actual"} + } + }, + "minimum_coverage_by_group": { + "type": "object", + "description": "Keyed by group name, then by criterion (lines, branches, methods).", + "additionalProperties": { + "type": "object", + "additionalProperties": {"$ref": "#/definitions/expected_actual"} + } + }, + "maximum_coverage": { + "type": "object", + "description": "Keyed by criterion: lines, branches, methods. `expected` is the configured maximum, `actual` is the measured value that exceeded it.", + "additionalProperties": {"$ref": "#/definitions/expected_actual"} + }, + "maximum_coverage_drop": { + "type": "object", + "description": "Keyed by criterion: lines, branches, methods.", + "additionalProperties": {"$ref": "#/definitions/maximum_actual"} + } + } + } + }, + "definitions": { + "totals": { + "type": "object", + "additionalProperties": false, + "anyOf": [ + {"required": ["lines"]}, + {"required": ["branches"]}, + {"required": ["methods"]} + ], + "properties": { + "lines": {"$ref": "#/definitions/line_statistic"}, + "branches": {"$ref": "#/definitions/coverage_statistic"}, + "methods": {"$ref": "#/definitions/coverage_statistic"} + } + }, + "source_file": { + "type": "object", + "additionalProperties": false, + "anyOf": [ + {"required": ["lines"]}, + {"required": ["branches"]}, + {"required": ["methods"]} + ], + "dependentRequired": { + "lines": ["lines_covered_percent", "covered_lines", "missed_lines", "omitted_lines", "total_lines"], + "lines_covered_percent": ["lines", "covered_lines", "missed_lines", "omitted_lines", "total_lines"], + "covered_lines": ["lines", "lines_covered_percent", "missed_lines", "omitted_lines", "total_lines"], + "missed_lines": ["lines", "lines_covered_percent", "covered_lines", "omitted_lines", "total_lines"], + "omitted_lines": ["lines", "lines_covered_percent", "covered_lines", "missed_lines", "total_lines"], + "total_lines": ["lines", "lines_covered_percent", "covered_lines", "missed_lines", "omitted_lines"], + "branches": ["branches_covered_percent", "covered_branches", "missed_branches", "total_branches"], + "branches_covered_percent": ["branches", "covered_branches", "missed_branches", "total_branches"], + "covered_branches": ["branches", "branches_covered_percent", "missed_branches", "total_branches"], + "missed_branches": ["branches", "branches_covered_percent", "covered_branches", "total_branches"], + "total_branches": ["branches", "branches_covered_percent", "covered_branches", "missed_branches"], + "methods": ["methods_covered_percent", "covered_methods", "missed_methods", "total_methods"], + "methods_covered_percent": ["methods", "covered_methods", "missed_methods", "total_methods"], + "covered_methods": ["methods", "methods_covered_percent", "missed_methods", "total_methods"], + "missed_methods": ["methods", "methods_covered_percent", "covered_methods", "total_methods"], + "total_methods": ["methods", "methods_covered_percent", "covered_methods", "missed_methods"] + }, + "properties": { + "lines": { + "type": "array", + "description": "Per-source-line coverage. Element index N corresponds to source line N+1. Integer hit-count, null for non-relevant (blank/comment) lines, or the string \"ignored\" for lines inside a simplecov:disable / :nocov: region. Present if and only if `meta.line_coverage` is true. When present, the five line stat fields (`lines_covered_percent`, `covered_lines`, `missed_lines`, `omitted_lines`, `total_lines`) are guaranteed to be present too.", + "items": {"$ref": "#/definitions/line_coverage"} + }, + "source": { + "type": "array", + "description": "Source lines of the file, in order (one entry per physical line). When the per-file `lines` array is present it has the same length, but `source` is emitted independently of line coverage. Present only when SimpleCov.source_in_json is true (the default); omitted when downstream tools opt out via `source_in_json false` to keep coverage.json smaller.", + "items": {"type": "string"} + }, + "lines_covered_percent": {"type": "number"}, + "covered_lines": {"type": "integer", "minimum": 0, "description": "Count of executable lines that were hit at least once."}, + "missed_lines": {"type": "integer", "minimum": 0, "description": "Count of executable lines that were never hit."}, + "omitted_lines": {"type": "integer", "minimum": 0, "description": "Count of non-executable lines (blank lines, comments). Mirrors `total.lines.omitted` at the per-file level. Excludes lines inside simplecov:disable / :nocov: regions, which surface as `\"ignored\"` in the `lines` array."}, + "total_lines": {"type": "integer", "minimum": 0, "description": "Count of executable lines: `covered_lines + missed_lines`. Does not include non-executable (omitted) lines or lines inside simplecov:disable / :nocov: regions, so this can be smaller than `lines.length`."}, + "branches": { + "type": "array", + "description": "Per-branch coverage. Present if and only if `meta.branch_coverage` is true. When present, the four branch stat fields (`branches_covered_percent`, `covered_branches`, `missed_branches`, `total_branches`) are guaranteed to be present too.", + "items": {"$ref": "#/definitions/branch"} + }, + "branches_covered_percent": {"type": "number"}, + "covered_branches": {"type": "integer", "minimum": 0}, + "missed_branches": {"type": "integer", "minimum": 0}, + "total_branches": {"type": "integer", "minimum": 0, "description": "Count of branches: `covered_branches + missed_branches`."}, + "methods": { + "type": "array", + "description": "Per-method coverage. Present if and only if `meta.method_coverage` is true. When present, the four method stat fields (`methods_covered_percent`, `covered_methods`, `missed_methods`, `total_methods`) are guaranteed to be present too.", + "items": {"$ref": "#/definitions/method"} + }, + "methods_covered_percent": {"type": "number"}, + "covered_methods": {"type": "integer", "minimum": 0}, + "missed_methods": {"type": "integer", "minimum": 0}, + "total_methods": {"type": "integer", "minimum": 0, "description": "Count of methods: `covered_methods + missed_methods`."} + } + }, + "branch": { + "type": "object", + "required": ["type", "start_line", "end_line", "coverage", "inline", "report_line"], + "additionalProperties": false, + "properties": { + "type": { + "type": "string", + "description": "Branch kind from Ruby's Coverage library (e.g. \"then\", \"else\", \"when\")." + }, + "start_line": {"type": "integer", "minimum": 1}, + "end_line": {"type": "integer", "minimum": 1}, + "coverage": {"$ref": "#/definitions/branch_method_coverage"}, + "inline": {"type": "boolean"}, + "report_line": {"type": "integer", "minimum": 1, "description": "Line of the conditional that owns this branch (the `if`, `case`, or `&&` line), not the start of the branch body. Useful for rendering annotations at the decision point."} + } + }, + "method": { + "type": "object", + "required": ["name", "start_line", "end_line", "coverage"], + "additionalProperties": false, + "properties": { + "name": { + "type": "string", + "description": "Qualified method name, e.g. \"Foo#bar\", \"Foo.bar\", or \"Foo::Bar#baz\"." + }, + "start_line": {"type": "integer", "minimum": 1}, + "end_line": {"type": "integer", "minimum": 1}, + "coverage": {"$ref": "#/definitions/branch_method_coverage"} + } + }, + "group": { + "type": "object", + "required": ["files"], + "additionalProperties": false, + "anyOf": [ + {"required": ["lines"]}, + {"required": ["branches"]}, + {"required": ["methods"]} + ], + "properties": { + "lines": {"$ref": "#/definitions/line_statistic"}, + "branches": {"$ref": "#/definitions/coverage_statistic"}, + "methods": {"$ref": "#/definitions/coverage_statistic"}, + "files": { + "type": "array", + "description": "Project-relative paths of the files that fell into this group.", + "items": {"type": "string"} + } + } + }, + "line_statistic": { + "type": "object", + "required": ["covered", "missed", "omitted", "total", "percent", "strength"], + "additionalProperties": false, + "properties": { + "covered": {"type": "integer", "minimum": 0}, + "missed": {"type": "integer", "minimum": 0}, + "omitted": { + "type": "integer", + "minimum": 0, + "description": "Lines that cannot be covered (blank, comment, etc.). Only present on line stats." + }, + "total": {"type": "integer", "minimum": 0, "description": "Executable lines: `covered + missed`. Does not include `omitted`."}, + "percent": {"type": "number"}, + "strength": {"type": "number", "description": "Average number of executions across covered lines (hits per covered line)."} + } + }, + "coverage_statistic": { + "type": "object", + "required": ["covered", "missed", "total", "percent", "strength"], + "additionalProperties": false, + "properties": { + "covered": {"type": "integer", "minimum": 0}, + "missed": {"type": "integer", "minimum": 0}, + "total": {"type": "integer", "minimum": 0, "description": "Total branches or methods: `covered + missed`."}, + "percent": {"type": "number"}, + "strength": {"type": "number", "description": "Average number of executions across covered branches or methods (hits per covered item)."} + } + }, + "line_coverage": { + "description": "Coverage value for a source line in the per-file `lines` array. Integer hit count, `null` for a non-executable line (blank or comment), or the literal string `\"ignored\"` for a line inside a simplecov:disable / :nocov: region.", + "oneOf": [ + {"type": "integer", "minimum": 0}, + {"type": "null"}, + {"type": "string", "const": "ignored"} + ] + }, + "branch_method_coverage": { + "description": "Coverage value for a branch or method `coverage` field. Integer hit count, or the literal string `\"ignored\"` for code inside a simplecov:disable / :nocov: region. Unlike line coverage, `null` never occurs: every branch and method maps to an executable construct.", + "oneOf": [ + {"type": "integer", "minimum": 0}, + {"type": "string", "const": "ignored"} + ] + }, + "expected_actual": { + "type": "object", + "required": ["expected", "actual"], + "additionalProperties": false, + "properties": { + "expected": {"type": "number"}, + "actual": {"type": "number"} + } + }, + "maximum_actual": { + "type": "object", + "required": ["maximum", "actual"], + "additionalProperties": false, + "properties": { + "maximum": {"type": "number"}, + "actual": {"type": "number"} + } + } + } +} diff --git a/schemas/coverage.schema.json b/schemas/coverage.schema.json new file mode 100644 index 00000000..8ebb5368 --- /dev/null +++ b/schemas/coverage.schema.json @@ -0,0 +1,300 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://raw.githubusercontent.com/simplecov-ruby/simplecov/main/schemas/coverage.schema.json", + "title": "SimpleCov coverage.json (latest)", + "description": "Convenience alias for the latest coverage.json schema. Mirrors schemas/coverage-v1.0.schema.json except for the $id (which is the unversioned URL). For long-lived integrations, pin to the versioned canonical (schemas/coverage-vX.Y.schema.json) so the contract you validate against does not silently shift when a new SimpleCov release bumps the schema.", + "type": "object", + "required": ["$schema", "meta", "total", "coverage", "groups", "errors"], + "additionalProperties": false, + "properties": { + "$schema": { + "type": "string", + "format": "uri", + "description": "URL of the JSON Schema this document conforms to. Pinned to the versioned canonical URL so documents stay validatable against the exact contract they were emitted under, even after the schema evolves." + }, + "meta": { + "type": "object", + "required": [ + "schema_version", + "simplecov_version", + "command_name", + "project_name", + "timestamp", + "root", + "commit", + "line_coverage", + "branch_coverage", + "method_coverage" + ], + "additionalProperties": false, + "properties": { + "schema_version": { + "type": "string", + "const": "1.0", + "description": "Schema major.minor. Additive changes bump minor; breaking changes bump major. Update this `const` whenever the schema version is bumped so documents claiming a different version don't quietly validate against this contract." + }, + "simplecov_version": { + "type": "string", + "description": "The version of the SimpleCov gem that produced this file." + }, + "command_name": {"type": "string"}, + "project_name": {"type": "string"}, + "timestamp": { + "type": "string", + "format": "date-time", + "description": "ISO 8601 timestamp with millisecond precision." + }, + "root": { + "type": "string", + "description": "Absolute path to SimpleCov.root at the time of write." + }, + "commit": { + "type": ["string", "null"], + "description": "Full git commit SHA of `root`'s HEAD when the report was generated, or null when the project isn't a git checkout (or git isn't available). Lets tools recover the exact source a report reflects, which matters most when `source_in_json` is false and the per-file `source` arrays are omitted." + }, + "line_coverage": {"type": "boolean"}, + "branch_coverage": {"type": "boolean"}, + "method_coverage": {"type": "boolean"} + } + }, + "total": {"$ref": "#/definitions/totals"}, + "coverage": { + "type": "object", + "description": "Map of project-relative file paths to per-file coverage data.", + "additionalProperties": {"$ref": "#/definitions/source_file"} + }, + "groups": { + "type": "object", + "description": "Map of group names to per-group totals plus the list of files in the group.", + "additionalProperties": {"$ref": "#/definitions/group"} + }, + "errors": { + "type": "object", + "description": "Threshold violations from minimum_coverage, minimum_coverage_by_file, minimum_coverage_by_group, maximum_coverage, and maximum_coverage_drop. Empty object when no thresholds were violated.", + "additionalProperties": false, + "properties": { + "minimum_coverage": { + "type": "object", + "description": "Keyed by criterion: lines, branches, methods.", + "additionalProperties": {"$ref": "#/definitions/expected_actual"} + }, + "minimum_coverage_by_file": { + "type": "object", + "description": "Keyed by project-relative filename, then by criterion (lines, branches, methods).", + "additionalProperties": { + "type": "object", + "additionalProperties": {"$ref": "#/definitions/expected_actual"} + } + }, + "minimum_coverage_by_group": { + "type": "object", + "description": "Keyed by group name, then by criterion (lines, branches, methods).", + "additionalProperties": { + "type": "object", + "additionalProperties": {"$ref": "#/definitions/expected_actual"} + } + }, + "maximum_coverage": { + "type": "object", + "description": "Keyed by criterion: lines, branches, methods. `expected` is the configured maximum, `actual` is the measured value that exceeded it.", + "additionalProperties": {"$ref": "#/definitions/expected_actual"} + }, + "maximum_coverage_drop": { + "type": "object", + "description": "Keyed by criterion: lines, branches, methods.", + "additionalProperties": {"$ref": "#/definitions/maximum_actual"} + } + } + } + }, + "definitions": { + "totals": { + "type": "object", + "additionalProperties": false, + "anyOf": [ + {"required": ["lines"]}, + {"required": ["branches"]}, + {"required": ["methods"]} + ], + "properties": { + "lines": {"$ref": "#/definitions/line_statistic"}, + "branches": {"$ref": "#/definitions/coverage_statistic"}, + "methods": {"$ref": "#/definitions/coverage_statistic"} + } + }, + "source_file": { + "type": "object", + "additionalProperties": false, + "anyOf": [ + {"required": ["lines"]}, + {"required": ["branches"]}, + {"required": ["methods"]} + ], + "dependentRequired": { + "lines": ["lines_covered_percent", "covered_lines", "missed_lines", "omitted_lines", "total_lines"], + "lines_covered_percent": ["lines", "covered_lines", "missed_lines", "omitted_lines", "total_lines"], + "covered_lines": ["lines", "lines_covered_percent", "missed_lines", "omitted_lines", "total_lines"], + "missed_lines": ["lines", "lines_covered_percent", "covered_lines", "omitted_lines", "total_lines"], + "omitted_lines": ["lines", "lines_covered_percent", "covered_lines", "missed_lines", "total_lines"], + "total_lines": ["lines", "lines_covered_percent", "covered_lines", "missed_lines", "omitted_lines"], + "branches": ["branches_covered_percent", "covered_branches", "missed_branches", "total_branches"], + "branches_covered_percent": ["branches", "covered_branches", "missed_branches", "total_branches"], + "covered_branches": ["branches", "branches_covered_percent", "missed_branches", "total_branches"], + "missed_branches": ["branches", "branches_covered_percent", "covered_branches", "total_branches"], + "total_branches": ["branches", "branches_covered_percent", "covered_branches", "missed_branches"], + "methods": ["methods_covered_percent", "covered_methods", "missed_methods", "total_methods"], + "methods_covered_percent": ["methods", "covered_methods", "missed_methods", "total_methods"], + "covered_methods": ["methods", "methods_covered_percent", "missed_methods", "total_methods"], + "missed_methods": ["methods", "methods_covered_percent", "covered_methods", "total_methods"], + "total_methods": ["methods", "methods_covered_percent", "covered_methods", "missed_methods"] + }, + "properties": { + "lines": { + "type": "array", + "description": "Per-source-line coverage. Element index N corresponds to source line N+1. Integer hit-count, null for non-relevant (blank/comment) lines, or the string \"ignored\" for lines inside a simplecov:disable / :nocov: region. Present if and only if `meta.line_coverage` is true. When present, the five line stat fields (`lines_covered_percent`, `covered_lines`, `missed_lines`, `omitted_lines`, `total_lines`) are guaranteed to be present too.", + "items": {"$ref": "#/definitions/line_coverage"} + }, + "source": { + "type": "array", + "description": "Source lines of the file, in order (one entry per physical line). When the per-file `lines` array is present it has the same length, but `source` is emitted independently of line coverage. Present only when SimpleCov.source_in_json is true (the default); omitted when downstream tools opt out via `source_in_json false` to keep coverage.json smaller.", + "items": {"type": "string"} + }, + "lines_covered_percent": {"type": "number"}, + "covered_lines": {"type": "integer", "minimum": 0, "description": "Count of executable lines that were hit at least once."}, + "missed_lines": {"type": "integer", "minimum": 0, "description": "Count of executable lines that were never hit."}, + "omitted_lines": {"type": "integer", "minimum": 0, "description": "Count of non-executable lines (blank lines, comments). Mirrors `total.lines.omitted` at the per-file level. Excludes lines inside simplecov:disable / :nocov: regions, which surface as `\"ignored\"` in the `lines` array."}, + "total_lines": {"type": "integer", "minimum": 0, "description": "Count of executable lines: `covered_lines + missed_lines`. Does not include non-executable (omitted) lines or lines inside simplecov:disable / :nocov: regions, so this can be smaller than `lines.length`."}, + "branches": { + "type": "array", + "description": "Per-branch coverage. Present if and only if `meta.branch_coverage` is true. When present, the four branch stat fields (`branches_covered_percent`, `covered_branches`, `missed_branches`, `total_branches`) are guaranteed to be present too.", + "items": {"$ref": "#/definitions/branch"} + }, + "branches_covered_percent": {"type": "number"}, + "covered_branches": {"type": "integer", "minimum": 0}, + "missed_branches": {"type": "integer", "minimum": 0}, + "total_branches": {"type": "integer", "minimum": 0, "description": "Count of branches: `covered_branches + missed_branches`."}, + "methods": { + "type": "array", + "description": "Per-method coverage. Present if and only if `meta.method_coverage` is true. When present, the four method stat fields (`methods_covered_percent`, `covered_methods`, `missed_methods`, `total_methods`) are guaranteed to be present too.", + "items": {"$ref": "#/definitions/method"} + }, + "methods_covered_percent": {"type": "number"}, + "covered_methods": {"type": "integer", "minimum": 0}, + "missed_methods": {"type": "integer", "minimum": 0}, + "total_methods": {"type": "integer", "minimum": 0, "description": "Count of methods: `covered_methods + missed_methods`."} + } + }, + "branch": { + "type": "object", + "required": ["type", "start_line", "end_line", "coverage", "inline", "report_line"], + "additionalProperties": false, + "properties": { + "type": { + "type": "string", + "description": "Branch kind from Ruby's Coverage library (e.g. \"then\", \"else\", \"when\")." + }, + "start_line": {"type": "integer", "minimum": 1}, + "end_line": {"type": "integer", "minimum": 1}, + "coverage": {"$ref": "#/definitions/branch_method_coverage"}, + "inline": {"type": "boolean"}, + "report_line": {"type": "integer", "minimum": 1, "description": "Line of the conditional that owns this branch (the `if`, `case`, or `&&` line), not the start of the branch body. Useful for rendering annotations at the decision point."} + } + }, + "method": { + "type": "object", + "required": ["name", "start_line", "end_line", "coverage"], + "additionalProperties": false, + "properties": { + "name": { + "type": "string", + "description": "Qualified method name, e.g. \"Foo#bar\", \"Foo.bar\", or \"Foo::Bar#baz\"." + }, + "start_line": {"type": "integer", "minimum": 1}, + "end_line": {"type": "integer", "minimum": 1}, + "coverage": {"$ref": "#/definitions/branch_method_coverage"} + } + }, + "group": { + "type": "object", + "required": ["files"], + "additionalProperties": false, + "anyOf": [ + {"required": ["lines"]}, + {"required": ["branches"]}, + {"required": ["methods"]} + ], + "properties": { + "lines": {"$ref": "#/definitions/line_statistic"}, + "branches": {"$ref": "#/definitions/coverage_statistic"}, + "methods": {"$ref": "#/definitions/coverage_statistic"}, + "files": { + "type": "array", + "description": "Project-relative paths of the files that fell into this group.", + "items": {"type": "string"} + } + } + }, + "line_statistic": { + "type": "object", + "required": ["covered", "missed", "omitted", "total", "percent", "strength"], + "additionalProperties": false, + "properties": { + "covered": {"type": "integer", "minimum": 0}, + "missed": {"type": "integer", "minimum": 0}, + "omitted": { + "type": "integer", + "minimum": 0, + "description": "Lines that cannot be covered (blank, comment, etc.). Only present on line stats." + }, + "total": {"type": "integer", "minimum": 0, "description": "Executable lines: `covered + missed`. Does not include `omitted`."}, + "percent": {"type": "number"}, + "strength": {"type": "number", "description": "Average number of executions across covered lines (hits per covered line)."} + } + }, + "coverage_statistic": { + "type": "object", + "required": ["covered", "missed", "total", "percent", "strength"], + "additionalProperties": false, + "properties": { + "covered": {"type": "integer", "minimum": 0}, + "missed": {"type": "integer", "minimum": 0}, + "total": {"type": "integer", "minimum": 0, "description": "Total branches or methods: `covered + missed`."}, + "percent": {"type": "number"}, + "strength": {"type": "number", "description": "Average number of executions across covered branches or methods (hits per covered item)."} + } + }, + "line_coverage": { + "description": "Coverage value for a source line in the per-file `lines` array. Integer hit count, `null` for a non-executable line (blank or comment), or the literal string `\"ignored\"` for a line inside a simplecov:disable / :nocov: region.", + "oneOf": [ + {"type": "integer", "minimum": 0}, + {"type": "null"}, + {"type": "string", "const": "ignored"} + ] + }, + "branch_method_coverage": { + "description": "Coverage value for a branch or method `coverage` field. Integer hit count, or the literal string `\"ignored\"` for code inside a simplecov:disable / :nocov: region. Unlike line coverage, `null` never occurs: every branch and method maps to an executable construct.", + "oneOf": [ + {"type": "integer", "minimum": 0}, + {"type": "string", "const": "ignored"} + ] + }, + "expected_actual": { + "type": "object", + "required": ["expected", "actual"], + "additionalProperties": false, + "properties": { + "expected": {"type": "number"}, + "actual": {"type": "number"} + } + }, + "maximum_actual": { + "type": "object", + "required": ["maximum", "actual"], + "additionalProperties": false, + "properties": { + "maximum": {"type": "number"}, + "actual": {"type": "number"} + } + } + } +} diff --git a/simplecov.gemspec b/simplecov.gemspec index 9db7c935..82c96d6f 100644 --- a/simplecov.gemspec +++ b/simplecov.gemspec @@ -52,7 +52,7 @@ Gem::Specification.new do |gem| gem.required_ruby_version = ">= 3.1" - gem.files = Dir["lib/**/*.*", "exe/*", "LICENSE", "CHANGELOG.md", "README.md", "doc/*"] + gem.files = Dir["{lib,schemas}/**/*.*", "exe/*", "LICENSE", "CHANGELOG.md", "README.md", "doc/*"] gem.bindir = "exe" gem.executables = ["simplecov"] gem.require_paths = ["lib"] diff --git a/spec/configuration_spec.rb b/spec/configuration_spec.rb index 933b468b..8411f80b 100644 --- a/spec/configuration_spec.rb +++ b/spec/configuration_spec.rb @@ -158,6 +158,23 @@ end end + describe "#source_in_json" do + it "defaults to true" do + expect(config.source_in_json).to be true + end + + it "reads back the assigned value" do + config.source_in_json false + expect(config.source_in_json).to be false + end + + it "round-trips a true assignment" do + config.source_in_json false + config.source_in_json true + expect(config.source_in_json).to be true + end + end + describe "#print_error_status (deprecated)" do it "warns when read and still returns the value" do config.print_error_status = false diff --git a/spec/fixtures/json/sample.json b/spec/fixtures/json/sample.json index 7d2da258..9c151a29 100644 --- a/spec/fixtures/json/sample.json +++ b/spec/fixtures/json/sample.json @@ -1,10 +1,14 @@ { + "$schema": "https://raw.githubusercontent.com/simplecov-ruby/simplecov/main/schemas/coverage-v1.0.schema.json", "meta": { + "schema_version": "1.0", "simplecov_version": "0.22.0", "command_name": "STUB_COMMAND_NAME", "project_name": "STUB_PROJECT_NAME", "timestamp": "2024-01-01T00:00:00.000+00:00", "root": "/STUB_WORKING_DIRECTORY", + "commit": "STUB_COMMIT_SHA", + "line_coverage": true, "branch_coverage": false, "method_coverage": false }, @@ -77,6 +81,7 @@ "lines_covered_percent": 90.0, "covered_lines": 9, "missed_lines": 1, + "omitted_lines": 10, "total_lines": 10 } }, diff --git a/spec/fixtures/json/sample_groups.json b/spec/fixtures/json/sample_groups.json index 6b395cec..e062ab44 100644 --- a/spec/fixtures/json/sample_groups.json +++ b/spec/fixtures/json/sample_groups.json @@ -1,10 +1,14 @@ { + "$schema": "https://raw.githubusercontent.com/simplecov-ruby/simplecov/main/schemas/coverage-v1.0.schema.json", "meta": { + "schema_version": "1.0", "simplecov_version": "0.22.0", "command_name": "STUB_COMMAND_NAME", "project_name": "STUB_PROJECT_NAME", "timestamp": "2024-01-01T00:00:00.000+00:00", "root": "/STUB_WORKING_DIRECTORY", + "commit": "STUB_COMMIT_SHA", + "line_coverage": true, "branch_coverage": false, "method_coverage": false }, @@ -77,6 +81,7 @@ "lines_covered_percent": 90.0, "covered_lines": 9, "missed_lines": 1, + "omitted_lines": 10, "total_lines": 10 } }, diff --git a/spec/fixtures/json/sample_with_branch.json b/spec/fixtures/json/sample_with_branch.json index a5757717..cf551574 100644 --- a/spec/fixtures/json/sample_with_branch.json +++ b/spec/fixtures/json/sample_with_branch.json @@ -1,10 +1,14 @@ { + "$schema": "https://raw.githubusercontent.com/simplecov-ruby/simplecov/main/schemas/coverage-v1.0.schema.json", "meta": { + "schema_version": "1.0", "simplecov_version": "0.22.0", "command_name": "STUB_COMMAND_NAME", "project_name": "STUB_PROJECT_NAME", "timestamp": "2024-01-01T00:00:00.000+00:00", "root": "/STUB_WORKING_DIRECTORY", + "commit": "STUB_COMMIT_SHA", + "line_coverage": true, "branch_coverage": true, "method_coverage": false }, @@ -84,6 +88,7 @@ "lines_covered_percent": 90.0, "covered_lines": 9, "missed_lines": 1, + "omitted_lines": 10, "total_lines": 10, "branches": [ { diff --git a/spec/fixtures/json/sample_with_method.json b/spec/fixtures/json/sample_with_method.json index a34b5246..5147c743 100644 --- a/spec/fixtures/json/sample_with_method.json +++ b/spec/fixtures/json/sample_with_method.json @@ -1,10 +1,14 @@ { + "$schema": "https://raw.githubusercontent.com/simplecov-ruby/simplecov/main/schemas/coverage-v1.0.schema.json", "meta": { + "schema_version": "1.0", "simplecov_version": "0.22.0", "command_name": "STUB_COMMAND_NAME", "project_name": "STUB_PROJECT_NAME", "timestamp": "2024-01-01T00:00:00.000+00:00", "root": "/STUB_WORKING_DIRECTORY", + "commit": "STUB_COMMIT_SHA", + "line_coverage": true, "branch_coverage": false, "method_coverage": true }, @@ -84,6 +88,7 @@ "lines_covered_percent": 90.0, "covered_lines": 9, "missed_lines": 1, + "omitted_lines": 10, "total_lines": 10, "methods": [ { diff --git a/spec/formatter/coverage_schema_spec.rb b/spec/formatter/coverage_schema_spec.rb new file mode 100644 index 00000000..a7820fd1 --- /dev/null +++ b/spec/formatter/coverage_schema_spec.rb @@ -0,0 +1,215 @@ +# frozen_string_literal: true + +require "helper" +require "json" +require "json_schemer" + +# Validates that every shape JSONFormatter is expected to emit conforms to +# the contract in schemas/coverage-v1.0.schema.json. The schema is +# published as the public contract for coverage.json consumers, this +# spec catches drift between code and schema at test time so neither +# can rot independently. +describe "coverage.json schema" do # rubocop:disable RSpec/DescribeClass + let(:schema_path) { File.expand_path("../../schemas/coverage-v1.0.schema.json", __dir__) } + let(:alias_path) { File.expand_path("../../schemas/coverage.schema.json", __dir__) } + let(:schema_doc) { JSON.parse(File.read(schema_path)) } + let(:alias_doc) { JSON.parse(File.read(alias_path)) } + let(:schemer) { JSONSchemer.schema(schema_doc) } + + def validate_against_schema(document) + schemer.validate(document).map { |e| "#{e['data_pointer']}: #{e['error']}" } + end + + it "is itself a valid JSON Schema (2020-12)" do + expect(JSONSchemer.draft202012.validate(schema_doc).to_a).to be_empty + end + + it "declares 2020-12 as its meta-schema" do + expect(schema_doc.fetch("$schema")).to eq("https://json-schema.org/draft/2020-12/schema") + end + + # The unversioned alias is a convenience pointer to the latest version. + # Allow only the metadata triple to diverge (title, description, $id) + # so the canonical and alias can't drift in shape unnoticed. + it "ships an unversioned alias that mirrors the latest versioned canonical" do + alias_metadata_keys = %w[$id title description] + expect(alias_doc.except(*alias_metadata_keys)).to eq(schema_doc.except(*alias_metadata_keys)) + end + + it "pins the versioned canonical's $id to a versioned URL" do + expect(schema_doc.fetch("$id")).to match(%r{/schemas/coverage-v\d+\.\d+\.schema\.json\z}) + end + + # Guards against drift: a document claiming a foreign schema_version + # must not quietly validate against whatever the current contract is. + # "0.0" is used because the schema started at 1.0 and only bumps + # upward — a pre-1.0 version string will never match any future + # `const`, so this test stays valid across version bumps. + it "rejects a document claiming a foreign schema_version" do + document = JSON.parse(File.read(source_fixture("json/sample.json"))) + document["meta"]["schema_version"] = "0.0" + errors = validate_against_schema(document) + expect(errors).not_to be_empty + end + + context "with shipped fixtures" do + %w[sample sample_with_branch sample_with_method sample_groups].each do |basename| + it "validates #{basename}.json" do + document = JSON.parse(File.read(source_fixture("json/#{basename}.json"))) + errors = validate_against_schema(document) + expect(errors).to be_empty, "schema validation failed:\n #{errors.join("\n ")}" + end + end + end + + context "with fresh JSONFormatter output" do + let(:fixed_time) { Time.new(2024, 1, 1, 0, 0, 0, "+00:00") } + let(:formatter) { SimpleCov::Formatter::JSONFormatter.new(silent: true) } + + before do + FileUtils.rm_f("tmp/coverage/coverage.json") + SimpleCov.process_start_time = Time.now + end + + after { SimpleCov.process_start_time = nil } + + def emit(result) + result.created_at = Time.new(2024, 1, 1, 0, 0, 0, "+00:00") + formatter.format(result) + JSON.parse(File.read("tmp/coverage/coverage.json")) + end + + it "validates a minimal line-coverage result" do + result = SimpleCov::Result.new({source_fixture("json/sample.rb") => {"lines" => [1, 0, nil]}}) + errors = validate_against_schema(emit(result)) + expect(errors).to be_empty, errors.join("\n") + end + + it "validates a result emitted with SimpleCov.source_in_json off (no per-file `source` array)" do + allow(SimpleCov).to receive(:source_in_json).and_return(false) + result = SimpleCov::Result.new({source_fixture("json/sample.rb") => {"lines" => [1, 0, nil]}}) + document = emit(result) + # Sanity: the `source` field really is omitted, not just empty. + expect(document.fetch("coverage").values.first).not_to have_key("source") + errors = validate_against_schema(document) + expect(errors).to be_empty, errors.join("\n") + end + + it "validates a result with branch coverage enabled" do + allow(SimpleCov).to receive(:branch_coverage?).and_return(true) + result = SimpleCov::Result.new({ + source_fixture("json/sample.rb") => { + "lines" => [nil, 1, 1, 1, 1, nil, nil, 1, 1, nil, nil, + 1, 1, 0, nil, 1, nil, nil, nil, nil, 1, 0, nil, nil, nil], + "branches" => { + [:if, 0, 13, 4, 17, 7] => { + [:then, 1, 14, 6, 14, 10] => 0, + [:else, 2, 16, 6, 16, 10] => 1 + } + } + } + }) + errors = validate_against_schema(emit(result)) + expect(errors).to be_empty, errors.join("\n") + end + + # `disable_coverage :line` is a real configuration. The formatter + # then drops all line keys from `total`, per-file payloads, and + # group totals. Guard against the schema drifting back into + # requiring those keys. + it "validates a result emitted with line coverage disabled" do + allow(SimpleCov).to receive(:coverage_criterion_enabled?).and_call_original + allow(SimpleCov).to receive(:coverage_criterion_enabled?).with(:line).and_return(false) + allow(SimpleCov).to receive(:coverage_criterion_enabled?).with(:oneshot_line).and_return(false) + allow(SimpleCov).to receive(:branch_coverage?).and_return(true) + result = SimpleCov::Result.new({ + source_fixture("json/sample.rb") => { + "lines" => [nil, 1, 1, 0], + "branches" => {[:if, 0, 1, 0, 4, 0] => { + [:then, 1, 2, 2, 2, 6] => 1, + [:else, 2, 3, 2, 3, 6] => 0 + }} + } + }) + document = emit(result) + expect(document.fetch("meta").fetch("line_coverage")).to be(false) + expect(document.fetch("total")).not_to have_key("lines") + expect(document.fetch("coverage").values.first).not_to have_key("lines") + errors = validate_against_schema(document) + expect(errors).to be_empty, errors.join("\n") + end + + it "validates a result with method coverage enabled" do + allow(SimpleCov).to receive(:method_coverage?).and_return(true) + result = SimpleCov::Result.new({ + source_fixture("json/sample.rb") => { + "lines" => [nil, 1, 1, 1, 1, nil, nil, 1, 1, nil, nil, + 1, 1, 0, nil, 1, nil, nil, nil, nil, 1, 0, nil, nil, nil], + "methods" => { + ["Foo", :initialize, 3, 2, 6, 5] => 1, + ["Foo", :bar, 8, 2, 10, 5] => 1, + ["Foo", :foo, 12, 2, 18, 5] => 1 + } + } + }) + errors = validate_against_schema(emit(result)) + expect(errors).to be_empty, errors.join("\n") + end + + # The `errors` object is part of the public contract too; validate each + # violation shape end-to-end so the schema can't drift away from what + # the formatter actually emits when thresholds trip. + context "with errors populated" do + let(:result) do + SimpleCov::Result.new({source_fixture("json/sample.rb") => {"lines" => [1, 0, 1]}}) + end + + it "validates a minimum_coverage violation" do + allow(SimpleCov).to receive(:minimum_coverage).and_return(line: 95) + document = emit(result) + expect(document.dig("errors", "minimum_coverage")).not_to be_nil + errors = validate_against_schema(document) + expect(errors).to be_empty, errors.join("\n") + end + + it "validates a minimum_coverage_by_file violation" do + allow(SimpleCov).to receive(:minimum_coverage_by_file).and_return(line: 95) + document = emit(result) + expect(document.dig("errors", "minimum_coverage_by_file")).not_to be_nil + errors = validate_against_schema(document) + expect(errors).to be_empty, errors.join("\n") + end + + it "validates a minimum_coverage_by_group violation" do + line_stats = SimpleCov::CoverageStatistics.new(covered: 7, missed: 3) + mock_file_list = instance_double(SimpleCov::FileList, + coverage_statistics: {line: line_stats}, + map: [source_fixture("json/sample.rb")]) + allow(result).to receive(:groups).and_return("Models" => mock_file_list) + allow(SimpleCov).to receive(:minimum_coverage_by_group).and_return("Models" => {line: 80}) + + document = emit(result) + expect(document.dig("errors", "minimum_coverage_by_group")).not_to be_nil + errors = validate_against_schema(document) + expect(errors).to be_empty, errors.join("\n") + end + + it "validates a maximum_coverage violation" do + allow(SimpleCov).to receive(:maximum_coverage).and_return(line: 50) + document = emit(result) + expect(document.dig("errors", "maximum_coverage")).not_to be_nil + errors = validate_against_schema(document) + expect(errors).to be_empty, errors.join("\n") + end + + it "validates a maximum_coverage_drop violation" do + allow(SimpleCov).to receive(:maximum_coverage_drop).and_return(line: 2) + allow(SimpleCov::LastRun).to receive(:read).and_return({result: {line: 95.0}}) + document = emit(result) + expect(document.dig("errors", "maximum_coverage_drop")).not_to be_nil + errors = validate_against_schema(document) + expect(errors).to be_empty, errors.join("\n") + end + end + end +end diff --git a/spec/formatter/html_formatter_spec.rb b/spec/formatter/html_formatter_spec.rb index d861b12b..a505c21d 100644 --- a/spec/formatter/html_formatter_spec.rb +++ b/spec/formatter/html_formatter_spec.rb @@ -119,6 +119,24 @@ def coverage_data(dir = coverage_dir) expect(file_data["source"]).not_to be_empty end + # The client-side viewer renders source from the embedded array, + # so coverage_data.js must keep `source` regardless of the + # `source_in_json` setting. The setting only controls the side-file + # coverage.json, which downstream tools that read source from disk + # can opt into shrinking. + it "keeps source in coverage_data.js even when SimpleCov.source_in_json is false" do + allow(SimpleCov).to receive(:source_in_json).and_return(false) + formatter.format(make_result) + expect(coverage_data["coverage"].values.first).to include("source") + end + + it "drops source from coverage.json when SimpleCov.source_in_json is false" do + allow(SimpleCov).to receive(:source_in_json).and_return(false) + formatter.format(make_result) + external = JSON.parse(File.read(File.join(coverage_dir, "coverage.json"))) + expect(external["coverage"].values.first).not_to include("source") + end + it "embeds the metadata section in the coverage payload" do meta = coverage_data["meta"] diff --git a/spec/formatter/json_formatter_spec.rb b/spec/formatter/json_formatter_spec.rb index e918e86b..2cc268ee 100644 --- a/spec/formatter/json_formatter_spec.rb +++ b/spec/formatter/json_formatter_spec.rb @@ -2,6 +2,7 @@ require "helper" require "fileutils" +require "open3" STUB_WORKING_DIRECTORY = "STUB_WORKING_DIRECTORY" @@ -9,6 +10,8 @@ STUB_PROJECT_NAME = "STUB_PROJECT_NAME" +STUB_COMMIT = "1234567890abcdef1234567890abcdef12345678" + RSpec.describe SimpleCov::Formatter::JSONFormatter do subject(:formatter) { described_class.new(silent: true) } @@ -29,6 +32,8 @@ before do FileUtils.rm_f("tmp/coverage/coverage.json") SimpleCov.process_start_time = Time.now + allow(Open3).to receive(:capture2e) + .and_return(["#{STUB_COMMIT}\n", instance_double(Process::Status, success?: true)]) end # Outside SimpleCov.start, process_start_time is nil. Anchor it so the @@ -150,6 +155,29 @@ end end + context "with the source_in_json toggle" do + let(:file_entry) { json_output.fetch("coverage").fetch(project_fixture_filename("json/sample.rb")) } + + it "includes the source array by default" do + formatter.format(result) + expect(file_entry).to have_key("source") + expect(file_entry["source"]).to be_an(Array) + expect(file_entry["source"]).not_to be_empty + end + + it "omits the source array when SimpleCov.source_in_json is false" do + allow(SimpleCov).to receive(:source_in_json).and_return(false) + formatter.format(result) + expect(file_entry).not_to have_key("source") + end + + it "still includes line / branch / method sections when source is omitted" do + allow(SimpleCov).to receive(:source_in_json).and_return(false) + formatter.format(result) + expect(file_entry).to include("lines", "covered_lines", "missed_lines", "total_lines", "lines_covered_percent") + end + end + context "with branch coverage" do let(:original_lines) do [nil, 1, 1, 1, 1, nil, nil, 1, 1, @@ -275,7 +303,7 @@ errors = json_output.fetch("errors") expect(errors).to eq( "minimum_coverage_by_file" => { - "lines" => {project_fixture_filename("json/sample.rb") => {"expected" => 95, "actual" => 90.0}} + project_fixture_filename("json/sample.rb") => {"lines" => {"expected" => 95, "actual" => 90.0}} } ) end @@ -307,7 +335,7 @@ errors = json_output.fetch("errors") expect(errors).to eq( "minimum_coverage_by_file" => { - "branches" => {project_fixture_filename("json/sample.rb") => {"expected" => 75, "actual" => 50.0}} + project_fixture_filename("json/sample.rb") => {"branches" => {"expected" => 75, "actual" => 50.0}} } ) end @@ -337,7 +365,7 @@ errors = json_output.fetch("errors") expect(errors).to eq( "minimum_coverage_by_file" => { - "lines" => {project_fixture_filename("json/sample.rb") => {"expected" => 100, "actual" => 90.0}} + project_fixture_filename("json/sample.rb") => {"lines" => {"expected" => 100, "actual" => 90.0}} } ) end @@ -555,6 +583,26 @@ end end + describe "meta.commit" do + it "records the git commit SHA of the project HEAD" do + formatter.format(result) + expect(json_output.fetch("meta").fetch("commit")).to eq(STUB_COMMIT) + end + + it "is null when the project is not a git checkout" do + allow(Open3).to receive(:capture2e) + .and_return(["fatal: not a git repository", instance_double(Process::Status, success?: false)]) + formatter.format(result) + expect(json_output.fetch("meta").fetch("commit")).to be_nil + end + + it "is null when git is not available" do + allow(Open3).to receive(:capture2e).and_raise(Errno::ENOENT) + formatter.format(result) + expect(json_output.fetch("meta").fetch("commit")).to be_nil + end + end + def enable_branch_coverage allow(SimpleCov).to receive(:branch_coverage?).and_return(true) end @@ -584,5 +632,6 @@ def replace_stubs(file) .gsub("\"/#{STUB_WORKING_DIRECTORY}\"", "\"#{current_working_directory}\"") .gsub("\"#{STUB_COMMAND_NAME}\"", "\"#{SimpleCov.command_name}\"") .gsub("\"#{STUB_PROJECT_NAME}\"", "\"#{SimpleCov.project_name}\"") + .gsub("\"STUB_COMMIT_SHA\"", "\"#{STUB_COMMIT}\"") end end