Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- A later run under the same command name keeps the test map an earlier run recorded for the tests it did not run. Rerunning one test file, or running `simplecov affected --run`, replaced the whole map, so a later `simplecov affected` selected nothing for a change that only the other tests covered, and exited 0. A test recorded again replaces its earlier lines. An earlier test whose file is gone is dropped, and one that touched a source file changed since is credited with every line of that file, since its recorded line numbers no longer hold. See [#1316].
- The `strict` profile no longer makes JRuby warn that branch and method coverage are not supported. It enabled both on every engine, so `Coverage.start` was asked for criteria JRuby cannot measure. It now enables only the criteria the runtime supports and pins a 100% threshold for each of those.
- `track_tests` records an RSpec shared example under RSpec's id for it in the spec file that includes its group, such as `spec/user_spec.rb[1:2:1]`. `rspec` takes this id to run the example again. Before, it recorded the file of the shared group, such as `spec/support/shared_examples.rb:2`, which is not a test file. Then `simplecov affected` selected that file, and not the spec files that include the group. With `granularity: :file`, those spec files recorded nothing for their shared examples. A test map recorded before this change keeps the old ids, so record it again from scratch.
- `track_tests` credits what a spec file's `before(:all)` hooks run to that spec file (`spec/user_spec.rb`), in a nested group too, whatever order the hooks were added in. RSpec runs those hooks once, before the group's first example starts, so their lines went to the spec file that ran before it, or to no test. `simplecov affected` then picked that earlier spec file and skipped the one that needs the hook.

## [1.3.2] - 2026-09-30

Expand Down
2 changes: 2 additions & 0 deletions docs/Configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -417,6 +417,8 @@ spec file triggers with `require "rails_helper"` belongs to every test rather th
Test files are recognized by name (`*_spec.rb`, `*_test.rb`, and `test_*.rb` other than `test_helper.rb`). JRuby
cannot report a file as it loads, so there a test file's loading belongs to no test.

What an RSpec group's `before(:all)` hooks run, in a nested group as well, is credited to the group's spec file.

A file a test loads for the first time is treated the same way: its class bodies, constants, and `def` lines run once
on behalf of every later test, so the loading test is credited only with the method body lines it ran. Telling them
apart takes Prism, so a Ruby that cannot load it, and any file that is not Ruby source (a template, say), credits the
Expand Down
13 changes: 10 additions & 3 deletions lib/simplecov/test_tracker.rb
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,12 @@ def suite_starting
open_outside_tests(nil)
end

# Credits the test file at `path` with its `before(:all)` hooks, which RSpec
# runs outside any test, instead of the test that ran before them.
def test_file_starting(path)
open_outside_tests(path, loading: false)
end

def poisoned?
@poisoned
end
Expand Down Expand Up @@ -147,7 +153,8 @@ def settle(id, nested_opening)
# process loaded, branch and method tables included), so recording settles
# at segment boundaries: consecutive tracks with one context id share one
# open segment and pay nothing in between. In-root code that runs between
# two segments is attributed to the later one, the documented price.
# two tracks goes to the earlier one, unless a framework hook marks a
# boundary first.
def open_segment(id, loading: false)
return if same_segment?(id, loading)

Expand All @@ -166,8 +173,8 @@ def same_segment?(id, loading)

# A nil id is code no test owns, whose segment records nothing, so only a
# named one's `loading` matters.
def open_outside_tests(id)
open_segment(id, loading: true) if @depth.zero? && !@poisoned
def open_outside_tests(id, loading: true)
open_segment(id, loading: loading) if @depth.zero? && !@poisoned
end

def close_segment(closing)
Expand Down
25 changes: 22 additions & 3 deletions lib/simplecov/test_tracker/framework_hooks.rb
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,24 @@ def install_rspec_hook(rspec = rspec_module)
config.around do |example|
SimpleCov.track_test(TestTracker.rspec_example_id(example)) { example.run }
end
config.prepend_before(:suite) { SimpleCov.test_tracker&.suite_starting }
config.prepend_before(:suite) do
SimpleCov.test_tracker&.suite_starting
# RSpec reports a group starting ahead of all its before(:all)
# hooks, whatever order they were added in. Registered here
# because asking for the reporter any earlier sets up the default
# formatter before the suite has chosen its own.
config.reporter.register_listener(TestTracker, :example_group_started)
end
end
end

# Called by RSpec's reporter as each group starts, nested ones too, so
# their before(:all) hooks go to their spec file rather than to the test
# before them.
def example_group_started(notification)
SimpleCov.test_tracker&.test_file_starting(rspec_path(notification.group.metadata.fetch(:rerun_file_path)))
end

# The RSpec this process has, if it has one. A Minitest-only suite has
# none, and no example can make this process into one: hiding the RSpec
# constant takes down the very runner asking the question.
Expand Down Expand Up @@ -97,8 +111,13 @@ def install_minitest_hook_when_loaded(root = Object)
# (`spec/a_spec.rb[1:2:1]`), which is its own and reruns it.
def rspec_example_id(example)
metadata = example.metadata
id = metadata.fetch(:shared_group_inclusion_backtrace).empty? ? metadata.fetch(:location) : example.id
id.delete_prefix("./")
rspec_path(metadata.fetch(:shared_group_inclusion_backtrace).empty? ? metadata.fetch(:location) : example.id)
end

# Relative the way RSpec prints it (`./spec/a_spec.rb` becomes
# `spec/a_spec.rb`).
def rspec_path(path)
path.delete_prefix("./")
end

# An autoload declaration fires `const_added` without the module existing
Expand Down
8 changes: 7 additions & 1 deletion sig/internal/simplecov/test_tracker.rbs
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ module SimpleCov

def suite_starting: () -> void

def test_file_starting: (String path) -> void

def poisoned?: () -> bool

def dropped_reason: () -> String?
Expand All @@ -54,6 +56,10 @@ module SimpleCov

def self.rspec_example_id: (untyped example) -> String

def self.example_group_started: (untyped notification) -> void

def self.rspec_path: (String path) -> String

def self.minitest_test_id: (untyped test) -> String

def self.definition_site: (untyped test) -> Array[untyped]?
Expand Down Expand Up @@ -92,7 +98,7 @@ module SimpleCov

def same_segment?: (String? id, bool loading) -> bool

def open_outside_tests: (String? id) -> void
def open_outside_tests: (String? id, ?loading: bool) -> void

def close_segment: (Hash[String, untyped] closing) -> void

Expand Down
56 changes: 56 additions & 0 deletions spec/sandbox/track_tests_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,50 @@ def angle
include_examples "a four-sided shape"
end
RUBY
write_file("lib/setup_once.rb", <<~RUBY)
module SetupOnce
def self.call
:set_up
end
end
RUBY
write_file("spec/setting_up_spec.rb", <<~RUBY)
require "spec_helper"
require_relative "../lib/setup_once"

RSpec.describe "SettingUp" do
before(:all) { SetupOnce.call }

it("runs after its group's setup") { expect(1).to eq(1) }
end
RUBY
write_file("lib/setup_inside.rb", <<~RUBY)
module SetupInside
def self.first
:set_up
end

def self.nested
:set_up
end
end
RUBY
write_file("spec/setting_up_inside_spec.rb", <<~RUBY)
require "spec_helper"
require_relative "../lib/setup_inside"

RSpec.describe "SettingUpInside" do
prepend_before(:all) { SetupInside.first }

it("runs after the group's setup") { expect(1).to eq(1) }

describe "a nested group" do
before(:all) { SetupInside.nested }

it("runs after the nested group's setup") { expect(1).to eq(1) }
end
end
RUBY
end

let!(:result) do
Expand Down Expand Up @@ -117,6 +161,18 @@ def angle
.to eq([%w[spec/square_spec.rb[1:1:1] spec/square_spec.rb[2:1]], %w[spec/square_spec.rb[1:1:2] spec/square_spec.rb[2:2]]])
end

it "credits a spec file with what its before(:all) hooks ran, not the spec file before it" do
expect(stored_contexts.covering(File.join(sandbox_dir, "lib/setup_once.rb"), 3))
.to eq(["spec/setting_up_spec.rb"])
end

it "credits a spec file with what its prepended and nested before(:all) hooks ran" do
setup_inside = File.join(sandbox_dir, "lib/setup_inside.rb")

expect([3, 7].map { |line| stored_contexts.covering(setup_inside, line) })
.to eq([["spec/setting_up_inside_spec.rb"]] * 2)
end

it "credits a spec file with what it ran while it loaded" do
expect(stored_contexts.covering(File.join(sandbox_dir, "lib/faked_project.rb"), 5))
.to include("spec/zz_defining_spec.rb")
Expand Down
94 changes: 89 additions & 5 deletions spec/simple_cov/test_tracker_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -617,6 +617,82 @@ def path(relative) = File.join(root, relative)
end
end

describe "test files starting" do
context "with a test file's group setup running after another file's test" do
before do
allow(Coverage).to receive(:peek_result).and_return(
{lib_file => {lines: [0, 0]}},
{lib_file => {lines: [1, 0]}},
{lib_file => {lines: [1, 1]}},
{lib_file => {lines: [1, 1]}}
)

tracker.track("spec/a_spec.rb:3") { :ran }
tracker.test_file_starting("spec/b_spec.rb")
tracker.track("spec/b_spec.rb:5") { :ran }
end

it "credits the setup to the test file that starts" do
expect(tracker.recorded_map.covering(lib_file, 2)).to eq(["spec/b_spec.rb"])
end

it "leaves the earlier test with what it ran" do
expect(tracker.recorded_map.covering(lib_file, 1)).to eq(["spec/a_spec.rb:3"])
end
end

it "leaves a test file starting inside a test to that test" do
stub_peeks({lib_file => {lines: [0]}}, {lib_file => {lines: [1]}})
tracker.track("spec/a_spec.rb:3") { tracker.test_file_starting("spec/b_spec.rb") }

expect(tracker.recorded_map.contexts).to eq(["spec/a_spec.rb:3"])
end

context "with file granularity" do
subject(:tracker) do
described_class.new(root_regex: /\A#{Regexp.escape(project_root + File::SEPARATOR)}/i, granularity: :file)
end

before do
allow(Coverage).to receive(:peek_result).and_return({lib_file => {lines: [0]}}, {lib_file => {lines: [1]}})
tracker.test_file_starting("spec/b_spec.rb")
tracker.track("spec/b_spec.rb:5") { :ran }
end

it "keeps the file's tests in the segment its setup opened" do
tracker.recorded_map

expect(Coverage).to have_received(:peek_result).twice
end

it "credits setup and tests to the file" do
expect(tracker.recorded_map.covering(lib_file, 1)).to eq(["spec/b_spec.rb"])
end
end
end

describe ".example_group_started" do
let(:live_tracker) { instance_double(described_class, test_file_starting: nil) }
let(:notification) do
group = class_double(RSpec::Core::ExampleGroup, metadata: {rerun_file_path: "./spec/b_spec.rb"})
RSpec::Core::Notifications::GroupNotification.new(group)
end

before { allow(SimpleCov).to receive(:test_tracker).and_return(live_tracker) }

it "starts the group's spec file on the live tracker" do
described_class.example_group_started(notification)

expect(live_tracker).to have_received(:test_file_starting).with("spec/b_spec.rb")
end

it "starts nothing when no tracker is live" do
allow(SimpleCov).to receive(:test_tracker).and_return(nil)

expect { described_class.example_group_started(notification) }.not_to raise_error
end
end

describe "file granularity", mutant_expression: ["SimpleCov::TestTracker*",
"SimpleCov::TestTracker#track"] do
subject(:tracker) do
Expand Down Expand Up @@ -858,11 +934,12 @@ def poison
end

let(:around_blocks) { [] }
let(:suite_blocks) { [] }
let(:prepended_blocks) { Hash.new { |blocks, scope| blocks[scope] = [] } }
let(:fake_config) do
config = instance_double(RSpec::Core::Configuration)
reporter = instance_double(RSpec::Core::Reporter, register_listener: true)
config = instance_double(RSpec::Core::Configuration, reporter: reporter)
allow(config).to receive(:around) { |&block| around_blocks << block }
allow(config).to receive(:prepend_before).with(:suite) { |&block| suite_blocks << block }
allow(config).to receive(:prepend_before) { |scope, &block| prepended_blocks[scope] << block }
config
end
let(:fake_rspec) do
Expand Down Expand Up @@ -916,7 +993,7 @@ def poison
live_tracker = instance_double(described_class, suite_starting: nil)
allow(SimpleCov).to receive(:test_tracker).and_return(live_tracker)
described_class.install_rspec_hook(fake_rspec)
suite_blocks.first.call
prepended_blocks[:suite].first.call

expect(live_tracker).to have_received(:suite_starting)
end
Expand All @@ -925,7 +1002,14 @@ def poison
described_class.install_rspec_hook(fake_rspec)
allow(SimpleCov).to receive(:test_tracker).and_return(nil)

expect { suite_blocks.first.call }.not_to raise_error
expect { prepended_blocks[:suite].first.call }.not_to raise_error
end

it "listens for each group starting once the suite starts" do
described_class.install_rspec_hook(fake_rspec)
prepended_blocks[:suite].first.call

expect(fake_config.reporter).to have_received(:register_listener).with(described_class, :example_group_started)
end

context "when installed twice in one process" do
Expand Down
Loading