Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions HISTORY.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# RELEASE HISTORY

## Unreleased

* Bring the HTML reporter in from rubytest-html, rewritten for the recorder:
`rubytest -f html`. It escapes what it writes, reports skipped tests and
cases, and shows captured output with each test.
* Bring the Rake task in from rubytest-rake: `require 'rubytest/rake'` for
`Test::Rake::TestTask`. The task now fails when the run does (it always
exited 0), runs only the default globs that match a file, and lets `TEST`
replace the task's files for one invocation.

## 0.10.0 / 2026-11-17

Results now go through a recorder to the reporters, and the reporters that
Expand Down
33 changes: 29 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,10 @@ test), `outline` (nested cases), `progress` (running count and timing), `tap`
(TAP version 13), and `tapy` and `tapj` (TAP-Y and TAP-J, the structured
streams that [tapout](https://github.com/rubyworks/tapout) reads). Select one
with `rubytest -f FORMAT`; `-y` and `-j` are short for `-f tapy` and `-f tapj`.
Formats load only when selected. The TAP formats capture test output: TAP
emits it as diagnostics, and TAP-Y and TAP-J record it with each test.
`html` writes the whole run as one self-contained HTML document
(`rubytest -f html > report.html`). Formats load only when selected. The TAP
and HTML formats capture test output: TAP emits it as diagnostics, and TAP-Y,
TAP-J and HTML record it with each test.


## Installation
Expand Down Expand Up @@ -122,8 +124,31 @@ named profile, use the `-p/--profile` option:
The configuration file can also be at `config/test.rb`, `Testfile`,
or `.test` (with optional `.rb` extension).

If you are using Rake, shelling out to `rubytest` keeps your test
environment pristine:
### Via Rake

Rubytest has a Rake task. In your Rakefile:

require 'rubytest/rake'

Test::Rake::TestTask.new do |run|
run.loadpath 'lib'
run.test_files << 'test/*_test.rb'
end

The block takes the same settings as `Test.configure`. With no files, the
task runs whichever of `test/**/test_*.rb`, `*_test.rb`, `case_*.rb` and
`*_case.rb` it finds, and with no load path it adds `lib`. The task is named
`test` unless you give it another name (`TestTask.new('spec')`).

To run only some tests for one invocation, set `TEST`:

$ rake test TEST=test/foo_test.rb

Where the platform can fork, the tests run in a child process, so what they
load stays out of Rake's. `rake test` fails if the run does.

A configuration file isn't read by the task. To use one, shell out to
`rubytest` instead:

desc "run tests"
task :test do
Expand Down
24 changes: 24 additions & 0 deletions demo/05_builtin_reporters.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,3 +116,27 @@ The command line has a shortcut for each.
cli.options.parse!([flag])
cli.format.assert == format
end

The HTML format writes one self-contained document once the run has
finished. Names, messages and captured output are escaped.

markup = probe_type.new('markup <b>&</b>') { print '<script>'; true }
html_suite = [Class.new(Array) { def to_s; 'a <case>'; end }.new([markup, failing]),
pending, skipped, skipped_case, broken_case]
success, output, report = render.call('html', html_suite)
success.assert == false

output.lines.first.chomp.assert == '<!DOCTYPE html>'
output.lines.last.chomp.assert == '</html>'
output.scan('<div class="result ').size.assert == report.total
output.scan('<section class="case">').size.assert == output.scan('</section>').size
%w[pass fail error todo skip].each do |status|
output.include?(%[<div class="result #{status}">]).assert == true
end

output.include?('markup &lt;b&gt;&amp;&lt;/b&gt;').assert == true
output.include?('<h2>a &lt;case&gt;</h2>').assert == true
output.include?('<pre class="stdout">&lt;script&gt;</pre>').assert == true
output.include?('<script>').assert == false
output.include?('Assertion: ').assert == true
output.include?('6 tests: 1 passing, 1 failures, 1 errors, 1 pending, 2 skipped.').assert == true
87 changes: 87 additions & 0 deletions demo/06_rake_task.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
## Rake task

`rubytest/rake` provides a task for running tests from a Rakefile.

require 'rubytest/rake'
require 'tmpdir'

project = Dir.mktmpdir('rubytest-rake')
Dir.mkdir(File.join(project, 'test'))

probe = lambda do |label, body|
<<-RUBY
probe = Object.new
probe.define_singleton_method(:call) { #{body} }
probe.define_singleton_method(:to_s) { #{label.inspect} }
$TEST_SUITE << probe
RUBY
end

File.write(File.join(project, 'test', 'good_test.rb'), probe.call('good', 'true'))
File.write(File.join(project, 'bad.rb'), probe.call('bad', "raise Assertion, 'wrong value'"))

status_of = lambda do |task|
Dir.chdir(project) do
begin
task.run
0
rescue SystemExit => exit
exit.status
end
end
end

The block configures the run, as with `Test.configure`. The task is an
ordinary Rake task.

passing = Test::Rake::TestTask.new('demo:pass') do |run|
run.files << 'test/good_test.rb'
run.format = 'test'
end

Rake::Task.task_defined?('demo:pass').assert == true
status_of.call(passing).assert == 0

A run that fails makes the task exit with status 1, so that `rake` fails too.

failing = Test::Rake::TestTask.new('demo:fail') do |run|
run.files << 'bad.rb'
run.format = 'test'
end

status_of.call(failing).assert == 1

Given no files, the task looks under `test/` for files named `test_*.rb`,
`*_test.rb`, `case_*.rb` or `*_case.rb`, and runs whichever of those it
finds. It also puts `lib` on the load path.

defaults = Test::Rake::TestTask.new('demo:defaults') { |run| run.format = 'test' }

status_of.call(defaults).assert == 0
defaults.config.files.assert == ['test/**/*_test.rb']
defaults.config.loadpath.assert == ['lib']

If it finds none, it says so.

empty = Dir.mktmpdir('rubytest-rake-empty')
nothing = Test::Rake::TestTask.new('demo:nothing') { |run| run.format = 'test' }
missing = begin
Dir.chdir(empty) { nothing.run }
nil
rescue ArgumentError => error
error
end

missing.message.assert.start_with?('no test files found')

The `TEST` environment variable selects the tests for one invocation, in
place of the task's own.

previous = ENV['TEST']
begin
ENV['TEST'] = 'bad.rb'
status_of.call(passing).assert == 1
passing.config.files.assert == ['bad.rb']
ensure
ENV['TEST'] = previous
end
144 changes: 144 additions & 0 deletions lib/rubytest/format/html.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
# encoding: UTF-8

module Test::Reporters

# HTML Test Reporter
#
# Writes the run as one self-contained HTML document, once the run has
# finished: a tally, then every result under the heading of its case.
# Test output is captured and shown with the test that wrote it.
#
class Html < Abstract

LABELS = {
:pass => 'PASS',
:fail => 'FAIL',
:error => 'ERROR',
:todo => 'TODO',
:skip => 'SKIP'
}

ESCAPES = {
'&' => '&amp;', '<' => '&lt;', '>' => '&gt;', '"' => '&quot;', "'" => '&#39;'
}

STYLE = <<-CSS
:root { color-scheme: light dark; --pass: #1a7f37; --fail: #cf222e; --todo: #9a6700; --skip: #57606a; --line: #d0d7de; }
@media (prefers-color-scheme: dark) { :root { --pass: #3fb950; --fail: #f85149; --todo: #d29922; --skip: #8b949e; --line: #30363d; } }
body { font-family: system-ui, sans-serif; line-height: 1.5; max-width: 60rem; margin: 2rem auto; padding: 0 1rem; }
h1 { margin-bottom: 0.25rem; }
.tally { font-weight: bold; }
.tally.passed { color: var(--pass); }
.tally.failed { color: var(--fail); }
section { border-left: 2px solid var(--line); padding-left: 1rem; margin: 1rem 0; }
.result { margin: 0.25rem 0; }
.status { display: inline-block; min-width: 3.5rem; font: bold 0.85em ui-monospace, monospace; }
.pass .status { color: var(--pass); }
.fail .status, .error .status { color: var(--fail); }
.todo .status { color: var(--todo); }
.skip .status { color: var(--skip); }
.time, .reason, footer { color: var(--skip); font-size: 0.85em; }
pre { margin: 0.25rem 0 0.5rem 3.5rem; padding: 0.5rem; border: 1px solid var(--line); overflow-x: auto; font-size: 0.85em; }
CSS

#
def initialize(runner, output: $stdout)
super(runner)
@output = output
@body = []
@depth = 0
end

# Test output would otherwise land in the middle of the document.
def capture_output?
true
end

#
def begin_suite(suite)
@body = []
@depth = 0
end

#
def begin_case(test_case)
title, *rest = test_case.to_s.strip.split("\n")
level = [@depth + 2, 6].min
@body << %[<section class="case">]
@body << %[<h#{level}>#{h title}</h#{level}>]
@body << %[<p>#{rest.map{ |line| h(line.strip) }.join('<br>')}</p>] unless rest.empty?
@depth += 1
end

#
def end_case(test_case)
@depth -= 1
@body << %[</section>]
end

# Add a completed result to the document.
def record(result)
label = result.test.to_s.strip.split("\n").first.to_s
label = "#{label} (case)" if result.kind == :case

line = %[<span class="status">#{LABELS.fetch(result.status)}</span> #{h label}]
line << %[ <span class="reason">#{h result.reason}</span>] if result.status == :skip && result.reason
line << %[ <span class="reason">#{h result.exception.message}</span>] if result.status == :todo && result.exception
line << %[ <span class="time">#{'%.5fs' % result.elapsed}</span>]

@body << %[<div class="result #{result.status}">#{line}]
if [:fail, :error].include?(result.status)
result.exceptions.each{ |exception| @body << %[<pre>#{h detail(exception, result.status == :error)}</pre>] }
end
@body << %[<pre class="stdout">#{h result.stdout}</pre>] unless result.stdout.empty?
@body << %[<pre class="stderr">#{h result.stderr}</pre>] unless result.stderr.empty?
@body << %[</div>]
end

# Write the document.
def finish(summary)
outcome = summary.success? ? 'passed' : 'failed'
tally = if summary.total.zero?
'No tests were run.'
else
counts = ::Test::Result::STATUSES.map{ |status| "#{summary.counts[status]} #{TITLES[status]}" }
"#{summary.total} tests: #{counts.join(', ')}."
end

@output.puts '<!DOCTYPE html>'
@output.puts '<html lang="en">'
@output.puts '<head>'
@output.puts '<meta charset="utf-8">'
@output.puts '<meta name="viewport" content="width=device-width, initial-scale=1">'
@output.puts '<title>Test Report</title>'
@output.puts "<style>\n#{STYLE.gsub(/^ /, '')}</style>"
@output.puts '</head>'
@output.puts '<body>'
@output.puts '<h1>Test Report</h1>'
@output.puts %[<p class="tally #{outcome}">#{h tally}</p>]
@output.puts @body.join("\n") unless @body.empty?
@output.puts %[<footer>Finished in #{'%.5fs' % summary.elapsed}. Generated by Rubytest #{h ::Test::VERSION} on #{h Time.now.strftime('%Y-%m-%d %H:%M:%S')}.</footer>]
@output.puts '</body>'
@output.puts '</html>'
end

private

# Escape text for use in HTML.
def h(text)
text.to_s.gsub(/[&<>"']/, ESCAPES)
end

# An exception's class, message and where it was raised, with the rest
# of its backtrace if asked.
def detail(exception, backtrace=false)
lines = ["#{exception.class}: #{exception.message}"]
trace = exception.backtrace ? clean_backtrace(exception) : []
trace = trace.first(1) unless backtrace
lines.concat(trace.map{ |line| " #{line}" })
lines.join("\n")
end

end

end
Loading
Loading