diff options
Diffstat (limited to 'regress/regress.pl.1')
| -rw-r--r-- | regress/regress.pl.1 | 189 |
1 files changed, 189 insertions, 0 deletions
diff --git a/regress/regress.pl.1 b/regress/regress.pl.1 new file mode 100644 index 00000000..dc2b91ba --- /dev/null +++ b/regress/regress.pl.1 @@ -0,0 +1,189 @@ +.\" $Id$ +.\" +.\" Copyright (c) 2017 Ingo Schwarze <schwarze@openbsd.org> +.\" +.\" Permission to use, copy, modify, and distribute this software for any +.\" purpose with or without fee is hereby granted, provided that the above +.\" copyright notice and this permission notice appear in all copies. +.\" +.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +.\" +.Dd $Mdocdate$ +.Dt REGRESS.PL 1 +.Os +.Sh NAME +.Nm regress.pl +.Nd portable steering script for mandoc regression tests +.Sh SYNOPSIS +.Nm ./regress.pl +.Oo +.Ar directory Ns Op Pf : Ar test +.Op Ar modifier ... +.Oc +.Sh DESCRIPTION +The +.Nm +steering script allows running the +.Xr mandoc 1 +regression suite on arbitrary operating systems, +even though the suite was designed for OpenBSD only. +.Pp +When run without an argument, +.Nm +runs the complete regression suite. +.Pp +When run with one argument, that argument can be: +.Bl -enum +.It +A single dot to run the complete suite. +.It +One of the top level directories, for example +.Pa mdoc , +to run the test suite for a complete language or feature group. +.It +A subdirectory, for example +.Pa man/IP , +to run the tests for a specific macro or an individual feature. +.It +A subdirectory with a test name appended with a colon, for example +.Pa char/unicode : Ns Pa named , +to run the tests for one particular input file. +.El +.Pp +Any additional arguments modify the way the tests are run. +The default is +.Cm all . +The following modifiers are available: +.Bl -tag -width verbose +.It Cm all +Run all kinds of subtests. +This implies all other modifiers except +.Cm verbose +and +.Cm clean . +.It Cm ascii +Run subtests for +.Fl T Cm ascii +output mode. +.It Cm clean +Remove all output files created by running the tests. +.It Cm html +Run subtests for +.Fl T Cm html +output mode. +.It Cm lint +Run subtests for +.Fl T Cm lint +warning and error output. +.It Cm man +Run subtests for +.Fl T Cm man +output mode. +.It Cm utf8 +Run subtests for +.Fl T Cm utf8 +output mode. +.It Cm verbose +Display approximate indications of what is being done. +.El +.Pp +The amount of summary lines shown can be modified by giving an +argument consisting of a single digit: +.Bl -tag -width verbose +.It Cm 3 +Show all summary lines for all directories entered. +Even without +.Cm verbose , +this generates more than hundred lines of output when running the +complete regression suite. +.It Cm 2 +This is the default. +It shows the summary lines for the +.Ar directory +given on the command line and its immediate children. +Except for +.Pa mdoc , +the output usually fits on one screen. +.It Cm 1 +Only show a single summary line for the whole run. +.It Cm 0 +Do not show any summary lines. +No output means success. +Success or failure can also be seen from the exit status. +.El +.Pp +All failed tests are always reported, even when the +.Cm 0 +modifier is given. +.Sh EXIT STATUS +.Ex -std +.Sh EXAMPLES +The recommended invocation for casual users: +.Pp +.Dl ./regress.pl +.Pp +Maximum output: +.Pp +.Dl ./regress.pl \&. verbose +.Pp +Complete check, but keep the tree clean: +.Pp +.Dl ./regress.pl \&. all clean +.Pp +Test all of +.Pa mdoc , +but don't print the usual 65 lines of output: +.Pp +.Dl ./regress.pl mdoc 1 +.Pp +Investigate a specific failure: +.Pp +.Dl ./regress.pl mdoc/Bd:broken man verbose +.Sh HISTORY +The +.Nm +script appeared in release 1.14.1 of the portable +.Sy mandoc +distribution. +.Sh AUTHORS +.An Ingo Schwarze Aq Mt schwarze@openbsd.org +.Sh CAVEATS +This script is not optimized for elegance. +Regression suites for other software should not copy the design. +.Pp +The problem it solves is that the +.Sy mandoc +regression suite is tightly integrated into the regression +testing system of the OpenBSD base system, which requires +both OpenBSD +.Xr make 1 , +working neither with POSIX make nor with GNU make, and which +also requires the OpenBSD-specific Makefile fragments in +.Pa /usr/share/mk . +The workaround of parsing the Makefiles by hand and constructing +the required command lines by hand is unavoidably messy; it's +the classic no-no of parsing a language with an ad-hoc incomplete +parser. +But the problem of providing this regression suite for other +operating systems stood unsolved for many years, and no cleaner +solution was found that could be implemented with reasonable effort. +So maybe this is better than nothing. +.Pp +The top-level Makefiles for running this regression suite on +OpenBSD are not included in the portable distribution. +They are too OpenBSD-specific to be useful elsewhere, +and on OpenBSD itself, the suite ought be run natively from +.Pa /usr/src/regress/usr.bin/mandoc +and not from the portable distribution. +.Pp +The +.Pa db +subdirectory of the regression suite is not included. +It uses a Makefile structure that differs vastly from the +rest of the suite. |