summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Further apropos(1) speed optimization was trickier than anticipated.Ingo Schwarze2014-04-113-0/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Contrary to what i initially thought, almost all time is now spent inside sqlite3(3) routines, and i found no easy way calling less of them. However, sqlite(3) spends substantial time in malloc(3), and even more (twice that) in its immediate malloc wrapper, sqlite3MemMalloc(), keeping track of all individual malloc chunk sizes. Typically about 90% of the malloced memory is used for purposes of the pagecache. By providing an mmap(3) MAP_ANON SQLITE_CONFIG_PAGECACHE, execution time decreases by 20-25% for simple (Nd and/or Nm) queries, 10-20% for non-NAME queries, and even apropos(1) resident memory size as reported by top(1) decreases by 20% for simple and by 60% for non-NAME queries. The new function, mansearch_setup(), spends no measurable time. The pagesize chosen is optimal: * Substantially smaller pages yield no gain at all. * Larger pages provide no additional benefit and just waste memory. The chosen number of pages in the cache is a compromise: * For simple queries, a handful of pages would suffice to get the full speed effect, at an apropos(1) resident memory size of about 2.0 MB. * For non-NAME queries, a large pagecache with 2k pages (2.5 MB) might gain a few more percent in speed, but at the expense of doubling the apropos(1) resident memory size for *all* queries. * The chosen number of 256 pages (330 kB) allows nearly full speed gain for all queries at the price of a 15% resident memory size increase.
* Next speed optimization step for the new apropos(1).Ingo Schwarze2014-04-104-88/+144
| | | | | | | | | | | | | | Split manual names out of the common "keys" table into their own "names" table. This reduces standard apropos(1) search times (i.e. searching for names and descriptions only) by typically about 70% for the full /usr/share/man database. (Yes, that multiplies with the previous optimization step, so both together have reduced search times by a factor of more than six. I'm not done yet, expect more to come.) Even with the minimal databases built with makewhatis(8) -Q, this step still reduces search times by 15-20%. For both cases, database sizes and build times hardly change (+/-2%).
* After careful gprof(1)ing of the new apropos(1), move the descriptionsIngo Schwarze2014-04-094-77/+95
| | | | | | | | | | | | back from the keys table to the mpages table: I found a good way to still use them in searches, without complication of the code. On my notebook, this reduces typical apropos(1) search times by about 40%, it reduces /usr/share/man database size by 6% in makewhatis(8) -Q mode and by 2% in standard mode (less overhead storing pointers to mpages), and it doesn't measurably change database build times (may even be going down by a percent or so because less data is being copied around in ohashes).
* Add a new term_flushln() flag TERMP_BRIND (if break, then indent)Ingo Schwarze2014-04-084-47/+42
| | | | | | | | | | to control indentation of continuation lines in TERMP_NOBREAK mode. In the past, this was always on; continue using it for .Bl, .Nm, .Fn, .Fo, and .HP, but no longer for .IP and .TP. I looked at this because sthen@ reported the issue in a manual of a Perl module from ports, but it affects base, too: This patch reduces groff-mandoc differences in base by more than 15%.
* If the SYNOPSIS section contains an excessively long .Nm,Ingo Schwarze2014-04-081-0/+2
| | | | | adjust the right margin to avoid running into an assertion; output in that case now agrees with groff, too.
* Fully implement the \B (validate numerical expression) andIngo Schwarze2014-04-083-38/+87
| | | | | | | | | | | partially implement the \w (measure text width) escape sequence in a way that makes them usable in numerical expressions and in conditional requests, similar to how \n (interpolate number register) and \* (expand user-defined string) are implemented. This lets mandoc(1) handle the baroque low-level roff code found at the beginning of the ggrep(1) manual. Thanks to pascal@ for the report.
* We already supported (outer) user-defined strings containing referencesIngo Schwarze2014-04-071-36/+44
| | | | | | | | to other (inner) user-defined strings in their values, such that the inner ones get expanded at expansion time of the outer ones (delayed evaluation). Now we also support specifying the name of an (outer) user-defined string to expand using the expanded values of some other (inner) user-defined strings (indirect reference).
* Make it clear which errors come from which tool.Ingo Schwarze2014-04-071-3/+6
| | | | joint work with sthen@
* Accept arbitrary argument delimiters for various roff(7) escape sequences.Ingo Schwarze2014-04-071-4/+4
| | | | Needed for example by the new Perl pod2man(1) preamble.
* Almost complete implementation of roff(7) numerical expressions.Ingo Schwarze2014-04-072-73/+286
| | | | | | | Support all binary operators except ';' (scale conversion). Fully support chained operations and nested parentheses. Use this for the .nr, .if, and .ie requests. While here, fix parsing of integer numbers in roff_getnum().
* bugfix: make sure all variables are properly initializedIngo Schwarze2014-04-051-5/+5
| | | | when rendering .ll (line length) requests. oops.
* Implement the roff(7) .rr (remove register) request.Ingo Schwarze2014-04-052-6/+38
| | | | | As reported by sthen@, the perl-5.18 pod2man(1) preamble thinks cool kids use that in manuals. I hope *you* know better.
* In -p (picky) mode, warn unless each filename (aka mlink)Ingo Schwarze2014-04-041-24/+29
| | | | | appears as a name in the NAME section. While here, garbage collect two unused variables, both called "match".
* Warn about missing mlinks.Ingo Schwarze2014-04-041-0/+40
| | | | | | This is really expensive, more than tripling database build times, so only do it when the -p (picky) option was given, but none of the following options were given: -Q (quick), -d, -u, or -t.
* Remember which names are in the NAME section.Ingo Schwarze2014-04-044-53/+60
| | | | | | | This helps to find missing MLINKS. Database build times do not change and database growth is minimal (1.2% with -Q, 0.7% without -Q in /usr/share/man), so making this optional would be pointless.
* merge OpenBSD rev. 1.20, describing new featuresIngo Schwarze2014-04-041-2/+20
|
* When the -n or -t flag is given to makewhatis(8),Ingo Schwarze2014-04-042-7/+41
| | | | | | write names and decriptions to stdout, in a format similar to apropos(1) output. Inspired by espie@'s makewhatis.
* Instead of silently doing nothing at all,Ingo Schwarze2014-04-031-0/+5
| | | | | | | | | | warn and return non-zero when the manpath is empty, that is, when /etc/man.conf is non-existent or unreadable AND the environment variable MANPATH is unset or empty AND no directories were given on the command line. Inspired by the error handling in espie@'s makewhatis(8), except that one doesn't know about MANPATH.
* Rename the -W option to -p (mnemonics: picky, print to stderr):Ingo Schwarze2014-04-032-17/+24
| | | | | | | That letter was already chosen by espie@ for OpenBSD 2.7, so avoid being gratuitiously different more than a decade later. Accept -v for backward compatibility with espie@'s makewhatis, even though it does nothing right now.
* The -v option of mandocdb(8) clashes with the -v option of espie@'sIngo Schwarze2014-04-032-19/+17
| | | | | makewhatis(8), which traditionally does something different, so rename it to -D (mnemonics: Debug, Dump, Display).
* document -Q and -T; from OpenBSDIngo Schwarze2014-04-031-6/+22
|
* Support the CONTEXT section for kernel manual pages found in Solaris andIngo Schwarze2014-03-314-0/+14
| | | | | OpenBSD manuals. It describes which contexts you can call functions in. from dlg@, ok jmc@ deraadt@
* Improve formatting of broken blocks in -Tman,Ingo Schwarze2014-03-301-2/+11
| | | | | somewhat similar to what mdoc_term.c already does for -Tascii. OpenBSD rev. 1.58.
* Allow ERRORS in section 4; OpenBSD rev. 1.121.Ingo Schwarze2014-03-301-2/+4
|
* Support relative arguments to .ll (increase or decrease line length).Ingo Schwarze2014-03-307-15/+56
|
* Implement the roff(7) .ll (line length) request.Ingo Schwarze2014-03-3019-11/+109
| | | | | Found by naddy@ in the textproc/enchant(1) port. Of course, do not use this in new manuals.
* Allow leading and trailing vertical lines,Ingo Schwarze2014-03-283-4/+16
| | | | | | | and format them in the same way as groff. While here, do not require whitespace before vertical lines in layout specifications. Issues found by bentley@ in mpv(1).
* Properly initialize malloc(3)ed memory.Ingo Schwarze2014-03-281-0/+1
| | | | | With this bug fix, partly unitialized memory could sometimes be returned, sometimes causing crashes by bogus free(3)s in apropos(1).
* Without bloating mandoc(1) itself, let mandocdb(8) support filesIngo Schwarze2014-03-261-16/+80
| | | | | called manN/X.N.gz and catN/X.0.gz, reading them through a pipe(2) from gunzip(1) -c. Asked for by various people in the past.
* Improve error reporting.Ingo Schwarze2014-03-261-33/+51
| | | | | | | | Simplify combining a custom format string with perror(), avoiding many manual calls to strerror(errno). For low-level failures, report attempted function calls. Do not abuse the say() filename argument for files outside the basedir, and even less for other text.
* Skip leading escape sequences in man_deroff(). Helps indexing ofIngo Schwarze2014-03-231-4/+11
| | | | some manuals containing overzealous escaping in their NAME section.
* Retire the old concat() function.Ingo Schwarze2014-03-231-77/+26
| | | | | | | | For .Sh, i wasn't even needed at all. For .Dd, .Nm, and .Os, use the new mdoc_deroff() instead. This gets rid of the last limited-size static buffers in this file, hence eliminates the last explicit MANDOCERR_MEM throwers here, and it shortens the code by 50 lines.
* If an .Nd block contains macros, avoid fragmented entries in mandocdb(8),Ingo Schwarze2014-03-233-25/+44
| | | | instead use the .Nd content recursively.
* If a man(7) NAME section contains macros, avoid truncated or emptyIngo Schwarze2014-03-233-46/+49
| | | | | | entries for .Nd in mandocdb(8), instead use the macro content recursively. This improves indexing of more than 200 manuals in Xenocara, i.e. more than 15%, in particular GL and some Xkb.
* needs mandoc_aux, tooIngo Schwarze2014-03-231-0/+1
|
* avoid repetitive code for asprintf error handlingIngo Schwarze2014-03-235-47/+17
|
* The files mandoc.c and mandoc.h contained both specialised low-levelIngo Schwarze2014-03-232-0/+137
| | | | | | functions used for multiple languages (mdoc, man, roff), for example mandoc_escape(), mandoc_getarg(), mandoc_eos(), and generic auxiliary functions. Split the auxiliaries out into their own file and header.
* The files mandoc.c and mandoc.h contained both specialised low-levelIngo Schwarze2014-03-2334-126/+88
| | | | | | | functions used for multiple languages (mdoc, man, roff), for example mandoc_escape(), mandoc_getarg(), mandoc_eos(), and generic auxiliary functions. Split the auxiliaries out into their own file and header. While here, do some #include cleanup.
* Remove currently unimplemented macros from the lists of used-definedIngo Schwarze2014-03-201-14/+16
| | | | | | | macros to be cleared during .Dd and .TH because clearing them at that point defeats the purpose of backup implementations provided in the manual page itself, some of which _do_ work with mandoc(1). While here, add the new .%C macro to the list to be cleared.
* Register pure .so pages as mlinks, not as mpages.Ingo Schwarze2014-03-191-19/+67
| | | | | | | This doesn't affect /usr/share/man, but improves /usr/X11R6/man: * Eliminates multiple apropos(1) output for such pages. * Reduces X11R6 database size from 450 kB to 240 kB (-47%). * Reduces X11R6 database build time from 1.68s to 1.00s (-40%).
* Without the MPARSE_SO option, if the file contains nothing but aIngo Schwarze2014-03-196-7/+24
| | | | | | single .so request, do not read the file pointed to, but instead let mparse_result() provide the file name pointed to as a return value. To be used by makewhatis(8) in the future.
* cope with mparse_alloc() interface changeIngo Schwarze2014-03-192-2/+2
|
* Generalize the mparse_alloc() and roff_alloc() functions by givingIngo Schwarze2014-03-196-52/+44
| | | | | | | | them an "options" argument, replacing the existing "inttype" and "quick" arguments, preparing for a future MPARSE_SO option. Store this argument in struct mparse and struct roff, replacing the existing "inttype", "parsetype", and "quick" members. No functional change except one tiny cosmetic fix in roff_TH().
* Allow checking that databases are up to date even when you have no writeIngo Schwarze2014-03-181-20/+94
| | | | | | permission on the databases, as requested by espie@ quite some time ago. But make sure to not slow database generation down when you do have write permission, and to not delay error reporting in -Q mode.
* in apropos(1) output, sort names and avoid multiple section numbersIngo Schwarze2014-03-171-6/+54
|
* Sync to OpenBSD:Ingo Schwarze2014-03-171-55/+151
| | | | | | | | * do not talk about shell globbing * describe logical operations * improve examples * add HISTORY * some wording improvements for clarity
* minor sync to OpenBSDIngo Schwarze2014-03-171-4/+26
|
* sync to OpenBSD: .ft is a roff(7) request, not a man(7) macro,Ingo Schwarze2014-03-172-6/+28
| | | | and it is not even used frequently in man(7)
* In -Tutf8 mode, make sure that hyphens get counted against the output lineIngo Schwarze2014-03-131-6/+7
| | | | | | | length even when they are breakable. Before this, a line containing N breakable hyphens could get up to N characters wider than the right margin in -Tutf8 output mode. Issue reported by tedu@ on <bugs at OpenBSD>.
* new bug report from wiz@ regarding Tcl*(3)Ingo Schwarze2014-03-081-0/+10
|
generated by cgit (git 2.34.1) at 2024-11-14 17:22:37 +0000