summaryrefslogtreecommitdiffstats
path: root/mandocdb.c
Commit message (Collapse)AuthorAgeFilesLines
...
* better error reporting in case of SQL errors: mention dir and fileIngo Schwarze2014-04-131-8/+8
|
* Next speed optimization step for the new apropos(1).Ingo Schwarze2014-04-101-26/+64
| | | | | | | | | | | | | | Split manual names out of the common "keys" table into their own "names" table. This reduces standard apropos(1) search times (i.e. searching for names and descriptions only) by typically about 70% for the full /usr/share/man database. (Yes, that multiplies with the previous optimization step, so both together have reduced search times by a factor of more than six. I'm not done yet, expect more to come.) Even with the minimal databases built with makewhatis(8) -Q, this step still reduces search times by 15-20%. For both cases, database sizes and build times hardly change (+/-2%).
* After careful gprof(1)ing of the new apropos(1), move the descriptionsIngo Schwarze2014-04-091-17/+8
| | | | | | | | | | | | back from the keys table to the mpages table: I found a good way to still use them in searches, without complication of the code. On my notebook, this reduces typical apropos(1) search times by about 40%, it reduces /usr/share/man database size by 6% in makewhatis(8) -Q mode and by 2% in standard mode (less overhead storing pointers to mpages), and it doesn't measurably change database build times (may even be going down by a percent or so because less data is being copied around in ohashes).
* In -p (picky) mode, warn unless each filename (aka mlink)Ingo Schwarze2014-04-041-24/+29
| | | | | appears as a name in the NAME section. While here, garbage collect two unused variables, both called "match".
* Warn about missing mlinks.Ingo Schwarze2014-04-041-0/+40
| | | | | | This is really expensive, more than tripling database build times, so only do it when the -p (picky) option was given, but none of the following options were given: -Q (quick), -d, -u, or -t.
* Remember which names are in the NAME section.Ingo Schwarze2014-04-041-5/+8
| | | | | | | This helps to find missing MLINKS. Database build times do not change and database growth is minimal (1.2% with -Q, 0.7% without -Q in /usr/share/man), so making this optional would be pointless.
* When the -n or -t flag is given to makewhatis(8),Ingo Schwarze2014-04-041-5/+39
| | | | | | write names and decriptions to stdout, in a format similar to apropos(1) output. Inspired by espie@'s makewhatis.
* Instead of silently doing nothing at all,Ingo Schwarze2014-04-031-0/+5
| | | | | | | | | | warn and return non-zero when the manpath is empty, that is, when /etc/man.conf is non-existent or unreadable AND the environment variable MANPATH is unset or empty AND no directories were given on the command line. Inspired by the error handling in espie@'s makewhatis(8), except that one doesn't know about MANPATH.
* Rename the -W option to -p (mnemonics: picky, print to stderr):Ingo Schwarze2014-04-031-7/+10
| | | | | | | That letter was already chosen by espie@ for OpenBSD 2.7, so avoid being gratuitiously different more than a decade later. Accept -v for backward compatibility with espie@'s makewhatis, even though it does nothing right now.
* The -v option of mandocdb(8) clashes with the -v option of espie@'sIngo Schwarze2014-04-031-12/+12
| | | | | makewhatis(8), which traditionally does something different, so rename it to -D (mnemonics: Debug, Dump, Display).
* Without bloating mandoc(1) itself, let mandocdb(8) support filesIngo Schwarze2014-03-261-16/+80
| | | | | called manN/X.N.gz and catN/X.0.gz, reading them through a pipe(2) from gunzip(1) -c. Asked for by various people in the past.
* Improve error reporting.Ingo Schwarze2014-03-261-33/+51
| | | | | | | | Simplify combining a custom format string with perror(), avoiding many manual calls to strerror(errno). For low-level failures, report attempted function calls. Do not abuse the say() filename argument for files outside the basedir, and even less for other text.
* If an .Nd block contains macros, avoid fragmented entries in mandocdb(8),Ingo Schwarze2014-03-231-25/+3
| | | | instead use the .Nd content recursively.
* If a man(7) NAME section contains macros, avoid truncated or emptyIngo Schwarze2014-03-231-46/+7
| | | | | | entries for .Nd in mandocdb(8), instead use the macro content recursively. This improves indexing of more than 200 manuals in Xenocara, i.e. more than 15%, in particular GL and some Xkb.
* avoid repetitive code for asprintf error handlingIngo Schwarze2014-03-231-4/+1
|
* The files mandoc.c and mandoc.h contained both specialised low-levelIngo Schwarze2014-03-231-0/+1
| | | | | | | functions used for multiple languages (mdoc, man, roff), for example mandoc_escape(), mandoc_getarg(), mandoc_eos(), and generic auxiliary functions. Split the auxiliaries out into their own file and header. While here, do some #include cleanup.
* Register pure .so pages as mlinks, not as mpages.Ingo Schwarze2014-03-191-19/+67
| | | | | | | This doesn't affect /usr/share/man, but improves /usr/X11R6/man: * Eliminates multiple apropos(1) output for such pages. * Reduces X11R6 database size from 450 kB to 240 kB (-47%). * Reduces X11R6 database build time from 1.68s to 1.00s (-40%).
* Without the MPARSE_SO option, if the file contains nothing but aIngo Schwarze2014-03-191-1/+1
| | | | | | single .so request, do not read the file pointed to, but instead let mparse_result() provide the file name pointed to as a return value. To be used by makewhatis(8) in the future.
* Generalize the mparse_alloc() and roff_alloc() functions by givingIngo Schwarze2014-03-191-5/+5
| | | | | | | | them an "options" argument, replacing the existing "inttype" and "quick" arguments, preparing for a future MPARSE_SO option. Store this argument in struct mparse and struct roff, replacing the existing "inttype", "parsetype", and "quick" members. No functional change except one tiny cosmetic fix in roff_TH().
* Allow checking that databases are up to date even when you have no writeIngo Schwarze2014-03-181-20/+94
| | | | | | permission on the databases, as requested by espie@ quite some time ago. But make sure to not slow database generation down when you do have write permission, and to not delay error reporting in -Q mode.
* Implement the \: (optional line break) escape sequence,Ingo Schwarze2014-01-221-5/+14
| | | | | | | documented in the Ossanna-Kernighan-Ritter troff manual and also supported by groff. Missing feature reported by Steffen Nurpmeso <sdaoden at gmail dot com>.
* Avoid the risk of trying to modify a literal string.Ingo Schwarze2014-01-191-2/+3
|
* Always store the arch in lower-case only.Ingo Schwarze2014-01-191-5/+9
| | | | Reduces database size by ~0.5%, and by ~1.5% with -Q.
* Support a second -v on mandocdb(8) to show keys while they are being added;Ingo Schwarze2014-01-191-1/+16
| | | | | i need that for debugging, in particular to be used with -t. To be able to do so, provide a global table of key names, for reuse.
* Cope with slightly broken NAME sections in man(7) pagesIngo Schwarze2014-01-181-0/+9
| | | | | | having a trailing comma after the last name, like this: ASN1_OBJECT_new, ASN1_OBJECT_free, - object allocation functions
* Drop the AUTOINCREMENT PRIMARY KEYs from the mlinks and keys tables.Ingo Schwarze2014-01-181-4/+2
| | | | | | | | | | They are completely unused, and i cannot imagine what they *could* ever be used for; but apparently, they are expensive to generate. Standard DB build time goes down by 10%, now at 1.9x of makewhatis. Standard DB size goes down by 4%, now at 11x of makewhatis. DB build time with -Q goes down by 15%, now at 0.28x of makewhatis. DB size with -Q goes down by 3%, now at 3.35x of makewhatis.
* Despite some experimenting, i'm unable to find any relevant effect ofIngo Schwarze2014-01-181-3/+1
| | | | | | | | | | | | creating an index for the keys table on apropos(1) search times; apparently, adding that index was premature optimization in the first place; so, stop adding that index. Its root gone, the following evil is reduced (/usr/share/man on my notebook) - DB build time with -Q goes down by 15%, now at 1/3 of makewhatis - DB size with -Q goes down by 35%, now at 3.5x of makewhatis - full DB build time goes down by 12%, now at 2.1x of makewhatis - full DB size goes down by 42%, now at 11.5x of makewhatis
* Do not sync to disk after each individual manual page (duh!),Ingo Schwarze2014-01-061-4/+6
| | | | | | | | | | | | only sync to disk one single time when all data is ready. Rebuild times for /usr/share/man/mandoc.db shrink on my notebook: In standard mode from 45 seconds to 11 seconds (75% reduction) In -Q mode from 25 seconds to 3.1 seconds (87% reduction) For comparison: makewhatis(8): 4.2 seconds That is, in -Q mode, we are now *faster* than the existing makewhatis(8), and careful profiling shows there is still a lot of room for improval.
* Fix mandocdb(8) -d and -u.Ingo Schwarze2014-01-061-14/+35
| | | | It was broken by recent optimizations.
* Rename dbindex() to dbadd() to be less confusing.Ingo Schwarze2014-01-061-5/+5
| | | | | The concept of an index file is gone since the switch to SQLite. No functional change.
* Remove the redundant "file" column from the "mlinks" table.Ingo Schwarze2014-01-061-3/+1
| | | | | The contents can easily be reconstructed from sec, arch, name, form. Shrinks the database by another 3% in standard mode and 9% in -Q mode.
* Drop Nd from the mpages table, it is still in the keys table.Ingo Schwarze2014-01-061-14/+1
| | | | | This shrinks the database in standard mode by 3%, in -Q mode by 9%, without loss of functionality.
* Add an option -Q (quick) to mandocdb(8)Ingo Schwarze2014-01-051-7/+11
| | | | | | | | | | | | | | | for accelerated generation of reduced-size databases. Implement this by allowing the parsers to optionally abort the parse sequence after the NAME section. While here, garbage collect the unused void *arg attribute of struct mparse and mparse_alloc() and fix some errors in mandoc(3). This reduces the processing time of mandocdb(8) on /usr/share/man by a factor of 2 and the database size by a factor of 4. However, it still takes 5 times the time and 6 times the space of makewhatis(8), so more work is clearly needed.
* Rip out the complete "reachable" checks, without replacement.Ingo Schwarze2014-01-051-68/+6
| | | | | | | | | | | | | | It's a pity i spent time during t2k13 writing this; however, when an entire concept is busted, let us not look back, There is no such thing as an unreachable page. Even if you are crazy enough to put a page starting with ".Dt NAMEI 9" into a file man1/cat.1, we now make sure that it can be found by all of the following: Nm=namei Nm=cat sec=1 sec=9 It will always be displayed as: cat(1) - pathname lookup So you know that you have to type `man cat` to get at it. That obsoletes the concept of "unreachable manuals" for good.
* Remove the obsolete file name column from the mpages table.Ingo Schwarze2014-01-051-13/+1
| | | | | This column wasn't helpful because one manpage can have multiple MLINKS. Use the file name column in the mlinks table, instead.
* Remove the obsolete sec and arch columns from the mpages table.Ingo Schwarze2014-01-051-7/+3
| | | | | They were confusing because a manpage can have MLINKS in different sections and architectures.
* Reimplement apropos -s NUM -S ARCH EXPR by internally converting it toIngo Schwarze2014-01-051-2/+4
| | | | | | | | | | | | | | | apropos \( EXPR \) -a 'sec~^NUM$' -a 'arch~^(ARCH|any)$' in preparation for removal of sec and arch from the mpage table. Almost no functional change except for the following bonus: This also makes sure that for cross-section and cross-arch MLINKs, all of the following work: apropos -s 1 encrypt apropos -s 8 encrypt apropos -s 1 makekey apropos -s 8 makekey While here, print error messages about invalid regexps to stderr.
* Put section and architecture info into the keys table,Ingo Schwarze2014-01-051-2/+10
| | | | | | | | | | in preparation for removing them from the mpages table, aiming for cleaner and more uniform interfaces. Database growth is below 4%, part of which will be reclaimed. As a bonus, this allows searches like: ./obj/apropos An=kettenis -a arch=ppc ./obj/apropos An=kettenis -a sec~[^4]
* Avoid "utf8" in the names of a function and a struct memberIngo Schwarze2014-01-021-18/+18
| | | | | that don't necessarily have anything to do with UTF-8. Just renaming, no functional change.
* Do not put UTF-8-encoded strings into the database by default, use ASCII.Ingo Schwarze2014-01-021-16/+34
| | | | Just like for mandoc(1), provide a -Tutf8 option for people who want that.
* Polish the mlink_add() interface:Ingo Schwarze2014-01-021-26/+20
| | | | | Allocate memory inside, not in the callers. No functional change.
* Check all MLINKS for consistency with the content of the manual page,Ingo Schwarze2014-01-021-42/+59
| | | | | not just the first one. This doesn't change how the check is done, but just which MLINKS are checked.
* Yet another regression introduced by Kristaps when he switched fromIngo Schwarze2013-12-311-33/+5
| | | | | | | | | | | | | | Berkeley DB to SQLite3: In the .In parser, the logic got inverted. The resulting NULL pointer access was found by clang; scan log provided by Ulrich Spoerlein <uqs at FreeBSD>. The best fix is to simply remove the whole, pointless custom handler function for .In and let the framework do its work. Now searching for included header files actually works. While here, remove the similarly pointless custom .St handler, fix the return value of the .Fd handler and disentangle the spaghetti in the .Nm handler.
* remove assignments that will be overwritten right afterwards,Ingo Schwarze2013-12-311-1/+0
| | | | | and remove pointless local variables; found in a clang output from Ulrich Spoerlein <uqs at FreeBSD>
* Oops, that segfaulted after deleting an mlink from the list.Ingo Schwarze2013-12-271-7/+9
| | | | Fix the loop logic in mlinks_undupe().
* Split mlinks_undupe() out of mpages_merge()Ingo Schwarze2013-12-271-36/+52
| | | | | such that the check for source manuals of the same name can be done for multiple mlinks pointing to the same preformatted mpage.
* Save the MLINK name into the database, too;Ingo Schwarze2013-12-271-2/+4
| | | | apropos(1) will need it to display its results.
* Write more than one mlink per mpage to the database.Ingo Schwarze2013-12-271-7/+10
| | | | Not yet used by apropos(1).
* Allow saving more than one mlink per mpage in the mlinks ohash.Ingo Schwarze2013-12-271-23/+0
| | | | | | We are still only using one of them for now. Actually, we are now using a different one, but the order the mlinks are found is random anyway.
* Enable the framework code to allow more than one mlink per mpage.Ingo Schwarze2013-12-271-2/+3
| | | | Not used yet.
generated by cgit (git 2.34.1) at 2024-11-19 04:42:24 +0000