| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
| |
* rename the halloc callback to calloc, provide overflow protection
* rename the hfree callback to free, drop the useless size argument
* prevent integer overflows in ohash_resize
|
| |
|
|
|
|
|
| |
* Change eight reallocs to reallocarray to be safe from overflows.
* Change one malloc to reallocarray to be safe from overflows.
* Change one calloc to reallocarray, no zeroing needed.
* Change the order of arguments of three callocs (aesthetical).
|
| |
|
|
| |
suggested by espie@.
|
| |
|
|
|
| |
remove trailing whitespace and blanks before tabs, improve some indenting;
no functional change
|
| |
|
|
|
| |
note this doesn't affect performance, SQLite generates the same byte code.
While here, make the calls to exprspec() easier to understand.
|
| |
|
|
| |
to call this kid by a different name here than in all other tables.
|
| |
|
|
|
|
| |
and SQLITE_DETERMINISTIC when creating deterministic functions;
best practice measure suggested by espie@ and jeremy@;
as expected by jeremy@, no measurable effect on performance.
|
| |
|
|
| |
At the end of mansearch(), fchdir() back to where we started from.
|
| |
|
|
|
| |
this is cleaner and helps to not scatter gmon.out files all over
the place when profiling.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Contrary to what i initially thought, almost all time is now spent
inside sqlite3(3) routines, and i found no easy way calling less of them.
However, sqlite(3) spends substantial time in malloc(3), and even more
(twice that) in its immediate malloc wrapper, sqlite3MemMalloc(),
keeping track of all individual malloc chunk sizes. Typically about
90% of the malloced memory is used for purposes of the pagecache.
By providing an mmap(3) MAP_ANON SQLITE_CONFIG_PAGECACHE, execution
time decreases by 20-25% for simple (Nd and/or Nm) queries, 10-20% for
non-NAME queries, and even apropos(1) resident memory size as reported
by top(1) decreases by 20% for simple and by 60% for non-NAME queries.
The new function, mansearch_setup(), spends no measurable time.
The pagesize chosen is optimal:
* Substantially smaller pages yield no gain at all.
* Larger pages provide no additional benefit and just waste memory.
The chosen number of pages in the cache is a compromise:
* For simple queries, a handful of pages would suffice to get the full
speed effect, at an apropos(1) resident memory size of about 2.0 MB.
* For non-NAME queries, a large pagecache with 2k pages (2.5 MB) might
gain a few more percent in speed, but at the expense of doubling the
apropos(1) resident memory size for *all* queries.
* The chosen number of 256 pages (330 kB) allows nearly full speed gain
for all queries at the price of a 15% resident memory size increase.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Split manual names out of the common "keys" table into their
own "names" table. This reduces standard apropos(1) search
times (i.e. searching for names and descriptions only) by
typically about 70% for the full /usr/share/man database.
(Yes, that multiplies with the previous optimization step,
so both together have reduced search times by a factor of
more than six. I'm not done yet, expect more to come.)
Even with the minimal databases built with makewhatis(8) -Q,
this step still reduces search times by 15-20%. For both cases,
database sizes and build times hardly change (+/-2%).
|
| |
|
|
|
|
|
|
|
|
|
|
| |
back from the keys table to the mpages table: I found a good way
to still use them in searches, without complication of the code.
On my notebook, this reduces typical apropos(1) search times by about 40%,
it reduces /usr/share/man database size by 6% in makewhatis(8) -Q mode
and by 2% in standard mode (less overhead storing pointers to mpages),
and it doesn't measurably change database build times (may even be
going down by a percent or so because less data is being copied
around in ohashes).
|
| |
|
|
|
| |
With this bug fix, partly unitialized memory could sometimes be
returned, sometimes causing crashes by bogus free(3)s in apropos(1).
|
| | |
|
| |
|
|
|
|
|
| |
functions used for multiple languages (mdoc, man, roff), for example
mandoc_escape(), mandoc_getarg(), mandoc_eos(), and generic auxiliary
functions. Split the auxiliaries out into their own file and header.
While here, do some #include cleanup.
|
| | |
|
| | |
|
| |
|
|
| |
No functional change.
|
| |
|
|
|
| |
The contents can easily be reconstructed from sec, arch, name, form.
Shrinks the database by another 3% in standard mode and 9% in -Q mode.
|
| |
|
|
|
| |
This shrinks the database in standard mode by 3%, in -Q mode by 9%,
without loss of functionality.
|
| |
|
|
|
| |
This column wasn't helpful because one manpage can have multiple MLINKS.
Use the file name column in the mlinks table, instead.
|
| |
|
|
|
| |
They were confusing because a manpage can have MLINKS in different
sections and architectures.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
apropos \( EXPR \) -a 'sec~^NUM$' -a 'arch~^(ARCH|any)$'
in preparation for removal of sec and arch from the mpage table.
Almost no functional change except for the following bonus:
This also makes sure that for cross-section and cross-arch MLINKs,
all of the following work:
apropos -s 1 encrypt
apropos -s 8 encrypt
apropos -s 1 makekey
apropos -s 8 makekey
While here, print error messages about invalid regexps to stderr.
|
| |
|
|
|
|
|
|
|
|
| |
in preparation for removing them from the mpages table,
aiming for cleaner and more uniform interfaces.
Database growth is below 4%, part of which will be reclaimed.
As a bonus, this allows searches like:
./obj/apropos An=kettenis -a arch=ppc
./obj/apropos An=kettenis -a sec~[^4]
|
| |
|
|
|
|
| |
the old implementation got lost in the Berkeley to SQLite switch.
Note that this is not just feature creep, but required for upcoming
database format cleanup and simplification.
|
| |
|
|
|
|
|
|
| |
This really takes us beyond what grep -R /usr/*/man/ can do
because now you can search for pages by *one* criterion and then
display the contents of *another* macro from those pages, like in
$ apropos -O Ox Fa~wchar
to get an impression how long wide character handling is available.
|
| |
|
|
| |
too long and unwieldy, but will grow more code soon. No functional change.
|
| |
|
|
|
|
| |
and return a list of names with sections, used by apropos(1) for display.
While here, improve uniformity of the interface by allocating the file
name dynamically, just like the names list and the description.
|
| |
|
|
|
|
| |
both because it contains nothing but a subset of the data of the
existing mpages table and because the relationship of mpage and mlink
entries is still 1:1. But all that will eventually change.
|
| |
|
|
| |
The ~ operator has to do regular expression search, not globbing.
|
| |
|
|
| |
The = operator has to do substring search, not word search.
|
| |
|
|
|
|
| |
Consistently use the PATH_MAX since it is specified by POSIX,
while MAXPATHLEN is not.
In preparation for using this at a few more places.
|
| |
|
|
| |
(via mansearch), and merge mandocdb.h into mansearch.h (and remove).
|
| |
|
|
|
|
|
|
|
|
| |
This include's espie@'s wholesale src/lib/libc/ohash directory from OpenBSD
into compat_ohash.c (with a single copyright/license notice at the top)
and src/include/ohash.h as compat_ohash.h.
The ohash_int.h part of compat_ohash.c has been changed only in that ohash.h
points to compat_ohash.h.
Added HAVE_OHASH test (test-ohash.c) to Makefile.
In mandocdb.c and mansearch.c, check HAVE_OHASH test for inclusion.
|
| |
|
|
|
|
| |
This makes it run about 5x faster.
While here, wrap some sqlite3 statements in #defines to extract errors.
(Really, the warning/error/etc. macros should be functionified.)
|
| |
|
|
| |
verbose if sqlite3 errors occur.
|
|
|
This is a much more minimal interface that stuffs all operations into
a single function.
It uses sqlite3 and ohash.
|
