summaryrefslogtreecommitdiffstats
path: root/mandocdb.c
Commit message (Collapse)AuthorAgeFilesLines
* Implement the \: (optional line break) escape sequence,Ingo Schwarze2014-01-221-5/+14
| | | | | | | documented in the Ossanna-Kernighan-Ritter troff manual and also supported by groff. Missing feature reported by Steffen Nurpmeso <sdaoden at gmail dot com>.
* Avoid the risk of trying to modify a literal string.Ingo Schwarze2014-01-191-2/+3
|
* Always store the arch in lower-case only.Ingo Schwarze2014-01-191-5/+9
| | | | Reduces database size by ~0.5%, and by ~1.5% with -Q.
* Support a second -v on mandocdb(8) to show keys while they are being added;Ingo Schwarze2014-01-191-1/+16
| | | | | i need that for debugging, in particular to be used with -t. To be able to do so, provide a global table of key names, for reuse.
* Cope with slightly broken NAME sections in man(7) pagesIngo Schwarze2014-01-181-0/+9
| | | | | | having a trailing comma after the last name, like this: ASN1_OBJECT_new, ASN1_OBJECT_free, - object allocation functions
* Drop the AUTOINCREMENT PRIMARY KEYs from the mlinks and keys tables.Ingo Schwarze2014-01-181-4/+2
| | | | | | | | | | They are completely unused, and i cannot imagine what they *could* ever be used for; but apparently, they are expensive to generate. Standard DB build time goes down by 10%, now at 1.9x of makewhatis. Standard DB size goes down by 4%, now at 11x of makewhatis. DB build time with -Q goes down by 15%, now at 0.28x of makewhatis. DB size with -Q goes down by 3%, now at 3.35x of makewhatis.
* Despite some experimenting, i'm unable to find any relevant effect ofIngo Schwarze2014-01-181-3/+1
| | | | | | | | | | | | creating an index for the keys table on apropos(1) search times; apparently, adding that index was premature optimization in the first place; so, stop adding that index. Its root gone, the following evil is reduced (/usr/share/man on my notebook) - DB build time with -Q goes down by 15%, now at 1/3 of makewhatis - DB size with -Q goes down by 35%, now at 3.5x of makewhatis - full DB build time goes down by 12%, now at 2.1x of makewhatis - full DB size goes down by 42%, now at 11.5x of makewhatis
* Do not sync to disk after each individual manual page (duh!),Ingo Schwarze2014-01-061-4/+6
| | | | | | | | | | | | only sync to disk one single time when all data is ready. Rebuild times for /usr/share/man/mandoc.db shrink on my notebook: In standard mode from 45 seconds to 11 seconds (75% reduction) In -Q mode from 25 seconds to 3.1 seconds (87% reduction) For comparison: makewhatis(8): 4.2 seconds That is, in -Q mode, we are now *faster* than the existing makewhatis(8), and careful profiling shows there is still a lot of room for improval.
* Fix mandocdb(8) -d and -u.Ingo Schwarze2014-01-061-14/+35
| | | | It was broken by recent optimizations.
* Rename dbindex() to dbadd() to be less confusing.Ingo Schwarze2014-01-061-5/+5
| | | | | The concept of an index file is gone since the switch to SQLite. No functional change.
* Remove the redundant "file" column from the "mlinks" table.Ingo Schwarze2014-01-061-3/+1
| | | | | The contents can easily be reconstructed from sec, arch, name, form. Shrinks the database by another 3% in standard mode and 9% in -Q mode.
* Drop Nd from the mpages table, it is still in the keys table.Ingo Schwarze2014-01-061-14/+1
| | | | | This shrinks the database in standard mode by 3%, in -Q mode by 9%, without loss of functionality.
* Add an option -Q (quick) to mandocdb(8)Ingo Schwarze2014-01-051-7/+11
| | | | | | | | | | | | | | | for accelerated generation of reduced-size databases. Implement this by allowing the parsers to optionally abort the parse sequence after the NAME section. While here, garbage collect the unused void *arg attribute of struct mparse and mparse_alloc() and fix some errors in mandoc(3). This reduces the processing time of mandocdb(8) on /usr/share/man by a factor of 2 and the database size by a factor of 4. However, it still takes 5 times the time and 6 times the space of makewhatis(8), so more work is clearly needed.
* Rip out the complete "reachable" checks, without replacement.Ingo Schwarze2014-01-051-68/+6
| | | | | | | | | | | | | | It's a pity i spent time during t2k13 writing this; however, when an entire concept is busted, let us not look back, There is no such thing as an unreachable page. Even if you are crazy enough to put a page starting with ".Dt NAMEI 9" into a file man1/cat.1, we now make sure that it can be found by all of the following: Nm=namei Nm=cat sec=1 sec=9 It will always be displayed as: cat(1) - pathname lookup So you know that you have to type `man cat` to get at it. That obsoletes the concept of "unreachable manuals" for good.
* Remove the obsolete file name column from the mpages table.Ingo Schwarze2014-01-051-13/+1
| | | | | This column wasn't helpful because one manpage can have multiple MLINKS. Use the file name column in the mlinks table, instead.
* Remove the obsolete sec and arch columns from the mpages table.Ingo Schwarze2014-01-051-7/+3
| | | | | They were confusing because a manpage can have MLINKS in different sections and architectures.
* Reimplement apropos -s NUM -S ARCH EXPR by internally converting it toIngo Schwarze2014-01-051-2/+4
| | | | | | | | | | | | | | | apropos \( EXPR \) -a 'sec~^NUM$' -a 'arch~^(ARCH|any)$' in preparation for removal of sec and arch from the mpage table. Almost no functional change except for the following bonus: This also makes sure that for cross-section and cross-arch MLINKs, all of the following work: apropos -s 1 encrypt apropos -s 8 encrypt apropos -s 1 makekey apropos -s 8 makekey While here, print error messages about invalid regexps to stderr.
* Put section and architecture info into the keys table,Ingo Schwarze2014-01-051-2/+10
| | | | | | | | | | in preparation for removing them from the mpages table, aiming for cleaner and more uniform interfaces. Database growth is below 4%, part of which will be reclaimed. As a bonus, this allows searches like: ./obj/apropos An=kettenis -a arch=ppc ./obj/apropos An=kettenis -a sec~[^4]
* Avoid "utf8" in the names of a function and a struct memberIngo Schwarze2014-01-021-18/+18
| | | | | that don't necessarily have anything to do with UTF-8. Just renaming, no functional change.
* Do not put UTF-8-encoded strings into the database by default, use ASCII.Ingo Schwarze2014-01-021-16/+34
| | | | Just like for mandoc(1), provide a -Tutf8 option for people who want that.
* Polish the mlink_add() interface:Ingo Schwarze2014-01-021-26/+20
| | | | | Allocate memory inside, not in the callers. No functional change.
* Check all MLINKS for consistency with the content of the manual page,Ingo Schwarze2014-01-021-42/+59
| | | | | not just the first one. This doesn't change how the check is done, but just which MLINKS are checked.
* Yet another regression introduced by Kristaps when he switched fromIngo Schwarze2013-12-311-33/+5
| | | | | | | | | | | | | | Berkeley DB to SQLite3: In the .In parser, the logic got inverted. The resulting NULL pointer access was found by clang; scan log provided by Ulrich Spoerlein <uqs at FreeBSD>. The best fix is to simply remove the whole, pointless custom handler function for .In and let the framework do its work. Now searching for included header files actually works. While here, remove the similarly pointless custom .St handler, fix the return value of the .Fd handler and disentangle the spaghetti in the .Nm handler.
* remove assignments that will be overwritten right afterwards,Ingo Schwarze2013-12-311-1/+0
| | | | | and remove pointless local variables; found in a clang output from Ulrich Spoerlein <uqs at FreeBSD>
* Oops, that segfaulted after deleting an mlink from the list.Ingo Schwarze2013-12-271-7/+9
| | | | Fix the loop logic in mlinks_undupe().
* Split mlinks_undupe() out of mpages_merge()Ingo Schwarze2013-12-271-36/+52
| | | | | such that the check for source manuals of the same name can be done for multiple mlinks pointing to the same preformatted mpage.
* Save the MLINK name into the database, too;Ingo Schwarze2013-12-271-2/+4
| | | | apropos(1) will need it to display its results.
* Write more than one mlink per mpage to the database.Ingo Schwarze2013-12-271-7/+10
| | | | Not yet used by apropos(1).
* Allow saving more than one mlink per mpage in the mlinks ohash.Ingo Schwarze2013-12-271-23/+0
| | | | | | We are still only using one of them for now. Actually, we are now using a different one, but the order the mlinks are found is random anyway.
* Enable the framework code to allow more than one mlink per mpage.Ingo Schwarze2013-12-271-2/+3
| | | | Not used yet.
* Clean up the interface of mlink_add().Ingo Schwarze2013-12-271-69/+66
| | | | | Consistently use "fsec" and "fform" for info derived from the file name. No functional change.
* Another step on the way to clear naming, this time regarding mlinks:Ingo Schwarze2013-12-271-76/+36
| | | | | | | | * rename global ohash filenames to mlinks * rename ofadd() to mlink_add() * fold fileadd() and inoadd() into mlink_add() * fold filecheck() into mpages_merge() Still no functional change.
* Split struct mlink out of struct mpage.Ingo Schwarze2013-12-271-82/+123
| | | | Still a 1:1 relation, no functional change yet.
* Add an additional mlinks table to the database, redundant for now,Ingo Schwarze2013-12-271-21/+52
| | | | | | both because it contains nothing but a subset of the data of the existing mpages table and because the relationship of mpage and mlink entries is still 1:1. But all that will eventually change.
* Drop the mpages_list, use the existing mpages ohash for iteration.Ingo Schwarze2013-12-261-23/+27
| | | | | No functional change except that the order of database entries changes, which doesn't matter anyway.
* The name "id" is terrible for a struct.Ingo Schwarze2013-12-261-14/+14
| | | | | Make this more searchable by calling it "inodev". No functional change.
* To better support MLINKS, we will have to split the "docs" databaseIngo Schwarze2013-12-261-178/+183
| | | | | | | | table into two tables, on for actual files on disk, one for (often multiple) directory entries pointing to them. That implies splitting struct of into two structs, to be called "mpage" and "mlink", respectively. As a preparation, globally rename "of" and "inos" to "mpage". No functional change.
* Stop parsing man(7) input when we found all we were searching for,Ingo Schwarze2013-12-261-1/+4
| | | | such that we don't trigger an assertion on a duplicate NAME section.
* The man(7) language has no syntax to specify architectures, but itIngo Schwarze2013-10-271-10/+11
| | | | | | | | | | can still be used to write architecture-specific manuals, of course. So just derive the architecture a man(7) manual belongs to from the directory where it is located and refrain from warning about each and every architecture-specific man(7) manual found. While here, delete some trailing whitespace in the neighbourhood.
* The code in ofmerge() only tried the source parsers if at least oneIngo Schwarze2013-10-271-1/+1
| | | | | | | | | | | | | | | | | | | of the path (/man1/ .. /man9/) or the file name suffix (*.1 .. *.9) indicated a source manual. That missed source manuals with unusual names in unusual locations. Instead, as the existing comment right above already suggests, try the source parsers unless both the path and the file name suffix unambiguously indicate a preformatted manual (/cat*/*.0). This change is not expensive in practice because no real-world system will have large numbers of preformatted pages outside /cat*/*.0. The only way to make information loss even less probable would be to try the source parsers on all files, even /cat*/*.0, which wouldn't buy us much because no real-world system will call source manuals /cat*/*.0, but it will be expensive in practice, because many real-world systems have large numbers of preformatted pages called /cat*/*.0.
* delete duplicate NULL check and polish style;Ingo Schwarze2013-10-271-5/+5
| | | | no functional change
* Fix an assertion in dbindex(): Null strings are never entered into theIngo Schwarze2013-10-181-1/+1
| | | | string table. Fortunately, they never need UTF-8 translation either.
* Manuals to be checked with "mandocdb -t" need not be in the currentIngo Schwarze2013-10-181-1/+1
| | | | directory or one of its subdirectories.
* For the strings table, ohash_init is only called in ofmerge(),Ingo Schwarze2013-07-021-16/+15
| | | | | so move the str_info structure into that function. No functional change.
* Turning off synchronous mode logically belongs to opening the database,Ingo Schwarze2013-07-021-9/+11
| | | | so move the statement into the function dbopen().
* Restore the check whether each page added to the databaseIngo Schwarze2013-07-021-18/+86
| | | | | is actually reachable by man(1). This check got lost when the database backend was changed from Berkeley to sqlite.
* The mdoc_handler flags are unused and will never be used.Ingo Schwarze2013-06-071-126/+123
| | | | | Having a mask is sufficient to trigger putmdockey. Simplify by dropping the flags; no functional change.
* In .Xr database entries, mention the manual section again;Ingo Schwarze2013-06-071-2/+13
| | | | | the section was dropped when switching from db to sqlite. Use the customary format foo(N).
* The string hash table is only needed to combine multiple occurencesIngo Schwarze2013-06-071-109/+67
| | | | | | | | | | | | | | | | | | | | | | of the same string within the same manual, so initialize and purge it for each manual in ofmerge() instead of one single time in main(). There is no point in saving manual names and descriptions in that table because each of them occurs only once, or very few times. The is no point in saving section numbers there because they are so much shorter than the descriptions. Testing with the complete tree /usr/share/man/ on my notebook shows that this change slightly reduces memory consumption by about 20% while there is no measurable difference in execution time. As a bonus, this allows to delete the functions stradd() and stradds(), the "next" member from struct str, and the global struct str *words. While adapting the places in the code using stradd(), i noticed that parsing of the mdoc(7) .Nd macro was completely broken and that for formatted manual pages with unusable NAME section, the description was never set in the struct of. This commit fixes both bugs as well.
* Optimize stradds() and putkeys() to not call ohash_qlookupi()Ingo Schwarze2013-06-061-44/+27
| | | | | | | and ohash_find() twice. As a bonus, this allows to drop hashget(). While here, rename index to slot to match the terminology in the ohash manual; it also prevents potential clashes with index(3). Drop the slot variable altogether where it is used only once.
generated by cgit (git 2.34.1) at 2024-11-13 00:50:25 +0000