diff options
Diffstat (limited to 'doc/docco.txt')
| -rw-r--r-- | doc/docco.txt | 717 |
1 files changed, 717 insertions, 0 deletions
diff --git a/doc/docco.txt b/doc/docco.txt new file mode 100644 index 0000000..1860f13 --- /dev/null +++ b/doc/docco.txt @@ -0,0 +1,717 @@ +Patch management scripts +Andrew Morton <akpm@digeo.com> +18 October 2002 + +This is a description of a bunch of shell scripts which I use for +managing kernel patches. They are quite powerful. They can be used on +projects other than the linux kernel. They are easy to use, and fast. + +You end up doing a ton of recompiling with these scripts, because +you're pushing and popping all the time. ccache takes away the pain of +all that. http://ccache.samba.org/ - be sure to put the cache +directory on the same fs as where you're working so that ccache can use +hardlinks. + +The key philosophical concept is that your primary output is patches. +Not ".c" files, not ".h" files. But patches. So patches are the +first-class object here. + +Installation +============ + +You place all the scripts somewhere in your path, or in +/usr/lib/patch-scripts. + +Terminology +=========== + +The patch scripts require three special directories called "pc", +"patches" and "txt". + +If the environment variable PATCHSCRIPTS is set, it is taken to to be +the directory in which those three directories reside. Typically, it +would be a relative pathname. So + + setenv PATCHSCRIPTS ./i-put-them-here + +would tell the patch scripts to look in ./i-put-them-here/pc, etc. + +If PATCHSCRIPTS is not set, and the directory ./patch-scripts is +present then the patch scripts will us ./patch-scripts/pc/, +./patch-scripts/patches/ and ./patch-scripts/txt/. + +Otherwise, the patch scripts use ./pc, ./patches and ./txt. + +In this document, the symbol $P is used to describe the directory which +holds the pc/, patches/ and txt/ directories, as determined by the +above search. + +It is expected that $P will always expand to a relative path. + +Concepts +======== + +All work occurs with a single directory tree. All commands are invoked +within the root of that tree. The scripts manage a "stack" of patches. + +Each patch is a changeset against the base tree plus the preceding patches. + +All patches are listed, in order, in the file ./series. You manage the +series file. Lines in the series file which start with `#' are ignored. + +Any currently-applied patches are described in the file +./applied-patches. The patch scripts manage this file. + +Each patch affects a number of files in the tree. These files are +listed in a "patch control" file. These .pc files live in the +directory $P/pc/ + +Patches are placed in the directory $P/patches/ + +Documentation for the patches is placed in $P/txt/ + +So for a particular patch "my-first-patch" the following will exist: + +- An entry "my-first-patch.patch" in ./series + +- An entry "my-first-patch" in ./applied-patches (if it's currently applied) + +- A file $P/pc/my-first-patch.pc which contains the names of the + files which my-first-patch modifies, adds or removes + +- A file $P/txt/my-first-patch.txt which contains the patch's + changelog. + +- A file $P/patches/my-first-patch.patch, which is the output of the + patch scripts. + +Operation +========= + +When a patch "my-patch" is applied with apatch, or with pushpatch +(which calls apatch), all the affected files (from $P/pc/my-patch.pc) +are copied to files with ~my-patch appended. So if $P/pc/my-patch.pc +contained + + kernel/sched.c + fs/inode.c + +then apatch will copy those files into kernel/sched.c~my-patch and +fs/inode.c~my-patch. It will then apply the patch to kernel/sched.c +and fs/inode.c + +When a diff is regenerated by refpatch (which calls mpatch), the diff +is made between kernel/sched.c and kernel/sched.c~my-patch. How do the +scripts know to use "~my-patch"? Because my-patch is the current +topmost patch. It's the last line in ./applied-patches. + +In this way, the whole thing is stackable. If you have four patches +applied, say "patch-1", "patch-2", "patch-3" and "patch-4", and if +patch-2 and patch-4 both touch kernel/sched.c then you will have: + + kernel/sched.c~patch-2 Original copy, before patch-2 + kernel/sched.c~patch-4 Copy before patch-4. Contains changes + from patch-2 + kernel/sched.c Current working copy. Contains changes + from patch-4. + +This means that your diff headers contain "~patch-name" in them, which +is convenient documentation. + +Walkthrough +=========== + +Let's start. + +Go into /usr/src/linux (or wherever) + + mkdir pc patches txt + +Now let's generate a patch + + fpatch my-patch kernel/sched.c + +OK, we've copied kernel/sched.c to kernel/sched.c~my-patch. We've +appended "my-patch" to ./applied-patches and we've put "kernel/sched.c" +into the patch control file, pc/my-patch.pc. + + Now edit kernel/sched.c a bit. + +Now we're ready to document the patch + + Now write txt/my-patch.txt + +Now generate the patch + + refpatch + +This will generate patches/my-patch.patch. Take a look. + +Now remove the patch + + poppatch + +applied-patches is now empty, and the patch is removed. + +Now let's add a file to my-patch and then generate my-second-patch: + + Add "my-patch.patch" to ./series (no blank lines in that file please) + + pushpatch + +OK, the patch is applied again. Let's add another file + + fpatch kernel/printk.c + +Note that here we gave fpatch a single argument. So rather than +opening a new patch, it adds kernel/printk.c to the existing topmost +patch. That's my-patch. + + Edit kernel/printk.c + +Refresh my-patch (you end up running refpatch a lot) + + refpatch + +Now start a second patch: + + fpatch my-second-patch kernel/sched.c + +Now take a look at applied-patches. Also do an `ls kernel/sched*'. + + Edit kernel/sched.c, to make some changes for my-second-patch + +Generate my-second-patch: + + refpatch + +Take a look in patches/my-second-patch.patch + +Don't forget to add "my-second-patch.patch" to the series file. + +And remove both patches: + + poppatch + poppatch + + +That's pretty much it, really. + + +Command reference +================= + +Generally, where any of these commands take a "patch-name", that can be +of the form txt/patch-name.txt, patch-name.pc, just patch-name or +whatever. The scripts will strip off a leading "txt/", "patches/" or +"pc/" and any trailing extension. This is so you can do + + apatch patches/a<tab> + +to conveniently use shell tabbing to select patch names. + + + +added-by-patch + + Some internal thing. + +apatch [-f] patch-name + + This is the low-level function which adds patches. It does the + copying into ~-files and updates the applied-patches file. It + applies the actual patch. + + apatch will do a patch --dry-run first and will refuse to apply the + patch if the dryrun fails. + + So when you are getting rejects you do this: + + pushpatch # This fails, due to rejects. Drat. + apatch -f patch-name # Force the patch + (or) pushpatch -f # Force the patch + + OK, you've now applied patch-name, but you have rejects. Go fix + those up and do + + refpatch + + And you're ready to move on. + +combine-series output-file + + It incrementally combinediffs all the patches in series to make a + complete patch for the series. Requires combinediff frmo patchutils. + + See http://cyberelk.net/tim/patchutils/ (Don't download the + "experimental" patchutils - it seems to only have half of the + commands in it. Go for "stable") + +cvs-take-patch + + I forget. + +export_patch + + export the patches listed in ./series to a set of files which + are named in such a way that the sort order is the same as the + order of the series file. + + Usage: export_patch directory [prefix] + + Example: + + Suppose ./series contains + + mango.patch + orange.patch + banana.patch + apple.patch + pear.patch + + export_patch ../mypatches fruit + + The patches would be copied to + + ../mypatches/p00001_fruit_mango.patch + ../mypatches/p00002_fruit_orange.patch + ../mypatches/p00003_fruit_banana.patch + ../mypatches/p00003_fruit_banana.patch + ../mypatches/p00003_fruit_banana.patch + + Named in this way, someone may easily apply them: + + cat mypatches/p*fruit* | patch -p1 + + If prefix is omitted, the patchnames will be transformed + such that "original.patch" becomes "pXXXXX_original.patch". + +fpatch [patch-name] foo.c + + If patch-name is given, fpatch will start a new patch which + modifies (or adds, or removes) the single file foo.c. It updates + ./applied-patches and creates pc/patch-name.pc. fpatch will copy + foo.c to foo.c~patch-name in preparation for edits of foo.c. + + If patch-name is not given then fpatch will add foo.c to the + current topmost patch. It will add "foo.c" to $P/pc/$(toppatch).pc. + It will copy foo.c to foo.c~$(toppatch). + +import_patch + + Imports a set of patch files, creating $P/pc, $P/txt, $P/patches and + ./series as necessary. It also creates $P/txt/*.txt by stripping + off the top of the patches (and removes any diffstat output it finds, + so that it can eat refpatch output and export_patch output.) The + imported patch names are appended to the series file. + + In creating the $P/txt/*.txt files, mail headers are stripped with + formail, preserving the "From:" and "Subject:" lines. "DESC" and + "EDESC" markers are added if they are not already present, using the + "From:" and "Subject:" lines for the DESC portion, if they are present. + (See "patchdesc" command, below, for more on these markers.) + + Also, it can rename the patch file as it is imported by stripping out + a pattern. This is useful if, as often is the case, you have patch + sets with filenames designed to help sort the patches into the correct + order, such as "p001_xxx_funky_stuff.patch" you can have it automatically + renamed to funky_stuff.patch on import, and let the series file manage + the ordering. + + Import_patch will uncompress patches (*.Z, *.bz2, *.gz) as necessary. + + Usage: + + import_patch [-p pattern] patchfile ... + + Example: + + % ls ../fruit/p*patch + ../fruit/p00001_northern_apple.patch + ../fruit/p00001_tropical_mango.patch + ../fruit/p00002_northern_pear.patch + ../fruit/p00002_tropical_orange.patch + ../fruit/p00003_tropical_banana.patch + % import_patch -p 'p[0-9]*_tropical_' ../fruit/p*tropical* + Recreated pc/mango.pc + Recreated pc/orange.pc + Recreated pc/banana.pc + % import_patch -p 'p[0-9]*_northern_' ../fruit/p*northern* + Recreated pc/apple.pc + Recreated pc/pear.pc + + Then you can "pushpatch; refpatch" 5 times. + +inpatch + + List the names of ths files which are affected by the current + topmost patch. + + This is basically + + cat pc/$(toppatch).pc + +join-patch patchname + + "joins" the named patch to the current topmost patch. + + Use this when you want to merge two patches into one. All the + files which `patchname' affects are added to pc/$(toppatch).pc (if + they are not already there) and patch `patchname' is applied. The + top patch remains unchanged. You'll need to run refpatch afterwards. + +mpatch + + A low-level thing to generate patches + +new-kernel + + Some thing I use for importing a new kernel from kernel.org + +p0-2-p1 + + Internal thing to convert patch -p0 form into patch -p1 + +patchdesc + + Generates a single-line description of a patch. + + The txt/my-patch.txt files have the following format: + + <start of file> + DESC + some short description + EDESC + + The long description + <end of file> + + I use + + patchdesc $(cat series) + + to generate short-form summaries of the patch series. + +patchfns + + Internal utilities + +pcpatch + + Standalone tool to generate a .pc file from a patch. + + Say someone sends you "his-patch.diff". What you do is: + + cp ~/his-patch.diff patches/his-patch.patch + pcpatch his-patch + + This generates $P/pc/his-patch.pc and you're all set. Add + "his-patch.patch" to ./series in the right place and start pushing. + +p_diff + + I forget + +poppatch + + Remove one or more patches from the current stack. This command + does *not* use the series file. It works purely against + applied-patches. + + Usage: + + poppatch + Remove the topmost patch + poppatch 10 + Remove ten patches + poppatch some-patch-name[.patch] + Remove patches until "some-patch-name" is top patch + +pstatus + + Shows status of patches + + Usage: + pstatus [patchfile ...] + + One line per patch is output showing: + 1: Patch number in the series file + 2: Whether the patch is currently applied + 3: Name of patch + 4: Status of the patch (needs pcpatch, changelog, refpatch) + + If no patchfiles are specified, $P/patches/*.patch + are assumed. + + Caveats: + A patch set which contains separate patches to add a file + and modify that same file may give spurious "Needs refpatch" + status for the patch which adds the file or the topmost patch. + +ptkdiff + + Two modes: + + ptkdiff - + + Run tkdiff against all the file affected + by $(toppatch). The diff is only for the changes made + by the top patch! ie: it's between "filename" and + "filename~toppatch-name". + + ptkdiff filename + + Just run tkdiff against that file, + showing the changes which are due to toppatch. + +pushpatch [-f] + + Apply the next patch, from the series file. + + This consults ./applied-patches to find out the top patch, then + consults ./series to find the next patch. And pushes it. + + pushpatch + + Apply the next patch + + pushpatch 10 + + Apply the next ten patches + + pushpatch some-patch-name + + Keep pushing patches until "some-patch-name" is toppatch + + pushpatch -f + + Push the next patch, ignoring rejects. + +refpatch + + regnerates the topmost patch. Reads all the affected files + from pc/$(toppatch).pc and diffs them against their tilde-files. + + Also pastes into the patch your patch documentation and + generates a diffstat summary. + +removed-by-patch + + Some thing. + +rename-patch + + CVS rename for patches. + +rolled-up-patch + + Bit of a hack. Is designed to generate a rolled-up diff of all + currently-applied patches. But it requires a ../linux-2.x.y tree to + diff against. Needs to be redone. + +rpatch + + Internal command + +split-patch + + Some thing someone write to split patches up. I don't use it. + +tag-series + + Assuming you keep pc/*, patches/* and txt/* under CVS revision + control, tag-series allows you to tag a patchset's individual + components. I use + + tag-series s2_5_44-mm3 pc/2.5.44-mm3-series + + which will attach the cvs tag "s2_5_44-mm3" to every .pc, .patch + and .txt file which is mentioned in the series file + "pc/2.5.44-mm3-series". + + It will also tag pc/2.5.44-mm3-series, which is a bit redundant + given that I use a different series file for each patchset release.. + + +toppatch + + Print the name of the topmost patch. From ./applied-patches + +touched-by-patch patch-filename + + List the names of files which are affected by a diff. + +unitdiff.py + + Rasmus Andersen's script to convert a diff into minimum-context + form. This form has a better chance of applying if you're getting + nasty rejects. But patch can and will make mistakes when fed + small-context input. + + +Work Practices +============== + +I keep the kernel tree, the $P/pc/, $P/patches/ and $P/txt/ contents under +CVS control. This is important... + +I have several "series" files. I keep these in $P/pc/foo-series and use + + ln -s pc/foo-series series + +when I'm working on foo. + +If someone sends me a patch I'll do: + + cp ~/whatever patches/his-patch.patch + pcpatch his-patch + apatch his-patch + + If apatch fails then run `apatch -f his-patch' and fix the rejects. + + refpatch + + to clean up any fuzz. + + poppatch + cvs add pc/his-patch.pc patches/his-patch.patch + cvs commit pc patches + + Now edit ./series and place "his-patch.patch" in the appropriate place. + + +If you're working on a particular patch (say, "dud-patch") and you +balls something up, just run: + + refpatch # Generate the crap patch + poppatch # Remove it all + rm patches/dud-patch.patch + cvs up patches/dud-patch.patch + +and all is well. + + +Getting updates from Linus +========================== + +What I do is to grab the latest -bk diff from +http://www.kernel.org/pub/linux/kernel/people/dwmw2/bk-2.5/ +and do: + + gzip -d < cs<tab> > patches/linus.patch + pcpatch linus + apatch linus | grep diff + + Now fix up all the files which got deleted, + because there's something wrong with bitkeeper diffs: + + cvs up -ko <missing files from the above diff> + + apatch linus + $EDITOR linus/linus.txt + + Add the changeset number to txt/linus.txt + + refpatch + poppatch + + Now add "linus.patch" as the first entry in your ./series file and + start pushing your other patches on top of that. + +BUGS +==== + +Tons and tons. The scripts are fragile, the error handling is ungraceful and +if you do something silly you can end up in a pickle. + +Generally the scripts are very careful to not wreck your files or your +patches. But they can get the ./applied-patches and ~-files into an +awkward state. + +Usually you can sort it out by copying the ~-files back onto the originals +and removing the last line from ./applied-patches. Or do a "refpatch ; +poppatch ; rm patches/troublesome-patch.patch ; cvs up patches". + +If it's really bad, just blow away the entire tree and do a new CVS checkout. + + +Working on non-kernel projects +============================== + +Well it's the same thing. Say you've downloaded a copy of util-linux +and you want to make a change: + + cd /usr/src + tar xvfz ~/util-linux.tar.gz + cd util-linux + mkdir pc patches txt + fpatch my-patch sys-utils/rdev.c + fpatch sys-utils/ipcs.8 + <edit, edit> + refpatch + <ship patches/my-patch.patch> + +How to balls things up +====================== + +Well here's one way. Suppose you have 20 patches applied, and three of +them (say, "p1", "p6" and "p11") all modify "foo.c". + +Now you go and change foo.c. + +Well, to which patch does that change belong? You need to decide. +Let's say you decide "p6". + +If you run `refpatch' when "p11" is toppatch then you lose. The diff +went into p11. + +What you can do is: + +1: + poppatch p6 + <edit> + refpatch + pushpatch p11 + <test> + + (See why ccache is looking good?) + +or + +2: + <edit> + <test> + poppatch p6 <hope like hell that the other patches remove cleanly> + refpatch + + +Another good way of ballsing up is to cheat. Say "oh I just want to make +this one-line change". And "oh, and this one". + +Now you're getting in a mess. It's much, much better to just use the system: + + fpatch junk file1 + fpatch file2 + <edit> + <play> + refpatch + poppatch + rm pc/junk.pc patches/junk.patch + +Merging with -mm kernels +======================== + +Haven't tried this, but it should work: + +- Grab all the patches from broken-out/, place them in your $P/patches/ + +- Copy my series file into ./series (or $P/pc/akpm-series and symlink it) + +- pushpatch 99 + +And you're off and running. The nice thing about this is that you can +send me incremental diffs to diffs which I already have. + +Or whatever. I'm fairly handy with diffs nowadays. Rejects are +expected. I just prefer to have "one concept per diff". + |