aboutsummaryrefslogtreecommitdiffstats
path: root/libbe
Commit message (Collapse)AuthorAgeFilesLines
* storage:util:mapfile: convert YAML settings to JSON.W. Trevor King2012-09-173-84/+181
| | | | | | | | | This is faster and still readable. I also add an upgrader to convert bug directories from: Bugs Everywhere Directory v1.4 to: Bugs Everywhere Directory v1.5 but I haven't upgraded the local bugdir yet.
* storage:vcs:git: move Git to ExecGit and add PygitGit using pygit2.W. Trevor King2012-09-131-11/+238
| | | | | | | | | | | | | | | | | | | | | | | | | | | | If `pygit2` is installed, use it (via PygitGit) rather than calling the stand-alone `git` executable for all the Git (via ExecGit) for all the Git storage stuff. This saves one serializing/deserializing and process setup/teardown. I timed each test suite by commenting out one of the `make_vcs_testcase_subclasses()` calls at the end of `git.py` and running $ time python test.py libbe.storage.vcs.git The ExecGit tests ran in 13.7s and the PygitGit tests ran in 3.6s for a 3.8x speedup. I had to stretch pygit2 a bit to get a clean fit, so if you want to test this, you'll need to build pygit2 with the following merge requests: Add 'pygit2.__version__' for easy access from client software. https://github.com/libgit2/pygit2/pull/128 commit 0238fb72dfdf2a2308f2da347717cbaafddc4b83 signature: Add keyword argument parsing to Signature(). https://github.com/libgit2/pygit2/pull/129 commit c934858b629f40221406f34166dd77e881b9d5fd repository: add Repository.revparse_single() https://github.com/libgit2/pygit2/pull/131 commit 3afdc8b2f59ed137531671fedde36f3a39cbcc9d
* storage:vcs:base: fix _gets_ -> _get_ typo in test_gets_existing_user_id.W. Trevor King2012-09-131-1/+1
| | | | | VCS_get_user_id_TestCase.test_gets_existing_user_id is now VCS_get_user_id_TestCase.test_get_existing_user_id
* storage:vcs:base: fix realtive -> relative typo in _u_abspath docstring.W. Trevor King2012-09-131-1/+1
|
* command:html: add --strip-email option.W. Trevor King2012-09-031-6/+21
|
* command:html: oops, use default arguments for 'notify' and 'auth'.W. Trevor King2012-09-031-2/+2
|
* command:html: fix super(HTML. self) -> super(HTML, self).W. Trevor King2012-09-031-1/+1
|
* ui:command_line: use `be help server` for help with `--server`.W. Trevor King2012-09-031-1/+1
| | | | | Use the recently created topic, which provides more details than the `serve-commands` help message.
* ui:command_line: display help.TOPICS during `be help`.W. Trevor King2012-09-031-2/+14
|
* command:help: add `repo` and `server` topics.W. Trevor King2012-09-031-1/+60
|
* command:help: match against command names (not module names).W. Trevor King2012-09-031-1/+1
|
* ui:command_line: use command names in command-list help message.W. Trevor King2012-09-031-2/+2
| | | | | | The old implementation used module names. For example: command name: import-xml module name: import_xml
* ui:command_line: add 'html' command to no-pager commands.W. Trevor King2012-09-031-1/+2
| | | | | | | | | | | | Also rename 'serve' -> 'serve-storaage'. This catches the server version up with: commit 0336db8d4052a319f1d959a33fc847814551e3f8 Author: W. Trevor King <wking@tremily.us> Date: Wed Aug 29 23:55:37 2012 -0400 command:serve-storage: rename `be serve` -> `be serve-storage`.
* util:wsgi: WSGITestCase should not subclass WSGICaller.W. Trevor King2012-09-031-10/+18
| | | | | | | | The old implementation caused a number of unittest errors. This commit adds a WSGICaller instance as WSGITestCase.caller, and provides a wrapper .getURL method which copies the status/headers/etc. from the caller back into the WSGITestCase.
* command:html: rewrite as a libbe.util.wsgi.ServerCommand.W. Trevor King2012-09-031-291/+368
| | | | | | | | | Now `be html` defaults to serving the BE repo live. With the `--export-html` option, it reverts to the earlier static dump behaviour. This will make it easier for others to embed a read-only HTML server in their own app (e.g. as a gitorious issue tracker).
* util:wsgi: Pull WSGICaller out of WSGITestCase.W. Trevor King2012-09-031-49/+56
|
* command:serve-storage: rename server_version to BE-storage-server/<VERSION>.W. Trevor King2012-09-031-1/+1
| | | | | | | | | | This catches the server version up with: commit 0336db8d4052a319f1d959a33fc847814551e3f8 Author: W. Trevor King <wking@tremily.us> Date: Wed Aug 29 23:55:37 2012 -0400 command:serve-storage: rename `be serve` -> `be serve-storage`.
* command:serve_commands: allow unspecified parameters (use defaults).W. Trevor King2012-09-031-2/+12
| | | | | | | | | | | | | | | Also raise UnknownCommand if there is no `command` key in the posted dict (malformed request). With the new code, you can run commands with: $ wget --post-data='command: list' http://localhost:8000/run/ instead of having to go through and specify all the parameters explicitly. This will make the command server more robust for use with older clients (who may not know about all the parameters that the server knows about). Parameters sent by the client that the server does not know about are silently ignored.
* util:wsgi: add HandlerErrorApp to return HTTP errors.W. Trevor King2012-09-031-0/+13
|
* util:wsgi: fix WSGI_AppObject.default_callback -> .default_handler typo.W. Trevor King2012-09-031-1/+1
|
* command:import_xml: fix `--r` -> `-r` in long help.W. Trevor King2012-09-031-2/+2
|
* command:import_xml: fix be-mbox-to-xml -> be-mail-to-xml.W. Trevor King2012-09-031-3/+3
| | | | | | | | | | This catches the doc up with misc/xml/be-mail-to-xml: commit c8985785eb741ff646082879f1ca5e9cfe3873b0 Author: W. Trevor King <wking@drexel.edu> Date: Wed Jan 20 15:22:28 2010 -0500 'be-mbox-to-xml' -> 'be-mail-to-xml' + support for several formats.
* command:serve-storage: rename `be serve` -> `be serve-storage`.W. Trevor King2012-08-294-11/+16
| | | | | | | | | | This will help avoid confusion between be serve-storage and be serve-commands
* Rewrite commands to use bugdirs instead of a single bugdir.W. Trevor King2012-08-2925-447/+831
| | | | | | | The bulk of the work is in regard to XML, with new BugDir.xml and .from_xml methods to support the new <bugdir> entity. I also split the guts import_xml's ._run method into sub-methods to make the import logic more obvious.
* util:wsgi: add BEExceptionApp for translating storage exceptions.W. Trevor King2012-08-293-11/+25
| | | | | | This fixes .test_get_initial_value for the HTTP backend, because the tests use TestingHTTP.getURL, which only catch HandlerError, not the more specific storage exceptions.
* bugdir: add `update` argument to BugDir.append().W. Trevor King2012-08-291-6/+8
| | | | | | | | | | | | | | | | | | | | This avoids a deepcopy error where the BugDir tries to update before the Bug has had it's uuid assigned: Traceback (most recent call last): ... File ".../libbe/command/merge.py", line 168, in _run newCommTree = copy.deepcopy(bugB.comment_root) File "/usr/lib64/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) ... File "/usr/lib64/python2.7/copy.py", line 352, in _reconstruct y.append(item) File ".../libbe/bugdir.py", line 263, in append self._bug_map_gen() File ".../libbe/bugdir.py", line 152, in _bug_map_gen map[bug.uuid] = bug AttributeError: 'Bug' object has no attribute 'uuid'
* bugdir: add BugDir.xml(), .from_xml(), and .append().W. Trevor King2012-08-291-3/+230
|
* bug: fix invalid root element comment -> bug in Bug.from_xml().W. Trevor King2012-08-291-1/+1
|
* bug: fix leading comment indentation in Bug.xml().W. Trevor King2012-08-291-3/+41
|
* bug: add doctest for Bug.xml().W. Trevor King2012-08-291-0/+54
|
* Allow symlinks in bugdir directories.W. Trevor King2012-08-281-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | This allows you to do things like: $ mkdir -p /tmp/joint-repo/.be $ cd /tml/joint-repo/.be $ ln -s ~/src/be/.be/version $ ln -s ~/src/be/.be/bea86499-824e-4e77-b085-2d581fa9ccab/ $ ln -s ~/src/BEurtle/.be/7017b289-f207-4e39-9746-f58323404eba/ $ be list without crashing with a: Traceback (most recent call last): File ".../libbe/storage/base.py", line 316, in children return self._children(*args, **kwargs) File ".../libbe/storage/vcs/base.py", line 820, in _children path = self.path(id, revision, relpath=False) File ".../libbe/storage/vcs/base.py", line 721, in path path = self._cached_path_id.path(id) File ".../libbe/storage/vcs/base.py", line 280, in path raise InvalidID(uuid) libbe.storage.base.InvalidID: 7017b289-f207-4e39-9746-f58323404eba in revision None Currently it only lists the first bug directory it comes across, but after this patch, it doesn't crash ;).
* util:plugin: modnames() should only list *.py or *.pyc files.W. Trevor King2012-08-271-1/+3
| | | | | By appending '.py' to *everything*, it was listing '__pycache__' for Python 3.
* libbe:util:wsgi: extract WSGI utilities into a separate module.W. Trevor King2012-08-274-1120/+1054
|
* command:serve_commands: remove duplicate cruft and get working unit tests.W. Trevor King2012-08-241-327/+19
|
* command:serve: add ability to pass raw POST data with getURL.W. Trevor King2012-08-241-12/+16
| | | | | | | | | | This is analagous to the earlier change to get_post_url: commit 0cd072b9710ee964e6f449abd9265d85e02f34d2 Author: W. Trevor King <wking@tremily.us> Date: Fri Aug 24 09:29:58 2012 -0400 util:http: add ability to pass raw POST data with get_post_url.
* command:serve_commands: new command for running a command server.W. Trevor King2012-08-243-4/+709
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is an initial step towards improving BE's efficiency. Previously, BE gets slow as the bug count increases for several commands (e.g. `be list`), because it takes time to load the bugdir information from disk at each invocation. If you use a remote repo (`be --repo http://localhost:8000/ list`), the server process may have already loaded the repo from disk, but now your listing process has to fetch everything over the wire. This is even worse than loading it from disk. With the new `be serve-commands` and `be --server URL ...` pair, the bugdir loading happens once on the server, and all the processing is also carried out on the server. This means that calls like `be --server http://localhost:8000/ list` will scale much better than other methods. For example: $ time be --server http://localhost:8000/ list > /dev/null real 0m2.234s user 0m0.548s sys 0m0.114s $ time be --server http://localhost:8000/ list > /dev/null real 0m0.730s user 0m0.548s sys 0m0.112s $ time be list > /dev/null real 0m2.453s user 0m2.289s sys 0m0.166s $ time be list > /dev/null real 0m2.521s user 0m2.350s sys 0m0.172s The first call to a cold server takes about the same time as a local call, because you need to load the bugs from the filesystem. However, later calls to a warm server are 3x faster, while later local calls are still slow. This is currently a minimal working implementation. There's a good deal of code in libbe.command.serve that I'd like to abstract out into a libbe.util library (since there's still a bunch of duplication between libbe.command.serve and libbe.command.serve_commands). The remote calls are also not as fast as I'd like, likely due to library load times. This commit just locks in an initial working implementation.
* command:base: only cleanup UseInterface.storage_callback if it's there.W. Trevor King2012-08-241-1/+2
| | | | | It's hard to see why it wouldn't be, but .setup_command handles the case where it's missing, so we should be consistent here.
* command:base: use is/is-not None instead of ==/!= None in setup_command.W. Trevor King2012-08-241-3/+3
| | | | More Pythonic.
* command:serve: wrap server in ExceptionApp regardless of SSL.W. Trevor King2012-08-241-1/+1
| | | | Previously, only the SSL server was using ExceptionApp.
* util:http: add ability to pass raw POST data with get_post_url.W. Trevor King2012-08-241-10/+17
|
* util:http: convert urllib2.URLError into HTTPError in get_post_url.W. Trevor King2012-08-241-5/+11
| | | | | Also rework liburl2.HTTPError handling to get both the reason and the error code into the HTTPError message.
* util:http: pull HTTP helpers from libbe.storage.http into their own module.W. Trevor King2012-08-243-106/+130
| | | | This way they can be shared with the upcoming Command._run_remote.
* storage:http: use an explicit relative import for base.W. Trevor King2012-08-241-2/+3
|
* storage:http: adjust HTTPError message in get_post_url.W. Trevor King2012-08-241-2/+2
| | | | | | We'll usually get this message if we fail to reach the server, but we can also get it if the server is so broken that it dies without returning a valid HTTP response.
* storage:http: add agent argument to get_post_url.W. Trevor King2012-08-241-2/+4
| | | | | This allows us to set an appropriate user agent if we use this function in other places (e.g. upcoming Command._run_remote).
* command:serve: add SilentRequestHandler to avoid double-logging requests.W. Trevor King2012-08-231-2/+8
| | | | | | | | | | | | | | | | They were being logged by the request handler, and then logged again by WSGI_Object.log_request. After this patch, requests are only logged when logger.level is at a <= level (e.g. closer to DEBUG) than the WSGI_Object.log_level (usually INFO). For example, in Serve_setup_logging, self.logger.setLevel(logging.INFO) will get request logs printed, but self.logger.setLevel(logging.WARNING) will not.
* libbe:util:plugin: cleanup recent zip/egg handling code.W. Trevor King2012-08-231-33/+16
| | | | | | | To test this, you'll need to build an egg. You can use setuptools to do this, by applying: <snip-patch>
* Cherrypick improved zipfile support from Niall Douglas.Niall Douglas (a [underscors] sourceforge {at} nedprod [dot] com)2012-08-231-9/+15
| | | | | | | | | | | WTK: This is the meat of Niall's commit 4632cb6d22faa7220540f92af67693084f80f033 Author: Niall Douglas ... Date: Wed Feb 22 17:46:06 2012 +0000 Fixed small bug where running from inside zip support was over preferring .pyc files
* Cherrypick initial zipfile support from Niall Douglas.Niall Douglas (a [underscors] sourceforge {at} nedprod [dot] com)2012-08-231-2/+27
| | | | | | | | | | | WTK: This is a portion of Niall's commit 7f7a7738bcbcfd06a026f2985c1823a4ba5eb55b Author: Niall Douglas ... Date: Tue Feb 21 20:35:28 2012 +0000 Several hacks to make BE compatible with bbfreeze and therefore compilable into a self contained directory
* storage:vcs:hg: remove grandchildren from Hg._vcs_listdir.W. Trevor King2012-08-231-1/+4
| | | | | | | | | | | | | | | This fixes: From: Phil Schumm Date: Thu, 23 Aug 2012 08:15:21 -0500 Subject: [Be-devel] -be diff- under Hg ... I've just noticed that -be diff- seems to yield different output under Git than it does under Hg (e.g., in a Mercurial +repository, -be diff- appears to show all changes as new bugs rather than modified). ...
generated by cgit (git 2.34.1) at 2024-09-20 12:23:08 +0000