aboutsummaryrefslogtreecommitdiffstats
path: root/storage/filesystem
Commit message (Collapse)AuthorAgeFilesLines
...
* git, storer: use a common storer.Options for storer and PlainOpenJavi Fontan2018-08-303-29/+17
| | | | Signed-off-by: Javi Fontan <jfontan@gmail.com>
* git: add Static option to PlainOpenJavi Fontan2018-08-303-6/+221
| | | | | | | | | | | | | | | | Also adds Static configuration to Storage and DotGit. This option means that the git repository is not expected to be modified while open and enables some optimizations. Each time a file is accessed the storer tries to open an object file for the requested hash. When this is done for a lot of objects it is expensive. With Static option a list of object files is generated the first time an object is accessed and used to check if exists instead of using system calls. A similar optimization is done for packfiles. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* storage/dotgit: use HasPrefix instead of SplitJavi Fontan2018-08-271-6/+9
| | | | | | Also reformatted function comment and fixed some typos. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* storage/dotgit: search for incoming dir only onceJavi Fontan2018-08-251-13/+28
| | | | | | | | | Search for incoming object directory was done once each time objects were accessed. This means a ReadDir of the objects path that is expensive. Now incoming directory is searched the first time an object is accessed and its name kept in DotGit to be reused. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* plumbing, storage: add bases to the common cacheJavi Fontan2018-08-221-2/+16
| | | | | | | | | | | | After clone only resolved deltas were added to the cache. This caused slowdowns in small repositories where most objects can be held in cache. It also makes packfiles reuse delta cache from the store. Previously it created a new delta cache each time a packfile object was created. This also slowed down a bit accessing objects and had an impact on memory consumption when bases are added to the cache. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* Merge pull request #887 from noxora/hook-supportMáximo Cuadros2018-08-212-3/+101
|\ | | | | added support for quarantine directory
| * dotgit: fix object delete testSantiago M. Mola2018-08-171-14/+18
| | | | | | | | Signed-off-by: Santiago M. Mola <santi@mola.io>
| * added hook supportnoxora2018-08-162-4/+98
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: noxora <ldecker@indeed.com> trying a possible fix to the delete test Signed-off-by: noxora <ldecker@indeed.com> still trying to fix this test Signed-off-by: noxora <ldecker@indeed.com> fixes did not work, seems to be a windows env problem Signed-off-by: noxora <ldecker@indeed.com>
* | plumbing: packfile, open and close packfile on FSObject readsMiguel Molina2018-08-093-12/+48
| | | | | | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* | storage: filesystem, add PackfileIter benchmark reading object contentMiguel Molina2018-08-091-0/+67
| | | | | | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* | storage: filesystem, close Packfile after iterating objectsMiguel Molina2018-08-091-1/+10
| | | | | | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* | storage: filesystem, benchmark PackfileIterMiguel Molina2018-08-092-15/+94
| | | | | | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* | plumbing: packfile, allow non-seekable sources on ParserMiguel Molina2018-08-081-1/+6
| | | | | | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* | *: use parser to populate non writable storages and bug fixesMiguel Molina2018-08-072-48/+35
| | | | | | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* | storage/filesystem: remove duplicated IndexStorageJavi Fontan2018-07-271-15/+0
| | | | | | | | Signed-off-by: Javi Fontan <jfontan@gmail.com>
* | Merge pull request #899 from erizocosmico/feature/new-packfileMiguel Molina2018-07-271-0/+15
|\ \ | | | | | | plumbing: packfile, new Packfile representation
| * | plumbing: packfile, new Packfile representationMiguel Molina2018-07-261-0/+15
| | | | | | | | | | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* | | storage/filesystem: add back IndexStorageJavi Fontan2018-07-271-0/+47
|/ / | | | | | | Signed-off-by: Javi Fontan <jfontan@gmail.com>
* | plumbing, storage: integrate new indexJavi Fontan2018-07-263-36/+46
| | | | | | | | | | | | Now dotgit.PackWriter uses the new packfile.Parser and index. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* | plumbing/format/idxfile: add new Index and MemoryIndexMiguel Molina2018-07-192-48/+1
|/ | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* storage/filesystem: avoid norwfs build flagSantiago M. Mola2018-06-215-94/+82
| | | | | | | | | | | | | | | | | norwfs build flag was used to work on filesystems that do not support neither opening a file in read/write mode or renaming a file (e.f. sivafs). This had two problems: - go-git could not be compiled to work properly both with regular filesystems and limited filesystems at the same time. - the norwfs trick was not available on Windows. This PR removes the norwfs build flag, as well as the windows conditional flag on the dotgit package. For the file open mode, we use the new billy capabilities, to check at runtime if the filesystem supports opening a file in read/write mode or not. For the renaming, we just try and fallback to alternative methods if billy.ErrNotSupported is returned. Signed-off-by: Santiago M. Mola <santi@mola.io>
* storage: filesystem, make ObjectStorage constructor publicMiguel Molina2018-06-083-8/+9
| | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* dotgit: Move package outside internal.Antonio Jesus Navarro Perez2018-06-0518-9/+9
| | | | Signed-off-by: Antonio Jesus Navarro Perez <antnavper@gmail.com>
* storage: filesystem, close shallow file when readMáximo Cuadros2018-04-171-0/+2
| | | | Signed-off-by: Máximo Cuadros <mcuadros@gmail.com>
* Merge pull request #807 from keybase/strib/src-d-ignore-non-hash-filesMáximo Cuadros2018-04-172-3/+23
|\ | | | | dotgit: ignore filenames that don't match a hash
| * dotgit: add test for bad file in pack directoryJeremy Stribling2018-04-161-1/+13
| | | | | | | | | | | | | | Suggested by mcuadros. Issue: src-d/go-git#807 Signed-off-by: Jeremy Stribling <strib@alum.mit.edu>
| * dotgit: ignore filenames that don't match a hashJeremy Stribling2018-04-131-2/+10
| | | | | | | | | | | | | | For both packfiles and object files. Issue: keybase/client#11366 Signed-off-by: Jeremy Stribling <strib@alum.mit.edu>
* | storage: dotgit, init fixtures in benchmark. Fixes #770Javi Fontan2018-04-161-0/+1
|/ | | | | | | fixtures is not initialized in BenchmarkRefMultipleTimes and caused panic. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* storage: filesystem/dotgit, fix typo in return paramJavi Fontan2018-03-281-1/+1
| | | | Signed-off-by: Javi Fontan <jfontan@gmail.com>
* *: Use CheckClose with named returnsJavi Fontan2018-03-275-15/+15
| | | | | | | | Previously some close errors were losts. This is specially problematic in go-git as lots of work is done here like generating indexes and moving packfiles. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* storage/filesystem: optimize packfile iteratorDenys Smirnov2018-03-031-22/+61
| | | | | | | | * do not store extra bool values in the seen map * open packfile iterators lazily Signed-off-by: Denys Smirnov <denys@sourced.tech>
* all: remove some unused codeDaniel Martí2018-03-011-2/+0
| | | | Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
* storage/filesystem/shallow: fix error checkingferhat elmas2018-02-011-1/+1
| | | | Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
* Merge pull request #700 from jfontan/improvement/support-non-rw-filesystemsMáximo Cuadros2018-01-026-33/+142
|\ | | | | Add a setRef and rewritePackedRefsWhileLocked versions that supports non rw fs
| * Add norwfs version of rewritePackedRefsWhileLockedJavi Fontan2017-12-224-4/+50
| | | | | | | | Signed-off-by: Javi Fontan <jfontan@gmail.com>
| * Check reference also in norwfs SetRefJavi Fontan2017-12-221-2/+25
| | | | | | | | Signed-off-by: Javi Fontan <jfontan@gmail.com>
| * Add comment to the norwfs version of SetRefJavi Fontan2017-12-211-0/+7
| | | | | | | | Signed-off-by: Javi Fontan <jfontan@gmail.com>
| * Add a setRef version that supports non rw fsJavi Fontan2017-12-213-29/+62
| | | | | | | | | | | | | | | | | | There are some filesystems that do not support opening the files in read and write modes at the same time. The method SetRef is split in files with an extra version that only writes the reference. It can be activated with -tags norwfs on building. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* | Make DeltaBaseCache privateJavi Fontan2017-12-201-6/+6
| | | | | | | | Signed-off-by: Javi Fontan <jfontan@gmail.com>
* | Enforce the use of cache in packfile decoderJavi Fontan2017-12-201-5/+2
|/ | | | | | | | | | | | | | | | | Decoder object can make use of an object cache to speed up processing. Previously the only way to specify it was changing manually the struct generated by NewDecodeForFile. This lead to some instances to be created without it and penalized performance. Now the cache should be explicitly passed to the constructor function. NewDecoder now creates objects with a cache using the default size. A new helper function was added to create cache objects with the default size as this becomes a common task now: cache.NewObjectLRUDefault() Signed-off-by: Javi Fontan <jfontan@gmail.com>
* *: simplicationferhat elmas2017-12-181-5/+1
| | | | | | - no unnecessary err/bool check, uses them directly Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>
* storage: filesystem, add support for git alternates (#663)Sunny2017-12-063-1/+118
| | | | This change adds a new method Alternates() in DotGit to check and query alternate source.
* dotgit: use Equal method of time.Time for equalityferhat elmas2017-12-041-1/+1
|
* dotgit: rewrite packed-refs while holding lockJeremy Stribling2017-11-303-60/+103
| | | | | | Windows file system doesn't let us rename over a file while holding that file's lock, so use rewrite as a last resort. It could result in a partially-written file, if there's a failure at the wrong time.
* dotgit: open+lock packed-refs file until it doesn't changeJeremy Stribling2017-11-291-18/+46
| | | | Windows doesn't like it when we re-open a file we already have locked.
* storage: some minor code cleanupJeremy Stribling2017-11-291-6/+3
| | | | | | Suggested by mcuadros. Issue: #669
* plumbing: add `HasEncodedObject` method to StorerJeremy Stribling2017-11-291-0/+26
| | | | | | | This allows the user to check whether an object exists, without reading all the object data from storage. Issue: KBFS-2445
* Make object repacking more configurableTaru Karttunen2017-11-292-4/+16
|
* Support for repacking objectsTaru Karttunen2017-11-292-13/+29
|
* First pass of prune designTaru Karttunen2017-11-292-10/+62
|
generated by cgit (git 2.34.1) at 2024-11-18 03:30:09 +0000