aboutsummaryrefslogtreecommitdiffstats
path: root/storage/filesystem/object.go
Commit message (Collapse)AuthorAgeFilesLines
* storage/filesystem: check file object before using cacheJavi Fontan2019-01-301-5/+4
| | | | | | | | | | If the cache is shared between several repositories getFromUnpacked can erroneously return an object from other repository. This decreases performance a little bit as there's an extra fs operation when the object is in the cache but is correct when the cache is shared. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* plumbing: format/packfile, performance optimizations for reading large ↵Filip Navara2018-11-281-23/+36
| | | | | | commit histories (#963) Signed-off-by: Filip Navara <navara@emclient.com>
* storage/filesystem: Added reindex method to reindex packfilesJavier Peletier2018-11-121-0/+5
| | | | Signed-off-by: Javier Peletier <jm@epiclabs.io>
* filesystem: add a new test for EncodedObjectSizeJeremy Stribling2018-10-121-3/+1
| | | | | | Suggested by taruti. Signed-off-by: Jeremy Stribling <strib@alum.mit.edu>
* object: get object size without reading whole objectJeremy Stribling2018-10-111-0/+75
| | | | Signed-off-by: Jeremy Stribling <strib@alum.mit.edu>
* storage/filesystem: add more doc to NewPackfileIterJavi Fontan2018-09-211-4/+7
| | | | Signed-off-by: Javi Fontan <jfontan@gmail.com>
* storage/filesystem: keep packs open in PackfileIterJavi Fontan2018-09-201-10/+23
| | | | | | | | PackfileIter was not taking into account the option KeepDescriptors and was always closing the file. This caused "file already closed" errors when iterating packfiles in with KeepDescriptors active. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* Expose Storage cache.kuba--2018-09-071-19/+10
| | | | Signed-off-by: kuba-- <kuba@sourced.tech>
* storage/dotgit: add KeepDescriptors optionJavi Fontan2018-09-041-1/+9
| | | | | | | | | | This option maintains packfile file descriptors opened after reading objects from them. It improves performance as it does not have to be opening packfiles each time an object is needed. Also adds Close to EncodedObjectStorer to close all the files manualy. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* storage/filesystem: move Options to filesytem and dotgitJavi Fontan2018-09-031-0/+12
| | | | Signed-off-by: Javi Fontan <jfontan@gmail.com>
* plumbing, storage: add bases to the common cacheJavi Fontan2018-08-221-2/+16
| | | | | | | | | | | | After clone only resolved deltas were added to the cache. This caused slowdowns in small repositories where most objects can be held in cache. It also makes packfiles reuse delta cache from the store. Previously it created a new delta cache each time a packfile object was created. This also slowed down a bit accessing objects and had an impact on memory consumption when bases are added to the cache. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* plumbing: packfile, open and close packfile on FSObject readsMiguel Molina2018-08-091-9/+6
| | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* storage: filesystem, close Packfile after iterating objectsMiguel Molina2018-08-091-1/+10
| | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* storage: filesystem, benchmark PackfileIterMiguel Molina2018-08-091-4/+26
| | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* *: use parser to populate non writable storages and bug fixesMiguel Molina2018-08-071-47/+30
| | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* plumbing, storage: integrate new indexJavi Fontan2018-07-261-17/+29
| | | | | | Now dotgit.PackWriter uses the new packfile.Parser and index. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* plumbing/format/idxfile: add new Index and MemoryIndexMiguel Molina2018-07-191-1/+1
| | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* storage: filesystem, make ObjectStorage constructor publicMiguel Molina2018-06-081-2/+3
| | | | Signed-off-by: Miguel Molina <miguel@erizocosmi.co>
* dotgit: Move package outside internal.Antonio Jesus Navarro Perez2018-06-051-1/+1
| | | | Signed-off-by: Antonio Jesus Navarro Perez <antnavper@gmail.com>
* *: Use CheckClose with named returnsJavi Fontan2018-03-271-4/+4
| | | | | | | | Previously some close errors were losts. This is specially problematic in go-git as lots of work is done here like generating indexes and moving packfiles. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* storage/filesystem: optimize packfile iteratorDenys Smirnov2018-03-031-22/+61
| | | | | | | | * do not store extra bool values in the seen map * open packfile iterators lazily Signed-off-by: Denys Smirnov <denys@sourced.tech>
* Make DeltaBaseCache privateJavi Fontan2017-12-201-6/+6
| | | | Signed-off-by: Javi Fontan <jfontan@gmail.com>
* Enforce the use of cache in packfile decoderJavi Fontan2017-12-201-5/+2
| | | | | | | | | | | | | | | | | Decoder object can make use of an object cache to speed up processing. Previously the only way to specify it was changing manually the struct generated by NewDecodeForFile. This lead to some instances to be created without it and penalized performance. Now the cache should be explicitly passed to the constructor function. NewDecoder now creates objects with a cache using the default size. A new helper function was added to create cache objects with the default size as this becomes a common task now: cache.NewObjectLRUDefault() Signed-off-by: Javi Fontan <jfontan@gmail.com>
* storage: filesystem, add support for git alternates (#663)Sunny2017-12-061-0/+21
| | | | This change adds a new method Alternates() in DotGit to check and query alternate source.
* storage: some minor code cleanupJeremy Stribling2017-11-291-6/+3
| | | | | | Suggested by mcuadros. Issue: #669
* plumbing: add `HasEncodedObject` method to StorerJeremy Stribling2017-11-291-0/+26
| | | | | | | This allows the user to check whether an object exists, without reading all the object data from storage. Issue: KBFS-2445
* Make object repacking more configurableTaru Karttunen2017-11-291-2/+2
|
* Support for repacking objectsTaru Karttunen2017-11-291-0/+8
|
* First pass of prune designTaru Karttunen2017-11-291-0/+24
|
* all: simplificationferhat elmas2017-11-291-2/+2
| | | | | | | | | | - no length for map initialization - don't check for boolean/error return - don't format string - use string method of bytes buffer instead of converting bytes to string - use `strings.Contains` instead of `strings.Index` - use `bytes.Equal` instead of `bytes.Compare`
* update to go-billy.v4 and go-git-fixtures.v3Máximo Cuadros2017-11-231-1/+1
| | | | Signed-off-by: Máximo Cuadros <mcuadros@gmail.com>
* Merge pull request #515 from smola/reuse-packed-objectsMáximo Cuadros2017-07-271-7/+97
|\ | | | | storage: reuse deltas from packfiles
| * storage: reuse deltas from packfilesSantiago M. Mola2017-07-271-7/+97
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * plumbing: add DeltaObject interface for EncodedObjects that are deltas and hold additional information about them, such as the hash of the base object. * plumbing/storer: add DeltaObjectStorer interface for object storers that can return DeltaObject. Note that calls to EncodedObject will never return instances of DeltaObject. That requires explicit calls to DeltaObject. * storage/filesystem: implement DeltaObjectStorer interface. * plumbing/packfile: packfile encoder now supports reusing deltas that are already computed (e.g. from an existing packfile) if the storage implements DeltaObjectStorer. Reusing deltas boosts performance of packfile generation (e.g. on push).
* | filesystem: reuse cache for packfile iteratorSantiago M. Mola2017-07-271-3/+4
|/
* plumbing/cache: change FIFO to LRU cacheSantiago M. Mola2017-07-271-1/+1
|
* storage/filesystem: reuse delta cacheSantiago M. Mola2017-07-271-1/+9
| | | | | Reuse delta base object cache for packfile decoders across multiple instances.
* packfile: create packfile.Index and reuse itSantiago M. Mola2017-07-261-33/+18
| | | | | | | | | | | | | | | There was an internal type (i.e. storage/filesystem.idx) to use as in-memory index for packfiles. This was not convenient to reuse in the packfile. This commit creates a new representation (format/packfile.Index) that can be converted to and from idxfile.Idxfile. A packfile.Index now contains the functionality that was scattered on storage/filesystem.idx and packfile.Decoder's internals. storage/filesystem now reuses packfile.Index instances and this also results in higher cache hit ratios when resolving deltas.
* storage/filesystem: check all Close errorsSantiago M. Mola2017-07-191-9/+12
|
* *: upgrade to go-billy.v3, mergeMáximo Cuadros2017-06-181-1/+1
|
* Lazily load object index.JP Sugarbroad2017-04-061-6/+22
| | | | fixes #327
* use go-billy.v2 versionAntonio Jesus Navarro Perez2017-03-101-1/+1
|
* project: move imports from srcd.works to gopkg.inAntonio Jesus Navarro Perez2017-03-071-9/+9
| | | | To be able to fix #261 we will move again to gopkg.in before v4 stable release.
* Fix missing objects if they where deltified using ref-deltaAntonio Jesus Navarro Perez2017-02-071-3/+6
| | | | | - Deleted invalid logic that returned nil if an ref-delta was not found into the decoder index. This logic was missing objects if it was deltified using ref-deltas. - Now, to avoid that problem, index is mandatory to decode correctly a packfile of a specific type. Decoder.SetOffsets method now is called into the EncodedObjectIterator to avoid this problem.
* new srcd.works/go-git.v4 pathMáximo Cuadros2017-01-301-7/+7
|
* storage: IndexStorer implementationMáximo Cuadros2017-01-281-10/+10
|
* packfile/decoder: speed up packfile iterator when specific type (#200)Antonio Navarro Perez2017-01-121-16/+22
|
* Extract billy (#173)Sergio Arbeo2016-12-191-3/+4
| | | | | | | | | | | | | | | * Extract billy Billy is a new library directly extracted from go-git. It abstract several storages systems in a filesystem interface. More in github.com/src-d/billy * Fix grouping in imports block * Update billy to v1 * Re-remove fs_implementation example
* move plumbing from top level package to plumbing (#183)Santiago M. Mola2016-12-141-20/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * plumbing: rename Object -> EncodedObject. * plumbing/storer: rename ObjectStorer -> EncodedObjectStorer. * move difftree to plumbing/difftree. * move diff -> utils/diff * make Object/Tag/Blob/Tree/Commit/File depend on storer. * Object and its implementations now depend only on storer.EncodedObjectStorer, not git.Repository. * Tests are decoupled accordingly. * move Object/Commit/File/Tag/Tree to plumbing/object. * move Object/Commit/File/Tag/Tree to plumbing/object. * move checkClose to utils/ioutil. * move RevListObjects to plumbing/revlist.Objects. * move DiffTree to plumbing/difftree package. * rename files with plural nouns to singular * plumbing/object: add GetBlob/GetCommit/GetTag/GetTree.
* transport: create Client interface (#132)Santiago M. Mola2016-11-231-2/+2
| | | | | | | | | | | | | | | * plumbing: move plumbing/client package to plumbing/transport. * transport: create Client interface. * A Client can instantiate any client transport service. * InstallProtocol installs a Client for a given protocol, instead of just a UploadPackService. * A Client can open a session for fetch-pack or send-pack for a specific Endpoint. * Adapt ssh and http clients to the new client interface. * updated doc
* new plumbing package (#118)Máximo Cuadros2016-11-081-48/+49
| | | * plumbing: now core was renamed to core, and formats and clients moved inside
generated by cgit (git 2.34.1) at 2024-11-07 19:11:09 +0000