aboutsummaryrefslogtreecommitdiffstats
path: root/plumbing/format
Commit message (Collapse)AuthorAgeFilesLines
* plumbing: format/packfile, Fix data race and resource leak.Ed Bardsley2019-06-032-11/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Two problems are fixed: - Buffers are not returned to the pool in the case of errors. - Per https://golang.org/pkg/bytes/#Buffer.Bytes, the slice returned from bytes.Buffer.Bytes() is only valid until the next modification of the buffer, so it must be copied before the buffer is returned to the pool. Running `go test -race` detected the following: ``` ================== WARNING: DATA RACE Write at 0x00c000224020 by goroutine 15: bytes.(*Buffer).WriteByte() /usr/local/Cellar/go/1.11.5/libexec/src/bytes/buffer.go:271 +0xc8 vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.diffDelta() vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/diff_delta.go:95 +0x505 vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.getDelta() vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/diff_delta.go:60 +0x4ae vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*deltaSelector).tryToDeltify() vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/delta_selector.go:309 +0x398 vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*deltaSelector).walk() vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/delta_selector.go:262 +0x33b vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*deltaSelector).ObjectsToPack.func1() vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/delta_selector.go:70 +0x6a Previous read at 0x00c000224020 by goroutine 8: runtime.slicecopy() /usr/local/Cellar/go/1.11.5/libexec/src/runtime/slice.go:221 +0x0 vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.getDelta() vendor/gopkg.in/src-d/go-git.v4/plumbing/memory.go:53 +0x5e2 vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*deltaSelector).tryToDeltify() vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/delta_selector.go:309 +0x398 vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*deltaSelector).walk() vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/delta_selector.go:262 +0x33b vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*deltaSelector).ObjectsToPack.func1() vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/delta_selector.go:70 +0x6a Goroutine 15 (running) created at: vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*deltaSelector).ObjectsToPack() vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/delta_selector.go:69 +0x761 vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*Encoder).Encode() vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/encoder.go:52 +0xb1 vendor/gopkg.in/src-d/go-git%2ev4.pushHashes.func1() vendor/gopkg.in/src-d/go-git.v4/remote.go:1026 +0x102 Goroutine 8 (finished) created at: vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*deltaSelector).ObjectsToPack() vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/delta_selector.go:69 +0x761 vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile.(*Encoder).Encode() vendor/gopkg.in/src-d/go-git.v4/plumbing/format/packfile/encoder.go:52 +0xb1 vendor/gopkg.in/src-d/go-git%2ev4.pushHashes.func1() vendor/gopkg.in/src-d/go-git.v4/remote.go:1026 +0x102 ================== ``` Signed-off-by: Ed Bardsley <ewb@uber.com>
* Remove unnecessary mmap usage from testsFilip Navara2019-05-071-6/+4
| | | | Signed-off-by: Filip Navara <filip.navara@gmail.com>
* Add doc.go for commitgraph packagesFilip Navara2019-05-073-0/+106
| | | | Signed-off-by: Filip Navara <filip.navara@gmail.com>
* Merge pull request #1136 from filipnavara/idxfile-nobufMáximo Cuadros2019-04-271-51/+28
|\ | | | | plumbing: format/idxfile, avoid creating temporary buffers to decode integers
| * plumbing: format/idxfile, avoid looking up the fanout mapping in the ↵Filip Navara2019-04-261-10/+7
| | | | | | | | | | | | iterator twice Signed-off-by: Filip Navara <filip.navara@gmail.com>
| * plumbing: format/idxfile, save another 18% of time in genOffsetHash by not ↵Filip Navara2019-04-251-14/+11
| | | | | | | | | | | | using iterator and not loading CRC Signed-off-by: Filip Navara <filip.navara@gmail.com>
| * plumbing: format/idxfile, avoid creating temporary buffers to decode integersFilip Navara2019-04-251-32/+15
| | | | | | | | Signed-off-by: Filip Navara <filip.navara@gmail.com>
* | plumbing: format/commitgraph, rename structs/fields to follow the terms used ↵Filip Navara2019-04-255-59/+59
| | | | | | | | | | | | by git more closely Signed-off-by: Filip Navara <navara@emclient.com>
* | plumbing: format/commitgraph, clean up error handlingFilip Navara2019-04-253-6/+4
|/ | | | Signed-off-by: Filip Navara <navara@emclient.com>
* Merge pull request #1130 from saracen/gitattributesMáximo Cuadros2019-04-248-0/+1043
|\ | | | | plumbing: format/gitattributes support
| * plumbing: format/gitattributes supportArran Walker2019-04-248-0/+1043
| | | | | | | | | | | | Implements gitattributes parsing, matching and attribute extraction. Signed-off-by: Arran Walker <arran.walker@fiveturns.org>
* | Merge pull request #1128 from filipnavara/commitgraph-fmtMáximo Cuadros2019-04-245-0/+689
|\ \ | | | | | | plumbing: format/commitgraph, add APIs for reading and writing commit-graph files
| * | Remove debug print, fix large edge count calculation to make the reencoded ↵Filip Navara2019-04-242-3/+1
| | | | | | | | | | | | | | | | | | file binary identical Signed-off-by: Filip Navara <filip.navara@gmail.com>
| * | Split OpenFileIndex into smaller functionsFilip Navara2019-04-242-32/+51
| | | | | | | | | | | | Signed-off-by: Filip Navara <filip.navara@gmail.com>
| * | Split Encoder into smaller functionsFilip Navara2019-04-242-55/+95
| | | | | | | | | | | | Signed-off-by: Filip Navara <filip.navara@gmail.com>
| * | Rename fixture tag to commit-graphFilip Navara2019-04-241-3/+3
| | | | | | | | | | | | Signed-off-by: Filip Navara <filip.navara@gmail.com>
| * | plumbing: format/commitgraph, add APIs for reading and writing commit-graph ↵Filip Navara2019-04-245-0/+632
| | | | | | | | | | | | | | | | | | files Signed-off-by: Filip Navara <filip.navara@gmail.com>
* | | Merge pull request #1121 from filipnavara/small-deltasMáximo Cuadros2019-04-241-56/+126
|\ \ \ | |/ / |/| | plumbing: packfile, apply small object reading optimization also for delta objects
| * | Populate the offsetToType cache in the iterator even for the skipped objectsFilip Navara2019-04-241-0/+3
| | | | | | | | | | | | Signed-off-by: Filip Navara <navara@emclient.com>
| * | Better fix for the getObjectType problem in the iteratorFilip Navara2019-04-241-6/+9
| | | | | | | | | | | | Signed-off-by: Filip Navara <navara@emclient.com>
| * | Remove duplicate cache lookup in getObjectContent, it is already done in ↵Filip Navara2019-04-241-13/+0
| | | | | | | | | | | | | | | | | | FSObject Signed-off-by: Filip Navara <navara@emclient.com>
| * | Fix an error where getObjectType would seek in file and mess up the position ↵Filip Navara2019-04-241-7/+11
| | | | | | | | | | | | | | | | | | used by getNextObject in the subsequent statement Signed-off-by: Filip Navara <navara@emclient.com>
| * | Reduce indentationFilip Navara2019-04-231-7/+5
| | | | | | | | | | | | Signed-off-by: Filip Navara <navara@emclient.com>
| * | Add one more cache check to the iteratorFilip Navara2019-04-231-2/+12
| | | | | | | | | | | | Signed-off-by: Filip Navara <navara@emclient.com>
| * | Address PR feedbackFilip Navara2019-04-231-2/+2
| | | | | | | | | | | | Signed-off-by: Filip Navara <filip.navara@gmail.com>
| * | Rename functionsFilip Navara2019-04-231-7/+9
| | | | | | | | | | | | Signed-off-by: Filip Navara <filip.navara@gmail.com>
| * | Avoid filling up the object cache from Packfile.GetByType with typ != ↵Filip Navara2019-04-231-17/+42
| | | | | | | | | | | | | | | | | | plumbing.AnyObject Signed-off-by: Filip Navara <filip.navara@gmail.com>
| * | Get the buffer from pool in fillOFSDeltaObjectContent instead of creating ↵Filip Navara2019-04-221-1/+1
| | | | | | | | | | | | | | | | | | new one Signed-off-by: Filip Navara <filip.navara@gmail.com>
| * | Update commentFilip Navara2019-04-211-3/+3
| | | | | | | | | | | | Signed-off-by: Filip Navara <navara@emclient.com>
| * | plumbing: packfile, apply small object reading optimization also for delta ↵Filip Navara2019-04-201-23/+61
| | | | | | | | | | | | | | | | | | objects Signed-off-by: Filip Navara <navara@emclient.com>
* | | Merge pull request #1124 from saracen/packfile-scanner-improved-readerMáximo Cuadros2019-04-233-105/+143
|\ \ \ | |_|/ |/| | plumbing: packfile/scanner, readability/performance improvements, zlib pooling
| * | plumbing: packfile/scanner, readability/performance improvements, zlib poolingArran Walker2019-04-223-105/+143
| |/ | | | | | | Signed-off-by: Arran Walker <arran.walker@fiveturns.org>
* | Merge pull request #1123 from saracen/object-storage-open-packfileMáximo Cuadros2019-04-231-0/+5
|\ \ | | | | | | filesystem: ObjectStorage, MaxOpenDescriptors option
| * | filesystem: ObjectStorage, MaxOpenDescriptors optionArran Walker2019-04-221-0/+5
| |/ | | | | | | | | | | | | | | The MaxOpenDescriptors option provides a middle ground solution between keeping all packfiles open (as offered by the KeepDescriptors option) and keeping none open. Signed-off-by: Arran Walker <arran.walker@fiveturns.org>
* | Merge pull request #1126 from saracen/index-performance-improvementsMáximo Cuadros2019-04-221-18/+19
|\ \ | | | | | | plumbing: format/index perf, buffered reads, reflection removal
| * | plumbing: format/index perf, buffered reads, reflection removalArran Walker2019-04-211-18/+19
| |/ | | | | | | | | | | | | | | | | | | Large performance increase by buffering reads. There were a few instances where binary.Read() would end up using reflection on &plumbing.Hash, rather than treating it as a byte slice. This has now been resolved. Signed-off-by: Arran Walker <arran.walker@fiveturns.org>
* / plumbing: idxfile, avoid unnecessary building of reverse offset/hash mapFilip Navara2019-04-201-4/+26
|/ | | | Signed-off-by: Filip Navara <navara@emclient.com>
* packfile: get object size correctly for delta objectsJeremy Stribling2019-01-312-1/+27
| | | | Signed-off-by: Jeremy Stribling <strib@alum.mit.edu>
* plumbing/packfile: test UpdateObjectStorage empty packfile errorJavi Fontan2018-12-031-0/+14
| | | | Signed-off-by: Javi Fontan <jfontan@gmail.com>
* git: return better error message when packfile cannot be downloadedJavi Fontan2018-11-301-1/+7
| | | | | | | | Previously the error message when the connection was closed while fetching was "object not found" and was misleading. Now when the packfile size is 0 the error "unable to fetch packfile" is returned. Signed-off-by: Javi Fontan <jfontan@gmail.com>
* plumbing: format/packfile, performance optimizations for reading large ↵Filip Navara2018-11-285-33/+89
| | | | | | commit histories (#963) Signed-off-by: Filip Navara <navara@emclient.com>
* Merge pull request #1025 from mcuadros/eoiev4.8.0Máximo Cuadros2018-11-194-4/+122
|\ | | | | plumbing: format/index: support for EOIE extension
| * plumbing: format/index: support for EOIE extension, by default on git v2.2.0Máximo Cuadros2018-11-194-4/+122
| | | | | | | | Signed-off-by: Máximo Cuadros <mcuadros@gmail.com>
* | Merge pull request #994 from epiclabs-io/fix-thin-packMáximo Cuadros2018-11-162-47/+95
|\ \ | | | | | | plumbing/format/packfile: Fix broken "thin" packfile support. Fixes #991
| * | plumbing/format/packfile: Added thin pack testJavier Peletier2018-11-121-0/+50
| | | | | | | | | | | | Signed-off-by: Javier Peletier <jm@epiclabs.io>
| * | plumbing/format/packfile: Fix broken "thin" packfile support. Fixes #991Javier Peletier2018-10-231-47/+45
| |/ | | | | | | Signed-off-by: Javier Peletier <jm@epiclabs.io>
* / Remove unused method (#1022)Antonio Navarro Perez2018-11-161-56/+0
|/ | | Signed-off-by: Antonio Jesus Navarro Perez <antnavper@gmail.com>
* packfile: add comment on GetSizeByOffsetJeremy Stribling2018-10-151-0/+2
| | | | | | | Suggested by mcuadros. Issue: src-d/go-git#982 Signed-off-by: Jeremy Stribling <strib@alum.mit.edu>
* object: get object size without reading whole objectJeremy Stribling2018-10-111-0/+16
| | | | Signed-off-by: Jeremy Stribling <strib@alum.mit.edu>
* all: remove extra 's' in "mismatch"Jongmin Kim2018-09-261-1/+1
| | | | Signed-off-by: Jongmin Kim <jmkim@pukyong.ac.kr>
generated by cgit (git 2.34.1) at 2024-11-06 03:27:10 +0000