WIP - feat(atc): support volume size awareness for volume streaming #8693
Conversation
Signed-off-by: Ming Xu <[email protected]>
Signed-off-by: Ming Xu <[email protected]>
Signed-off-by: Ming Xu <[email protected]>
| "worker": volume.worker.DBWorker().Name(), | ||
| "size": bcSourceVolume.Size(), | ||
| }) | ||
| } |
There was a problem hiding this comment.
When volume.DBVolume().Size() > 0, where bcSourceVolume is initialized?
There was a problem hiding this comment.
bcSourceVolume is initialized in line 719(https://github.com/concourse/concourse/pull/8693/files#diff-2034974614561f2b31bff2d4e9572aa96a162533d58b49a6cc06cfd74625d905R719).
| logger.Error("failed-to-find-source-volume-for-streaming", err) | ||
| return Volume{}, err | ||
| } | ||
| if inputPath == "for image" { |
There was a problem hiding this comment.
I think we should open this feature to all volumes. If you only want to apply to image volume, then this check should go to line 720.
There was a problem hiding this comment.
Based on the metrics, streamed image volume accounts for more than 80% of the total streamed volume(image & non-image).
Also, calculate&cache image volume size should make more sense than non-image volume, because non-image volume size may change swiftly but image volume size could be considered as unchanged.
| } | ||
| } | ||
|
|
||
| func (vs *VolumeServer) GetVolumeWithSize(w http.ResponseWriter, req *http.Request) { |
There was a problem hiding this comment.
This function has completely duplicate to GetVolume(). We should add an private function:
func (vs *VolumeServer) getVolume(w http.ResponseWriter, req *http.Request, withSize bool) {
}
func (vs *VolumeServer) GetVolume(w http.ResponseWriter, req *http.Request) {
return vs.getVolume(w, req, false)
}
func (vs *VolumeServer) GetVolumeWithSize(w http.ResponseWriter, req *http.Request) {
return vs.getVolume(w, req, true)
}There was a problem hiding this comment.
Where GetVolumeWithSize() is used?
There was a problem hiding this comment.
Sure, updated.
Where GetVolumeWithSize() is used?
It's used ingetVolumeResponse()for sending GetVolume or GetVolumeWithSize request.
worker/baggageclaim/client/client.go
Signed-off-by: Ming Xu <[email protected]>
|
I wonder what is the relationship between this PR and #8756 (the latter being in release https://github.com/concourse/concourse/releases/tag/v7.10.0) ? |
What does this PR accomplish?
Bug Fix | Feature | Documentation
We received multiple user report error like
failed to create container on worker: context deadline exceededin our environment recently, the root cause seems to be user pipeline often stream huge volume unintentionally. And sometimes because of intermittent network instability, the volume streaming could be flaky especially when the volume is huge. Adding volume size awareness for volume streaming could be helpful in this situation, for example user could notice the unexpected huge volume size and then optimize the pipeline by remove unnecessary files in the volume.Changes proposed by this PR:
volumetable to avoid duplicate volume size detection.Notes to reviewer:
The screenshot contains
streaming volume volume1 ($volumeSize) from worker1...Release Note
Contributor Checklist
Reviewer Checklist
BOSH and
Helm packaging; otherwise, ignored for
the integration
tests
(for example, if they are Garden configs that are not displayed in the
--helptext).