Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wip] capture output from run to use elsewhere #138

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

coryb
Copy link
Contributor

@coryb coryb commented May 4, 2020

This is my current thinking, adding a capture option::run builtin to collect the stdout from the call to allow aliasing. In this example, we have capture as now where I capture the output from the date command. Then it is used later in an echo command.

fs _runDate() {
	image "busybox"
	run "date" with option {
		ignoreCache
		capture as now
	}
}

fs default() {
	image "busybox"
	run "echo" now with ignoreCache
}

The concept seems to at least work:

#1 compiling [default]
#1 ...

#2 docker-image://docker.io/library/busybox:latest
#2 resolve docker.io/library/busybox:latest 2.0s done
#2 DONE 2.0s

#2 docker-image://docker.io/library/busybox:latest
#2 CACHED

#3 /bin/sh -c date
#3 0.103 Mon May  4 07:22:31 UTC 2020
#3 DONE 0.1s

#1 compiling [default]
#1 DONE 2.3s

#2 docker-image://docker.io/library/busybox:latest
#2 resolve docker.io/library/busybox:latest 1.9s done
#2 DONE 1.9s

#2 docker-image://docker.io/library/busybox:latest
#2 CACHED

#4 echo 'Mon May  4 07:22:31 UTC 2020'
#4 0.101 Mon May  4 07:22:31 UTC 2020
#4 0.101
#4 DONE 0.1s

Looking for feedback, there is probably a more elegant way to do this, or some complications I have missing.

Copy link
Contributor

@hinshun hinshun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's two problems with this:

  1. Previously we defined aliases as inheriting the parent function's signature. How do we know that capture has an alias type of string? I think we need a design issue that answers those type of questions, otherwise the language won't have consistency.
  2. This captures vertex logs and I think we should understand what the caveats that come with this. There are probably differences between the actual stdout from the process and the vertex logs we get as a client.

solver/solve.go Outdated
return
}
for _, log := range status.Logs {
info.OutputCapture.Write(log.Data)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will capture logs of more than just the run statement. You'll need to isolate the logs based on vertex digest.

@coryb
Copy link
Contributor Author

coryb commented May 5, 2020

  1. Previously we defined aliases as inheriting the parent function's signature. How do we know that capture has an alias type of string? I think we need a design issue that answers those type of questions, otherwise the language won't have consistency.

Yeah, this is a general problem with aliases. I agree it is very unclear that aliasing an option::run might return either an fs (ie alias mount) or string (ie alias capture). I don't really have any ideas on how to resolve this.

  1. This captures vertex logs and I think we should understand what the caveats that come with this. There are probably differences between the actual stdout from the process and the vertex logs we get as a client.

I was able to resolve this by recording the vertex digest that the capture statement was applied to. Now when running:

fs _runDate() {
	image "busybox"
	run "echo '==> DO NOT CAPTURE <=='"
	run "date" with option {
		ignoreCache
		capture as now
	}
}

we no longer capture the output from the previous echo statement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants