[BUG] VCL managed snippets

[davidnrba]

PROBLEM

So Im trying to get the syntax of the vcl files reviewed with Falco. But I get these weird errors that I don't know how to fix.

Basically I have a Main.vcl file and then multiple snippets. For example I have one named datadome-init.vcl which has the following method declared inside of it:

sub set_origin_header {
    ...
 }

But then I have another snippet named: datadome-pass.vcl that has the following code:

include "snippet::datadome-init";
call set_origin_header;

However when I run falco I get the following errors:

🔥 [ERROR] Snippet snippet::datadome-init is not found on fastly managed snippets (include/module-not-found)
in snippet::datadome-pass at line 2, position 1
 1|# Start of `pass.vcl` for DataDome-2.15
 2|include "snippet::datadome-init";
   ^^^^^^^
 3|call set_origin_header;

snippet::datadome-pass &{0x1400061c7e0 0 5 42 0x140006aa330 [# Start of `pass.vcl` for DataDome-2.15 include "snippet::datadome-init"; call set_origin_header; ] snippet::datadome-pass []}
🔥 [ERROR] Subroutine set_origin_header is not defined. Did you forget to include it or have a typo in the name? (call-statement/subroutine-notfound)
in snippet::datadome-pass at line 3, position 1
 2|include "snippet::datadome-init";
 3|call set_origin_header;
   ^^^^
 4|

I have also tried to add:
include "snippet::datadome-init.vcl";
include "snippet::set_origin_header";
with no luck

The directory structure is:

... / vcl / - Main.vcl
           |
           - datadome-pass.vcl
           |
           - datadome-init.vcl

Expected behavior

Basically to show no error since the code works in the fastly UI with no errors.

Desktop (please complete the following information):

• OS: MacOs
• Shell zsh

[ysugimoto]

@davidnrba Thank you for reaching out to us.

The VCL snippet means the code snippets which is managed on Fastly, not a local file.
So, to use your datadome-pass module, you can use its module name without snippet:: prefix:

include "datadome-pass";

The snippet:: prefix means to include a module of Fastly managed snippet, both regular and dynamic one.
https://docs.fastly.com/en/guides/about-vcl-snippets

On the other hand, I could see the meaningless debug output like snippet::datadome-pass &{0x1400061c7e0 0 5 42 0x140006aa330 [# Start of pass.vcl for DataDome-2.15 include "snippet::datadome-init"; call set_origin_header; ] snippet::datadome-pass []} , we'll remove this output, thanks.

[davidnrba]

Hi I still get an error when including the datadome-init in the datadome-pass:

include "datadome-init.vcl";
call set_origin_header;

🔥 [ERROR] Failed to resolve include module: datadome-init (include/module-load-failed)
in snippet::datadome-pass at line 2, position 1
 1|# Start of `pass.vcl` for DataDome-2.15
 2|include "datadome-init.vcl";
   ^^^^^^^
 3|call set_origin_header;

🔥 [ERROR] Subroutine set_origin_header is not defined. Did you forget to include it or have a typo in the name? (call-statement/subroutine-notfound)
in snippet::datadome-pass at line 3, position 1
 2|include "datadome-init.vcl";
 3|call set_origin_header;
   ^^^^
 4|

I have also tried including:
include "datadome-init";
And also tried including the datadome-pass in the datadome init and in the main
include "datadome-pass";

But nothing has worked

I was thinking that maybe is something related to this, but I dont know how to fix it since they are in different vcl files.

Thanks again for your help!

[ysugimoto]

@davidnrba interesting, so could you show me the all VCLs and snippets? If can, I'll look into it.

[ysugimoto]

Ah, probably it solves...
On falco implementation, We don't allow file module inclusion from VCL snippet.
it is clearly declared here

Therefore, you could not include datedome-init.vcl (file module) via snippet::datadome-pass (VCL snippet).
This is because we would prevent cyclic inclusion, and I have never seen your case in my experience.

However, does your use case works on Fastly? If so, we should support it.

[davidnrba]

Yes, so Im working on setting up CICD implementation with terraform to manage Fastly resources as code. So I took all the snippets exactly as they were set up in Fastly UI already, and put them into terraform. The interesting thing is that the way it was set up is without any includes whatsoever, just VCL Snippets and a main VCL file. (Everything is currently working great in the UI so everything mentioned here is supported by Fastly)

Looks like Fastly puts or appends them all together in a big VCL file and thats why it works. For example in our case we have a service lets say 'FastlyService' in the Fastly UI, this service has a Main.vcl and then 15 vcl snippets of code:

However at the top of the Fastly UI there is also a button to show all the vcl code:

In this VCL code we have all the configuration required for the 'FastlyService' including all the VCL snippets and the Main.vcl appended to the code:

This would make sense because for example we have another vcl file named: datadome-recv.vcl which content is just:

if (...){
  ...
} else {
  ...
}

But Falco detects it as an error because the snippet starts with an if statement:

💥  Unexpected token "if" found
in snippets/datadome-recv.vcl at line 4, position 1
 1|# Start of `recv.vcl` for DataDome-2.15
 2|# Configure the regular expression below to match URLs that
 3|# should be checked by DataDome
 4|if (f

So I think there should be maybe an option in Falco where you can just specify at the beginning of the Main.vcl file all the other files you are using (kind of like the include tag), and just lint it normally as how it would with all the files together in one.

Only problem with this solution is that Im not sure about how Fastly knows which VCL Snippets goes before which in the "big" VCL File. For example we might have the Main.vcl that calls both the datadome-init and the datadome-pass, but the datadome-pass also calls datadome-init. So if they are placed in the code for example like:

main()
datadome-pass()
datadome-init()

then it would rise an error as mentioned here. And instead it should be placed like:

datadome-init()
datadome-pass()
main()

(since main calls datadome-pass and init and pass calls init)

As for sending you the VCL Files so you can test with them, could I do it through email or some other private channel so its not public in this issue?

Also I noticed that I get a lot more errors by running Falco from a Terraform plan than just Falco to the Main.vcl and the snippets (and sometimes even different ones). But I can do it either way so Ill just stick to running Falco like (let me know if Im missing any option in this command):

falco -I . Main.vcl

[ysugimoto]

Looks like Fastly puts or appends them all together in a big VCL file and thats why it works.

Yes, I understand that Fastly embeds VCL snippets which corresponds to phase by finding macro like # FASTLY recv .
When you can create VCL snippet, Fastly provides which phase do you want to embed snippet by dropdown box, here is the UI example:

And snippet.type field is corresponding in Terraform.
When you have created with none, you can include it manually like include "snippet::[snippet name]" .

So back to the current case, it seems you include a snippet that has if statement in the root statement. VCL and falco disallow if statement in the root statement hence falco raises the error that you put the comment.

However, probably we don't care about init placement, do you specify your VCL snippets with init placement? If so, it may reproduce your problem.

Anyway, I'm interested in your problem so I appreciate you could send me your VCLs. Could you send it through the email that is written in my GitHub account? Of course, this is a good chance to create some public channel to talk about this project (Discord, Slack, or something), thanks.

[richardmarshall]

Of course, this is a good chance to create some public channel to talk about this project (Discord, Slack, or something)

@ysugimoto I know this is an old issue but seems still relevant, would a #falco channel on the Gophers slack be a good spot for this?