Issues with HTML Reporter (and solutions)

[piterwilson]

Hello everyone,

My team is using XCLogParser to profile build times in one of our apps and we're running into and issue: for some of our developers (myself included) the html reporter is not giving information collected with -Xfrontend -debug-time-function-bodies and -Xfrontend -debug-time-expression-type-checking

The issue is strange because for some of our developers the tools works without issue. I dug into the source code and have managed to fix the issue in a local copy. I will detail my findings bellow with the full disclaimer that I don't understand the compiler enough to know why some things are happening they way they are.

What I noticed is that using dump I could see the function body time and type check information but the html reporter always showed this:

Issue 1, correctly detecting a compile log

The first issue I found is that the compile steps on my log were not being picked up by the parser. I found the cause to be Sources/XCLogParser/parser/BuildStep.swift line 100

case Prefix("CompileSwift "):
    return .swiftCompilation

Because when I checked what my logs output, the prefix was actually "SwiftDriverJob-Compile ".

The fixed version looks like this:

case Prefix("CompileSwift "), Prefix("SwiftDriverJob-Compile "):
    return .swiftCompilation

This is what I don't understand though: why is it that way?

Issue 2, correctly picking up the function type check and compile information.

After getting the parser to pick up the compilation steps, I would still see that function type check and compile information was not reported.

I could see that the report now showed the overall compilation time per file, but not information on the individual functions.

I found the issue to be in the HtmlReporter class itself. It looks like it doesn't pick up the sub steps inside a BuildStep in a recursive way. I think the compile steps in my case were nested deeper than the main BuildStep so in the end these were not picked up.

So I added the following:

private func getSubstepsRecursive(in build: BuildStep) -> [BuildStep] {
    var steps: [BuildStep] = []
    steps.append(contentsOf: build.subSteps)
    for subStep in build.subSteps {
        steps.append(contentsOf: getSubstepsRecursive(in: subStep))
    }
    return steps
}

It's rather naive code and I am open to suggestions on how to improve it. The main gist is that it will go recursively and extract all substeps and when that is done for all the parts that loop steps in the reporter things looked normal again.

I used the new function in writeTopFiles(build: BuildStep, toDir buildDir: String) and writeMainFiles(build: BuildStep, toDir buildDir: String)

If these are OK assumptions and I didn't miss anything, I can submit a PR to fix things.

Thanks!

[piterwilson]

Ok, so this is presumably the cause for the lack of proper traversing of substeps:

[lumo2707]

Are there any news? I also have the same issue here.
Did you create a PR for that?
@piterwilson

[jcospina-wm]

Are there any news? I also have the same issue here.

Did you create a PR for that?

@piterwilson

The PR is linked in this issue.

Honestly, I couldn't figure out how to properly fix this and there doesn't seem to be a lot of activity in this project to guide in the right direction.

In the end, I just went into the raw logs themselves using the X-frontend options and focused on a few "serious offenders" in my project, which did the trick for me.

[PatrikBillgren]

Hi, can you please verify if the issue is solved on the latest master? Thanks

[Bobbyphtr]

Hi @piterwilson, your Issue 1 seems to be connected with my Issue. On my logs output, it is now Swift Compile.