Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] L3CACHE group does not work on AMD EPYC 7763 #623

Open
Keluaa opened this issue May 16, 2024 · 2 comments
Open

[BUG] L3CACHE group does not work on AMD EPYC 7763 #623

Keluaa opened this issue May 16, 2024 · 2 comments
Labels
bug

Comments

@Keluaa
Copy link

Keluaa commented May 16, 2024

Describe the bug
The events L3_CACHE_REQ and L3_CACHE_REQ_MISS of the L3CACHE group are not found on AMD EPYC 7763, making this group unusable.

To Reproduce

  • LIKWID command and/or API usage
$ likwid-perfctr -C 0 -g L3CACHE echo "test"
--------------------------------------------------------------------------------
CPU name:	AMD EPYC 7763 64-Core Processor                
CPU type:	AMD K19 (Zen3) architecture
CPU clock:	2.45 GHz
WARN: Event L3_CACHE_REQ not found for current architecture
WARN: Event L3_CACHE_REQ_MISS not found for current architecture
--------------------------------------------------------------------------------
...
  • LIKWID version and download source (Github, FTP, package manger, ...)
    5.3.0
  • Operating system
    RHEL 8.8
  • Does your application use libraries like MPI, OpenMP or Pthreads?
    No
  • Are you using the MarkerAPI (CPU code instrumentation) or the NvMarkerAPI (Nvidia GPU code instrumentation)?
    No

To Reproduce with a LIKWID command
Please supply the output of the command with -V 3 added to the command:

likwid-perfctr output 
$ likwid-perfctr -V 3 -C 0 -g L3CACHE echo "test"
DEBUG - [hwloc_init_cpuInfo:359] HWLOC CpuInfo Family 25 Model 1 Stepping 1 Vendor 0x0 Part 0x0 isIntel 0 numHWThreads 256 activeHWThreads 256
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 0 Thread 0 Core 0 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 128 Thread 1 Core 0 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 1 Thread 0 Core 1 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 129 Thread 1 Core 1 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 2 Thread 0 Core 2 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 130 Thread 1 Core 2 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 3 Thread 0 Core 3 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 131 Thread 1 Core 3 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 4 Thread 0 Core 4 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 132 Thread 1 Core 4 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 5 Thread 0 Core 5 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 133 Thread 1 Core 5 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 6 Thread 0 Core 6 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 134 Thread 1 Core 6 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 7 Thread 0 Core 7 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 135 Thread 1 Core 7 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 8 Thread 0 Core 8 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 136 Thread 1 Core 8 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 9 Thread 0 Core 9 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 137 Thread 1 Core 9 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 10 Thread 0 Core 10 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 138 Thread 1 Core 10 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 11 Thread 0 Core 11 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 139 Thread 1 Core 11 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 12 Thread 0 Core 12 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 140 Thread 1 Core 12 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 13 Thread 0 Core 13 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 141 Thread 1 Core 13 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 14 Thread 0 Core 14 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 142 Thread 1 Core 14 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 15 Thread 0 Core 15 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 143 Thread 1 Core 15 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 16 Thread 0 Core 16 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 144 Thread 1 Core 16 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 17 Thread 0 Core 17 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 145 Thread 1 Core 17 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 18 Thread 0 Core 18 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 146 Thread 1 Core 18 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 19 Thread 0 Core 19 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 147 Thread 1 Core 19 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 20 Thread 0 Core 20 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 148 Thread 1 Core 20 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 21 Thread 0 Core 21 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 149 Thread 1 Core 21 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 22 Thread 0 Core 22 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 150 Thread 1 Core 22 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 23 Thread 0 Core 23 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 151 Thread 1 Core 23 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 24 Thread 0 Core 24 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 152 Thread 1 Core 24 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 25 Thread 0 Core 25 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 153 Thread 1 Core 25 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 26 Thread 0 Core 26 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 154 Thread 1 Core 26 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 27 Thread 0 Core 27 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 155 Thread 1 Core 27 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 28 Thread 0 Core 28 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 156 Thread 1 Core 28 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 29 Thread 0 Core 29 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 157 Thread 1 Core 29 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 30 Thread 0 Core 30 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 158 Thread 1 Core 30 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 31 Thread 0 Core 31 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 159 Thread 1 Core 31 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 32 Thread 0 Core 32 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 160 Thread 1 Core 32 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 33 Thread 0 Core 33 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 161 Thread 1 Core 33 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 34 Thread 0 Core 34 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 162 Thread 1 Core 34 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 35 Thread 0 Core 35 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 163 Thread 1 Core 35 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 36 Thread 0 Core 36 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 164 Thread 1 Core 36 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 37 Thread 0 Core 37 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 165 Thread 1 Core 37 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 38 Thread 0 Core 38 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 166 Thread 1 Core 38 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 39 Thread 0 Core 39 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 167 Thread 1 Core 39 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 40 Thread 0 Core 40 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 168 Thread 1 Core 40 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 41 Thread 0 Core 41 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 169 Thread 1 Core 41 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 42 Thread 0 Core 42 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 170 Thread 1 Core 42 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 43 Thread 0 Core 43 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 171 Thread 1 Core 43 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 44 Thread 0 Core 44 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 172 Thread 1 Core 44 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 45 Thread 0 Core 45 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 173 Thread 1 Core 45 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 46 Thread 0 Core 46 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 174 Thread 1 Core 46 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 47 Thread 0 Core 47 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 175 Thread 1 Core 47 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 48 Thread 0 Core 48 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 176 Thread 1 Core 48 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 49 Thread 0 Core 49 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 177 Thread 1 Core 49 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 50 Thread 0 Core 50 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 178 Thread 1 Core 50 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 51 Thread 0 Core 51 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 179 Thread 1 Core 51 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 52 Thread 0 Core 52 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 180 Thread 1 Core 52 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 53 Thread 0 Core 53 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 181 Thread 1 Core 53 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 54 Thread 0 Core 54 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 182 Thread 1 Core 54 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 55 Thread 0 Core 55 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 183 Thread 1 Core 55 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 56 Thread 0 Core 56 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 184 Thread 1 Core 56 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 57 Thread 0 Core 57 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 185 Thread 1 Core 57 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 58 Thread 0 Core 58 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 186 Thread 1 Core 58 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 59 Thread 0 Core 59 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 187 Thread 1 Core 59 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 60 Thread 0 Core 60 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 188 Thread 1 Core 60 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 61 Thread 0 Core 61 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 189 Thread 1 Core 61 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 62 Thread 0 Core 62 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 190 Thread 1 Core 62 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 63 Thread 0 Core 63 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 191 Thread 1 Core 63 Die 0 Socket 0 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 64 Thread 0 Core 64 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 192 Thread 1 Core 64 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 65 Thread 0 Core 65 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 193 Thread 1 Core 65 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 66 Thread 0 Core 66 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 194 Thread 1 Core 66 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 67 Thread 0 Core 67 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 195 Thread 1 Core 67 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 68 Thread 0 Core 68 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 196 Thread 1 Core 68 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 69 Thread 0 Core 69 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 197 Thread 1 Core 69 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 70 Thread 0 Core 70 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 198 Thread 1 Core 70 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 71 Thread 0 Core 71 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 199 Thread 1 Core 71 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 72 Thread 0 Core 72 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 200 Thread 1 Core 72 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 73 Thread 0 Core 73 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 201 Thread 1 Core 73 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 74 Thread 0 Core 74 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 202 Thread 1 Core 74 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 75 Thread 0 Core 75 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 203 Thread 1 Core 75 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 76 Thread 0 Core 76 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 204 Thread 1 Core 76 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 77 Thread 0 Core 77 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 205 Thread 1 Core 77 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 78 Thread 0 Core 78 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 206 Thread 1 Core 78 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 79 Thread 0 Core 79 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 207 Thread 1 Core 79 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 80 Thread 0 Core 80 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 208 Thread 1 Core 80 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 81 Thread 0 Core 81 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 209 Thread 1 Core 81 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 82 Thread 0 Core 82 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 210 Thread 1 Core 82 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 83 Thread 0 Core 83 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 211 Thread 1 Core 83 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 84 Thread 0 Core 84 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 212 Thread 1 Core 84 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 85 Thread 0 Core 85 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 213 Thread 1 Core 85 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 86 Thread 0 Core 86 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 214 Thread 1 Core 86 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 87 Thread 0 Core 87 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 215 Thread 1 Core 87 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 88 Thread 0 Core 88 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 216 Thread 1 Core 88 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 89 Thread 0 Core 89 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 217 Thread 1 Core 89 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 90 Thread 0 Core 90 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 218 Thread 1 Core 90 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 91 Thread 0 Core 91 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 219 Thread 1 Core 91 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 92 Thread 0 Core 92 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 220 Thread 1 Core 92 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 93 Thread 0 Core 93 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 221 Thread 1 Core 93 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 94 Thread 0 Core 94 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 222 Thread 1 Core 94 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 95 Thread 0 Core 95 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 223 Thread 1 Core 95 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 96 Thread 0 Core 96 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 224 Thread 1 Core 96 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 97 Thread 0 Core 97 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 225 Thread 1 Core 97 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 98 Thread 0 Core 98 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 226 Thread 1 Core 98 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 99 Thread 0 Core 99 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 227 Thread 1 Core 99 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 100 Thread 0 Core 100 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 228 Thread 1 Core 100 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 101 Thread 0 Core 101 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 229 Thread 1 Core 101 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 102 Thread 0 Core 102 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 230 Thread 1 Core 102 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 103 Thread 0 Core 103 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 231 Thread 1 Core 103 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 104 Thread 0 Core 104 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 232 Thread 1 Core 104 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 105 Thread 0 Core 105 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 233 Thread 1 Core 105 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 106 Thread 0 Core 106 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 234 Thread 1 Core 106 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 107 Thread 0 Core 107 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 235 Thread 1 Core 107 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 108 Thread 0 Core 108 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 236 Thread 1 Core 108 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 109 Thread 0 Core 109 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 237 Thread 1 Core 109 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 110 Thread 0 Core 110 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 238 Thread 1 Core 110 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 111 Thread 0 Core 111 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 239 Thread 1 Core 111 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 112 Thread 0 Core 112 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 240 Thread 1 Core 112 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 113 Thread 0 Core 113 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 241 Thread 1 Core 113 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 114 Thread 0 Core 114 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 242 Thread 1 Core 114 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 115 Thread 0 Core 115 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 243 Thread 1 Core 115 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 116 Thread 0 Core 116 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 244 Thread 1 Core 116 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 117 Thread 0 Core 117 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 245 Thread 1 Core 117 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 118 Thread 0 Core 118 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 246 Thread 1 Core 118 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 119 Thread 0 Core 119 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 247 Thread 1 Core 119 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 120 Thread 0 Core 120 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 248 Thread 1 Core 120 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 121 Thread 0 Core 121 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 249 Thread 1 Core 121 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 122 Thread 0 Core 122 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 250 Thread 1 Core 122 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 123 Thread 0 Core 123 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 251 Thread 1 Core 123 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 124 Thread 0 Core 124 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 252 Thread 1 Core 124 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 125 Thread 0 Core 125 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 253 Thread 1 Core 125 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 126 Thread 0 Core 126 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 254 Thread 1 Core 126 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 127 Thread 0 Core 127 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_nodeTopology:568] HWLOC Thread Pool PU 255 Thread 1 Core 127 Die 0 Socket 1 inCpuSet 1
DEBUG - [hwloc_init_cacheTopology:798] HWLOC Cache Pool ID 0 Level 1 Size 32768 Threads 2
DEBUG - [hwloc_init_cacheTopology:798] HWLOC Cache Pool ID 1 Level 2 Size 524288 Threads 2
DEBUG - [hwloc_init_cacheTopology:798] HWLOC Cache Pool ID 2 Level 3 Size 33554432 Threads 16
DEBUG - [affinity_init:547] Affinity: Socket domains 2
DEBUG - [affinity_init:549] Affinity: CPU die domains 2
DEBUG - [affinity_init:554] Affinity: CPU cores per LLC 8
DEBUG - [affinity_init:557] Affinity: Cache domains 16
DEBUG - [affinity_init:561] Affinity: NUMA domains 4
DEBUG - [affinity_init:562] Affinity: All domains 25
DEBUG - [affinity_addNodeDomain:370] Affinity domain N: 256 HW threads on 128 cores
DEBUG - [affinity_addSocketDomain:401] Affinity domain S0: 128 HW threads on 64 cores
DEBUG - [affinity_addSocketDomain:401] Affinity domain S1: 128 HW threads on 64 cores
DEBUG - [affinity_addDieDomain:438] Affinity domain D0: 128 HW threads on 64 cores
DEBUG - [affinity_addDieDomain:438] Affinity domain D1: 128 HW threads on 64 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C0: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C1: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C2: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C3: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C4: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C5: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C6: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C7: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C8: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C9: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C10: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C11: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C12: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C13: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C14: 16 HW threads on 8 cores
DEBUG - [affinity_addCacheDomain:474] Affinity domain C15: 16 HW threads on 8 cores
DEBUG - [affinity_addMemoryDomain:504] Affinity domain M0: 64 HW threads on 32 cores
DEBUG - [affinity_addMemoryDomain:504] Affinity domain M1: 64 HW threads on 32 cores
DEBUG - [affinity_addMemoryDomain:504] Affinity domain M2: 64 HW threads on 32 cores
DEBUG - [affinity_addMemoryDomain:504] Affinity domain M3: 64 HW threads on 32 cores
DEBUG - [create_lookups:290] T 0 T2C 0 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 1 T2C 1 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 2 T2C 2 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 3 T2C 3 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 4 T2C 4 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 5 T2C 5 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 6 T2C 6 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 7 T2C 7 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 8 T2C 8 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 9 T2C 9 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 10 T2C 10 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 11 T2C 11 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 12 T2C 12 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 13 T2C 13 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 14 T2C 14 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 15 T2C 15 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 16 T2C 16 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 17 T2C 17 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 18 T2C 18 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 19 T2C 19 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 20 T2C 20 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 21 T2C 21 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 22 T2C 22 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 23 T2C 23 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 24 T2C 24 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 25 T2C 25 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 26 T2C 26 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 27 T2C 27 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 28 T2C 28 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 29 T2C 29 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 30 T2C 30 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 31 T2C 31 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 32 T2C 32 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 33 T2C 33 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 34 T2C 34 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 35 T2C 35 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 36 T2C 36 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 37 T2C 37 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 38 T2C 38 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 39 T2C 39 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 40 T2C 40 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 41 T2C 41 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 42 T2C 42 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 43 T2C 43 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 44 T2C 44 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 45 T2C 45 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 46 T2C 46 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 47 T2C 47 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 48 T2C 48 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 49 T2C 49 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 50 T2C 50 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 51 T2C 51 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 52 T2C 52 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 53 T2C 53 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 54 T2C 54 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 55 T2C 55 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 56 T2C 56 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 57 T2C 57 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 58 T2C 58 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 59 T2C 59 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 60 T2C 60 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 61 T2C 61 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 62 T2C 62 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 63 T2C 63 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 64 T2C 64 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 65 T2C 65 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 66 T2C 66 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 67 T2C 67 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 68 T2C 68 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 69 T2C 69 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 70 T2C 70 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 71 T2C 71 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 72 T2C 72 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 73 T2C 73 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 74 T2C 74 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 75 T2C 75 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 76 T2C 76 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 77 T2C 77 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 78 T2C 78 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 79 T2C 79 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 80 T2C 80 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 81 T2C 81 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 82 T2C 82 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 83 T2C 83 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 84 T2C 84 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 85 T2C 85 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 86 T2C 86 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 87 T2C 87 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 88 T2C 88 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 89 T2C 89 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 90 T2C 90 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 91 T2C 91 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 92 T2C 92 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 93 T2C 93 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 94 T2C 94 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 95 T2C 95 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 96 T2C 96 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 97 T2C 97 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 98 T2C 98 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 99 T2C 99 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 100 T2C 100 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 101 T2C 101 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 102 T2C 102 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 103 T2C 103 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 104 T2C 104 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 105 T2C 105 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 106 T2C 106 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 107 T2C 107 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 108 T2C 108 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 109 T2C 109 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 110 T2C 110 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 111 T2C 111 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 112 T2C 112 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 113 T2C 113 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 114 T2C 114 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 115 T2C 115 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 116 T2C 116 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 117 T2C 117 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 118 T2C 118 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 119 T2C 119 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 120 T2C 120 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 121 T2C 121 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 122 T2C 122 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 123 T2C 123 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 124 T2C 124 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 125 T2C 125 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 126 T2C 126 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 127 T2C 127 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 128 T2C 0 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 129 T2C 1 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 130 T2C 2 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 131 T2C 3 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 132 T2C 4 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 133 T2C 5 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 134 T2C 6 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 135 T2C 7 T2S 0 T2D 0 T2LLC 0 T2M 0
DEBUG - [create_lookups:290] T 136 T2C 8 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 137 T2C 9 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 138 T2C 10 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 139 T2C 11 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 140 T2C 12 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 141 T2C 13 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 142 T2C 14 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 143 T2C 15 T2S 0 T2D 0 T2LLC 1 T2M 0
DEBUG - [create_lookups:290] T 144 T2C 16 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 145 T2C 17 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 146 T2C 18 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 147 T2C 19 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 148 T2C 20 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 149 T2C 21 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 150 T2C 22 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 151 T2C 23 T2S 0 T2D 0 T2LLC 2 T2M 0
DEBUG - [create_lookups:290] T 152 T2C 24 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 153 T2C 25 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 154 T2C 26 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 155 T2C 27 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 156 T2C 28 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 157 T2C 29 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 158 T2C 30 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 159 T2C 31 T2S 0 T2D 0 T2LLC 3 T2M 0
DEBUG - [create_lookups:290] T 160 T2C 32 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 161 T2C 33 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 162 T2C 34 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 163 T2C 35 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 164 T2C 36 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 165 T2C 37 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 166 T2C 38 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 167 T2C 39 T2S 0 T2D 0 T2LLC 4 T2M 1
DEBUG - [create_lookups:290] T 168 T2C 40 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 169 T2C 41 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 170 T2C 42 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 171 T2C 43 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 172 T2C 44 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 173 T2C 45 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 174 T2C 46 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 175 T2C 47 T2S 0 T2D 0 T2LLC 5 T2M 1
DEBUG - [create_lookups:290] T 176 T2C 48 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 177 T2C 49 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 178 T2C 50 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 179 T2C 51 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 180 T2C 52 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 181 T2C 53 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 182 T2C 54 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 183 T2C 55 T2S 0 T2D 0 T2LLC 6 T2M 1
DEBUG - [create_lookups:290] T 184 T2C 56 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 185 T2C 57 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 186 T2C 58 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 187 T2C 59 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 188 T2C 60 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 189 T2C 61 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 190 T2C 62 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 191 T2C 63 T2S 0 T2D 0 T2LLC 7 T2M 1
DEBUG - [create_lookups:290] T 192 T2C 64 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 193 T2C 65 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 194 T2C 66 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 195 T2C 67 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 196 T2C 68 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 197 T2C 69 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 198 T2C 70 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 199 T2C 71 T2S 1 T2D 1 T2LLC 8 T2M 2
DEBUG - [create_lookups:290] T 200 T2C 72 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 201 T2C 73 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 202 T2C 74 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 203 T2C 75 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 204 T2C 76 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 205 T2C 77 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 206 T2C 78 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 207 T2C 79 T2S 1 T2D 1 T2LLC 9 T2M 2
DEBUG - [create_lookups:290] T 208 T2C 80 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 209 T2C 81 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 210 T2C 82 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 211 T2C 83 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 212 T2C 84 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 213 T2C 85 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 214 T2C 86 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 215 T2C 87 T2S 1 T2D 1 T2LLC 10 T2M 2
DEBUG - [create_lookups:290] T 216 T2C 88 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 217 T2C 89 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 218 T2C 90 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 219 T2C 91 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 220 T2C 92 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 221 T2C 93 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 222 T2C 94 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 223 T2C 95 T2S 1 T2D 1 T2LLC 11 T2M 2
DEBUG - [create_lookups:290] T 224 T2C 96 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 225 T2C 97 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 226 T2C 98 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 227 T2C 99 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 228 T2C 100 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 229 T2C 101 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 230 T2C 102 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 231 T2C 103 T2S 1 T2D 1 T2LLC 12 T2M 3
DEBUG - [create_lookups:290] T 232 T2C 104 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 233 T2C 105 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 234 T2C 106 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 235 T2C 107 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 236 T2C 108 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 237 T2C 109 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 238 T2C 110 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 239 T2C 111 T2S 1 T2D 1 T2LLC 13 T2M 3
DEBUG - [create_lookups:290] T 240 T2C 112 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 241 T2C 113 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 242 T2C 114 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 243 T2C 115 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 244 T2C 116 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 245 T2C 117 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 246 T2C 118 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 247 T2C 119 T2S 1 T2D 1 T2LLC 14 T2M 3
DEBUG - [create_lookups:290] T 248 T2C 120 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 249 T2C 121 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 250 T2C 122 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 251 T2C 123 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 252 T2C 124 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 253 T2C 125 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 254 T2C 126 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [create_lookups:290] T 255 T2C 127 T2S 1 T2D 1 T2LLC 15 T2M 3
DEBUG - [cuda_topo_init:148] CUDA cannot be found and initialized (cuInit failed)
--------------------------------------------------------------------------------
CPU name:	AMD EPYC 7763 64-Core Processor                
CPU type:	AMD K19 (Zen3) architecture
CPU clock:	2.45 GHz
CPU family:	25
CPU model:	1
CPU short:	zen3
CPU stepping:	1
CPU features:	FP MMX SSE SSE2 HTT MMX RDTSCP MONITOR SSSE FMA SSE4.1 SSE4.2 AES AVX RDRAND AVX2 RDSEED SSE3 
CPU arch:	x86_64
--------------------------------------------------------------------------------
[likwid-pin] Main PID -> hwthread 0 - OK
Executing: echo test
DEBUG - [perfmon_addEventSet:2324] Currently 1 groups of 2 active
DEBUG - [perfgroup_readGroup:873] Reading group L3CACHE from /home/briandl/work/likwid/install_dir_debug/share/likwid/perfgroups/zen3/L3CACHE.txt
DEBUG - [perfmon_addEventSet:2385] Eventstring RETIRED_INSTRUCTIONS:PMC0,CPU_CLOCKS_UNHALTED:PMC1,L3_CACHE_REQ:CPMC0,L3_MISS_REQ:CPMC1,L3_CACHE_REQ_MISS:CPMC2
DEBUG - [perfmon_addEventSet:2508] Added event RETIRED_INSTRUCTIONS for counter PMC0 to group 0
DEBUG - [perfmon_addEventSet:2508] Added event CPU_CLOCKS_UNHALTED for counter PMC1 to group 0
WARN: Event L3_CACHE_REQ not found for current architecture
DEBUG - [perfmon_addEventSet:2508] Added event L3_MISS_REQ for counter CPMC1 to group 0
WARN: Event L3_CACHE_REQ_MISS not found for current architecture
DEBUG - [perfmon_setupCountersThread_perfevent:1084] SETUP_PMC [0] Register 0x3 , Flags: 0xC0 
DEBUG - [perfmon_setupCountersThread_perfevent:1416] perf_event_open: cpu_id=0 pid=-1 flags=0
DEBUG - [perfmon_setupCountersThread_perfevent:1084] SETUP_PMC [0] Register 0x4 , Flags: 0x76 
DEBUG - [perfmon_setupCountersThread_perfevent:1416] perf_event_open: cpu_id=0 pid=-1 flags=0
DEBUG - [perf_uncore_setup:726] Get information for uncore counters from folder /sys/bus/event_source/devices/amd_l3
DEBUG - [perf_uncore_setup:825] Format umask from 8-15 with value 0xFF
DEBUG - [perf_uncore_setup:829] Adding 0xFF00 to 0x9A
DEBUG - [perfmon_setupCountersThread_perfevent:1399] SETUP_UNCORE [0] Register 0xA , Flags: 0xFF9A 
DEBUG - [perfmon_setupCountersThread_perfevent:1426] perf_event_open: cpu_id=0 pid=-1 flags=0 type=11 config=0xFF9A disabled=1 inherit=1 exclusive=0 config1=0x0 config2=0x0
--------------------------------------------------------------------------------
DEBUG - [perfmon_startCountersThread_perfevent:1472] RESET_COUNTER [0] Register 0x0 , Flags: 0x0 
DEBUG - [perfmon_startCountersThread_perfevent:1483] START_COUNTER [0] Register 0x0 , Flags: 0x0 
DEBUG - [perfmon_startCountersThread_perfevent:1472] RESET_COUNTER [0] Register 0x0 , Flags: 0x0 
DEBUG - [perfmon_startCountersThread_perfevent:1483] START_COUNTER [0] Register 0x0 , Flags: 0x0 
DEBUG - [perfmon_startCountersThread_perfevent:1472] RESET_COUNTER [0] Register 0x0 , Flags: 0x0 
DEBUG - [perfmon_startCountersThread_perfevent:1483] START_COUNTER [0] Register 0x0 , Flags: 0x0 
DEBUG - [perfmon_readCountersThread_perfevent:1559] FREEZE_COUNTER [0] Register 0x3 , Flags: 0x0 
DEBUG - [perfmon_readCountersThread_perfevent:1563] READ_COUNTER [0] Register 0x3 , Flags: 0xE9E7 
DEBUG - [perfmon_readCountersThread_perfevent:1586] UNFREEZE_COUNTER [0] Register 0x3 , Flags: 0x0 
DEBUG - [perfmon_readCountersThread_perfevent:1559] FREEZE_COUNTER [0] Register 0x4 , Flags: 0x0 
DEBUG - [perfmon_readCountersThread_perfevent:1563] READ_COUNTER [0] Register 0x4 , Flags: 0x220B5 
DEBUG - [perfmon_readCountersThread_perfevent:1586] UNFREEZE_COUNTER [0] Register 0x4 , Flags: 0x0 
DEBUG - [perfmon_readCountersThread_perfevent:1559] FREEZE_COUNTER [0] Register 0x5 , Flags: 0x0 
DEBUG - [perfmon_readCountersThread_perfevent:1563] READ_COUNTER [0] Register 0x5 , Flags: 0x8CE 
DEBUG - [perfmon_readCountersThread_perfevent:1586] UNFREEZE_COUNTER [0] Register 0x5 , Flags: 0x0 
test
DEBUG - [perfmon_stopCountersThread_perfevent:1508] FREEZE_COUNTER [0] Register 0x3 , Flags: 0x0 
DEBUG - [perfmon_stopCountersThread_perfevent:1512] READ_COUNTER [0] Register 0x3 , Flags: 0xA99D3 
DEBUG - [perfmon_stopCountersThread_perfevent:1537] RESET_COUNTER [0] Register 0x3 , Flags: 0x0 
DEBUG - [perfmon_stopCountersThread_perfevent:1508] FREEZE_COUNTER [0] Register 0x4 , Flags: 0x0 
DEBUG - [perfmon_stopCountersThread_perfevent:1512] READ_COUNTER [0] Register 0x4 , Flags: 0xD5E61 
DEBUG - [perfmon_stopCountersThread_perfevent:1537] RESET_COUNTER [0] Register 0x4 , Flags: 0x0 
DEBUG - [perfmon_stopCountersThread_perfevent:1508] FREEZE_COUNTER [0] Register 0x5 , Flags: 0x0 
DEBUG - [perfmon_stopCountersThread_perfevent:1512] READ_COUNTER [0] Register 0x5 , Flags: 0xCF11 
DEBUG - [perfmon_stopCountersThread_perfevent:1537] RESET_COUNTER [0] Register 0x5 , Flags: 0x0 
--------------------------------------------------------------------------------
Group 1: L3CACHE
+----------------------+---------+------------+
|         Event        | Counter | HWThread 0 |
+----------------------+---------+------------+
| RETIRED_INSTRUCTIONS |   PMC0  |     694739 |
|  CPU_CLOCKS_UNHALTED |   PMC1  |     876129 |
|     L3_CACHE_REQ     |  CPMC0  |      -     |
|      L3_MISS_REQ     |  CPMC1  |      53009 |
|   L3_CACHE_REQ_MISS  |  CPMC2  |      -     |
+----------------------+---------+------------+

+---------------------+------------+
|        Metric       | HWThread 0 |
+---------------------+------------+
| Runtime (RDTSC) [s] |     0.0069 |
|         CPI         |     inf    |
|   L3 request rate   |          0 |
|     L3 miss rate    |          0 |
|    L3 miss ratio    |      -     |
+---------------------+------------+

Additional context

likwid-perfctr -e gives me only the following L3 events:

L3_ACCESS_ALL_TYPES, 0x4, 0xFF, CPMC
L3_ACCESS_MISS, 0x4, 0x1, CPMC
L3_MISS_LAT, 0x90, 0x0, CPMC
L3_MISS_REQ, 0x9A, 0xFF, CPMC

L3_CACHE_REQ and L3_CACHE_REQ_MISS are not part of that list. I suppose that this why the group is not working.

Replacing L3_CACHE_REQ by L3_ACCESS_ALL_TYPES and using L3_MISS_REQ instead of L3_CACHE_REQ_MISS in L3CACHE.txt gives some credible output with no warnings. I do not know if those two events are correct however.
@Keluaa Keluaa added the bug label May 16, 2024
@TomTheBear
Copy link
Member

TomTheBear commented May 16, 2024 •
edited

Thanks for reporting.

Your selection sounds reasonable. The official documentation uses L3_ACCESS_ALL_TYPES and L3_ACCESS_MISS:
grafik

The L3_MISS_REQ event is not officially documented. I update the event list and the group.

@TomTheBear
Copy link
Member

I added the L3_ACCESS_HIT event and updated the group but the event list needs further updates. There are multiple L3 latency event but I have to make sure that the options are properly set by default. I'm travelling at the moment, therefore I re-open the issue to take care of it next week when I have a proper keyboard in front of me.

@TomTheBear TomTheBear reopened this May 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@TomTheBear @Keluaa