KeyError when using dopca.py

[changzhi1990]

Verbose output from perf-collect

In my environment, I would to collect both kubelet and kube-apiserver by using perf-collect. Here is the detailed:

Collect kubelet:
sdp@b49691d6a544:~/zhi/perfspect/kubelet$ sudo ../perf-collect --timeout 10 --pid 2664002
2023-12-21 21:35:59,548 INFO: nmi_watchdog temporarily disabled. Will re-enable after collection.
2023-12-21 21:35:59,549 INFO: Run mode: pid
2023-12-21 21:36:02,292 INFO: changing default perf mux interval to 125ms
2023-12-21 21:36:06,177 INFO: Not collecting uncore events in this run mode
2023-12-21 21:36:06,177 INFO: Architecture: emeraldrapids
2023-12-21 21:36:06,177 INFO: Model: Genuine Intel(R) 0000
2023-12-21 21:36:06,177 INFO: Kernel version: Linux version 5.15.0-91-generic (buildd@lcy02-amd64-045) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38)
2023-12-21 21:36:06,249 INFO: Cores per socket: 56
2023-12-21 21:36:06,269 INFO: Socket: 2
2023-12-21 21:36:06,286 INFO: Hyperthreading on: True
2023-12-21 21:36:06,286 INFO: IMC count: 8
2023-12-21 21:36:06,286 INFO: CHA per socket: 56
2023-12-21 21:36:06,287 INFO: UPI count: 4
2023-12-21 21:36:06,287 INFO: PerfSpect version: 1.3.11
2023-12-21 21:36:06,296 INFO: Collection started!
PID/TID switch overriding SYSTEM
2023-12-21 21:36:21,315 INFO: Collection complete!
2023-12-21 21:36:22,747 INFO: nmi_watchdog re-enabled.
2023-12-21 21:36:22,748 INFO: changing perf mux interval back to default
2023-12-21 21:36:26,642 INFO: perf stat dumped to perfstat.csv


Collect kube-apiserver:
sdp@b49691d6a544:~/zhi/perfspect/kubeapi$ sudo ../perf-collect --timeout 10 --pid 3892
2023-12-21 21:37:15,894 INFO: nmi_watchdog temporarily disabled. Will re-enable after collection.
2023-12-21 21:37:15,895 INFO: Run mode: pid
2023-12-21 21:37:18,787 INFO: changing default perf mux interval to 125ms
2023-12-21 21:37:22,684 INFO: Not collecting uncore events in this run mode
2023-12-21 21:37:22,685 INFO: Architecture: emeraldrapids
2023-12-21 21:37:22,685 INFO: Model: Genuine Intel(R) 0000
2023-12-21 21:37:22,685 INFO: Kernel version: Linux version 5.15.0-91-generic (buildd@lcy02-amd64-045) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38)
2023-12-21 21:37:22,781 INFO: Cores per socket: 56
2023-12-21 21:37:22,805 INFO: Socket: 2
2023-12-21 21:37:22,853 INFO: Hyperthreading on: True
2023-12-21 21:37:22,854 INFO: IMC count: 8
2023-12-21 21:37:22,854 INFO: CHA per socket: 56
2023-12-21 21:37:22,854 INFO: UPI count: 4
2023-12-21 21:37:22,854 INFO: PerfSpect version: 1.3.11
2023-12-21 21:37:22,870 INFO: Collection started!
PID/TID switch overriding SYSTEM
2023-12-21 21:37:37,889 INFO: Collection complete!
2023-12-21 21:37:39,318 INFO: nmi_watchdog re-enabled.
2023-12-21 21:37:39,319 INFO: changing perf mux interval back to default
2023-12-21 21:37:43,192 INFO: perf stat dumped to perfstat.csv

Verbose output from perf-postprocess

sdp@b49691d6a544:~/zhi/perfspect/kubelet$ ../perf-postprocess -r perfstat.csv
2023-12-21 21:36:42,516 WARNING: Failed to convert cstate_core/c6-residency/ metric value: <not supported> to integer. Skipping
2023-12-21 21:36:42,516 WARNING: Failed to convert cstate_core/c6-residency/ metric value: <not supported> to integer. Skipping
2023-12-21 21:36:42,516 INFO: Formatting event data
2023-12-21 21:36:42,529 INFO: processing 2 samples in System mode
2023-12-21 21:36:42,751 INFO: Generated results file(s) in: /home/sdp/zhi/perfspect/kubelet
2023-12-21 21:36:42,751 INFO: Done!

sdp@b49691d6a544:~/zhi/perfspect/kubeapi$ ../perf-postprocess -r perfstat.csv
2023-12-21 21:37:50,961 WARNING: Failed to convert cstate_core/c6-residency/ metric value: <not supported> to integer. Skipping
2023-12-21 21:37:50,961 WARNING: Failed to convert cstate_core/c6-residency/ metric value: <not supported> to integer. Skipping
2023-12-21 21:37:50,961 INFO: Formatting event data
2023-12-21 21:37:50,974 INFO: processing 2 samples in System mode
2023-12-21 21:37:51,199 INFO: Generated results file(s) in: /home/sdp/zhi/perfspect/kubeapi
2023-12-21 21:37:51,199 INFO: Done!

What steps can reproduce the bug?

sdp@b49691d6a544:~/zhi/PerfSpect/similarity-analyzer$ sudo python3 dopca.py -f "../../perfspect/kubeapi/metric_out.sys.average.csv,../../perfspect/kubele
t/metric_out.sys.average.csv" -o sim-workload1 > log
[scatterd] >INFO> starting similarity analyzer 1.1.0
[scatterd] >INFO> data formatter collated pmu metrics at sim-workload1.csv file
[scatterd] >INFO> starting PCA
[scatterd] >INFO> PCA completed
[scatterd] >INFO> PCA plot initiated
/usr/local/lib/python3.10/dist-packages/matplotlib/text.py:1461: FutureWarning: Calling float on a single element Series is deprecated and will raise a T
ypeError in the future. Use float(ser.iloc[0]) instead
  x = float(self.convert_xunits(x))
/usr/local/lib/python3.10/dist-packages/matplotlib/text.py:1463: FutureWarning: Calling float on a single element Series is deprecated and will raise a T
ypeError in the future. Use float(ser.iloc[0]) instead
  y = float(self.convert_yunits(y))
/usr/local/lib/python3.10/dist-packages/matplotlib/text.py:753: FutureWarning: Calling float on a single element Series is deprecated and will raise a Ty
peError in the future. Use float(ser.iloc[0]) instead
  posx = float(self.convert_xunits(self._x))
/usr/local/lib/python3.10/dist-packages/matplotlib/text.py:754: FutureWarning: Calling float on a single element Series is deprecated and will raise a Ty
peError in the future. Use float(ser.iloc[0]) instead
  posy = float(self.convert_yunits(self._y))
[scatterd] >INFO> PCA plot saved at sim-workload1_pca.png
[scatterd] >INFO> starting PCA-2
[scatterd] >INFO> Add density layer
/usr/local/lib/python3.10/dist-packages/scatterd/scatterd.py:228: UserWarning: KDE cannot be estimated (0 variance or perfect covariance). Pass `warn_sin
gular=False` to disable this warning.
  ax = sns.kdeplot(x=X[:, 0], y=X[:, 1], ax=ax, **args_density)
[scatterd] >INFO> Create scatterplot
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/indexes/base.py", line 3791, in get_loc
    return self._engine.get_loc(casted_key)
  File "index.pyx", line 152, in pandas._libs.index.IndexEngine.get_loc
  File "index.pyx", line 181, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 7080, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 7088, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'PC2'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/sdp/zhi/PerfSpect/similarity-analyzer/dopca.py", line 396, in <module>
    final_df = dopca_density(
  File "/home/sdp/zhi/PerfSpect/similarity-analyzer/dopca.py", line 221, in dopca_density
    plt.text(row["PC1"], row["PC2"], workload_names[c], fontsize=16)
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/series.py", line 1040, in __getitem__
    return self._get_value(key)
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/series.py", line 1156, in _get_value
    loc = self.index.get_loc(label)
  File "/usr/local/lib/python3.10/dist-packages/pandas/core/indexes/base.py", line 3798, in get_loc
    raise KeyError(key) from err
KeyError: 'PC2'

Additional information

I don't know why the similarity-analyze reports KeyError. Could someone give me some advice? Thanks!