Mac OS sensor/support

[luiscruz]

Bug description

Cannot build using Mac OS.

To Reproduce

cargo build

Relevant Output

error[E0433]: failed to resolve: could not find `linux` in `os`
  --> /Users/luiscruz/.cargo/registry/src/github.com-1ecc6299db9ec823/procfs-0.8.1/src/process/mod.rs:62:14
   |
62 | use std::os::linux::fs::MetadataExt;
   |              ^^^^^ could not find `linux` in `os`

error[E0599]: no method named `st_uid` found for struct `Metadata` in the current scope
   --> /Users/luiscruz/.cargo/registry/src/github.com-1ecc6299db9ec823/procfs-0.8.1/src/process/mod.rs:562:23
    |
562 |             owner: md.st_uid(),
    |                       ^^^^^^ method not found in `Metadata`
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
            `use std::os::macos::fs::MetadataExt;`

error[E0599]: no method named `st_mode` found for struct `Metadata` in the current scope
   --> /Users/luiscruz/.cargo/registry/src/github.com-1ecc6299db9ec823/procfs-0.8.1/src/process/mod.rs:763:30
    |
763 |                     mode: md.st_mode() & libc::S_IRWXU,
    |                              ^^^^^^^ method not found in `Metadata`
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
            `use std::os::macos::fs::MetadataExt;`

error[E0308]: mismatched types
   --> /Users/luiscruz/.cargo/registry/src/github.com-1ecc6299db9ec823/procfs-0.8.1/src/process/mod.rs:182:22
    |
182 |         const READ = libc::S_IRUSR;
    |                      ^^^^^^^^^^^^^ expected `u32`, found `u16`

error[E0308]: mismatched types
   --> /Users/luiscruz/.cargo/registry/src/github.com-1ecc6299db9ec823/procfs-0.8.1/src/process/mod.rs:183:23
    |
183 |         const WRITE = libc::S_IWUSR;
    |                       ^^^^^^^^^^^^^ expected `u32`, found `u16`

error[E0308]: mismatched types
   --> /Users/luiscruz/.cargo/registry/src/github.com-1ecc6299db9ec823/procfs-0.8.1/src/process/mod.rs:184:25
    |
184 |         const EXECUTE = libc::S_IXUSR;
    |                         ^^^^^^^^^^^^^ expected `u32`, found `u16`

error: aborting due to 6 previous errors

Some errors have detailed explanations: E0308, E0433, E0599.
For more information about an error, try `rustc --explain E0308`.
error: could not compile `procfs`

Environment

• Mac OS 10.15.7
• Kernel version: 19.6.0

Context

I have just realised that this project is targeting linux distributions only. That kinda makes sense from a production perspective. From a dev perspective is quite limiting. Any ideas?

[bpetit]

Hi,

Thanks for the try. This was somehow expected. The powercaprapl sensor is based on a linux kernel module. We will need to create a new sensor for mac os I guess. Discussions about that have started in the ARM related issue (about apple m1) that may be redundant but just to mention.

I have an old macbook air somewhere so I may start digging on that topic but not before february/march. If anyone with a macbook (maybe better to start with x86 macbooks to tackle one topic at a time) is willing to give it a shot before, it would be awesome :)

[luiscruz]

I'd be happy to help but that might be a bit too much for me…
Still, it feels like it should be possible to build without having a Mac OS sensor ready. I can imagine that a Mac OS sensor will not be compatible with Linux, but it should still be possible to build it on a Linux machine.

[mrchrisadams]

Hi @luiscruz I looked into some of this myself over the holidays.

My notes are here for OS X, but the TLDR is that if we want RAPL data, we likely need to look for the output from apple's powermetrics cli command.

#35

[mrchrisadams]

Ah, here's what you can get out. You can access the RAPL code with a command like so:

sudo powermetrics -i 2 -n 1 -s cpu_power --format plist

this runs powermetrics, pulling out only the cpu_power info, and formats it as a dialect of XML (the --format plist) bit, taking a sample for 2 milliseconds, and doing it once.

I can't see a way to read a stream of info through :(

[florimondmanca]

FWIW I just realized I'm oftentimes using CoconutBattery, which is a macOS utility to view battery state and usage. It has a "show watt usage" thing. I'd be curious to know how it does that. It doesn't seem to be open source (hmm… should I be using it then? 😝), but the author's email is at the bottom of the landing page. (I don't know their github @, else we could have pinged them here…)

[bpetit]

Great to see there are userspace tools that get such metrics, this is a good sign !

Looking at your message @florimondmanca I just want to mention that battery based metrics, may be a bit different than rapl. Like building a macos sensor based on battery metrics wouldn't work on a mac mini server for instance. It's not necessary an issue, if we name the sensor appropriately and document it to indicate that this is only suitable for a laptop. (it reminds me of powertop, that gets metrics from battery related sensors if available and fallbacks to something else if not...)

@mrchrisadams it would be interessting to look at what libraries/bsd internals powermetrics calls. Maybe there we could get a stream/more suitable sort of data ?

[bpetit]

@luiscruz yes maybe we could build. But the powercaprapl sensor being the only available right now it wouldn't be very helpful for mac users to have a binary that gets no metrics :/

[mrchrisadams]

So, on OS X there is this:

https://github.com/beltex/dshb - a nice open source CLI exposing the similar stats as coconut battery.

Under the hood, that uses this library, in Swift:

https://github.com/beltex/SystemKit

And that in turn uses an open sourced library from apple:
https://opensource.apple.com/source/PowerManagement/

And this issue here in system kit is about where I ran out of steam when looking at it.
beltex/SystemKit#2

I think you'd need to find someone who really understands the powermetrics code to provide pointers, and get an idea of what it's reading to get these numbers.

I don't think the powermetrics is open, and but this post here might shed some more light:
https://www.cmdsec.com/macos-performance-monitoring-collection/

The metrics are exposed, because iStat pro shows me stuff like this on my mac:

But I don't have experience working this close to the metal with OS X.

[bpetit]

That may be a good news for developing scaphandre support on the m1:

At least for me as I don't have one.

[davidmytton]

The procfs crate is designed specifically for Linux. From the brief discussion in eminence/procfs#42 it looks like an alternative approach is needed for macOS, probably using conditional compilation to bring in a separate platform specific library on macOS and Windows.

[bpetit]

The procfs crate is designed specifically for Linux. From the brief discussion in eminence/procfs#42 it looks like an alternative approach is needed for macOS, probably using conditional compilation to bring in a separate platform specific library on macOS and Windows.

Absolutely. We need to rewrite a part of the sensors logic. I should start this refactoring next week as I plan to work on the windows support (#74).

[mrchrisadams]

From reading @luiscruz 's recent, accessible, and quite fun to read piece on measuring energy in computers - syspower is a small OSS programme that can read power usage on non-Apple-Silicon Macs:

https://github.com/s4y/syspower/

It's written in C++, is less than 200 lines long, and quite well documented.

[florimondmanca]

@mrchrisadams 👍 I just tried it out on my macOS laptop, seems to work brilliantly.

The default command line program outputs a periodic bundle as follows:

PSTR (System Total):    13.2656
PCPC (CPU package CPU): 2.88281
PCPG (CPU package GPU): 0.0898438
PCPT (CPU package Tot): 13.2656

All values are in watts (W), according to the code comments. The value ~13W seems reasonable to me for a mostly idle MBP 2012. Also:

• If I dim my screen to the max, PSTR goes down to ~9W.
• If I start a while True: pass loop in Python in a separate terminal, PSTR goes to ~21W (+8 W).
• Put another loop up, PSTR goes ~28W (+7 W).

Which seems consistent.

It'd be fun to try and integrate this into scaphandre as a proof-of-concept, perhaps?

Edit: okay, seems like the refactor based on conditional compilation is a prerequisite to this in any case.

[luiscruz]

Nice! If you come across a solution for Apple M1 CPUs please let me know! 🙏

[Vic063]

Hello,

I just looked into Intel Power Gadgets which uses a kernel driver to read/write from/into MSRs.
Sadly, I do not own an Apple computer so I only tested on a VM running under qemu/KVM.

Obviously, required MSRs are not available but I have a list of the used MSRs (which are mainly the same used by the Windows version):
[244872.306699] kvm [927770]: ignored rdmsr: 0x606 data 0x0
[244872.306699] kvm [927770]: ignored rdmsr: 0x610 data 0x0
[244872.306699] kvm [927770]: ignored rdmsr: 0x611 data 0x0
[244872.306699] kvm [927770]: ignored rdmsr: 0x614 data 0x0
[244872.306710] kvm [927770]: ignored rdmsr: 0x38f data 0x0
[244872.306714] kvm [927770]: ignored rdmsr: 0x38d data 0x0
[244872.306724] kvm [927770]: ignored rdmsr: 0x38f data 0x0

However, I use to run a modified Linux kernel which implements these MSRs so it could be possible to inject (a/k/a write) values into these MSRs and make them available to Scaphandre inside the VM (I already did that for Windows VMs).

I also looked into the solution purposed by @mrchrisadams but syspower does not return any value on my Mac OS VM. I did not dig more than that.

Now I'm thinking about writing an OS X driver with the capability to read MSRs values, like the Windows driver. As the Scaphandre engine is being rewritten to support multiple OSes, the same method would be applied to read from OS X or Windows NT, right @bpetit?

[mrchrisadams]

hi @Vic063 - I've never come across the term MSR, so I'm having a hard time following along on the last comment.

Would you mind sharing a link explaining what they are?

[Vic063]

Hello @mrchrisadams
sorry, MSR stands for Model Specific Register. Since ~1993, Intel releases processors which embed some specific registers which are used for specific jobs.

For example, MSRs linked to RAPL can be internally used to protect the processor from over-voltage when someone tries to overclock its CPU. You can find more explanations here: [https://en.wikipedia.org/wiki/Model-specific_register](Model-specific register - Wikipedia)

[bpetit]

Hi !

Yes @Vic063 I think we could use the same methodology in the Scaphandre codebase, as for Windows, to plug to a dedicated MSR based driver. Even if we find a less invasive way to monitor power consumption on MacOS at some point, it may be included in a new sensor, in addition to this one anyway. So your help would be very helpful here 😁 (we also have to release the windows driver codebase soon :P).

I also add to this topic this recent very interesting blog post : https://www.rdegges.com/2022/how-to-calculate-the-energy-consumption-of-a-mac/

It seems the author talks mostly about battery data, which is something we could have used for Linux technically, but didn't use because it didn't fit a server use case. However, on this MacOs topic, I have the feeling that most of the use cases are for laptops (if anyone has a mac mini or a mac server raise a hand ! 😂). So maybe, if getting RAPL data on macos is too time consuming, we could use the data from battery consumption as a first iteration ? WDYT ? (I don't say we should do it, but having a sensor based on those data seems interesting too, in this context, especially for MacOS and M1 CPU that are ARM CPU and don't provide RAPL for example...).

We could also join him as he seems very interested in the topic ! 🤝

[rdegges]

Happy to test stuff out for ya -- the approach I outlined in that article is using Apple's ioreg CLI utility which inspects hardware devices and returns information. They have a low-level API within their "battery" metrics that allows you to read the current amount of watts being pulled from a power adapter -- which is what I used to measure total consumption over time.

In my experience, this is the best way to do it if you want to get an accurate reading (I verified it was accurate using hardware monitors) for the OVERALL energy consumption of the device. The downside (I think?) for your usecase is that it returns the number of watts overall, it isn't looking at how many volts particular components are using at any given moment (CPU vs GPU etc.).

[davidmytton]

Firefox 104 introduced a new power profiler for Windows (Windows 11 or Windows 10 with an EMI) and Mac (Apple Silicon only). This provides output in watts in the profiler UI, but the internals are measuring in picowatt-hour. On Mac it's using the task_info API.

Discussed in https://www.green-coding.org/blog/firefox-104-energy-measurements/