Merge pull request #699 from SignalK/esp_log

Use native ESP-IDF logging

.clang-format

@@ -1,3 +1,9 @@
 ---
 BasedOnStyle: Google
 IndentWidth: 2
+IncludeBlocksStyle: Regroup
+IncludeCategories:
+ - Regex: '"sensesp\.h"'
+ Priority: -1000
+ - Regex: '"esp32-hal-log\.h"'
+ Priority: -900

.github/workflows/test.yml

@@ -4,7 +4,6 @@ on: [push, pull_request]
 
 jobs:
  build:
-
  runs-on: ubuntu-latest
  strategy:
  matrix:
@@ -26,13 +25,19 @@ jobs:
  target_device:
  - esp32dev
  steps:
- - uses: actions/checkout@v1
- - name: Set up Python
- uses: actions/setup-python@v1
+ - uses: actions/checkout@v3
+ - uses: actions/cache@v3
+ with:
+ path: |
+ ~/.cache/pip
+ ~/.platformio/.cache
+ key: ${{ runner.os }}-pio
+ - uses: actions/setup-python@v4
+ with:
+ python-version: "3.11"
  - name: Install dependencies
  run: |
- python -m pip install --upgrade pip
- pip install platformio
+ pip install --upgrade platformio
  - name: Run PlatformIO
  run: ci/run-ci.sh
  env:

ci/platformio-esp32dev.ini

@@ -15,3 +15,9 @@ board_build.partitions = min_spiffs.csv
 monitor_filters = esp32_exception_decoder
 build_flags =
  -D LED_BUILTIN=2
+ -D CORE_DEBUG_LEVEL=ARDUHAL_LOG_LEVEL_VERBOSE
+ ; Arduino Core bug workaround: define the log tag for the Arduino
+ ; logging macros.
+ -D TAG='"Arduino"'
+ ; Use the ESP-IDF logging library - required by SensESP.
+ -D USE_ESP_IDF_LOG

docs/pages/migration/index.md

@@ -4,14 +4,77 @@ title: Migrating From Version 1
 nav_order: 70
 ---
 
-# Migrating SensESP Version 1 Projects to Version 2
+# Migration Guides for SensESP Major Versions
+
+## Migrating SensESP Version 2 Projects to Version 3
+
+
+
+### Logging
+
+Previous SensESP versions logged to the serial port using the `debugX` functions, where `X` is the log level.
+This pattern was inherited from the RemoteDebug library.
+SensESP v3 has switched to using standard ESP-IDF logging functions.
+They allow redirecting log messages to different outputs, which will be used in future SensESP versions to provide logging to the web interface.
+
+To enable logging in SensESP v3, change the `build_flags` in your `platformio.ini` file to include the following:
+
+```ini
+build_flags =
+ -D LED_BUILTIN=2
+ ; Max (and default) debugging level in Arduino ESP32 Core
+ -D CORE_DEBUG_LEVEL=ARDUHAL_LOG_LEVEL_VERBOSE
+ ; Arduino Core bug workaround: define the log tag for the Arduino
+ ; logging macros.
+ -D TAG='"ARDUINO"'
+ ; Use the ESP-IDF logging library - required by SensESP.
+ -D USE_ESP_IDF_LOG
+```
+
+The `debugX` functions are still available, but they are now just wrappers around the ESP-IDF logging functions.
+In any new code, use the `ESP_LOGX` functions, where `X` is the log level.
+These require a tag argument, which is a string that identifies the source of the log message.
+You can use any tag you like, but for simple programs, you can use the `__FILE__` macro, which expands to the name of the current file.
+
+The allowed log levels are:
+
+- `NONE`: No log output
+- `ERROR`: Critical errors, software module can not recover on its own
+- `WARN`: Error conditions from which recovery measures have been taken
+- `INFO`: Information messages which describe normal flow of events
+- `DEBUG`: Extra information which is not necessary for normal use (values, pointers, sizes, etc).
+- `VERBOSE`: Bigger chunks of debugging information, or frequent messages which can potentially flood the output.
+
+Here is an example of how to use the `ESP_LOGX` functions:
+
+```c++
+ESP_LOGI(__FILE__, "Initializing NMEA2000");
+...
+ESP_LOGD(__FILE__, "Sending value %d to fobulator %s", value, fobulator_name);
+...
+ESP_LOGE(__FILE__, "Failed to initialize NMEA2000");
+```
+
+To enable logging in previous SensESP versions, you had to call `SetupSerialDebug(115200)` as the first line in your `setup()` function.
+In new SensESP versions, replace this with the new `SetupLogging()` function call to set logging defaults.
+The old `SetupSerialDebug` function is still available, but it is now just a wrapper around `SetupLogging`.
+
+It is possible to change the log level for individual tags.
+Here is an example of how to set the overall log level to `INFO` and the log level for the `main.cpp` tag to `DEBUG`:
+
+```c++
+esp_log_level_set("*", ESP_LOG_INFO);
+esp_log_level_set("main.cpp", ESP_LOG_DEBUG);
+```
+
+## Migrating SensESP Version 1 Projects to Version 2
 
 SensESP version 2 has a number of backwards-incompatible changes compared to the previous version.
 Most of the changes are trivial, while some others require a bit of more work to update the code.
 
 This document walks through the most important changes, and explains how to update your project to the new version.
 
-## ESP8266 Support Removed
+### ESP8266 Support Removed
 
 If your project uses ESP8266 hardware, you will either have to update to an ESP32 device, *or* you can keep using SensESP version 1. To peg your project to SensESP v1, change the SensESP dependency in your project's `platformio.ini` file `lib_deps` section to this:
 
@@ -20,9 +83,9 @@ lib_deps =
  SignalK/SensESP @ ^1.0.8
 ```
 
-## Main Program Structure
+### Main Program Structure
 
-### Setup and Loop Functions
+#### Setup and Loop Functions
 
 SensESP builds on [ReactESP](https://github.com/mairas/ReactESP), which is an event-based framework for developing ESP32 firmware.
 Previous versions of ReactESP defined the Arduino Framework default `setup()` and `loop()` functions internally and relied on a lambda function for initializing the program and defining the top-level functionality:
@@ -56,7 +119,7 @@ void setup() {
 void loop() { app.tick(); }
 ```
 
-### Namespace Usage
+#### Namespace Usage
 
 In projects with a lot of dependencies, it is common that some upstream library exports some very generic symbol names, which then causes conflicts or hard-to-debug issues in the code being developed. The standard C++ approach to mitigate these issues is to use a namespace.
 
@@ -87,7 +150,7 @@ namespace sensesp {
 }
 ```
 
-## External Sensors
+### External Sensors
 
 All Sensor classes requiring external libraries have been removed.
 Reducing the number of external dependencies improves code stability and improves build times.
@@ -97,7 +160,7 @@ Most, however, have been removed in favor of a more generic approach, namely the
 The `RepeatSensor` class allows you to easily interface any external hardware sensor libraries with SensESP.
 See the `RepeatSensor` tutorials ([part 1](../tutorials/bmp280), [part 2](../tutorials/bmp280)) for more details.
 
-## Renamed Classes and Types
+### Renamed Classes and Types
 
 Type-specific Consumer and Producer class names have been renamed to more closely match the native C++ types.
 
@@ -109,15 +172,15 @@ Similarly, names with other types have been renamed to more closely match the st
 
 To better reflect the intent and the functionality, the `Enable` class has been renamed to `Startable`.
 
-## Class Interface Changes
+### Class Interface Changes
 
 Some class public interfaces have been changed.
 
 The [`DigitalInputState`](https://signalk.org/SensESP/generated/docs/classsensesp_1_1_digital_input_state.html) constructor no longer accepts the `interrupt_type` argument because that class never used interrupts.
 
 The [`DigitalInputChange`](https://signalk.org/SensESP/generated/docs/classsensesp_1_1_digital_input_change.html) implementation has been simplified and the constructor no longer requres the `read_delay` argument.
 
-## System Info Sensors
+### System Info Sensors
 
 SensESP v1 had so called "standard sensors" that transmit information on the operation of the device: free memory, number of event loop executions per second, device IP address, and so on.
 The standard sensors were initialized using a SensESP constructor or builder bitfield argument:
@@ -135,7 +198,7 @@ sensesp_app = builder.enable_free_mem_sensor()
  ->get_app();
 ```
 
-## Remote Debugger Disabled
+### Remote Debugger Disabled
 
 The Remote Debugger allows you to connect to the device over telnet and view the log messages and even reset the device, all without a USB cable connection.
 Even though this is a neat and useful feature, it was not widely known and uses *a lot* of memory.
@@ -161,7 +224,7 @@ build_flags =
  -D DEBUG_DISABLED
 ```
 
-## Over-The-Air (OTA) Firmware Updates
+### Over-The-Air (OTA) Firmware Updates
 
 OTA firmware updates have been supported already for a long time.
 To improve security, OTA updates are now enabled only if an OTA password is defined in the App builder:

examples/analog_input.cpp

@@ -17,9 +17,7 @@ reactesp::ReactESP app;
 // The setup function performs one-time application initialization.
 void setup() {
 // Some initialization boilerplate when in debug mode...
-#ifndef SERIAL_DEBUG_DISABLED
- SetupSerialDebug(115200);
-#endif
+ SetupLogging();
 
  // Create the global SensESPApp() object.
  SensESPAppBuilder builder;

examples/async_repeat_sensor.cpp

@@ -17,10 +17,7 @@ reactesp::ReactESP app;
 
 // The setup function performs one-time application initialization.
 void setup() {
-// Some initialization boilerplate when in debug mode...
-#ifndef SERIAL_DEBUG_DISABLED
- SetupSerialDebug(115200);
-#endif
+ SetupLogging();
 
  // Create the global SensESPApp() object.
  SensESPAppBuilder builder;

examples/chain_counter.cpp

@@ -25,7 +25,7 @@ using namespace sensesp;
 ReactESP app;
 
 void setup() {
- SetupSerialDebug(115200);
+ SetupLogging();
 
  SensESPAppBuilder builder;
  sensesp_app = builder.set_hostname("ChainCounter")

examples/constant_sensor.cpp

@@ -20,10 +20,7 @@ using namespace sensesp;
 reactesp::ReactESP app;
 
 void setup() {
- // Some initialization boilerplate when in debug mode...
-#ifndef SERIAL_DEBUG_DISABLED
- SetupSerialDebug(115200);
-#endif
+ SetupLogging();
 
  // Create the builder object
  SensESPAppBuilder builder;

examples/freertos_tasks.cpp

@@ -35,10 +35,7 @@ void ToggleTestOutputPin(void *parameter) {
 
 // The setup function performs one-time application initialization.
 void setup() {
-// Some initialization boilerplate when in debug mode...
-#ifndef SERIAL_DEBUG_DISABLED
- SetupSerialDebug(115200);
-#endif
+ SetupLogging();
 
  SensESPMinimalAppBuilder builder;
  SensESPMinimalApp *sensesp_app = builder.set_hostname("async")->get_app();

examples/fuel_level_sensor/fuel_level_sensor_example.cpp

@@ -10,9 +10,7 @@ using namespace sensesp;
 ReactESP app;
 
 void setup() {
-#ifndef SERIAL_DEBUG_DISABLED
- SetupSerialDebug(115200);
-#endif
+ SetupLogging();
 
  // Set up sensesp
  SensESPAppBuilder builder;

examples/hysteresis.cpp

@@ -12,7 +12,7 @@ using namespace sensesp;
 ReactESP app;
 
 void setup() {
- SetupSerialDebug(115200);
+ SetupLogging();
 
  SensESPAppBuilder builder;
  sensesp_app = builder.set_hostname("sensesp-hysteresis-example")

examples/lambda_transform.cpp

@@ -11,7 +11,7 @@ using namespace sensesp;
 ReactESP app;
 
 void setup() {
- SetupSerialDebug(115200);
+ SetupLogging();
 
  // Create a new SensESPApp object. This is the direct constructor call, and
  // an equivalent alternative to using the SensESPAppBuilder class.

examples/manual_networking.cpp

@@ -24,7 +24,7 @@ const uint8_t input_pin1 = 0;
 ReactESP app;
 
 void setup() {
- SetupSerialDebug(115200);
+ SetupLogging();
 
  SensESPMinimalAppBuilder builder;
  auto sensesp_app = builder.set_hostname("counter-test")->get_app();

examples/milone_level_sensor/milone_level_sensor.cpp

@@ -47,9 +47,7 @@ ReactESP app;
 
 void setup() {
 // Some initialization boilerplate when in debug mode...
-#ifndef SERIAL_DEBUG_DISABLED
- SetupSerialDebug(115200);
-#endif
+ SetupLogging();
 
  // Create a builder object
  SensESPAppBuilder builder;

examples/minimal_app.cpp

@@ -29,7 +29,7 @@ const uint8_t output_pin2 = 21;
 ReactESP app;
 
 void setup() {
- SetupSerialDebug(115200);
+ SetupLogging();
 
  SensESPMinimalAppBuilder builder;
  auto sensesp_app = builder.set_hostname("counter-test")->get_app();

examples/platformio.ini

@@ -15,7 +15,7 @@
 
 [platformio]
 ;set default_envs to whichever board(s) you use. Build/Run/etc processes those envs
-default_envs = 
+default_envs =
  esp32dev
 
 [env]
@@ -46,11 +46,19 @@ monitor_filters = esp32_exception_decoder
 [env:esp32dev]
 extends = espressif32_base
 board = esp32dev
-; Verify that this is the correct pin number for your board!
-build_flags = 
+
+build_flags =
+ ; Verify that this is the correct pin number for your board!
  -D LED_BUILTIN = 2
+ ; Max (and default) debugging level in Arduino ESP32 Core
+ -D CORE_DEBUG_LEVEL=ARDUHAL_LOG_LEVEL_VERBOSE
+ ; Arduino Core bug workaround: define the log tag for the Arduino
+ ; logging macros.
+ -D TAG='"Arduino"'
+ ; Use the ESP-IDF logging library - required by SensESP.
+ -D USE_ESP_IDF_LOG
+
 ; uncomment and change these if PlatformIO can't auto-detect
 ; the ports
 ;upload_port = /dev/tty.SLAB_USBtoUART
 ;monitor_port = /dev/tty.SLAB_USBtoUART
-

examples/raw_json.cpp

@@ -14,9 +14,7 @@ reactesp::ReactESP app;
 ObservableValue<bool> toggler;
 
 void setup() {
-#ifndef SERIAL_DEBUG_DISABLED
- SetupSerialDebug(115200);
-#endif
+ SetupLogging();
 
  SensESPAppBuilder builder;
  SensESPApp *sensesp_app = builder.set_hostname("json_demo")

examples/relay_control.cpp

@@ -23,9 +23,7 @@ ReactESP app;
 
 void setup() {
 // Some initialization boilerplate when in debug mode...
-#ifndef SERIAL_DEBUG_DISABLED
- SetupSerialDebug(115200);
-#endif
+ SetupLogging();
 
  // Create a builder object
  SensESPAppBuilder builder;