Description

We should review and update the way we are capturing the output of our cloud modules (Azure, AWS y GCP) as well as the Docker-Listener module.

To better understand why this is necessary, first it is important to know how these modules are implemented. They are made of two different component:

• The core part, written in C, which launches the modules, catches the output and prints it to the ossec.log.
• The module itself, written in python, which executes the integration and generates some output to be shown in the stdout.

The issue here is that the way the output is written and sent to the ossec.log is not consistent between modules. In addition to that, the output of the different modules will be ignored, even if there are warning or error messages, if the debug mode is not enabled, which is disabled by default.

Here is a detailed explanation on how every module works currently:

AWS

The python module runs and its output, which is written using print statements, is caught. However, for the output to be dumped to the ossec.log at least one of the following conditions must be met:

• If there is an internal error in the C part the output will be written to ossec.log if debug mode is 1 or higher.
• If the execution ends with an error code 1 the output will be written to ossec.log if the Unknown error string was present.
• If the execution ends with an error code 2 the output will be written to ossec.log if the aws.py: error: string was present.
• If the execution ends with an error code 3 or higher the output will be written to ossec.log as a WARNING, regardless of if it was an ERROR.
• If the execution ends successfully with an return value 0 or fails and returns a negative value the output will be written to the ossec.log only if debug mode is set to 2.

GCP

The python module runs and its output, which is written using logging statements, is caught. However, for the output to be dumped to the ossec.log at least one of the following conditions must be met:

• If debug level is 0 only CRITICAL, ERROR and WARNING messages will be written
• If debug level is 1 it will show INFO messages in addition to the previously listed.
• If debug level is 2 it will show DEBUG messages in addition to the previously listed.

This is done by looking for the WARNING, ERROR, INFO strings present in the output. If not present, the output is skipped.

Azure

The azure module uses logging in a similar way to how the GCP was implemented. However, the C implementation is not expecting output from this module and it is not processing it, hence nothing will be written into the ossec.log at all. This is because the older implementation wrote its output directly to a custom file and now it is set to send it to the stdout but the core part must be updated to expect and catch this output.

Docker Listener

There is output written using print statements that will be shown only when running the module manually. The module sent his output to the analysis engine in the form of events.

Proposed solution

We do not want to write directly on the ossec.log, as this task is the responsibility of modulesd, but we should keep the logic of determining when a message should be shown and when not depending on the debug level in the modules part. This logic currently already exists. The solution would be to just update the C part to just capture the output it receives as it is, without filtering it at all, and write it to the ossec.log.

This solution involves updating the four modules as well as the core part and their unit testing.

Related Issues

• Review Logging package to test integration to be used as the logging method of external integrations #16619
• Develop logging tool or interface to standardize logs for all cloud integration modules #16717
• Refactor AWS module to use Logger tool and log relevant information #16737
• Refactor GCP module to use Logger tool and log relevant information #16738
• Refactor Azure module to use Logger tool and log relevant information #16739
• Module docker-listener not executed every <interval> it stays blocked in wpopenl #18906
• Develop a modularized log parser for wazuh-modulesd #17049
• Investigate and enhance external modules logging messages #16314
• Review cloud modules logging messages #23843
• Fix AWS integration tests after logging refactor #23841

Branch

• https://github.com/wazuh/wazuh/tree/16717-improve-cloud-modules-logging