Is it possible to distinguish between the sql exporter process status and the target database?

[karenlava]

We are currently planning to use sql exporter as a remote exporter.

Currently, the "up" state changes to 0 even when the target database engine is down, and the up state changes to 0 even when the exporter is down. Therefore, it is difficult to determine whether the actual database is down using only the up variable. Is there a way to distinguish between database down and exporter down, like "mssql_up"?

Thank you.

[fsiler]

When you say up, are you referring to the one produced by sql_exporter, or the one produced by your upstream Prometheus instance?

[karenlava]

I think that prometheus produce it.

As explained below, up=0 is recorded both when the database is inaccessible and when the exporter is down. Is there a separate way for me to distinguish between the two situations?

When the database is unreachable, /metrics responds with HTTP code 500 Internal Server Error, causing Prometheus to record up=0 for that scrape. Only metrics defined by collectors are exported on the /metrics endpoint. SQL Exporter process metrics are exported at /sql_exporter_metrics.

[burningalchemist]

Hey @karenlava,

I think you might want to name your target in the configuration like here:

# The target to monitor and the list of collectors to execute on it.
target:
  name: "target_name"
  data_source_name: 'dsn://example'
  collectors: [collectors_example]

Once configured, there should appear another up metric (it's exposed by sql_exporter) with the target name label (and any other static labels you might include as well), that corresponds to the database connection state. The name should be put explicitly for a single target mode. In case of using jobs (multi-target mode), the labels are populated automatically.

So, If you query the up metric with respective labels (e.g. target label), you should be able to receive correct time series that might help distinguish statuses.

In addition, you can also scrape /sql_exporter_metrics endpoint for internal sql_exporter metrics. If this endpoint is unreachable, then sql_exporter process is for sure down.

Also, in the upcoming release we also expose scrape_errors_total metric that should indicate scrape success status.

Please, give it a try.

I'll also convert this issue into a discussion.