PyObservability

Platform Supported

Deployments

Kick off

Recommendations

• Install python 3.11 or above
• Use a dedicated virtual environment

Install PyObservability

python -m pip install pyobservability

Initiate - IDE

import pyobservability


if __name__ == '__main__':
    pyobservability.start()

Initiate - CLI

pyobservability start

Use pyobservability --help for usage instructions.

Containerized Deployment

docker pull thevickypedia/pyobservability:latest

docker run \
    --name observability \
    -p 8080:80 \
    -v /home/user/config:/config \
    --restart=no \
    thevickypedia/pyobservability

Grafana Dashboard powered by Prometheus

PyObservability can optionally stream metrics through a /metrics endpoint in Prometheus format, which can be scraped by Prometheus and visualized in Grafana. The dashboard is powered by Prometheus, which is used to scrape the metrics from the targets and store them in a time-series database.

Steps to set up the Grafana dashboard:

1. Create a config directory in the current working directory to store the configuration files for the server and Grafana containers.
2. Copy samples/secrets.json to the config directory and update it with the appropriate values for your environment.
3. COPY samples/docker-compose-server.yml and samples/docker-compose-grafana.yml to current working directory.
4. Run ./docker-launch.sh to start the server and Grafana containers using Docker Compose.

   NOTE: Set env variables [OR] update the docker-compose-*.yml files with the desired port numbers and config directory location before running the script.

5. Access the Grafana dashboard at http://localhost:3000 and log in with the credentials specified in the secrets.json file.
6. Add Prometheus as a data source in Grafana using the URL http://host.docker.internal:9090 (or http://localhost:9090 if running on Linux).
7. Import the sample Grafana dashboard JSON file into Grafana to visualize the metrics.

Environment Variables

Sourcing environment variables from an env file

By default, PyObservability will look for a .env file in the current working directory. JSON file format is also supported with a custom kwarg or env var env_file pointing to the filepath.

Mandatory

• TARGETS - Target URLs running PyNinja in the following format.
  • TARGETS='[{"name":"node1","base_url":"http://192.168.1.10:8000","apikey":"token1"},{"name":"node2","base_url":"http://192.168.1.11:8000"}]'

Defaults

• HOST - Host IP to run PyObservability. Defaults to 127.0.0.1 or 0.0.0.0
• PORT - Port number to run PyObservability. Defaults to 8080
• INTERVAL - Polling interval to retrieve server information.

Optional

• USERNAME - Username to authenticate the monitoring page.
• PASSWORD - Password to authenticate the monitoring page.
• TIMEOUT - Timeout (in seconds) for UI authentication. Defaults to 5m.
• LEGACY_UI - Enable legacy UI. Defaults to False, displaying a Grafana like dashboard.

Logging

PyObservability uses uvicorn logger by default. Following options can be used to override the default logger.

• LOG - Lazy config to use the default log format. Can either be file or stdout.
• DEBUG - Enables debug level logging. Defaults to False.
• LOGS_PATH - Directory path to store log files if LOG is set to file.
• LOG_CONFIG - Path to a custom logging configuration file.

Uptime Kuma

Uptime Kuma integration can be enabled by setting the following environment variables.

• KUMA_URL - Base URL of the Uptime Kuma server.
• KUMA_USERNAME - Username to authenticate with Uptime Kuma.
• KUMA_PASSWORD - Password to authenticate with Uptime Kuma.
• KUMA_TIMEOUT - Timeout (in seconds) for Uptime Kuma authentication. Defaults to 5s.

GitHub Runners

GitHub Runners integration can be enabled by setting the following environment variables.

• GIT_ORG - GitHub organization name or username.
• GIT_TOKEN - GitHub token with read:org permissions.

Prometheus Metrics

Enabling prometheus metrics will expose a /metrics endpoint in Prometheus format, which can be scraped by Prometheus and visualized in Grafana.
This endpoint is automatically secured with the same credentials as the monitoring page if authentication is enabled.

• PROMETHEUS_ENABLED - Enable Prometheus metrics endpoint. Defaults to False.

Warning

Enabling prometheus metrics will increase the resource usage in all the monitored nodes, as the metrics are constantly streamed as long as the server is running. It is recommended to use this option with a high polling interval to reduce the resource usage.

License & copyright

© Vignesh Rao

Licensed under the MIT License