What's Changed

• [-] monitor all sources if --group cmdopt is omitted, fixes #843 (#847)
• [*] improve stat-statements Grafana v11 Postgres dashboards by @f9n in #846

Full Changelog: v3.6.0...v3.7.0

What to do next?

• Star the repository
• Read the documentation
• Check out the blog
• Follow us on Linkedin

What's Changed

• [+] add "default" group processing by @pashagolub in #798
• [+] add measurements written log message to RPC sink by @0xgouda in #829
• [+] add TLS encryption to RPC channel by @0xgouda in #796
• [+] bump go.etcd.io/etcd/v3 from 3.6.0 to 3.6.2 by @dependabot[bot] in #789 #837
• [+] bump pashagolub/pgxmock/v4 from 4.7.0 to 4.8.0 by @dependabot[bot] in #806
• [+] bump shirou/gopsutil/v4 from 4.25.5 to 4.25.6 by @dependabot[bot] in #827
• [+] bump testcontainers/testcontainers-go from 0.37.0 to 0.38.0 by @dependabot[bot] in #839
• [+] optimize the size of MeasurementEnvelope by @pashagolub in #793
• [*] exclude MetricDef from MeasurementEnvelope by @pashagolub in #787
• [*] improve --batching-delay parameter description, closes #821 by @pashagolub in #828
• [-] fix --group command-line option, closes #792 by @pashagolub in #794
• [-] fix instance_up metric, closes #759 by @pashagolub in #791
• [-] fix master only postgres source shutdown due to catched InRecovery state logic by @0xgouda in #813
• [-] fix Reap() method to update Recovery cache after processing the previous value by @0xgouda in #815
• [-] fix Reap() to not require Source.MinDbSizeMb to be >= 8 by @0xgouda in #818
• [-] fix Source.Equal() presets comparison, closes #830 by @0xgouda in #831
• [-] fix typo in reapMetricMeasurements() log message by @0xgouda in #807
• [-] fix JSONB null casting errors in postgres/v11 dashboards by @f9n in #838
• [-] fix JSONB null casting errors in "Top Tables" and "Health-check" dashboards by @kmoppel in #833
• [-] improve Prometheus scrapping, fixes #749 by @pashagolub in #790
• [-] remove unnecessary check from FetchMetric() by @0xgouda in #808

New Contributors

• @f9n made their first contribution in #838

Full Changelog: v3.5.0...v3.6.0

What to do next?

• Star the repository
• Read the documentation
• Check out the blog
• Follow us on Linkedin

What's Changed

• [+] add pprof build profile for debugging by @pashagolub in #779
• [+] bump shirou/gopsutil/v4 from 4.25.4 to 4.25.5 by @dependabot in #774
• [+] decrease memory allocations for metrics.MeasurementEnvelope by @pashagolub in #768
• [+] define sync metric operations types as constants by @0xgouda in #781 #782
• [+] move sources sync to the reaper by @pashagolub in #769
• [+] rewrite ParseLogs() with timers and eliminate extra checks by @pashagolub in #770
• [+] switch to json-iterator/go from encoding/json by @pashagolub in #776
• [*] change json sink to use one global encoder by @0xgouda in #772
• [*] remove metrics definitions from Postgres sink by @pashagolub in #773
• [*] use pgx.Rows directly in DetectConfigurationChanges() by @pashagolub in #775
• [-] add additional checks to LogParse() by @pashagolub in #784
• [-] remove misleading VOLUME /var/lib/postgresql declaration by @pashagolub in #780

New Contributors

• @0xgouda made their first contribution in #772

Full Changelog: v3.4.0...v3.5.0

What to do next?

• Star the repository
• Read the documentation
• Check out the blog
• Follow us on Linkedin

What's Changed

• [+] bump jackc/pgx/v5 from 5.7.4 to 5.7.5 by @dependabot in #763
• [+] bump go.etcd.io/etcd/v3 from 3.5.21 to 3.6.0 by @dependabot in #762
• [*] use consistent password in install docs by @df7cb in #765
• [-] fix dbSourcesReaderWriter.updateDatabase() to include is_enabled by @pashagolub in #760
• [-] fix QueryMeasurements nil pointer dereference, fixes #766 by @pashagolub in #767

New Contributors

• @df7cb made their first contribution in #765

Full Changelog: v3.3.0...v3.4.0

What to do next?

• Star the repository
• Read the documentation
• Check out the blog
• Follow us on Linkedin

What's Changed

• [+] add pgwatch source resolve command by @pashagolub in #723
• [+] add a SystemD service template by @kmoppel in #704
• [+] add pgpool service to docker compose files by @pashagolub in #739
• [+] add replication node to docker compose file by @pashagolub in #734
• [+] add tests for pgwatch metric print commands by @pashagolub in #708
• [+] add tests for convert_metrics utility by @pashagolub in #744
• [+] add tests for pgwatch by @pashagolub in #748
• [+] add tests for webserver by @pashagolub in #754
• [+] add tests for etcd-based Patroni clusters resolver by @pashagolub in #751
• [+] bump cybertec-postgresql/pgx-migrator from 1.0.2 to 1.2.0 by @dependabot in #695 #727
• [+] bump go.etcd.io/etcd/client/v3 from 3.5.20 to 3.5.21 by @dependabot in #700
• [+] bump golang-jwt/jwt to v5.2.2 by @pashagolub in #717
• [+] bump golangci/golangci-lint-action from 6 to 7 by @dependabot in #690
• [+] bump http-proxy-middleware from 2.0.7 to 2.0.9 by @dependabot in #742
• [+] bump pashagolub/pgxmock/v4 from 4.6.0 to 4.7.0 by @dependabot in #726
• [+] bump prometheus/client_golang from 1.21.1 to 1.22.0 by @dependabot in #715
• [+] bump shirou/gopsutil/v4 from 4.25.2 to 4.25.4 by @dependabot in #705 #753
• [+] bump testcontainers/testcontainers-go from 0.35.0 to 0.37.0 by @dependabot in #701 #746
• [+] bump Go to v1.24 by @pashagolub in #698
• [+] improve sources test coverage by @pashagolub in #721
• [+] use new InstanceMetricCache implementation in the Reaper by @pashagolub in #716
• [+] use new concurrent metric definitions implementation by @pashagolub in #709
• [-] fix pgwatcg source ping command by @pashagolub in #718
• [-] fix anonymous Grafana access in Docker, fixes #724 by @iuuuuan in #725
• [-] fix duplicated definition of symbol dlopen in #758
• [-] fix growing measurement cache introduced in #716 by @pashagolub in #737
• [-] fix log message for rows written in JSONWriter by @pashagolub in #738
• [-] fix --metrics command-option description, fixes #756 in #757
• [-] fix Pgpool-II support, closes #730 by @pashagolub in #740
• [*] deprecate IsSuperuser source option by @pashagolub in #706
• [*] drop support for discontinued Azure Single Server by @pashagolub in #736
• [*] fix hard coded epoch_ns measurement field name by @pashagolub in #712
• [*] fix old archives.format_overrides.format goreleaser option by @pashagolub in #694
• [*] move psutil functionality to reaper package by @pashagolub in #714
• [*] refactor TryCreateMetricsFetchingHelpers by @pashagolub in #743
• [*] remove unused db.GetTableColumns() by @pashagolub in #707
• [*] replace MetricFetchConfig with SourceConn in FetchMetrics() by @pashagolub in #741

New Contributors

• @iuuuuan made their first contribution in #725

Full Changelog: v3.2.0...v3.3.0

What to do next?

• Star the repository
• Read the documentation
• Check out the blog
• Follow us on Linkedin

What's Changed

• [!] refactor reaper package and sinks.MultiWriter by @pashagolub in #667
• [+] add Reaper.WriteMonitoredSources() and QueryMeasurents() by @pashagolub in #678
• [+] add migration instructions for configuration database, closes #654 by @pashagolub in #655
• [+] allow to add EDB Postgres under monitoring, closes #649 by @pashagolub in #650
• [+] bump @babel/helpers from 7.20.13 to 7.26.10 by @dependabot in #669
• [+] bump @babel/runtime from 7.20.13 to 7.26.10 by @dependabot in #666
• [+] bump axios from 1.7.4 to 1.8.2 by @dependabot in #668
• [+] bump go.etcd.io/etcd/client/v3 from 3.5.18 to 3.5.20 by @dependabot in #662 #687
• [+] bump jackc/pgx/v5 from 5.7.2 to 5.7.4 by @dependabot in #688 #691
• [+] bump pashagolub/pgxmock/v4 from 4.5.0 to 4.6.0 by @dependabot in #683
• [+] bump prometheus/client_golang from 1.20.5 to 1.21.1 by @dependabot in #656 #660
• [+] bump shirou/gopsutil/v4 from 4.25.1 to 4.25.2 by @dependabot in #659
• [+] install Grafana with apt in Docker and bump to v10.4.15 by @pashagolub in #647
• [*] fix typo in documentation by @mookie- in #686
• [*] remove deprecated reaper constants by @pashagolub in #670
• [*] remove obsolete unreachable map by @pashagolub in #672
• [*] rename sources.MonitoredDatabase to SourceConn by @pashagolub in #671
• [*] replace configMap arguments with interval by @pashagolub in #674
• [*] updating non-functional links in docs by @shubham-j-sde in #681
• [*] use dark style logo by @pashagolub in #651
• [*] use logging source field for log parsing instead of fmt modifiers by @pashagolub in #658
• [-] fix adding new metric to configuration db via web UI, fixes #664 by @pashagolub in #665
• [-] fix connection issues for discovered quoted db names, fixes #675 by @pashagolub in #676
• [-] fix datasource uid for Grafana v11 dashboards by @pashagolub in #689
• [-] fix duplicate object key in postgres/v11/system-stats.json by @AjayPawarExist in #685
• [-] fix linter warning "impossible condition: nil != nil" by @pashagolub in #652
• [-] remove pg_stat_file() dependencies from metrics by @pashagolub in #677
• [-] remove obsolete undersizedDBs and recoveryIgnoredDBs maps by @pashagolub in #673

New Contributors

• @mookie- made their first contribution in #686
• @shubham-j-sde made their first contribution in #681
• @AjayPawarExist made their first contribution in #685

Full Changelog: v3.1.0...v3.2.0

What to do next?

• Star the repository
• Read the documentation
• Check out the blog
• Follow us on Linkedin

What's Changed

• [+] refactor config command and sub-commands by @pashagolub in #621
• [+] persist WebUI visible column settings, fixes #581 by @ay-bh in #635
• [+] make fallback logger use default options by @pashagolub in #620
• [+] improve pgwatch-demo Docker image by @pashagolub in #643
• [+] improve docker compose files by @pashagolub in #642
• [+] fail early if Prometheus sink cannot be served by @pashagolub in #627
• [+] bump jackc/pgx/v5 from 5.7.1 to 5.7.2 by @dependabot in #608
• [+] bump testcontainers/testcontainers-go from 0.34.0 to 0.35.0 by @dependabot in #617
• [+] bump shirou/gopsutil/v4 from 4.24.11 to 4.25.1 by @dependabot in #609 #632
• [+] bump pashagolub/pgxmock/v4 from 4.4.0 to 4.5.0 by @dependabot in #640
• [+] bump go.etcd.io/etcd/client/v3 from 3.5.17 to 3.5.18 by @dependabot in #628
• [+] add packages doc.go files by @pashagolub in #623
• [+] add /readiness and /liveness endpoints, closes #611 by @pashagolub in #615
• [*] renormalize line endings by @pashagolub in #604
• [*] Prometheus Grafana v11 dashboard updates by @kmoppel in #626
• [*] mask password in the source connection string, fixes #610 by @tvecerakb in #618
• [*] align text fields to the left in WebUI grids by @pashagolub in #619
• [*] add init_sql for metrics with external dependencies by @pashagolub in #641
• [-] remove passwords from sink log messages, fixes #612 by @pashagolub in #614
• [-] fix updating metric definition in database configuration, fixes #644 by @pashagolub in #645
• [-] fix duplicate source names in YAML configs, fixes #630 by @amrdb in #633
• [-] fix concatenated output of pgwatch metric print-init by @agelwarg in #631
• [-] fix checkpointer dashboards for PostgreSQL 17, fixes #613 by @ay-bh in #636
• [-] fix PostgresWriter.EnsureMetricTime(), fixes #622 by @pashagolub in #624
• [-] fix "invalid command -- ping" for PgBouncer source, closes #625 by @pashagolub in #639

New Contributors

• @tvecerakb made their first contribution in #618
• @agelwarg made their first contribution in #631
• @amrdb made their first contribution in #633
• @kmoppel made their first contribution in #626
• @ay-bh made their first contribution in #635

Full Changelog: v3.0.0...v3.1.0

What to do next?

• Star the repository
• Read the documentation
• Check out the blog
• Follow us on Linkedin

What's New

This major release introduces several significant updates, new features, bug fixes, and enhancements:

Important Changes

• Documentation: The documentation has been updated and moved to https://pgwat.ch for improved readability and navigation. Special part dedicated to the pgwatch development has been added.
• Parallel Sinks: Support for simultaneous storing metric measurements to several destinations, e.g. database and file at once.
• Custom Storage: Leverage our remote sink (RPC) interface to seamlessly integrate any storage format or data stream, such as Parquet files or Kafka, directly into pgwatch.
• Windows Support Added: Windows support has been integrated, broadening the compatibility of the software.
• Docker Images Rename: The pgwatch2-postgres Docker image has been renamed to pgwatch-demo, pgwatch2-daemon to pgwatch.
• Enhanced Logging: An advanced logging system has been implemented to improve error tracking and monitoring.
• Grafana Upgrade: Support for the latest Grafana v9.x and v10 has been added, including a switch to Grafana's REST API for dashboard provisioning.
• Etcd v3 Migration: The system has been upgraded to Etcd v3, ensuring better performance and compatibility.
• PostgreSQL 17 Metrics: New metrics for PostgreSQL 17 have been introduced.
• Automatic Schema Rollout: A new automatic schema rollout mechanism has been implemented, streamlining database management.
• Build and Test Workflow: A new "Build and Test" workflow has been added to automate testing and building processes.
• WebUI Enhancements: The WebUI now includes options like test connections, a log tab, and a clone button for the database grid.
• REST API: New REST API endpoints have been added, including endpoints such as source, /metric, /preset, log.
• Docker Improvements: The Docker images and related scripts have been improved, including the introduction of a new Dockerfile based on the official Postgres image and Docker Compose support.
• Test Coverage: Test coverage statistics are now available to ensure code quality and reliability.
• Environment Variables Update: The PW2_ environment variable prefixes have been updated to PW3_.
• Code Refactoring: The main.go file has been streamlined by extracting functionality into separate packages.
• Deprecations: Removed support for InfluxDB, Graphite datastore, and other outdated features.

Dependency Updates

Numerous dependencies have been updated, including upgrades to Golang, PostgreSQL, Prometheus, and various Go modules. These upgrades ensure better performance, security, and compatibility with other tools.

This release represents a significant step forward in functionality, performance, and security, with a strong focus on expanding compatibility and improving the user experience.

Changelog

• [!] add Measurements group and use URIs for --sink options by @pashagolub in #368
• [!] add Grafana v11 dashboards by @muni106 in #407
• [!] add metrics for PostgreSQL 16 by @pashagolub in #246
• [!] add RPC sink support, closes #464 by @destrex271 in #465
• [!] add support for jackc/pgx, closes #171 by @pashagolub in #204
• [!] add support for application commands by @pashagolub in #504
• [!] add support for log tab in web UI, closes #170 by @pashagolub in #206
• [!] add support for PostgreSQL v17 by @pashagolub in #497
• [!] add support for the latest Grafana v9.x, closes #174 by @pashagolub in #186
• [!] add windows support, closes #3 by @pashagolub in #4
• [!] allow multiple parallel metric storages by @pashagolub in #313
• [!] bump Grafana to v10, closes #227 by @pashagolub in #232
• [!] decouple sources package from main, closes #357 by @pashagolub in #358
• [!] decouple metrics into separate package, closes #372 by @pashagolub in #408
• [!] differentiate the concepts of source and monitored database by @pashagolub in #472
• [!] extract psutil into separate package by @pashagolub in #144
• [!] extract configuration handling into separate package, closes #138 by @pashagolub in #139
• [!] fix Go linting, closes #142 by @pashagolub in #143
• [!] implement automatic schema rollout machinery, closes #180 by @pashagolub in #225
• [!] implement enhanced logging by @pashagolub in #146
• [!] implement new Dockerfile based on official Postgres image, closes #210 by @pashagolub in #226
• [!] implement upgrade machinery, closes #179 by @pashagolub in #451
• [!] implement webui authentication, closes #216 by @denys-holub in #251
• [!] move measurements reaper code out of main package, closes #418 by @pashagolub in #419
• [!] move metric batching to sinks from the gatherer by @pashagolub in #346
• [!] rename pgwatch3-postgres Docker image to pgwatch3, closes #20 by @pashagolub in #21
• [!] restructure code tree, closes #518 by @pashagolub in #521
• [!] restructure Docker related assets by @pashagolub in #535
• [!] switch configuration to connection string from separate parameters, closes #255 by @pashagolub in #322
• [!] switch to Grafana REST API for dashboard provisioning, closes #188 by @pashagolub in #189
• [!] transform Sphinx manual to mkdocs-material by @pashagolub in #476
• [!] upgrade to Etcd v3, resolves #59 by @pashagolub in #157
• [+] add "Build and Test" workflow, closes #5 by @pashagolub in #6
• [+] add /metric endpoint, closes #93 by @pashagolub in #113
• [+] add arm64 binaries to release assets, closes #512 by @pashagolub in #517
• [+] add convert app to migrate v2 metrics definitions to v3 by @pashagolub in #373
• [+] add pgbouncer_clients metric by @pashagolub in #440
• [+] add Preset configs tab, closes #135 by @denys-holub in #136
• [+] add RPCSyncRequest to api by @pashagolub in #528
• [+] add usename to replication metric by @pashagolub in #444
• [+] add bootstrap tests by @pashagolub in #277
• [+] add built-in http server, closes #28 by @pashagolub in #30
• [+] add clone button to database grid, closes #150 by @denys-holub in #159
• [+] add commands config [init|upgrade], closes #514 by @pashagolub in #515
• [+] add concurrency clause to Build GHA workflow by @pashagolub in #269
• [+] add Docker Compose file, closes #240 by @pashagolub in #242
• [+] add icon and logo for web ui, closes #66 by @denys-holub in #67
• [+] add log tab to the main window, closes #68 by @denys-holub in #71
• [+] add main metrics descriptions, closes #547 by @pashagolub in #548
• [+] add metrics tests by @pashagolub in #478
• [+] add new metrics for invalid and unused indexes by @pashagolub in #286
• [+] add pgAdmin4 to docker-compose.yml by @ondar in #432
• [+] add Prometheus service to Docker Compose file by @pashagolub in #314
• [+] add support for /PATCH preset endpoint, closes #163 by @pashagolub in #187
• [+] add support for GET /preset endpoint, closes #132 by @pashagolub in #134
• [+] add support for POST /preset endpoint by @pashagolub in #177
• [+] add test cases to webserver api by @muni106 in #397
• [+] add test coverage statistics, closes #273 by @pashagolub in #274
• [+] add tests for db package by @pashagolub in #463
• [+] add tests for sinks by @pashagolub in #449
• [+] add tests for postgres sources.ReaderWriter implementation by @pashagolub in #469
• [+] add tests for YAML sources.ReaderWriter implementation by @pashagolub in #470
• [+] add TimescaleDB metric storage based docker image, resolves #176 by @pashagolub in #183
• [+] add trigger on metric to update presets, closes #185 by @pashagolub in #370
• [+] add web UI start options, closes #236 by @pashagolub in #237
• [+] allow CORS from https://localhost:4000 for webui debugging by @pashagolub in #439
• [+] bump axios from 1.6.0 to 1.7.4 by @dependabot in #522
• [+] bump express from 4.19.2 to 4.21.0 in webui by @dependabot in #544
• [+] bump jackc/pgx/v5 from 5.6.0 to 5.7.1 by @dependabot in #543
• [+] bump micromatch from 4.0.5 to 4.0.8 by @dependabot in #520
• [+] bump prometheus/client_golang from 1.20.1 to 1.20.4 by @dependabot in #519 #539 #545
• [+] bump rollup from 2.79.1 to 2.79.2 in webui by @dependabot in #550
• [+] bump shirou/gopsutil/v4 from 4.24.7 to 4.24.9 by @dependabot in #538 #551
• [+] bump yaml.v2 to yaml.v3, closes #359 by @pashagolub in #360
• [+] bump webpack from 5.76.1 to 5.94.0 by @dependabot in #537
• [+] bump Go to v1.23 by @pashagolub in #555
• [+] create common error & loading components, closes #125 by @denys-holub in #126
• [+] get rid of control channel map, closes #434 by @pashagolub in #435
• [+] highlight active WebUI tab, closes #199 by @denys-holub in #200
• [+] implement GET /metric & DELETE /metric to web UI, closes #116 by @denys-holub in #117
• [+] implement POST /metric & PATCH /metric to web UI, closes #119 by @denys-holub in #120
• [+] improve Build & Test GHA workflow by @pashagolub in #481
• [+] improve Developer Documentation by @pashagolub in #487
• [+] improve Docker images and related scripts by @pashagolub in #158
• [+] improve Docker workflow by @pashagolub in #122
• [+] include webui into pgwatch3 binary, resolves #74 by @pashagolub in #75
• [+] make pgwatch2-daemon Docker image use alpine base image, closes #23 by @pashagolub in #25
• [+] move helpers to metrics definition by @pashagolub in #374
• [+] produce "latest" and semver tags for all Docker images by @pashagolub in #556
• [+] refactor sinks as a standalone package by @pashagolub in #345
• [+] refactor metric definitions handling by @pashagolub in #342
• [+] replace checkboxes in monitoring grid with switches, closes #160 by @denys-holub in #164
• [+] sync changes from pgwatch2...