Features

• Force reconfiguration now supports updating any in-progress move, regardless of its state. This makes it much easier to use as an escape hatch to recover partitions in various stuck scenarios that would otherwise be difficult to recover. by @bharathv in #26669
• rpk cluster config list: new command to list all cluster configuration properties in the cluster by @andresaristizabal in #26825

Bug Fixes

• Fixed race condition in metadata auto create topics by @michael-redpanda in #26944
• Fixes a bug in which a partition for a tiered storage enabled topic could provide an incorrect response to an OffsetForLeaderEpoch request by @WillemKauf in #26880
• #26838 Fixes a bug in timequeries performed over local storage which could lead to inconsistent or undefined results. by @WillemKauf in #26843
• repair initial leadership appointment machinery by @joe-redpanda in #26852
• rpk: fixes a bug where issuing batched FindCoordinator requests consistently failed (used in rpk topic describe when describing multiple topics). by @r-vasquez in #26850

Improvements

• Per-partition cloud storage scrubber metrics will now appear only when the feature is turned on at cluster level (cloud_storage_enable_scrubbing == true) by @oleiman in #26846
• Redpanda will now validate that an Iceberg REST endpoint is set when cluster configuration is altered to enable the Iceberg REST catalog. by @wdberkeley in #26856
• rpk container now uses Redpanda Console version 3.1.2 by @r-vasquez in #26885
• PR #26692 [v25.1.x] storage: quiet logging when no tiered storage topics by @dotnwat
• PR #26848 [v25.1.x] iceberg/config: rename iceberg_rest_catalog_aws_credentials_source by @wdberkeley
• PR #26923 [v25.1.x] Fixes RandomNodeOpeartions decommission stopped making progress test by @mmaslankaprv
• PR #26935 [v25.1.x] rptest: remove test_create_or_delete_topics_while_busy by @nvartolomei

Full Changelog: v25.1.8...v25.1.9

Bug Fixes

• Fixed an issue with consumer groups with manually assigned consumers. When an OffsetDeleteRequest was sent on such a group, a GROUP_SUBSCRIBED_TO_TOPIC error was returned. by @IoannisRP in #26705
• Increase the default self check timeout from 5s to 10s to leave time to retry DNS lookups if they time out during a self check operation. by @pgellert in #26775
• Resolves a memory leak scenario in node_status_backend by resetting connections which make no progress by @joe-redpanda in #26802
• Return the correct error response if the RPC to the leader for deleting ACLs fails. by @BenPope in #26785
• This fixes a bug in Redpanda's self-check functionality, where the self-check would occasionally fail with 'Uploaded key/payload could not be found in cloud storage item list.' despite the object being successfully uploaded. This issue occurred when testing against an Azure ABS tiered storage endpoint. by @pgellert in #26728
• #26739 Fixes a bug in which a segment produced by adjacent merge compaction did not have its batch cache reset, leading to potentially stale reads in the storage layer. by @WillemKauf in #26741
• #26820 Fixes a bug where data loss could occur during FPM w/ tiered storage disabled by @oleiman in #26821

Improvements

• Make segment download timeouts configurable in cloud cache hydration by @oleiman in #26779
• PR #26744 [v24.3.x] [CORE-12729] debug/bundle: forward kubernetes env vars to rpk by @IoannisRP
• PR #26815 [v24.3.x] [CORE-8805] dt/archival: Decrease manifest upload interval to avoid race by @oleiman
• PR #26769 [v24.3.x] raft/heartbeat_manager: timeout memory leak fix by @joe-redpanda

Full Changelog: v24.3.17...v24.3.18

Bug Fixes

• Resolves a memory leak scenario in node_status_backend by resetting connections which make no progress by @joe-redpanda in #26801
• Return the correct error response if the RPC to the leader for deleting ACLs fails. by @BenPope in #26784
• #26819 Fixes a bug where data loss could occur during FPM w/ tiered storage disabled by @oleiman in #26822
• PR #26817 [v24.2.x] [CORE-8805] dt/archival: Decrease manifest upload interval to avoid race by @oleiman
• PR #26770 [v24.2.x] raft/heartbeat_manager: timeout memory leak fix by @joe-redpanda

Full Changelog: v24.2.26...v24.2.27

Bug Fixes

• Fixed an issue with consumer groups with manually assigned consumers. When an OffsetDeleteRequest was sent on such a group, a GROUP_SUBSCRIBED_TO_TOPIC error was returned. by @IoannisRP in #26704
• Fixes a bug where data loss could occur during FPM w/ tiered storage disabled by @oleiman in #26818
• Fixes a hang in RPC dispatch that may result in failed replication and leadership transfers. by @bharathv in #26805
• Increase the default self check timeout from 5s to 10s to leave time to retry DNS lookups if they time out during a self check operation. by @pgellert in #26776
• Resolves a memory leak scenario in node_status_backend by resetting connections which make no progress by @joe-redpanda in #26800
• Return the correct error response if the RPC to the leader for deleting ACLs fails. by @BenPope in #26783
• This fixes a bug in Redpanda's self-check functionality, where the self-check would occasionally fail with 'Uploaded key/payload could not be found in cloud storage item list.' despite the object being successfully uploaded. This issue occurred when testing against an Azure ABS tiered storage endpoint. by @pgellert in #26713
• #26738 Fixes a bug in which a segment produced by adjacent merge compaction did not have its batch cache reset, leading to potentially stale reads in the storage layer. by @WillemKauf in #26740

Improvements

• Fall back to the previously uploaded cluster manifest's group offset snapshot if uploading the group offsets fails for a consumer offsets topic partition. by @pgellert in #26793
• Make segment download timeouts configurable in cloud cache hydration by @oleiman in #26780
• PR #26641 [v25.1.x] [CORE-8392] http: Add shutdown connection error code by @Lazin
• PR #26699 [backport v25.1.x] iceberg/config: mark iceberg auth options as restored (default) by @wdberkeley
• PR #26734 [v25.1.x] [CORE-12729] debug/bundle: forward kubernetes env vars to rpk by @IoannisRP
• PR #26771 [v25.1.x] raft/heartbeat_manager: timeout memory leak fix by @joe-redpanda

Full Changelog: v25.1.7...v25.1.8

Bug Fixes

• Fixes unbounded memory usage in some transaction use caes by @bharathv in #26681

Improvements

• PR #26649 [v25.1.x] Backport AWS Glue REST catalog support by @wdberkeley
• PR #26666 [v25.1.x] Added defensive checks when materializing batch records by @mmaslankaprv
• PR #26674 [v25.1.x] c/rm_frontend: more nuanced mapping of error when locking writes by @mmaslankaprv
• PR #26701 [backport v25.1.x] datalake: add default partition spec kludge for AWS Glue by @wdberkeley

Full Changelog: v25.1.6...v25.1.7

Bug Fixes

• Fix Avro translation to Iceberg when root Avro schema is a primitive type. by @nvartolomei in #26461
• Fix an issue where audit log could lock down a cluster, if miss-configured. Now it is always possible to disable it. by @IoannisRP in #26652
• Iceberg integration: Encode avro record field names to avoid using disallowed characters (like dots with default partitioning hour(redpanda.timestamp). This makes it possible to read Iceberg data with latest version of DuckDB. by @nvartolomei in #26535
• prevents Redpanda from crashing when reading invalid record data by @mmaslankaprv in #26492

Improvements

• Adds support for the Iceberg table properties write.metadata.path and write.data.path. When an Iceberg catalog defines these properties, Redpanda will use them to determine where to write Iceberg table metadata and data, respectively, instead of using default locations based on the table location. by @wdberkeley in #26440
• Allows direct uploading of debug bundles collected with rpk debug remote-bundle by @JFlath in #26515
• Cut down the amount of time spent in fstat() syscalls during storage layer housekeeping & cut down the amount of time spent in fstat() syscalls in the storage layer EVEN MORE IN GENERAL! by @WillemKauf in #26656
• Fixed large allocation issues when handling OffsetCommits by @mmaslankaprv in #26414
• Fixes an issue in which users could experience oversized allocations during a DescribeGroup request. by @WillemKauf in #26532
• ability to control batch cache settings for __consumer_offsets topic by @mmaslankaprv in #26558

Full Changelog: v25.1.5...v25.1.6

Bug Fixes

• Fix an issue where audit log could lock down a cluster, if miss-configured. Now it is always possible to disable it. by @IoannisRP in #26651
• Fixes unbounded memory usage in some transaction use caes by @bharathv in #26682

Improvements

• ability to control batch cache settings for __consumer_offsets topic by @mmaslankaprv in #26658
• PR #25397 [v24.3.x] [CORE-8946] cloud_storage: Update process_anomalies method by @Lazin
• PR #26177 [v24.3.x] r/consensus: do not block leadership completely in maintenance mode by @mmaslankaprv
• PR #26526 [v24.3.x] ducktape: Respect rpk timeout in rpk by @StephanDollberg
• PR #26548 [v24.3.x] [CORE-12155] Introduce external timeout for cloud_storage client leases by @oleiman
• PR #26554 [v24.3.x] Fix race between bootstrap and shutdown by @bashtanov
• PR #26578 [v24.3.x] csc/client_pool: Add null checks in lease watchdog handler by @oleiman
• PR #26599 [v24.3.x] kc/consumer: fixed resource leak when coordinator changes by @mmaslankaprv
• PR #26638 [v24.3.x] raft: Handle exceptions in backgroun_apply_fiber by @Lazin
• PR #26640 [v24.3.x] [CORE-8392] http: Add shutdown connection error code by @Lazin
• PR #26665 [v24.3.x] Added defensive checks when materializing batch records by @mmaslankaprv

Full Changelog: v24.3.16...v24.3.17

Bug Fixes

• Fixes unbounded memory usage in some transaction use caes by @bharathv in #26683
• prevents Redpanda from crashing when reading invalid record data by @mmaslankaprv in #26493
• PR #26433 [v24.2.x] storage: fix index state truncate overflow by @andrwng

Full Changelog: v24.2.25...v24.2.26

Bug Fixes

• prevents Redpanda from crashing when reading invalid record data by @mmaslankaprv in #26494

Improvements

• Allows direct uploading of debug bundles collected with rpk debug remote-bundle by @JFlath in #26514
• Fixes an issue in which users could experience oversized allocations during a DescribeGroup request. by @WillemKauf in #26531
• PR #26444 [v24.3.x] storage: call reserve() in storage::range() by @WillemKauf

Full Changelog: v24.3.15...v24.3.16

Bug Fixes

• Allow partition balancing to opearte in case when space management was enabled, but local target capacity was unset. by @ztlpn in #26306
• Enable TCP keepalive for cloud storage connections. by @Lazin in #26409
• Fix Redpanda crash if partition_autobalancing_concurrent_moves was set to 0. by @ztlpn in #26306
• #26190 Fixes a bug in which a broker would crash during sliding window compaction when started with log_compaction_use_sliding_window=false and its value was later set to true without restarting. by @WillemKauf in #26196
• partition_autobalancing_mode=off now stops on-demand partition rebalance as well. by @ztlpn in #26306
• PR #26082 [v24.3.x] archival: Fix archival_stm_snapshot installation by @Lazin
• PR #26277 [v24.3.x] r/consensus: stop consumable offset monitor by @mmaslankaprv
• PR #26307 [v24.3.x] Fix archival STM shutdown race by @bashtanov
• PR #26432 [v24.3.x] storage: fix index state truncate overflow by @andrwng

Improvements

• Improved handling rf=1 partitions health reporting by @mmaslankaprv in #26178
• In AlterPartitionReassignmentsResponse per-partition response REASSIGNMENT_IN_PROGRESS error code is used if a reassignment is requested while Partition Balancer is moving partition replicas. by @bashtanov in #26349
• Made it easier to detect and diagnose node operation issues by @mmaslankaprv in #26176
• #26202 Adds the storage_log_adjacent_segments_compacted metric for better observability into adjacent segment compaction. by @WillemKauf in #26204
• #26252 rpk: decommission-status reports reallocation failure details by @daisukebe in #26263
• rpk debug bundle: improve reliability of debug bundle collection in k8s environments. by @r-vasquez in #26163
• PR #26108 [v24.3.x] make ntp_callbacks actually support multiple callbacks by @bashtanov
• PR #26173 [v24.3.x] storage: add _segment_cleanly_compacted to probe (Manual backport) by @WillemKauf
• PR #26176 [v24.3.x] Decommission status improvements by @mmaslankaprv
• PR #26223 [v24.3.x] c/archival: wakeup upload loop after flush by @ztlpn
• PR #26230 [v24.3.x] storage: output full segment in WARN log in offset_to_filepos.cc by @WillemKauf
• PR #26339 [v24.3.x] c/partition_manager: added log entries when partition is being shutdown by @mmaslankaprv

Full Changelog: v24.3.14...v24.3.15