Longhorn v1.9.1 Release Notes

Longhorn 1.9.1 introduces several improvements and bug fixes that are intended to improve system quality, resilience, stability and security.

The Longhorn team appreciates your contributions and expects to receive feedback regarding this release.

Installation

You can install Longhorn using a variety of tools, including Rancher, Kubectl, and Helm. For more information about installation methods and requirements, see Quick Installation in the Longhorn documentation.

Upgrade

Longhorn only allows upgrades from supported versions. For more information about upgrade paths and procedures, see Upgrade in the Longhorn documentation.

Post-Release Known Issues

For information about issues identified after this release, see Release-Known-Issues.

Resolved Issues

Feature

• [BACKPORT][v1.9.1][FEATURE] Standardized way to override container image registry 11068 - @marcosbc @roger-ryao
• [BACKPORT][v1.9.1][FEATURE] Standardized way to specify image pull secrets 11072 - @marcosbc @chriscchien

Improvement

• [BACKPORT][v1.9.1][IMPROVEMENT] Remove the Patch preserveUnknownFields: false for CRDs 11280 - @derekbit @chriscchien
• [BACKPORT][v1.9.1][IMPROVEMENT] Improve the disk space un-schedulable condition message 11212 - @yangchiu @davidcheng0922
• [BACKPORT][v1.9.1][IMPROVEMENT] Improve the condition message of engine image check 11196 - @derekbit @chriscchien
• [BACKPORT][v1.9.1][IMPROVEMENT] Improve the logging when detecting multiple backup volumes of the same volume on the same backup target 11225 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.9.1][IMPROVEMENT] extra invalid BackupVolumeCR may be created during cluster split-brain 11168 - @mantissahz @roger-ryao
• [BACKPORT][v1.9.1][IMPROVEMENT] Full replica rebuilding when a node goes down for a while and then comes back 11069 - @mantissahz
• [BACKPORT][v1.9.1][IMPROVEMENT] Adding retry logic for longhorn-csi-plugin when it trying to contact the longhorn-manager pods 10914 - @PhanLe1010 @roger-ryao

Bug

• [BACKPORT][v1.9.1][BUG] Incorrect value of remove-snapshots-during-filesystem-trim in longhorn chart/values.yaml 11266 - @derekbit @chriscchien
• [BACKPORT][v1.9.1][BUG] privateRegistry.registryUrl does not work when overriding specific image registries 11258 - @marcosbc @chriscchien
• [BACKPORT][v1.9.1][BUG] system backup error 11235 - @c3y1huang @roger-ryao
• [BACKPORT][v1.9.1][BUG] Volume expansion fails with "unsupported disk encryption format ext4" 11184 - @COLDTURNIP @mantissahz @roger-ryao
• [BACKPORT][v1.9.1][BUG] The Backup YAML example in the Longhorn doc does not work 11217 - @mantissahz @nzhan126
• [BACKPORT][v1.9.1][BUG] CSI Plugin restart triggers unintended restart of migratable RWX volume workloads 11164 - @c3y1huang @roger-ryao
• [BACKPORT][v1.9.1][BUG] in the browser UI: Volume -> Clone Volume results in the broken browser page 11180 - @houhoucoop @roger-ryao
• [BACKPORT][v1.9.1][BUG] Test case test_engine_image_not_fully_deployed_perform_volume_operations failed: unable to detach a volume 10917 - @mantissahz @chriscchien
• [BACKPORT][v1.9.1][BUG] Creating support-bundle panic NPE 11170 - @c3y1huang @roger-ryao
• [BACKPORT][v1.9.1][BUG] Unable to Build Longhorn-Share-Manager Image Due to CMAKE Compatibility 11162 - @derekbit @roger-ryao
• [BACKPORT][v1.9.1][BUG] Recurring jobs fail when assigned to default group 11020 - @c3y1huang @chriscchien
• [BACKPORT][v1.9.1][BUG] SPDK API bdev_lvol_detach_parent does not work as expected 11047 - @DamiaSan @roger-ryao
• [BACKPORT][v1.9.1][BUG] unable to clean up the backing image volume replica after node eviction 11056 - @COLDTURNIP @roger-ryao
• [BACKPORT][v1.9.1][BUG] backing image volume replica NPE crash during evicting node 11035 - @COLDTURNIP @chriscchien

Misc

• [HOTFIX] Create hotfixed image for longhorn-manager:v1.9.0 11140 - @derekbit @chriscchien
• [BACKPORT][v1.9.1][TASK] Ensure support-bundle-kit builds use vendored dependencies 11118 - @yangchiu @c3y1huang

New Contributors

• @davidcheng0922
• @marcosbc
• @nzhan126

Contributors

• @COLDTURNIP
• @DamiaSan
• @PhanLe1010
• @c3y1huang
• @chriscchien
• @derekbit
• @houhoucoop
• @innobead
• @mantissahz
• @roger-ryao
• @yangchiu
• @sushant-suse
• @rebeccazzzz
• @forbesguthrie
• @asettle

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Feature

• [BACKPORT][v1.9.1][FEATURE] Standardized way to override container image registry 11068 - @marcosbc @roger-ryao
• [BACKPORT][v1.9.1][FEATURE] Standardized way to specify image pull secrets 11072 - @marcosbc @chriscchien

Improvement

• [BACKPORT][v1.9.1][IMPROVEMENT] Remove the Patch preserveUnknownFields: false for CRDs 11280 - @derekbit @chriscchien
• [BACKPORT][v1.9.1][IMPROVEMENT] Improve the disk space un-schedulable condition message 11212 - @yangchiu @davidcheng0922
• [BACKPORT][v1.9.1][IMPROVEMENT] Improve the condition message of engine image check 11196 - @derekbit @chriscchien
• [BACKPORT][v1.9.1][IMPROVEMENT] Improve the logging when detecting multiple backup volumes of the same volume on the same backup target 11225 - @PhanLe1010 @chriscchien
• [BACKPORT][v1.9.1][IMPROVEMENT] extra invalid BackupVolumeCR may be created during cluster split-brain 11168 - @mantissahz @roger-ryao
• [BACKPORT][v1.9.1][IMPROVEMENT] Full replica rebuilding when a node goes down for a while and then comes back 11069 - @mantissahz
• [BACKPORT][v1.9.1][IMPROVEMENT] Adding retry logic for longhorn-csi-plugin when it trying to contact the longhorn-manager pods 10914 - @PhanLe1010 @roger-ryao

Bug

• [BACKPORT][v1.9.1][BUG] Incorrect value of remove-snapshots-during-filesystem-trim in longhorn chart/values.yaml 11266 - @derekbit @chriscchien
• [BACKPORT][v1.9.1][BUG] privateRegistry.registryUrl does not work when overriding specific image registries 11258 - @marcosbc @chriscchien
• [BACKPORT][v1.9.1][BUG] system backup error 11235 - @c3y1huang @roger-ryao
• [BACKPORT][v1.9.1][BUG] Volume expansion fails with "unsupported disk encryption format ext4" 11184 - @COLDTURNIP @mantissahz @roger-ryao
• [BACKPORT][v1.9.1][BUG] The Backup YAML example in the Longhorn doc does not work 11217 - @mantissahz @nzhan126
• [BACKPORT][v1.9.1][BUG] CSI Plugin restart triggers unintended restart of migratable RWX volume workloads 11164 - @c3y1huang @roger-ryao
• [BACKPORT][v1.9.1][BUG] in the browser UI: Volume -> Clone Volume results in the broken browser page 11180 - @houhoucoop @roger-ryao
• [BACKPORT][v1.9.1][BUG] Test case test_engine_image_not_fully_deployed_perform_volume_operations failed: unable to detach a volume 10917 - @mantissahz @chriscchien
• [BACKPORT][v1.9.1][BUG] Creating support-bundle panic NPE 11170 - @c3y1huang @roger-ryao
• [BACKPORT][v1.9.1][BUG] Unable to Build Longhorn-Share-Manager Image Due to CMAKE Compatibility 11162 - @derekbit @roger-ryao
• [BACKPORT][v1.9.1][BUG] Recurring jobs fail when assigned to default group 11020 - @c3y1huang @chriscchien
• [BACKPORT][v1.9.1][BUG] SPDK API bdev_lvol_detach_parent does not work as expected 11047 - @DamiaSan @roger-ryao
• [BACKPORT][v1.9.1][BUG] unable to clean up the backing image volume replica after node eviction 11056 - @COLDTURNIP @roger-ryao
• [BACKPORT][v1.9.1][BUG] backing image volume replica NPE crash during evicting node 11035 - @COLDTURNIP @chriscchien

Misc

• [HOTFIX] Create hotfixed image for longhorn-manager:v1.9.0 11140 - @derekbit @chriscchien
• [BACKPORT][v1.9.1][TASK] Ensure support-bundle-kit builds use vendored dependencies 11118 - @yangchiu @c3y1huang

Contributors

• @COLDTURNIP
• @DamiaSan
• @PhanLe1010
• @c3y1huang
• @chriscchien
• @davidcheng0922
• @derekbit
• @houhoucoop
• @innobead
• @mantissahz
• @marcosbc
• @nzhan126
• @roger-ryao
• @yangchiu

Longhorn v1.8.2 Release Notes

Longhorn 1.8.2 introduces several improvements and bug fixes that are intended to improve system quality, resilience, stability and security.

The Longhorn team appreciates your contributions and expects to receive feedback regarding this release.

Installation

You can install Longhorn using a variety of tools, including Rancher, Kubectl, and Helm. For more information about installation methods and requirements, see Quick Installation in the Longhorn documentation.

Upgrade

Longhorn only allows upgrades from supported versions. For more information about upgrade paths and procedures, see Upgrade in the Longhorn documentation.

Post-Release Known Issues

For information about issues identified after this release, see Release-Known-Issues.

Resolved Issues

Improvement

• [BACKPORT][v1.8.2][IMPROVEMENT] Adding retry logic for longhorn-csi-plugin when it is trying to contact the longhorn-manager pods 11027 - @PhanLe1010 @roger-ryao
• [BACKPORT][v1.8.2][IMPROVEMENT] add strict field validation to the update option in upgrade path 10648 - @ChanYiLin
• [BACKPORT][v1.8.2][IMPROVEMENT] Move SettingNameV2DataEngineHugepageLimit to danger zone settings 10568 - @derekbit @chriscchien
• [BACKPORT][v1.8.2][IMPROVEMENT] Reduce auto balancing logging noise for detached volumes 10692 - @roger-ryao
• [BACKPORT][v1.8.2][IMPROVEMENT] Improve the Warning Message When Failed to Remove Block-Type Disks 10576 - @yangchiu @ChanYiLin

Bug

• [BACKPORT][v1.8.2][BUG] unable to clean up the backing image volume replica after node eviction 11057 - @COLDTURNIP @roger-ryao
• [BACKPORT][v1.8.2][BUG] backing image volume replica NPE crash during evicting node 11036 - @COLDTURNIP @chriscchien
• [BUG] V2 Backing image failed after upgrade from v1.8.1 to v1.8.2-rc1 10969 - @COLDTURNIP @roger-ryao
• [BACKPORT][v1.8.2][BUG] Error on git checkout in a container 10975 - @derekbit @chriscchien
• [BACKPORT][v1.8.2][BUG] Helm persistence.backupTargetName not referenced in storageclass template 10964 - @yangchiu @mantissahz
• [BACKPORT][v1.8.2][BUG] MultiUnmapper floods logs with warnings about size mismatch. 10565 - @shuo-wu @roger-ryao
• [BACKPORT][v1.8.2][BUG] Test case test_snapshot_prune_and_coalesce_simultaneously_with_backing_image fails 10822 - @yangchiu @c3y1huang
• [BACKPORT][v1.8.2][BUG] System backup could get stuck in CreatingBackingImageBackups indefinitely 10748 - @yangchiu @ChanYiLin
• [BACKPORT][v1.8.2][BUG] Failed to terminate namespace longhorn-system if there is a support bundle ReadyForDownload 10732 - @yangchiu @c3y1huang
• [BACKPORT][v1.8.2][BUG] [v1.9.0-rc1] DR volume does not sync with latest backup when activation 10842 - @c3y1huang @chriscchien
• [BACKPORT][v1.8.2][BUG] Can NOT delete an oversized Not Ready volume 10742 - @WebberHuang1118 @chriscchien
• [BACKPORT][v1.8.2][BUG][UI] Bulk backup creation with a detached volume returns error 405 and error messages show in browser console 10725 - @yangchiu @a110605
• [BACKPORT][v1.8.2][BUG] Naming collision when creating the name of the new backing image manager 10618 - @yangchiu @ChanYiLin
• [BACKPORT][v1.8.2][BUG] I/O errors on Longhorn v1.7.2 volume during VM migration while upgrading Harvester v1.4.1 10549 - @derekbit @roger-ryao
• [BACKPORT][v1.8.2][BUG] Adding a non-existing disk to a node will cause the longhorn-manager to crash 10750 - @ChanYiLin @roger-ryao
• [BACKPORT][v1.8.2][BUG] After node down and force delete the terminating deployment pod, volume can not attach success 10713 - @c3y1huang @chriscchien
• [BACKPORT][v1.8.2][BUG] Instance manager image build fail 10654 - @shuo-wu
• [BACKPORT][v1.8.2][BUG] Longhorn Volume Encryption Not Working in Talos 1.9.x 10605 - @c3y1huang @roger-ryao
• [BACKPORT][v1.8.2][BUG] integer divide by zero in replica scheduler 10504 - @c3y1huang

Misc

• [TASK] Fix CVE issues for v1.8.2 10913 - @c3y1huang

Contributors

• @COLDTURNIP
• @ChanYiLin
• @WebberHuang1118
• @a110605
• @c3y1huang
• @chriscchien
• @derekbit
• @innobead
• @mantissahz
• @roger-ryao
• @shuo-wu
• @yangchiu
• @sushant-suse
• @jillian-maroket
• @rebeccazzzz
• @forbesguthrie
• @asettle

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.8.2][IMPROVEMENT] Adding retry logic for longhorn-csi-plugin when it trying to contact the longhorn-manager pods 11027 - @PhanLe1010 @roger-ryao
• [BACKPORT][v1.8.2][IMPROVEMENT] add strict field validation to the update option in upgrade path 10648 - @ChanYiLin
• [BACKPORT][v1.8.2][IMPROVEMENT] Move SettingNameV2DataEngineHugepageLimit to danger zone settings 10568 - @derekbit @chriscchien
• [BACKPORT][v1.8.2][IMPROVEMENT] Reduce auto balancing logging noise for detached volumes 10692 - @roger-ryao
• [BACKPORT][v1.8.2][IMPROVEMENT] Improve the Warning Message When Failed to Remove Block-Type Disks 10576 - @yangchiu @ChanYiLin

Bug

• [BACKPORT][v1.8.2][BUG] unable to clean up the backing image volume replica after node eviction 11057 - @COLDTURNIP @roger-ryao
• [BACKPORT][v1.8.2][BUG] backing image volume replica NPE crash during evicting node 11036 - @COLDTURNIP @chriscchien
• [BUG] V2 Backing image failed after upgrade from v1.8.1 to v1.8.2-rc1 10969 - @COLDTURNIP @roger-ryao
• [BACKPORT][v1.8.2][BUG] Error on git checkout in a container 10975 - @derekbit @chriscchien
• [BACKPORT][v1.8.2][BUG] Helm persistence.backupTargetName not referenced in storageclass template 10964 - @yangchiu @mantissahz
• [BACKPORT][v1.8.2][BUG] MultiUnmapper floods logs with warnings about size mismatch. 10565 - @shuo-wu @roger-ryao
• [BACKPORT][v1.8.2][BUG] Test case test_snapshot_prune_and_coalesce_simultaneously_with_backing_image fails 10822 - @yangchiu @c3y1huang
• [BACKPORT][v1.8.2][BUG] System backup could get stuck in CreatingBackingImageBackups indefinitely 10748 - @yangchiu @ChanYiLin
• [BACKPORT][v1.8.2][BUG] Failed to terminate namesapce longhorn-system if there is a support bundle ReadyForDownload 10732 - @yangchiu @c3y1huang
• [BACKPORT][v1.8.2][BUG] [v1.9.0-rc1] DR volume does not sync with latest backup when activation 10842 - @c3y1huang @chriscchien
• [BACKPORT][v1.8.2][BUG] Can NOT delete an oversized Not Ready volume 10742 - @WebberHuang1118 @chriscchien
• [BACKPORT][v1.8.2][BUG][UI] Bulk backup creation with a detached volume returns error 405 and error messages show in browser console 10725 - @yangchiu @a110605
• [BACKPORT][v1.8.2][BUG] Naming collision when creating the name of the new backing image manager 10618 - @yangchiu @ChanYiLin
• [BACKPORT][v1.8.2][BUG] I/O errors on Longhorn v1.7.2 volume during VM migration while upgrading Harvester v1.4.1 10549 - @derekbit @roger-ryao
• [BACKPORT][v1.8.2][BUG] Adding a non-existing disk to a node will cause the longhorn-manager to crash 10750 - @ChanYiLin @roger-ryao
• [BACKPORT][v1.8.2][BUG] After node down and force delete the terminating deployment pod, volume can not attach success 10713 - @c3y1huang @chriscchien
• [BACKPORT][v1.8.2][BUG] Instance manager image build fail 10654 - @shuo-wu
• [BACKPORT][v1.8.2][BUG] Longhorn Volume Encryption Not Working in Talos 1.9.x 10605 - @c3y1huang @roger-ryao
• [BACKPORT][v1.8.2][BUG] integer divide by zero in replica scheduler 10504 - @c3y1huang

Contributors

• @COLDTURNIP
• @ChanYiLin
• @PhanLe1010
• @WebberHuang1118
• @a110605
• @c3y1huang
• @chriscchien
• @derekbit
• @innobead
• @mantissahz
• @roger-ryao
• @shuo-wu
• @yangchiu

Longhorn v1.9.0 Release Notes

Longhorn v1.9.0 introduces new features, enhancements, and bug fixes aimed at improving system stability and user experience. Key highlights include V2 Data Engine improvements, orphaned instance deletion, offline replica rebuilding, recurring system backups, and enhanced observability of Longhorn resources.

The Longhorn team appreciates your contributions and anticipates receiving feedback regarding this release.

For terminology and background on Longhorn releases, see Releases.

Warning

The longhorn-manager v1.9.0 is impacted by a regression issue, which causes failures of recurring jobs. To resolve this issue, replace longhorn-manager:v1.9.0 with the hotfixed image longhorn-manager:v1.9.0-hotfix-1.

You can apply the update by following these steps:

1. Disable the upgrade version check

   • Helm users: Set upgradeVersionCheck to false in the values.yaml file.
   • Manifest users: Remove the --upgrade-version-check flag from the deployment manifest.

2. Update the longhorn-manager image

   • Change the image tag from v1.9.0 to v1.9.0-hotfix-1 in the appropriate file:
     • For Helm: Update values.yaml
     • For manifests: Update the deployment manifest directly.

3. Proceed with the upgrade

   • Apply the changes using your standard Helm upgrade command or reapply the updated manifest.

Removal

Environment Check Script

The environment_check.sh script, deprecated in v1.7.0, has been removed in v1.9.0. Use the Longhorn Command Line Tool (longhornctl) to check your environment for potential issues.

Orphan Resource Auto-Deletion

The orphan-auto-deletion setting has been replaced by orphan-resource-auto-deletion in v1.9.0. To replicate the previous behavior, include replica-data in the orphan-resource-auto-deletion value. During the upgrade, the original orphan-auto-deletion setting is automatically migrated.

For more information, see Orphan Data Cleanup.

Deprecated Fields in longhorn.io/v1beta2 CRDs

Deprecated fields have been removed from the CRDs. For details, see issue #6684.

Deprecation & Incompatibilities

longhorn.io/v1beta1 API

The v1beta1 version of the Longhorn API is marked unserved and unsupported in v1.9.0 and will be removed in v1.10.0.

For more details, see Issue #10250.

Breaking Change in V2 Backing Image

Starting with Longhorn v1.9.0, V2 backing images are incompatible with earlier versions due to naming conflicts in the extended attributes (xattrs) used by SPDK backing image logical volumes. As a result, V2 backing images must be deleted and recreated during the upgrade process. Since backing images cannot be deleted while volumes using them still exist, you must first back up, delete, and later restore those volumes as the following steps:

• Before upgrading to v1.9.0:
  • Verify that backup targets are functioning properly.
  • Create full backups of all volumes that use a V2 backing image.
  • Detach and delete these volumes after the backups complete.
  • In the Backing Image page, save the specifications of all V2 backing images, including the name and the image source.
  • Delete all V2 backing images.
• After upgrading:
  • Recreate the V2 backing images using the same names and image sources.
  • Restore the volumes from your backups.

For more details, see Issue #10805.

Primary Highlights

New V2 Data Engine Features

While the V2 Data Engine remains experimental in this release, several core functions have been significantly improved:

• Support UBLK Frontend: Support for UBLK frontend in the V2 Data Engine, which allows for better performance and resource utilization.
• Storage Network: Introduces support for storage networks in the V2 Data Engine to allow network segregation.
• Offline Replica Rebuilding: Support for offline replica rebuilding, which allows degraded volumes to automatically recover replicas even while the volume is detached. This capability ensures high data availability without manual intervention.

Recurring System Backup

Starting with Longhorn v1.9.0, you can create a recurring job for system backup creation.

Documentation | GitHub Issue

Offline Replica Rebuilding

Longhorn introduces offline replica rebuilding, a feature that allows degraded volumes to automatically recover replicas even while the volume is detached. This capability minimizes the need for manual recovery steps, accelerates restoration, and ensures high data availability. By default, offline replica rebuilding is disabled. To enable it, set the offline-replica-rebuilding setting to true in the Longhorn UI or CLI.

Documentation | GitHub Issue

Orphaned Instance Deletion

Longhorn can now track and remove orphaned instances, which are leftover resources like replicas or engines that are no longer associated with an active volume. These instances may accumulate due to unexpected failures or incomplete cleanup.

To reduce resource usage and maintain system performance, Longhorn supports both automatic and manual cleanup. By default, this feature is disabled. To enable it, set the orphan-resource-auto-deletion setting to instance in the Longhorn UI or CLI.

Documentation | GitHub Issue

Improved Metrics for Replica, Engine, and Rebuild Status

Longhorn improves observability with new Prometheus metrics that expose the status and identity of Replica and Engine CRs, along with rebuild activity. These metrics make it easier to monitor rebuilds across the cluster.

For more information, see #10550 and #10722.

Installation

You can install Longhorn using a variety of tools, including Rancher, Kubectl, and Helm. For more information about installation methods and requirements, see Quick Installation in the Longhorn documentation.

Upgrade

Longhorn only allows upgrades from supported versions. For more information about upgrade paths and procedures, see Upgrade in the Longhorn documentation.

Post-Release Known Issues

For information about issues identified after this release, see Release-Known-Issues.

Highlight

• [FEATURE] Cleanup orphaned volume runtime resources if the v1 resources already deleted 6764 - @COLDTURNIP @chriscchien
• [FEATURE] v2 volume supports UBLK frontend 9456 - @PhanLe1010 @chriscchien
• [FEATURE] V1 and V2 volume offline replica rebuilding 8443 - @mantissahz @roger-ryao
• [TASK] Migrate v1beta1 CR to v1beta2 10250 - @COLDTURNIP @roger-ryao
• [FEATURE] Storage network with V2 data engine 6450 - @c3y1huang @roger-ryao
• [FEATURE] Recurring system backup 6534 - @yangchiu @c3y1huang

Feature

• [FEATURE] Delta Replica Rebuilding using Delta Snapshot: SPDK API Development 10799 - @yangchiu @DamiaSan
• [FEATURE] Running replicas field in volume table 10817 - @xelab04 @roger-ryao
• [FEATURE] Longhorn UI supports orphaned instance CRs management 10760 - @yangchiu @houhoucoop
• [FEATURE] Allow auto deleting snapshot when a backup is created from that snapshot. 9213 - @yangchiu @mantissahz
• [FEATURE] Add missing metrics of number of volumes/replicas by node/cluster 7599 - @c3y1huang @roger-ryao

Improvement

• [IMPROVEMENT] Remove unnecessary lasso dependency 10856 - @derekbit @chriscchien
• [IMPROVEMENT] Configurable wait interval for orphaned instance CR creation 10904 - @derekbit @chriscchien
• [IMPROVEMENT] Prevent creating orphans while deleting the v1 instances 10888 - @COLDTURNIP @chriscchien
• [IMPROVEMENT] prevent false logs from deleting volume with offline rebuilding is disabled. 10889 - @...

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Highlight

• [FEATURE] Cleanup orphaned volume runtime resources if the v1 resources already deleted 6764 - @COLDTURNIP @chriscchien
• [FEATURE] v2 volume supports UBLK frontend 9456 - @PhanLe1010 @chriscchien
• [FEATURE] V1 and V2 volume offline replica rebuilding 8443 - @mantissahz @roger-ryao
• [UI][FEATURE] V1 and V2 volume offline replica rebuilding 10581 - @houhoucoop @roger-ryao
• [TASK] Migrate v1beta1 CR to v1beta2 10250 - @COLDTURNIP @roger-ryao
• [FEATURE] Storage network with V2 data engine 6450 - @c3y1huang @roger-ryao
• [FEATURE] Recurring system backup 6534 - @yangchiu @c3y1huang
• [UI][FEATURE] Recurring system backup 10262 - @yangchiu @houhoucoop

Feature

• [FEATURE] Delta Replica Rebuilding using Delta Snapshot: SPDK API Development 10799 - @yangchiu @DamiaSan
• [FEATURE] Running replicas field in volume table 10817 - @xelab04 @roger-ryao
• [FEATURE] Longhorn UI supports orphaned instance CRs management 10760 - @yangchiu @houhoucoop
• [FEATURE] Allow auto deleting snapshot when a backup is created from that snapshot. 9213 - @yangchiu @mantissahz
• [FEATURE] Add missing metrics of number of volumes/replicas by node/cluster 7599 - @c3y1huang @roger-ryao

Improvement

• [IMPROVEMENT] Remove unnecessary lasso dependency 10856 - @derekbit @chriscchien
• [IMPROVEMENT] Configurable wait interval for orphaned instance CR creation 10904 - @derekbit @chriscchien
• [IMPROVEMENT] Prevent creating orphans while deleting the v1 instances 10888 - @COLDTURNIP @chriscchien
• [IMPROVEMENT] prevent false logs from deleting volume with offline rebuilding is disabled. 10889 - @mantissahz @chriscchien
• [IMPROVEMENT] Add Prometheus metrics for Replica and Engine CRs 10722 - @hookak @chriscchien
• [IMPROVEMENT] Export longhorn engine rebuild status as prometheus metrics 10550 - @hookak @chriscchien
• [IMPROVEMENT] Disable Snapshot Checksum Calculation for Single-Replica V1 Volume 10518 - @derekbit @chriscchien
• [IMPROVEMENT] add extraObject in charts 10835 - @DrummyFloyd @chriscchien
• [IMPROVEMENT] Disable the v2 snapshot hashing while it is being deleted 10563 - @shuo-wu @roger-ryao
• [IMPROVEMENT] v2 checksum calculation and update should follow the v1 flow 10480 - @shuo-wu @roger-ryao
• [IMPROVEMENT] add strict field validation to the update option in upgrade path 10644 - @ChanYiLin @chriscchien
• [IMPROVEMENT] Show snapshot size during in-progress backup 9783 - @yangchiu @houhoucoop
• [IMPROVEMENT] Don't synchronize all filesystem before snapshotting a v2 volume 9023 - @yangchiu @DamiaSan
• [IMPROVEMENT] spdk_tgt can cancel lvol checksum calculation while there is high priority task 10421 - @yangchiu @DamiaSan
• [IMPROVEMENT] Lvol is not force-removed if Blob is busy 10474 - @yangchiu @DamiaSan
• [IMPROVEMENT] Remove deprecated fields from CRDs 6684 - @derekbit @roger-ryao
• [IMPROVEMENT] Longhorn CLI fails to recognize Raspbian OS 10676 - @bachmanity1 @roger-ryao
• [IMPROVEMENT] Reduce auto balancing logging noise for detached volumes 10691 - @dihmandrake @roger-ryao
• [IMPROVEMENT] Remove the upper bound of v2-data-engine-guaranteed-instance-manager-cpu 10662 - @derekbit @roger-ryao
• [IMPROVEMENT] Clean up BackupTarget condition message handling 8224 - @chriscchien @houhoucoop
• [IMPROVEMENT] Longhorn CLI supports SLES micro 9256 - @yangchiu @DamiaSan
• [IMPROVEMENT] Allow volumeBindingMode to be set from helm values 10592 - @ruant @roger-ryao
• [DOC] Prepare a knowledge base for backing image trouble shooting during upgrade 10590 - @ChanYiLin @chriscchien
• [IMPROVEMENT] Missing Prometheus Metrics for Engine v2 Volumes 10472 - @hookak @roger-ryao
• [IMPROVEMENT] Create Volume UI improvement, Automatically Filter Backing Image Based on v1 or v2 Selection 10086 - @houhoucoop @roger-ryao
• [UI][IMPROVEMENT] Improve the Warning Message When Failed to Remove Block-Type Disks 10580 - @houhoucoop @roger-ryao
• [IMPROVEMENT] Pass full backup mode option to CSI volume snapshot type backup 9785 - @ChanYiLin @roger-ryao
• [UI][IMPROVEMENT] Clean up BackupTarget condition message handling 10579 - @houhoucoop
• [IMPROVEMENT] Improve the Warning Message When Failed to Remove Block-Type Disks 10522 - @yangchiu @ChanYiLin
• [IMPROVEMENT] Move SettingNameV2DataEngineHugepageLimit to danger zone settings 7746 - @derekbit @chriscchien
• [IMPROVEMENT] Include the /proc/mounts file and multipath.config in the support-bundle 6754 - @c3y1huang @roger-ryao
• [IMPROVEMENT] Use code-generator/kube_codegen.sh to generate K8s stubs and CRDs 7944 - @derekbit @chriscchien
• [IMPROVEMENT] CRD & API code generator decouple from Go conventional source path 10556 - @COLDTURNIP
• [IMPROVEMENT] Support configurable upgrade-responder URL 10437 - @derekbit @roger-ryao
• [IMPROVEMENT] Settings change validation should go back to using Volume state to determine "are all volumes detached" 10233 - @yangchiu @james-munson
• [IMPROVEMENT] Improve the UX of updating danger zone settings 8070 - @yangchiu @mantissahz

Bug

• [BUG] Error on git checkout in a container 10621 - @derekbit @chriscchien
• [BUG] Test case test_snapshot_prune_and_coalesce_simultaneously_with_backing_image fails 10808 - @yangchiu @c3y1huang
• [BUG] Failed to terminate namespace longhorn-system if there is a support bundle ReadyForDownload 10731 - @yangchiu @c3y1huang
• [BUG] Helm persistence.backupTargetName not referenced in storageclass template 10961 - @yangchiu @mantissahz
• [BUG] SPDK API lvol_get_snapshot_range_checksums cannot get the correct result 10950 - @shuo-wu @roger-ryao
• [BUG] V2 Backing image not ready after upgrade from v1.8.1 to v1.9.x 10805 - @COLDTURNIP @chriscchien
• [BUG] v2 replica rebuilding will miss the backing image 10909 - @shuo-wu @chriscchien
• [BUG] minor spacing issues found with yamllint in the helm chart 10681 - @codekow
• [BUG][UI] Snapshots of v2 volume with backing image aren't shown on the Snapshots and Backups graph 10526 - @derekbit @chriscchien
• [BUG] Deleted orphan data still renders on the page until page refresh 10803 - @COLDTURNIP @chriscchien @houhoucoop
• [BUG] DR volume does not sync with latest backup when activation 10824 - @c3y1huang @chriscchien
• [BUG][v1.9.0-rc1] Block disks become temporarily unavailable after the upgrading from v1.8.1 to v1.9.0-rc1 10821 - @mantissahz
• [BUG] Naming collision when creating the name of the new backing image manager 10616 - @yangchiu @ChanYiLin
• [BUG] v2 volume replica status error after snapshot deletion with `Immediate Data Integrit...

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.8.2][IMPROVEMENT] add strict field validation to the update option in upgrade path 10648 - @ChanYiLin
• [BACKPORT][v1.8.2][IMPROVEMENT] Move SettingNameV2DataEngineHugepageLimit to danger zone settings 10568 - @derekbit @chriscchien
• [BACKPORT][v1.8.2][IMPROVEMENT] Reduce auto balancing logging noise for detached volumes 10692 - @roger-ryao
• [BACKPORT][v1.8.2][IMPROVEMENT] Improve the Warning Message When Failed to Remove Block-Type Disks 10576 - @yangchiu @ChanYiLin

Bug

• [BUG] V2 Backing image failed after upgrade from v1.8.1 to v1.8.2-rc1 10969 - @COLDTURNIP
• [BACKPORT][v1.8.2][BUG] Error on git checkout in a container 10975 - @derekbit @chriscchien
• [BACKPORT][v1.8.2][BUG] Helm persistence.backupTargetName not referenced in storageclass template 10964 - @yangchiu @mantissahz
• [BACKPORT][v1.8.2][BUG] MultiUnmapper floods logs with warnings about size mismatch. 10565 - @shuo-wu @roger-ryao
• [BACKPORT][v1.8.2][BUG] Test case test_snapshot_prune_and_coalesce_simultaneously_with_backing_image fails 10822 - @yangchiu @c3y1huang
• [BACKPORT][v1.8.2][BUG] System backup could get stuck in CreatingBackingImageBackups indefinitely 10748 - @yangchiu @ChanYiLin
• [BACKPORT][v1.8.2][BUG] Failed to terminate namesapce longhorn-system if there is a support bundle ReadyForDownload 10732 - @yangchiu @c3y1huang
• [BACKPORT][v1.8.2][BUG] [v1.9.0-rc1] DR volume does not sync with latest backup when activation 10842 - @c3y1huang @chriscchien
• [BACKPORT][v1.8.2][BUG] Can NOT delete an oversized Not Ready volume 10742 - @WebberHuang1118 @chriscchien
• [BACKPORT][v1.8.2][BUG][UI] Bulk backup creation with a detached volume returns error 405 and error messages show in browser console 10725 - @yangchiu @a110605
• [BACKPORT][v1.8.2][BUG] Naming collision when creating the name of the new backing image manager 10618 - @yangchiu @ChanYiLin
• [BACKPORT][v1.8.2][BUG] I/O errors on Longhorn v1.7.2 volume during VM migration while upgrading Harvester v1.4.1 10549 - @derekbit @roger-ryao
• [BACKPORT][v1.8.2][BUG] Adding a non-existing disk to a node will cause the longhorn-manager to crash 10750 - @ChanYiLin @roger-ryao
• [BACKPORT][v1.8.2][BUG] After node down and force delete the terminating deployment pod, volume can not attach success 10713 - @c3y1huang @chriscchien
• [BACKPORT][v1.8.2][BUG] Instance manager image build fail 10654 - @shuo-wu
• [BACKPORT][v1.8.2][BUG] Longhorn Volume Encryption Not Working in Talos 1.9.x 10605 - @c3y1huang @roger-ryao
• [BACKPORT][v1.8.2][BUG] integer divide by zero in replica scheduler 10504 - @c3y1huang

Contributors

• @COLDTURNIP
• @ChanYiLin
• @WebberHuang1118
• @a110605
• @c3y1huang
• @chriscchien
• @derekbit
• @innobead
• @mantissahz
• @roger-ryao
• @shuo-wu
• @yangchiu

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Improvement

• [BACKPORT][v1.8.2][IMPROVEMENT] add strict field validation to the update option in upgrade path 10648 - @ChanYiLin
• [BACKPORT][v1.8.2][IMPROVEMENT] Move SettingNameV2DataEngineHugepageLimit to danger zone settings 10568 - @derekbit @chriscchien
• [BACKPORT][v1.8.2][IMPROVEMENT] Reduce auto balancing logging noise for detached volumes 10692 - @roger-ryao
• [BACKPORT][v1.8.2][IMPROVEMENT] Improve the Warning Message When Failed to Remove Block-Type Disks 10576 - @yangchiu @ChanYiLin

Bug

• [BACKPORT][v1.8.2][BUG] MultiUnmapper floods logs with warnings about size mismatch. 10565 - @shuo-wu @roger-ryao
• [BACKPORT][v1.8.2][BUG] Test case test_snapshot_prune_and_coalesce_simultaneously_with_backing_image fails 10822 - @yangchiu @c3y1huang
• [BACKPORT][v1.8.2][BUG] System backup could get stuck in CreatingBackingImageBackups indefinitely 10748 - @yangchiu @ChanYiLin
• [BACKPORT][v1.8.2][BUG] Failed to terminate namesapce longhorn-system if there is a support bundle ReadyForDownload 10732 - @yangchiu @c3y1huang
• [BACKPORT][v1.8.2][BUG] [v1.9.0-rc1] DR volume does not sync with latest backup when activation 10842 - @c3y1huang @chriscchien
• [BACKPORT][v1.8.2][BUG] Can NOT delete an oversized Not Ready volume 10742 - @WebberHuang1118 @chriscchien
• [BACKPORT][v1.8.2][BUG][UI] Bulk backup creation with a detached volume returns error 405 and error messages show in browser console 10725 - @yangchiu @a110605
• [BACKPORT][v1.8.2][BUG] Naming collision when creating the name of the new backing image manager 10618 - @yangchiu @ChanYiLin
• [BACKPORT][v1.8.2][BUG] I/O errors on Longhorn v1.7.2 volume during VM migration while upgrading Harvester v1.4.1 10549 - @derekbit @roger-ryao
• [BACKPORT][v1.8.2][BUG] Adding a non-existing disk to a node will cause the longhorn-manager to crash 10750 - @ChanYiLin @roger-ryao
• [BACKPORT][v1.8.2][BUG] After node down and force delete the terminating deployment pod, volume can not attach success 10713 - @c3y1huang @chriscchien
• [BACKPORT][v1.8.2][BUG] Instance manager image build fail 10654 - @shuo-wu
• [BACKPORT][v1.8.2][BUG] Longhorn Volume Encryption Not Working in Talos 1.9.x 10605 - @c3y1huang @roger-ryao
• [BACKPORT][v1.8.2][BUG] integer divide by zero in replica scheduler 10504 - @c3y1huang

Contributors

• @ChanYiLin
• @WebberHuang1118
• @a110605
• @c3y1huang
• @chriscchien
• @derekbit
• @innobead
• @roger-ryao
• @shuo-wu
• @yangchiu

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Highlight

• [FEATURE] Cleanup orphaned volume runtime resources if the v1 resources already deleted 6764 - @COLDTURNIP @chriscchien
• [FEATURE] v2 volume supports UBLK frontend 9456 - @PhanLe1010 @chriscchien
• [FEATURE] V1 and V2 volume offline replica rebuilding 8443 - @mantissahz @roger-ryao
• [UI][FEATURE] V1 and V2 volume offline replica rebuilding 10581 - @houhoucoop @roger-ryao
• [TASK] Migrate v1beta1 CR to v1beta2 10250 - @COLDTURNIP @roger-ryao
• [FEATURE] Storage network with V2 data engine 6450 - @c3y1huang @roger-ryao
• [FEATURE] Recurring system backup 6534 - @yangchiu @c3y1huang
• [UI][FEATURE] Recurring system backup 10262 - @yangchiu @houhoucoop

Feature

• [FEATURE] Delta Replica Rebuilding using Delta Snapshot: SPDK API Development 10799 - @yangchiu @DamiaSan
• [FEATURE] Running replicas field in volume table 10817 - @xelab04 @roger-ryao
• [FEATURE] Longhorn UI supports orphaned instance CRs management 10760 - @yangchiu @houhoucoop
• [FEATURE] Allow auto deleting snapshot when a backup is created from that snapshot. 9213 - @yangchiu @mantissahz
• [FEATURE] Add missing metrics of number of volumes/replicas by node/cluster 7599 - @c3y1huang @roger-ryao

Improvement

• [IMPROVEMENT] Prevent creating orphans while deleting the v1 instances 10888 - @COLDTURNIP @chriscchien
• [IMPROVEMENT] Configurable wait interval for orphaned instance CR creation 10904 - @derekbit @chriscchien
• [IMPROVEMENT] prevent false logs from deleting volume with offline rebuilding is disabled. 10889 - @mantissahz @chriscchien
• [IMPROVEMENT] Remove unnecessary lasso dependency 10856 - @derekbit @chriscchien
• [IMPROVEMENT] Add Prometheus metrics for Replica and Engine CRs 10722 - @hookak @chriscchien
• [IMPROVEMENT] Export longhorn engine rebuild status as prometheus metrics 10550 - @hookak @chriscchien
• [IMPROVEMENT] Disable Snapshot Checksum Calculation for Single-Replica V1 Volume 10518 - @derekbit @chriscchien
• [IMPROVEMENT] add extraObject in charts 10835 - @DrummyFloyd @chriscchien
• [IMPROVEMENT] Disable the v2 snapshot hashing while it is being deleted 10563 - @shuo-wu @roger-ryao
• [IMPROVEMENT] v2 checksum calculation and update should follow the v1 flow 10480 - @shuo-wu @roger-ryao
• [IMPROVEMENT] add strict field validation to the update option in upgrade path 10644 - @ChanYiLin @chriscchien
• [IMPROVEMENT] Show snapshot size during in-progress backup 9783 - @yangchiu @houhoucoop
• [IMPROVEMENT] Don't synchronize all filesystem before snapshotting a v2 volume 9023 - @yangchiu @DamiaSan
• [IMPROVEMENT] spdk_tgt can cancel lvol checksum calculation while there is high priority task 10421 - @yangchiu @DamiaSan
• [IMPROVEMENT] Lvol is not force-removed if Blob is busy 10474 - @yangchiu @DamiaSan
• [IMPROVEMENT] Remove deprecated fields from CRDs 6684 - @derekbit @roger-ryao
• [IMPROVEMENT] Longhorn CLI fails to recognize Raspbian OS 10676 - @bachmanity1 @roger-ryao
• [IMPROVEMENT] Reduce auto balancing logging noise for detached volumes 10691 - @dihmandrake @roger-ryao
• [IMPROVEMENT] Remove the upper bound of v2-data-engine-guaranteed-instance-manager-cpu 10662 - @derekbit @roger-ryao
• [IMPROVEMENT] Clean up BackupTarget condition message handling 8224 - @chriscchien @houhoucoop
• [IMPROVEMENT] Longhorn CLI supports SLES micro 9256 - @yangchiu @DamiaSan
• [IMPROVEMENT] Allow volumeBindingMode to be set from helm values 10592 - @ruant @roger-ryao
• [DOC] Prepare a knowledge base for backing image trouble shooting during upgrade 10590 - @ChanYiLin @chriscchien
• [IMPROVEMENT] Missing Prometheus Metrics for Engine v2 Volumes 10472 - @hookak @roger-ryao
• [IMPROVEMENT] Create Volume UI improvement, Automatically Filter Backing Image Based on v1 or v2 Selection 10086 - @houhoucoop @roger-ryao
• [UI][IMPROVEMENT] Improve the Warning Message When Failed to Remove Block-Type Disks 10580 - @houhoucoop @roger-ryao
• [IMPROVEMENT] Pass full backup mode option to CSI volume snapshot type backup 9785 - @ChanYiLin @roger-ryao
• [UI][IMPROVEMENT] Clean up BackupTarget condition message handling 10579 - @houhoucoop
• [IMPROVEMENT] Improve the Warning Message When Failed to Remove Block-Type Disks 10522 - @yangchiu @ChanYiLin
• [IMPROVEMENT] Move SettingNameV2DataEngineHugepageLimit to danger zone settings 7746 - @derekbit @chriscchien
• [IMPROVEMENT] Include the /proc/mounts file and multipath.config in the support-bundle 6754 - @c3y1huang @roger-ryao
• [IMPROVEMENT] Use code-generator/kube_codegen.sh to generate K8s stubs and CRDs 7944 - @derekbit @chriscchien
• [IMPROVEMENT] CRD & API code generator decouple from Go conventional source path 10556 - @COLDTURNIP
• [IMPROVEMENT] Support configurable upgrade-responder URL 10437 - @derekbit @roger-ryao
• [IMPROVEMENT] Settings change validation should go back to using Volume state to determine "are all volumes detached" 10233 - @yangchiu @james-munson
• [IMPROVEMENT] Code conventions in the Longhorn project with golangci-lint 8955 -
• [IMPROVEMENT] Improve the UX of updating danger zone settings 8070 - @yangchiu @mantissahz

Bug

• [BUG] v2 replica rebuilding will miss the backing image 10909 - @shuo-wu @chriscchien
• [BUG] Test case test_snapshot_prune_and_coalesce_simultaneously_with_backing_image fails 10808 - @yangchiu @c3y1huang
• [BUG] V2 Backing image not ready after upgrade from v1.8.1 to v1.9.x 10805 - @COLDTURNIP @chriscchien
• [BUG] minor spacing issues found with yamllint in the helm chart 10681 - @codekow
• [BUG][UI] Snapshots of v2 volume with backing image aren't shown on the Snapshots and Backups graph 10526 - @derekbit @chriscchien
• [BUG] Deleted orphan data still renders on the page until page refresh 10803 - @COLDTURNIP @chriscchien @houhoucoop
• [BUG] DR volume does not sync with latest backup when activation 10824 - @c3y1huang @chriscchien
• [BUG][v1.9.0-rc1] Block disks become temporarily unavailable after the upgrading from v1.8.1 to v1.9.0-rc1 10821 - @mantissahz
• [BUG] Naming collision when creating the name of the new backing image manager 10616 - @yangchiu @ChanYiLin
• [BUG] v2 volume replica status error after snapshot deletion with Immediate Data Integrity Check Enabled 10798 - @shuo-wu
• [BUG] Enabling V2 Data Engine setting, v2 instance manager doesn't start after certain negative factor operations 10791 - @COLDTURNIP @yangchiu
• [BUG] spdk emits Device or resource busy while registering lvol checksum calculation 10140 - @shuo-wu @roger-ryao
• [BUG] v2 instance managers keep crashing on ma...

DON'T UPGRADE from/to any RC/Preview/Sprint releases because the operation is not supported.

Resolved Issues in this release

Highlight

• [FEATURE] Support creation of PVC from CSI snapshot without copying data for v2 data engine 7794 - @PhanLe1010
• [FEATURE] V1 and V2 volume offline replica rebuilding 8443 - @mantissahz @roger-ryao
• [FEATURE] Delta Replica Rebuilding using Delta Snapshot: Control and Data Planes 10037 - @shuo-wu
• [FEATURE] v2 volume supports UBLK frontend 9456 - @PhanLe1010 @chriscchien
• [UI][FEATURE] V1 and V2 volume offline replica rebuilding 10581 - @houhoucoop @roger-ryao
• [TASK] Migrate v1beta1 CR to v1beta2 10250 - @COLDTURNIP @roger-ryao
• [FEATURE] Storage network with V2 data engine 6450 - @c3y1huang @roger-ryao
• [FEATURE] Recurring system backup 6534 - @yangchiu @c3y1huang
• [UI][FEATURE] Recurring system backup 10262 - @yangchiu @houhoucoop

Feature

• [FEATURE] Cleanup orphaned volume runtime resources if the resources already deleted 6764 - @COLDTURNIP @chriscchien
• [FEATURE] Running replicas field in volume table 10817 - @xelab04 @roger-ryao
• [FEATURE] Longhorn UI supports orphaned instance CRs management 10760 - @yangchiu @houhoucoop
• [FEATURE] Allow auto deleting snapshot when a backup is created from that snapshot. 9213 - @yangchiu @mantissahz
• [FEATURE] Add missing metrics of number of volumes/replicas by node/cluster 7599 - @c3y1huang @roger-ryao

Improvement

• [IMPROVEMENT] Add Prometheus metrics for Replica and Engine CRs 10722 - @hookak @chriscchien
• [IMPROVEMENT] Export longhorn engine rebuild status as prometheus metrics 10550 - @hookak @chriscchien
• [IMPROVEMENT] Disable Snapshot Checksum Calculation for Single-Replica V1 Volume 10518 - @derekbit @chriscchien
• [IMPROVEMENT] Remove unnecessary lasso dependency 10856 - @derekbit @chriscchien
• [IMPROVEMENT] add extraObject in charts 10835 - @DrummyFloyd @chriscchien
• [IMPROVEMENT] Disable the v2 snapshot hashing while it is being deleted 10563 - @shuo-wu @roger-ryao
• [IMPROVEMENT] v2 checksum calculation and update should follow the v1 flow 10480 - @shuo-wu @roger-ryao
• [IMPROVEMENT] add strict field validation to the update option in upgrade path 10644 - @ChanYiLin @chriscchien
• [IMPROVEMENT] Show snapshot size during in-progress backup 9783 - @yangchiu @houhoucoop
• [IMPROVEMENT] Don't synchronize all filesystem before snapshotting a v2 volume 9023 - @yangchiu @DamiaSan
• [IMPROVEMENT] spdk_tgt can cancel lvol checksum calculation while there is high priority task 10421 - @yangchiu @DamiaSan
• [IMPROVEMENT] Lvol is not force-removed if Blob is busy 10474 - @yangchiu @DamiaSan
• [IMPROVEMENT] Remove deprecated fields from CRDs 6684 - @derekbit @roger-ryao
• [IMPROVEMENT] Longhorn CLI fails to recognize Raspbian OS 10676 - @bachmanity1 @roger-ryao
• [IMPROVEMENT] Reduce auto balancing logging noise for detached volumes 10691 - @dihmandrake @roger-ryao
• [IMPROVEMENT] Remove the upper bound of v2-data-engine-guaranteed-instance-manager-cpu 10662 - @derekbit @roger-ryao
• [IMPROVEMENT] Clean up BackupTarget condition message handling 8224 - @chriscchien @houhoucoop
• [IMPROVEMENT] Longhorn CLI supports SLES micro 9256 - @yangchiu @DamiaSan
• [IMPROVEMENT] Allow volumeBindingMode to be set from helm values 10592 - @ruant @roger-ryao
• [DOC] Prepare a knowledge base for backing image trouble shooting during upgrade 10590 - @ChanYiLin @chriscchien
• [IMPROVEMENT] Missing Prometheus Metrics for Engine v2 Volumes 10472 - @hookak @roger-ryao
• [IMPROVEMENT] Create Volume UI improvement, Automatically Filter Backing Image Based on v1 or v2 Selection 10086 - @houhoucoop @roger-ryao
• [UI][IMPROVEMENT] Improve the Warning Message When Failed to Remove Block-Type Disks 10580 - @houhoucoop @roger-ryao
• [IMPROVEMENT] Pass full backup mode option to CSI volume snapshot type backup 9785 - @ChanYiLin @roger-ryao
• [UI][IMPROVEMENT] Clean up BackupTarget condition message handling 10579 - @houhoucoop
• [IMPROVEMENT] Improve the Warning Message When Failed to Remove Block-Type Disks 10522 - @yangchiu @ChanYiLin
• [IMPROVEMENT] Move SettingNameV2DataEngineHugepageLimit to danger zone settings 7746 - @derekbit @chriscchien
• [IMPROVEMENT] Include the /proc/mounts file and multipath.config in the support-bundle 6754 - @c3y1huang @roger-ryao
• [IMPROVEMENT] Use code-generator/kube_codegen.sh to generate K8s stubs and CRDs 7944 - @derekbit @chriscchien
• [IMPROVEMENT] CRD & API code generator decouple from Go conventional source path 10556 - @COLDTURNIP
• [IMPROVEMENT] Support configurable upgrade-responder URL 10437 - @derekbit @roger-ryao
• [IMPROVEMENT] Settings change validation should go back to using Volume state to determine "are all volumes detached" 10233 - @yangchiu @james-munson
• [IMPROVEMENT] Code conventions in the Longhorn project with golangci-lint 8955 -
• [IMPROVEMENT] Improve the UX of updating danger zone settings 8070 - @yangchiu @mantissahz

Bug

• [BUG] backup target settings causes longhorn-manager crash loop during upgrade to v1.9.0 10864 - @COLDTURNIP
• [BUG][v1.9.0-rc1] v2 volumes don't reuse failed replicas as expected after a node goes down 10828 - @yangchiu @shuo-wu
• [BUG][v1.9.0-rc1] Unexpected orphaned data are created after v2 instance managers deleted 10829 - @yangchiu
• [BUG] v2 volume with backing image fails to recover to healthy state after deleting instance manager during replica rebuilding 10521 - @yangchiu
• [BUG] V2 Backing image not ready after upgrade from v1.8.1 to v1.9.x 10805 - @COLDTURNIP @chriscchien
• [BUG] Test case test_snapshot_prune_and_coalesce_simultaneously_with_backing_image fails 10808 - @yangchiu @c3y1huang
• [BUG][UI] Snapshots of v2 volume with backing image aren't shown on the Snapshots and Backups graph 10526 - @derekbit @chriscchien
• [BUG] Deleted orphan data still renders on the page until page refresh 10803 - @COLDTURNIP @chriscchien @houhoucoop
• [BUG] v2 Engine loops in detaching and attaching state after rebuilding 10396 - @shuo-wu @roger-ryao
• [BUG] DR volume does not sync with latest backup when activation 10824 - @c3y1huang @chriscchien
• [BUG][v1.9.0-rc1] While running component resilience robot test suite, a v2 instance manager gets stuck in Terminating state, preventing block disk from being schedulable on this node 10810 - @DamiaSan
• [BUG][v1.9.0-rc1] Block disks become temporarily unavailable after the upgrading from v1.8.1 to v1.9.0-rc1 10821 - @mantissahz
• [BUG] Naming collision when creating the name of the new backing image manager 10616 - @yangchiu @ChanYiLin
• [BUG] v2 volume replica status error after snapshot deletion with Immediate Data Integrity Check Enable...