Is your feature request related to a problem?

• The actual size of volumes is very high and disks are getting low on available disk space.
• the performance is affected because instead of distributing the volumes across multiple disks, one disk is used to store all the volumes (a better option would be to balance volumes based on IOPS usage, but for now this is science fiction :D)

Describe the solution you'd like

It would be awesome if Longhorn would be able to:

1. allocate new volumes based on available disk space and/or based on actual size usage .
2. evict a volume from a disk

Describe alternatives you've considered

1. Set best-effort as data locality and then constrain the deployment on different nodes (this solution does not fix the problem entirely because we can't configure which disk to be used).
2. To evict a volume, increase the replication factor to 2, wait a few hours, delete the old replica, set replication factor to 1

#3840 #3887