What's Changed

Added

• bitswap/network/httpnet: New WithMetricsLabelsForEndpoints allows defining which hosts/endpoints can be used for labelling metrics that support such label. '*' enables this for all endpoints receiving HTTP requests, but may cause metric cardinality growth when too many endpoints exist. These labels allow tracking, for example, number or requests per response status AND endpoint used. Non-labelled request hosts are labelled with same value: other.

Changed

• DontHaveTimeoutConfig's default MinTimeout is changed from 0 to 50ms #959 #965.
• upgrade to go-libp2p v0.42.0

Fixed

• bitswap/client: Fix sending extra wants #968 + #975
• routing/http/client: Improve URL handling for delegated routing endpoints #971

Security

• fix panic when incoming Bitswap protobuf message does not contain Wantlist #961

Full Changelog: v0.32.0...v0.33.0

This release was brought to you by the Shipyard team.

What's Changed

Added

• provider includes metrics on the number of keys provided so far(reprovider_provide_count) and the number of keys reprovided so far (reprovider_reprovide_count) #944
• bitswap/client: New metrics:
  • ipfs_bitswap_wanthaves_broadcast: Count of want-haves broadcasts
  • ipfs_bitswap_haves_received: Count of total have responses
  • ipfs_bitswap_bcast_skips_total{: Count of broadcasts skipped as part of spam reduction logic (see "Changed" below)
  • ipfs_bitswap_unique_blocks_received: Count of non-duplicate blocks recieved

Changed

• provider: previously, the code in this module was logging to reprovider.simple, provider.batched and provider.queue facilities. They have now been consolidated in a single provider logging facility, along with some adjustments to logging levels and extra debug statements.
• bitswap/client: Added an opt-in ability to reduce bitswap broadcast volume by limiting broadcasts to peers that have previously responded as having wanted blocks and peers on local network. The following bitswap client options are available to configure the behavior of broadcast reduction:
  • BroadcastControlEnable enables or disables broadcast reduction logic. Setting this to false restores the previous broadcast behavior of sending broadcasts to all peers, and ignores all other BroadcastControl options. Default is false (disabled).
  • BroadcastControlMaxPeers sets a hard limit on the number of peers to send broadcasts to. A value of 0 means no broadcasts are sent. A value of -1 means there is no limit. Default is -1 (unlimited).
  • BroadcastControlLocalPeers enables or disables broadcast control for peers on the local network. If false, then always broadcast to peers on the local network. If true, apply broadcast control to local peers. Default is false (always broadcast to local peers).
  • BroadcastControlPeeredPeers enables or disables broadcast control for peers configured for peering. If false, then always broadcast to peers configured for peering. If true, apply broadcast control to peered peers. Default is false (always broadcast to peered peers).
  • BroadcastControlMaxRandomPeers sets the number of peers to broadcast to anyway, even though broadcast control logic has determined that they are not broadcast targets. Setting this to a non-zero value ensures at least this number of random peers receives a broadcast. This may be helpful in cases where peers that are not receiving broadcasts may have wanted blocks. Default is 0 (no random broadcasts).
  • BroadcastControlSendToPendingPeers enables or disables sending broadcasts to any peers to which there is a pending message to send. When true (enabled), this sends broadcasts to many more peers, but does so in a way that does not increase the number of separate broadcast messages. There is still the increased cost of the recipients having to process and respond to the broadcasts. Default is false.

Removed

• bitswap/server do not allow override of peer ledger with WithPeerLedger #938

Fixed

• gateway: Fixed suffix range-requests and updated tests to gateway-conformance v0.8 #922

Full Changelog: v0.31.0...v0.32.0

This release was brought to you by the Shipyard team.

What's Changed

Added

• bitswap/httpnet: New option WithMaxDontHaveErrors(int) (defaults to 100)
  will trigger disconnections from bitswap peers that cannot provide any of
  the blocks that are requested for the given number of requests in a
  row.. This is meant to limit bitswap HTTP-based optimistic requests for
  blocks to discovered endpoints, which were before considered permanently
  peered upon discovery.

Changed

• upgrade to go-libp2p-kad-dht v0.33.1
• deprecated WithPeerLedger option for bitswap server. Will remove option in next release. See issue #928
• gateway: update backend car traversal to use go-car/v2 #925
  • This change may prohibit fetching DAGs from CAR files that have no roots.

Full Changelog: v0.30.0...v0.31.0

This release was brought to you by the Shipyard team.

What's Changed

Added

• Control over UnixFS DAG Width
  • We have made some changes to allow setting custom max width of UnixFS DAGs. This enables users to produce DAGs that follow current and future community conventions (see the related discussion).
  • ipld/unixfs: DagModifier now allows specifying file DAG Width (MaxLinks) #898
  • ipld/unixfs/io/directory: We have made some changes to unixfs directory tooling #906
    • We have exposed creator methods for new BasicDirectory and HAMTDirectory, that complement the existing NewDirectory() which creates dynamic directories.
    • We have added WithCidBuilder(...), WithMaxLinks(...), WithMaxHAMTFanout(...) and WithStat(...) as options to these new methods so that empty directories can be initilized as wished from the get-go.
    • WithMaxLinks(...) and WithMaxHAMTFanout(...) are new options that allow to set a limit to the number of children that a directory DAG node can have. For details on what they exactly do for each of the directory type, please check the documentation.
  • mfs supports as well the new MaxLinks and MaxHAMTFanout options. They have been made part of the MkdirOptions object and the methods NewEmptyDirectory() and NewEmptyRoot() have been added to facilitate the initialization of MFS objects. #906
• provider: added support for walking partial DAGs in offline mode #905
  • a KeyChanFunc that traverses DAGs from a given root (NewDAGProvider).
  • a KeyChanFunc that buffers all the CIDs in memory from another KeyChanFunc (NewBufferedProvider).
  • fetcher/impl/blockservice: new option SkipNotFound for the IPLD fetcher. It will skip not found nodes when traversing the DAG. This allows offline traversal of DAGs when using, for example, an offline blockservice.
  • This enables use case of providing lazy-loaded, partially local DAGs (like ipfs files in Kubo's MFS implementation, see kubo#10386)
• gateway: generated HTML with UnixFS directory listings now include a button for copying CIDs of child entities #899
• bitswap/server: Add ability to enable/disable bitswap server using WithServerEnabled bitswap option (#911)[https://github.com//pull/911]

Changed

• upgrade to go-libp2p v0.41.1
• bitswap/network: Add a new requests_in_flight metric gauge that measures how many bitswap streams are being written or read at a given time.
• improve speed of data onboarding by batching/bufering provider queue writes #888
• provider: providing queue is now independent from reprovides, speeding up initial provides #907
  • renamed provider.ReproviderStats.TotalProvides => provider.ReproviderStats.TotalReprovides
  • renamed provider.ReproviderStats.AvgProvideDuration => provider.ReproviderStats.AvgReprovideDuration
• provider/queue deduplicates CIDs #910

Fixed

• gateway: query parameters are now supported and preserved in redirects triggered by a _redirects file #886
• provider: adjusted first reprovide timing after node reboot #890
• gateway: validate configuration and warn when UseSubdomains=true is used with IP-based hostnames #903

Full Changelog: v0.29.1...v0.30.0

This release was brought to you by the Shipyard team.

What's Changed

Changed

• bitswap/httpnet: do not follow redirects #878
• provider: provider helper that buffers results to fix issues with slow re-providing #870

Fixed

• provider: don't reprovide if reprovideInterval is set to 0 #871
• gateway: Skip DNSLink lookup for IP addresses to avoid unnecessary DNS queries #880
• bitswap/client: Fix unintentional ignoring DontHaveTimeoutConfig #872

Full Changelog: v0.29.0...v0.29.1

What's Changed

Added

• feat(bitswap/client): MinTimeout for DontHaveTimeoutConfig #865
• ✨ httpnet: Transparent HTTP-block retrieval support over Trustless Gateways [#747]((#747):
  • Complements Bitswap as a block-retrieval mechanism, implementing bitswap/network.
  • Understands peers found in provider records with /.../http endpoints (trustless gateway).
  • Treats them as "Bitswap" peers, except instead of using Bitswap it makes HTTP/2 requests to discover (HEAD) and retrieve (GET) individual blocks (?format=raw).
  • A bitswap/network proxy implementation allows co-existance with standard bitswap/network/bsnet.
  • httpnet is not enabled by default. Upstream implementations may use it by modifying how they create the Bitswap network and initialize bitswap.

Changed

• ipns: The DefaultRecordTTL changed from 1h to 5m #859
• upgrade to go-libp2p v0.41.0
• upgrade to go-libp2p-kad-dht [v0.30.2](github.com/libp2p/go-libp2p-kad-dht v0.30.2)
• upgrade to go-datastore v0.8.2 - includes API updates and removes go-process
• bitswap/client reduce lock scope of PeerManageer to help performance #860
• NewFromIpfsHost() and the rest of the BitSwapNetwork implementation has moved from github.com/ipfs/boxo/bitswap/network to github.com/ipfs/boxo/bitswap/network/bsnet.

Removed

• Removed dependency on github.com/hashicorp/go-multierror so that boxo only depends on one multi-error package, go.uber.org/multierr. #867

Full Changelog: v0.28.0...v0.29.0

This release was brought to you by the Shipyard team.

What's Changed

Added

• bitswap/client: Add DontHaveTimeoutConfig type alias and func DontHaveTimeoutConfig() to expose config defined in internal package.

Changed

• move ipld/unixfs from gogo protobuf #841
• provider: Prevent multiple instances of reprovider.Reprovide() from running at the same time. #834
• upgrade to go-libp2p v0.40.0
• upgrade to go-libp2p-kad-dht [v0.29.0](github.com/libp2p/go-libp2p-kad-dht v0.29.0)
• move bitswap and filestore away from gogo protobuf #839
• updated Go in go.mod to 1.23 #848

Note: This release contains changes to protocol buffer library code. If you depend on deterministic CIDs then please double-check,, before upgrading, that this release does not generate different CIDs.

Full Changelog: v0.27.4...v0.28.0

This release was brought to you by the Shipyard team.

What's Changed

Fixed

• Fix memory leaks due to not cleaning up wantlists #829, #833
• ipns: Improved interop with legacy clients by restoring support for []byte CID in Value field. Value() will convert it to a valid path.Path. Empty Value() will produce NoopPath (/ipfs/bafkqaaa) to avoid breaking existing code that expects a valid record to always produce a valid content path. #830

Full Changelog: v0.27.3...v0.27.4

This release was brought to you by the Shipyard team.

What's Changed

Added

• provider: Added ReprovideInterval and LastRun stats to the Reprovider #815

Removed

• bitswap/cllient: Remove unused tracking of CID for interested sessions. #821

Fixed

• bitswap/client: Fix runaway goroutine creation under high load. Under high load conditions, goroutines are created faster than they can complete and the more goroutines creates the slower them complete. This creates a positive feedback cycle that ends in OOM. The fix dynamically adjusts message send scheduling to avoid the runaway condition. #817
• bitswap/cllient: Fix resource leak caused by recording the presence of blocks that no session cares about. #822

Full Changelog: v0.27.2...v0.27.3

This release was brought to you by the Shipyard team.

What's Changed

Fixed

• bitswap/client: Reverted attempt to send cancels with excluded peer due to additional issues with wantlist accounting #809

Full Changelog: v0.27.1...v0.27.2

This release was brought to you by the Shipyard team.