v2.5.0

@x448

This release adds important new features, optimizations, bug fixes, and 8 new contributors. Fuzz tests passed 5+ billion execs for CBOR encoding and decoding functions.

Notable changes are split into 3 categories.

⭐ Notable Changes to Review Before Upgrading

These new features and bug fixes were cherry-picked to highlight for review (for projects using older version).

PR 370: Add SimpleValue type to more fully support CBOR Simple Values, including values not assigned by IANA and...
PR 376: Add ByteString type to support CBOR maps with byte string keys because Go doesn't allow []byte as map keys and...
PR 379: Make Decoder.Decode() return io.ErrUnexpectedEOF instead of io.EOF on EOF if current CBOR data item is incomplete.
PR 380: Make Unmarshal() and Valid() return cbor.ExtraneousDataError (instead of ignoring extraneous data if any remain).
PR 387: Retry in Decoder if io.Reader's Read() returns 0 bytes read with nil error. Add tests to get 100% code coverage for stream.go.

⭐ More Notable Changes

PR 342: Add DecOptions.UTF8 to decode invalid UTF-8. Default is unchanged (reject invalid UTF-8 and return error).
PR 355 Allow MaxNestedLevels setting to be configured up to 65535.
PR 352, 377: Add EncOptions.NilContainersMode to encode nil Go maps and slices as either CBOR nil (default) or empty container.
PR 381: Add Decoder.Skip() to skip CBOR data item in CBOR Sequences (RFC 8742).
PR 386 Add functions for Extended Diagnostic Notation (RFC 8610 Appendix G).
PR 398 Add UnmarshalFirst() to decode CBOR data item and also return remaining bytes.
PR 400 Deprecate Valid and add Wellformed to replace it.
PR 402 Add UnmarshalFirst() to DecMode interface.
PR 412 Add Decoder.Buffered to return remaining data in buffer.

🚀 Notable Changes Affecting Speed or Memory Use

PR 335: Reuse underlying array if RawMessage has sufficient capacity.
PR 382: Return buffer to pool in Encode(). It adds a bit of overhead to Encode() but NewEncoder().Encode() is faster and uses less memory.

Benchmark comparison for PR 382 shows significant improvement in speed and memory use.

🚀 (click to expand) 🚀

Benchmarks provided by @x448 for NewEncoder().Encode() optimized in PR 382.

$ benchstat bench-v2.4.0.log bench-f9e6291.log 
goos: linux
goarch: amd64
pkg: github.com/fxamacker/cbor/v2
cpu: 12th Gen Intel(R) Core(TM) i7-12700H
                                                     │ bench-v2.4.0.log │  bench-f9e6291.log                  │
                                                     │      sec/op      │   sec/op     vs base                │
NewEncoderEncode/Go_bool_to_CBOR_bool-20                   236.70n ± 2%   58.04n ± 1%  -75.48% (p=0.000 n=10)
NewEncoderEncode/Go_uint64_to_CBOR_positive_int-20         238.00n ± 2%   63.93n ± 1%  -73.14% (p=0.000 n=10)
NewEncoderEncode/Go_int64_to_CBOR_negative_int-20          238.65n ± 2%   64.88n ± 1%  -72.81% (p=0.000 n=10)
NewEncoderEncode/Go_float64_to_CBOR_float-20               242.00n ± 2%   63.00n ± 1%  -73.97% (p=0.000 n=10)
NewEncoderEncode/Go_[]uint8_to_CBOR_bytes-20               245.60n ± 1%   68.55n ± 1%  -72.09% (p=0.000 n=10)
NewEncoderEncode/Go_string_to_CBOR_text-20                 243.20n ± 3%   68.39n ± 1%  -71.88% (p=0.000 n=10)
NewEncoderEncode/Go_[]int_to_CBOR_array-20                 563.0n ± 2%    378.3n ± 0%  -32.81% (p=0.000 n=10)
NewEncoderEncode/Go_map[string]string_to_CBOR_map-20       2.043µ ± 2%    1.906µ ± 2%   -6.75% (p=0.000 n=10)
geomean                                                    349.7n         122.7n       -64.92%
                                                     │ bench-v2.4.0.log │    bench-f9e6291.log                │
                                                     │       B/op       │    B/op     vs base                 │
NewEncoderEncode/Go_bool_to_CBOR_bool-20                     128.0 ± 0%     0.0 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_uint64_to_CBOR_positive_int-20           128.0 ± 0%     0.0 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_int64_to_CBOR_negative_int-20            128.0 ± 0%     0.0 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_float64_to_CBOR_float-20                 128.0 ± 0%     0.0 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_[]uint8_to_CBOR_bytes-20                 128.0 ± 0%     0.0 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_string_to_CBOR_text-20                   128.0 ± 0%     0.0 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_[]int_to_CBOR_array-20                   128.0 ± 0%     0.0 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_map[string]string_to_CBOR_map-20         544.0 ± 0%   416.0 ± 0%   -23.53% (p=0.000 n=10)
geomean                                                      153.4                    ?                       ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean
                                                     │ bench-v2.4.0.log │    bench-f9e6291.log                │
                                                     │    allocs/op     │ allocs/op   vs base                 │
NewEncoderEncode/Go_bool_to_CBOR_bool-20                     2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_uint64_to_CBOR_positive_int-20           2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_int64_to_CBOR_negative_int-20            2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_float64_to_CBOR_float-20                 2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_[]uint8_to_CBOR_bytes-20                 2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_string_to_CBOR_text-20                   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_[]int_to_CBOR_array-20                   2.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_map[string]string_to_CBOR_map-20         28.00 ± 0%   26.00 ± 0%    -7.14% (p=0.000 n=10)
geomean                                                      2.782                    ?                       ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean

What's Changed

All merged pull requests are listed here (including the already highlighted ones).

(click to expand)

Changes to Code

Most coding changes here were already mentioned.

Reuse underlying array if RawMessage has sufficient capacity by @zensh in #335
Add decoding option to allow invalid UTF-8 by @fxamacker in #342
Allow MaxNestedLevels to be up to 65535 by @immesys in #355
Add support for unassigned/reserved CBOR simple values by @fxamacker in #370
Add ByteString type to support any CBOR byte string by @fxamacker in #376
add option to enforce nil container marshaling as empty containers by @dedefer in #352
Refactor NilContainersMode option by @fxamacker in #377
Make Decoder.Decode() return io.UnexpectedEOF instead of io.EOF if CBOR data item is truncated by @fxamacker in #379
Fix handling of extra data in Unmarshal() & Valid() by @fxamacker in #380
Add Decoder.Skip() to skip CBOR data item in CBOR Sequences (RFC 8742) by @fxamacker in #381
Return buffer to pool when using Encoder by @fxamacker in #382
Retry in decoder if Read() returns 0 bytes read with nil error by @fxamacker in #387
Deprecate Valid() and add Wellformed() to replace it by @fxamacker in #400
Add UnmarshalFirst by @immesys in #398
Add UnmarshalFirst to DecMode interface by @fxamacker in #402
Add functions for Extended Diagnostic Notation (RFC 8610 Appendix G) by @zensh in #386
Add more tests for Diagnose and DiagnoseFirst by @fxamacker in #405
Fix Diagnose to return io.EOF error on empty data by @fxamacker in #410
Add Decoder.Buffered to return buffered data by @fxamacker in #412
Fix Diagnose() to return io.EOF error on empty data by @fxamacker in #410

Changes to CI, Comments, and Docs

Remove trailing whitespaces in .golangci.yml by @CodingVoid in #333
Update ci.yml by @fxamacker in #334
Update ci.yml by @x448 in #339
Update ci.yml by @x448 in #340
Remove default permissions from GitHub Actions workflows by @x448 in #341
Fix ci for stream mode branch by @fxamacker in #344
Create dependabot.yml by @x448 in #346
Bump github/codeql-action from 1 to 2 by @dependabot in #347
Update pull_request_template.md by @x448 in #350
Fix typos by @deeglaze in #353
Mention security assessment by NCC Group by @fxamacker in #358
godoc.org link in Readme should point to v2 by @jdharms in #361
Add Go 1.19 to ci.yml test matrix by @fxamacker in #363
bump safer-golangci-lint.yml to 1.46.2 by @x448 in #365
Bump Go to 1.19 in ci-go-cover.yml by @x448 in #367
Bump safer-golangci-lint.yml to 1.49.0.1 by @x448 in #368
Mention 1276 repos depend on fxamacker/cbor/v2 by @x448 in #371
docs: fix EncMode example in the package comment by @creachadair in #375
Update fxamacker_cbor_banner.png for v2.5 by @x448 in #383
bump safer-golangci-lint.yml to 1.51.1 by @x448 in #389
Create govulncheck.yml by @fxamacker in #393
Update README for cbor v2.5.0-beta2 by @fxamacker in #394
Bump actions/setup-go from 3 to 4 by @dependabot in #395
Update ci-go-cover.yml by @x448 in #404
Update README for CBOR Sequences RFC 8742 by @fxamacker in #406
Update README.md for v2.5.0-beta3 by @fxamacker in #407
Bump safer-golangci-lint to 1.52.2 by @x448 in #408
Update README.md for cbor v2.5.0-beta4 by @fxamacker in #411
Update README.md for v2.5.0-beta5 by @fxamacker in #413
Bump CI govulncheck to v1.0.0 by @fxamacker in #414
Remove ignored UTF8 setting in diagMode() by @fxamacker in #417
Update README for CBOR codec v2.5.0-rc1 by @fxamacker in #418
Update README.md by @fxamacker in #419
Update intro and cbor quick start in README by @fxamacker in #421
Update README.md for v2.5.0 by @fxamacker in #422
Update CONTRIBUTING.md by @fxamacker in #423
Update README.md for v2.5.0 by @fxamacker in #424

New Contributors (alphabetically)

@agaffney made their first contribution in #376
@CodingVoid made their first contribution in #333
@creachadair made their first contribution in #375
@dedefer made their first contribution in #352
@deeglaze made their first contribution in #353
@immesys made their first contribution in #355
@jdharms made their first contribution in #361
@zensh made their first contribution in #335

Non-coding contibutions were made by opening notable issues that directly or indirectly improved this release.

@burdiyan opened issue and followups that led to Encode() returning buffer to pool as default behavior.
@espoal opened issue and followups that led to adding Skip() feature for CBOR Sequences (RFC 8742).
@immesys opened multiple issues and provided helpful feedback in addition to merged PRs.
@qmuntal opened issue and @x448 added feedback that led to improved support for CBOR Simple Values.
@x448 opened multiple issues and provided helpful feedback in addition to merged PRs.
@zensh opened multiple issues and provided helpful feedback in addition to merged PRs.

Full Changelog: v2.4.0...v2.5.0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!