CSS Counter Styles Level 3 says:

The following stylesheet fragment provides the normative definition of these predefined counter styles:
...

@counter-style square {
 symbols: \25FE;
...

Unicode code point 25FE has Emoji_Presentation=Yes:
https://www.unicode.org/Public/UCD/latest/ucd/emoji/emoji-data.txt

This has the effect that a compliant UA will try to find a glyph for it from a color-emoji font rather than the current default text font, which leads to the following problems:

1. it isn't rendered using the current CSS foreground color
2. it's usually way too large in comparison to list-style-type: disc/circle
   (see this testcase for some examples - 25FE is the third character from the left)

Currently, many UAs render list-style-type: square/disc/circle/... using custom non-standard methods (typically a special internal box type) so the above is a minor problem. However, we're changing Gecko to use generated text per the CSS Counter Styles spec instead, which makes this a major problem: the rendering for 25FE is so ugly that it's unshippable for this use. (Point 1 above can also lead to the glyph being invisible when the background is black.)

So I think this symbol needs to be changed to something else... I've tested the filled squares in Geometric Shapes code block with many different fonts and it appears 25AA would be the best choice. It's rendered in the foreground color and it's similar in size and alignment to disc/circle (2022/25E6). It also appears to be supported by most common fonts. (25AA is the second character from the left in the testcase mentioned above)

(Fwiw, even with the above change the default rendering is still too ugly in some fonts for us to ship that as the default rendering. So we'll use an internal font for these counter styles by default (that makes it very close to our current legacy rendering), but we'll let authors override that with a font of their choice if they want to.)