Repository navigation
Official CC-BY-4.0 license text is not being matched correctly by LicenseCompareHelper.isTextStandardLicense() #233
Description
When org.spdx.utility.compare.LicenseCompareHelper.isTextStandardLicense().isDifferenceFound() is run on the official CC-BY-4.0 license text, it (incorrectly) returns true (i.e. the standard license was not matched). When I run org.spdx.utility.compare.LicenseCompareHelper.isTextStandardLicense().getDifferenceMessage() on it, I get:
Normal text of license does not match starting at line #5 column #9 "commons" when comparing to template text "
By exercising the Licensed Rights (defined below), You accept and agree to be bound by the terms a...". Last optional text was not found due to the optional difference:
Normal text of license does not match starting at line #5 column #29 "(" when comparing to template text " Creative Commons Attribution 4.0 International Public License
". Last optional text was not found due to the optional difference:
Normal text of license does not match starting at line #42 column #5 "example" when comparing to template text "
Considerations for the public: By using one of our public licenses, a licensor grants the public p...". Last optional text was not found due to the optional difference:
Normal text of license does not match starting at line #42 column #5 "example" when comparing to template text "
Considerations for the public: By using one of our public licenses, a licensor grants the public p...". Last optional text was not found due to the optional difference:
Normal text of license does not match starting at line #1 column #12 "4" when comparing to template text "Creative Commons"
I don't see any mismatches when looking at the CC-BY-4.0 SPDX template however.
This was reproduced with Spdx-Java-Library v1.11 and SPDX license list v3.23.
Activity
pmonks commentedon Mar 26, 2024
Note: this also seems to be occurring for some of the
CC-BY-*-4.0variations, including:CC-BY-SA-4.0CC-BY-NC-4.0CC-BY-NC-SA-4.0CC-BY-ND-4.0CC-BY-NC-ND-4.0(these are just the subset of
CC-BY-*-4.0licenses that I'm testing against - there may be issues with other variations too)goneall commentedon Mar 29, 2024
Thanks @pmonks for the analysis - I'm currently traveling for the next week or so - I'll take a look once I'm back
pmonks commentedon Mar 29, 2024
As always no rush! I've been traveling a lot myself in recent weeks, so have also been less active than usual.
goneall commentedon Feb 26, 2025
Ok - a bit longer than I expected, but I'm finally doing some analysis.
No conclusions yet, but I did notice that the template had nested "optional" rules. This isn't technically allowed.
The reason for the nesting is that anything within a
titleTexttag is translated to optional. With the template, there is anoptionaltag which creates the nested optional.goneall commentedon Feb 26, 2025
I found the issue - in the Official text, two dashes are used
--whereas in the license list XML file single dashes are used-.I know we ignore multiple dashes (e.g. 3 or more) since they are typically separators for text, but the code currently doesn't normalize 2 dashes and 1 dash.
Question: Should it? Or should we update the XML file to match one or two?
pmonks commentedon Feb 27, 2025
@goneall that vaguely reminds me - was there an "ignorable punctuation" issue in the license list XML project? I seem to vaguely recall commenting on such a thing a year or two ago. If I'm remembering correctly it was going to be something of an "official" ruling on this topic.
I've been really busy lately (and that will continue for another month or so), but I may be able to look for it tomorrow or Friday.
pmonks commentedon Feb 28, 2025
I was thinking of spdx/license-list-XML#1617
goneall commentedon Feb 28, 2025
Thanks @pmonks for checking into it
Ah yes - I remember that issue. The fix was to match 3 or more dashes - this particular license uses 2 dashes :(
In this specific context, it looks like 2 dashes should be treated the same as a single dash.
I can't think of a situation where an extra dash would make a different in a license text, but it may make sense to run this by the legal team just to make sure.
No rush - I'm just catching up on the issues now that I've (mostly) completed the upgrade to SPDX 3.0.
goneall commentedon Apr 14, 2025
@pmonks - Any thoughts on this? We could add a markup to CC-BY-4.0 to handles this specific situation, I'm a bit nervous about matching 2 dashes as a single dash without running it by the legal team. We can also punt the decision to the next release of the library (I'm working through the issues for another release).
pmonks commentedon Apr 14, 2025
@goneall yeah this now seems like it's not a bug in Spdx-Java-Library, but either a bug in the template, or something that needs to be included in further (legal team) discussions around ignorable punctuation. Either way, it should probably be moved elsewhere (but I'm not sure where - sorry! 😬).
goneall commentedon Apr 15, 2025
@pmonks I can open a similar issue in the license-list-XML repo framing it as either a template issue or a change in matching guidelines that two dashes are the same as a single dash - this would be a distinct issue from the line separators which are 3 or more dashes.
goneall commentedon Apr 15, 2025
I created this issue in the license-list-XML repo: spdx/license-list-XML#2704
Closing this issue - we can open a separate issue if the decision is made to change the license matching guidelines.