[BUG] Compilation error after execution of updateKeywords

The execution of the updateKeywords task adds the following tokens to RelObjectNameWithoutValue:

JSqlParser/src/main/jjtree/net/sf/jsqlparser/parser/JSqlParserCC.jjt

Lines 529 to 551 in 411a3da

    
           | <#TYPE_BIT:  "BISTRING"> 
        
           | <#TYPE_BLOB: "BLOB" | "BYTEA" | <K_BINARY> | "VARBINARY" | <K_BYTES> > 
        
           | <#TYPE_BOOLEAN: "BOOLEAN" | "BOOL" > 
        
           | <#TYPE_ENUM: "ENUM" > 
        
           | <#TYPE_MAP: "MAP" > 
        
           | <#TYPE_DECIMAL: "DECIMAL" | "NUMBER" | "NUMERIC" > 
        
           | <#TYPE_TINYINT: "TINYINT" | "INT1" > 
        
           | <#TYPE_SMALLINT: "SMALLINT" | "INT2" | "SHORT" > 
        
           | <#TYPE_INTEGER: ( "INTEGER" | "INT" | "INT4" | <K_SIGNED> | <K_UNSIGNED> ) > 
        
           | <#TYPE_BIGINT: "BIGINT" | "INT8" | "LONG" > 
        
           | <#TYPE_HUGEINT: "HUGEINT" > 
        
           | <#TYPE_UTINYINT: "UTINYINT" > 
        
           | <#TYPE_USMALLINT: "USMALLINT" > 
        
           | <#TYPE_UINTEGER: "UINTEGER" > 
        
           | <#TYPE_UBIGINT: "UBIGINT" > 
        
           | <#TYPE_UHUGEINT: "UHUGEINT" > 
        
           | <#TYPE_REAL: "REAL" | "FLOAT4" | "FLOAT"> 
        
           | <#TYPE_DOUBLE: "DOUBLE" | "PRECISION"  | "FLOAT8" | "FLOAT64"> 
        
           | <#TYPE_VARCHAR: "NVARCHAR" | "VARCHAR" | "NCHAR" | <K_CHAR> | "BPCHAR" | "STRING" | "TEXT" | <K_CHARACTER> | "VARYING"> 
        
           | <#TYPE_TIME: "TIMETZ" > 
        
           | <#TYPE_TIMESTAMP: "TIMESTAMP_NS" | "TIMESTAMP_MS" | "TIMESTAMP_S"  > 
        
           | <#TYPE_UUID: "UUID">

This causes the javacc-jjtree goal to fail:

Error: Line 1962, Column 313: String token "BISTRING" has been defined as a private regular expression.
Error: Line 1962, Column 1163: String token "ENUM" has been defined as a private regular expression.
Error: Line 1962, Column 1500: String token "HUGEINT" has been defined as a private regular expression.
Error: Line 1962, Column 1815: String token "MAP" has been defined as a private regular expression.
Error: Line 1962, Column 3401: String token "TIMETZ" has been defined as a private regular expression.
Error: Line 1962, Column 3527: String token "UBIGINT" has been defined as a private regular expression.
Error: Line 1962, Column 3542: String token "UHUGEINT" has been defined as a private regular expression.
Error: Line 1962, Column 3558: String token "UINTEGER" has been defined as a private regular expression.
Error: Line 1962, Column 3672: String token "USMALLINT" has been defined as a private regular expression.
Error: Line 1962, Column 3689: String token "UTINYINT" has been defined as a private regular expression.
Error: Line 1962, Column 3705: String token "UUID" has been defined as a private regular expression.

Thank you for reporting, I actually broke it myself when introducing complex compound Data Type tokens.
I will definitely fix it, but this will take a couple of days since I am in transit right now. Sorry for any inconvenience.

fix: Avoid private tokens to be white listed

Thank you for the quick reply.
I tried to implement and provide a solution. The idea is to parse and iterate over each token definition and don't whitelist anything which is defined within a private token.
Let me know your thoughts on this: #2044

However, there is still an issue, related to #2041. After running the updateKeywords now, PRECISION and BASE64 will be removed. PRECISION is removed, because it is just part of a private token. BASE64 is removed, because numbers in the token values are not being whitelisted due to the condition tokenValue.matches("[A-Za-z]+").

Avoid private tokens to be white listed and allow any word character …

Navigation Menu

Repository navigation

[BUG] Compilation error after execution of updateKeywords #2040

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	\| <#TYPE_BIT: "BISTRING">
	\| <#TYPE_BLOB: "BLOB" \| "BYTEA" \| <K_BINARY> \| "VARBINARY" \| <K_BYTES> >
	\| <#TYPE_BOOLEAN: "BOOLEAN" \| "BOOL" >
	\| <#TYPE_ENUM: "ENUM" >
	\| <#TYPE_MAP: "MAP" >
	\| <#TYPE_DECIMAL: "DECIMAL" \| "NUMBER" \| "NUMERIC" >
	\| <#TYPE_TINYINT: "TINYINT" \| "INT1" >
	\| <#TYPE_SMALLINT: "SMALLINT" \| "INT2" \| "SHORT" >
	\| <#TYPE_INTEGER: ( "INTEGER" \| "INT" \| "INT4" \| <K_SIGNED> \| <K_UNSIGNED> ) >
	\| <#TYPE_BIGINT: "BIGINT" \| "INT8" \| "LONG" >
	\| <#TYPE_HUGEINT: "HUGEINT" >
	\| <#TYPE_UTINYINT: "UTINYINT" >
	\| <#TYPE_USMALLINT: "USMALLINT" >
	\| <#TYPE_UINTEGER: "UINTEGER" >
	\| <#TYPE_UBIGINT: "UBIGINT" >
	\| <#TYPE_UHUGEINT: "UHUGEINT" >
	\| <#TYPE_REAL: "REAL" \| "FLOAT4" \| "FLOAT">
	\| <#TYPE_DOUBLE: "DOUBLE" \| "PRECISION" \| "FLOAT8" \| "FLOAT64">
	\| <#TYPE_VARCHAR: "NVARCHAR" \| "VARCHAR" \| "NCHAR" \| <K_CHAR> \| "BPCHAR" \| "STRING" \| "TEXT" \| <K_CHARACTER> \| "VARYING">
	\| <#TYPE_TIME: "TIMETZ" >
	\| <#TYPE_TIMESTAMP: "TIMESTAMP_NS" \| "TIMESTAMP_MS" \| "TIMESTAMP_S" >

	\| <#TYPE_UUID: "UUID">

[BUG] Compilation error after execution of updateKeywords #2040

Description

Activity

manticore-projects commented on Jul 18, 2024

ssteinhauser commented on Jul 19, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions