Custom Codec doesn't work with DOMAIN types

asyncpg version: 0.18.3
PostgreSQL version: 11.3
Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?: Local
Python version: 3.7.3
Platform: MacOS
Do you use pgbouncer?: No
Did you install asyncpg with pip?: Yes
If you built asyncpg locally, which version of Cython did you use?: PyPI
Can the issue be reproduced under both asyncio and
uvloop?: Don't Know

When creating a DOMAIN, and using a custom codec, it calls the native type's codec, as opposed to the custom code. Here is the test code that is supposed to test for this:

https://github.com/MagicStack/asyncpg/blob/master/tests/test_codecs.py#L1224-L1239

However, the lambda functions aren't called. It appears the data type is recognized as an int.

Postgres seems to always report the base type OID in the RowDescription message. Interestingly, ParameterDescription contains the correct domain OID, so we can set up a custom encoder, but not a custom decoder. Given this, I think we should declare custom codecs on domains to be unsupported altogether.

Relevant threads on pgsql-hackers:

https://www.postgresql.org/message-id/flat/27307.1047485980%40sss.pgh.pa.us#b8053ec75f4e5c33d60d43fe5e3f15b8

https://www.postgresql.org/message-id/flat/9DB5FAA2-A800-11D7-866B-0030656EE7B2%40icx.net#96c06d0a3f45f1125c87a4f67c749739

Original commit: postgres/postgres@d9b679c

That’s unfortunate. I’m sure you’ve gotten this before, however, with some rules on the decoder function couldn’t we support composite types?

If we removed this check:

asyncpg/asyncpg/connection.py

Lines 968 to 971 in f4aa9bf

    
           if not introspection.is_scalar_type(typeinfo): 
        
               raise ValueError( 
        
                   'cannot use custom codec on non-scalar type {}.{}'.format( 
        
                       schema, typename))

And required the decoder function to do something like x.strip('()').split(','), as well as making the decoder handle Python type conversion, couldn’t asyncpg support composites?

Text I/O for composites is gnarly and wouldn't work very well, since asyncpg would refuse to do text I/O for the component types. Theoretically, this might be possible using the 'tuple' format protocol, but there's quite a few changes to be made to make this work, and I don't see a compelling use case for it.

I hear your point around the text I/O processing. The use case was around having a composite type so that one can input the result into a dataclass. row_to_json works for now, as the json type can be registered and decoded to a dictionary. However, it is beneficial to have errors if what is returned doesn't fit into a specific structure. I can close this if you don't think there is value here.

This issue is originally about domains, and that still needs to be resolved properly, so let's keep this open.

Regarding custom codecs for composite types -- feel free to open a separate issue (with the description of your use case). It's not impossible to implement and there's some value in that, so maybe someone will chime in as well.

Right! Sorry, my slow responses made me lose sight of the originating issue. I'll create a new ticket.

Prohibit custom codecs on domains

asyncpg v0.22.0

Navigation Menu

Repository navigation

Custom Codec doesn't work with DOMAIN types #457

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if not introspection.is_scalar_type(typeinfo):
	raise ValueError(
	'cannot use custom codec on non-scalar type {}.{}'.format(
	schema, typename))

Custom Codec doesn't work with DOMAIN types #457

Description

Activity

elprans commented on Jul 8, 2019

elprans commented on Jul 8, 2019

kjmph commented on Jul 8, 2019

elprans commented on Oct 3, 2019

kjmph commented on Oct 29, 2019

elprans commented on Oct 29, 2019

kjmph commented on Oct 29, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions