You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi,
I've come accross the need to deepcast a nested LUA table that might contain empty tables which should be interpreted either as empty lists or empty maps. There does not seem to be a way to make a distinction yet since the deepcast heuristic does not work on empty table.
For example I'd like to be able to write something along the lines of
One option I see would be to inject a special value via the call to empty_list that gets pattern matched to an empty list in deep_cast but maybe there is a better way?
Perhaps I should add some context about why I need to do that; the goal is to have user-provided scripts that can specify a JSON object via their returned table (maybe the better option is to serialize to a string directly in LUA and deserialize it back in Elixir but I'd like to see if there is an alternative).
So if I understand correctly your idea, the user would have to provide their own list of which keys in the table need to be encoded as empty lists, say
This might be difficult to maintain, especially for nested tables.
After some discussions, one option that came up from Allow differentiation of arrays and objects for proper empty-object serialization was to store the fact that a table needs to be treated as an empty list in Elixir on the metatable. I am not sure if this is possible, let alone a good idea as this would probably move things deep inside luerl?
What do you think about the possibility to specify a static value to deep_cast that gets interpreted as an empty list in the pattern matching, something like
@anjou-low given that empty_list_token could be any Elixir term, I don't see how this would work given that decoded Lua tables will only have encodeable values in them (and thus no atoms, only nil allowed)
Another option for you is that we could have the caller of Lua.Table.deep_cast/1 specify whether or not to treat empty tables as lists or maps. The obvious issue here being that it would uniformly treat all empty tables the same way, no matter which it is.
I'm not clear on your usecase, but you could also get clever with the new Lua.set_private/3 API, storing references to tables and having an internal datastructure that decides how to cast each table based on each reference. I wouldn't want to officially support that, but depending on your usecase, could be possible.
After some discussions, one option that came up from mpx/lua-cjson#11 was to store the fact that a table needs to be treated as an empty list in Elixir on the metatable. I am not sure if this is possible, let alone a good idea as this would probably move things deep inside luerl?
I find it unlikely that this would be accepted into Luerl, but you could ask in the Slack / Discord
Hi, sorry... So we decided to serialize the table to JSON directly in LUA with a library that supports differentiating between empty objects and empty arrays, i.e. {nil} will be interpreted as [].
Maybe I should clarify what we are doing for the sake of completeness but it is ok for me if you close the issue. The goal is to allow users to define webhooks with custom logic in LUA; their scripts should output a JSON response's body and sometimes the distinction between returning {} or [] is important depending of the service calling the webhook.
I think the problem with this approach in our situation is that it requires the writer of the script to maintain the empty_encoder, i.e they have to provide us a way to construct the function to use when we deep_cast their table since we don't know in advance the structure of their data.
This is not something we want to do as this would make scripts too hard to maintain for the end users, especially considering nested keys, i.e. differentiating between ["a"] and ["b", "a"] in the table
{a = {}, b = {a = {}}}
While this could be achieved by passing a list of keys to the empty_encoder, it would quickly make it hard to deal with.
We will stick with doing the JSON serialization on the LUA side for now. I suppose there could be a use for the empty_encoder option in deep_cast but we don't have one yet and if we are the only ones who might need it, I don't want to be responsible for any extra work this might cause you 😛
So feel free to close the issue as is and again thanks for your help!
@anjou-low I think this the wrong way to think about the problem. I agree that your users shouldn't be worried about serialization, and it is instead up to your application to decide how to address this problem, either through schemas, a convention, or some heuristic.
I will move forward with the ability for users to provide a function for deciding how to encode empty values, and leave it there.
Activity
davydog187 commentedon May 19, 2025
The issue here is that there isn't a concept of a list in Lua, it's just a table with integer keys starting at 1, by convention.
The only option here would be to allow you to choose the encoding based on the key.
WDYT?
anjou-low commentedon May 21, 2025
Thanks for your answer and sorry for the delay.
Perhaps I should add some context about why I need to do that; the goal is to have user-provided scripts that can specify a JSON object via their returned table (maybe the better option is to serialize to a string directly in LUA and deserialize it back in Elixir but I'd like to see if there is an alternative).
So if I understand correctly your idea, the user would have to provide their own list of which keys in the table need to be encoded as empty lists, say
This might be difficult to maintain, especially for nested tables.
After some discussions, one option that came up from Allow differentiation of arrays and objects for proper empty-object serialization was to store the fact that a table needs to be treated as an empty list in Elixir on the
metatable. I am not sure if this is possible, let alone a good idea as this would probably move things deep inside luerl?What do you think about the possibility to specify a static value to
deep_castthat gets interpreted as an empty list in the pattern matching, something likedavydog187 commentedon May 21, 2025
@anjou-low given that
empty_list_tokencould be any Elixir term, I don't see how this would work given that decoded Lua tables will only have encodeable values in them (and thus no atoms, onlynilallowed)Another option for you is that we could have the caller of
Lua.Table.deep_cast/1specify whether or not to treat empty tables as lists or maps. The obvious issue here being that it would uniformly treat all empty tables the same way, no matter which it is.I'm not clear on your usecase, but you could also get clever with the new
Lua.set_private/3API, storing references to tables and having an internal datastructure that decides how to cast each table based on each reference. I wouldn't want to officially support that, but depending on your usecase, could be possible.I find it unlikely that this would be accepted into Luerl, but you could ask in the Slack / Discord
davydog187 commentedon Jun 3, 2025
@anjou-low ping :)
anjou-low commentedon Jun 4, 2025
Hi, sorry... So we decided to serialize the table to JSON directly in LUA with a library that supports differentiating between empty objects and empty arrays, i.e.
{nil}will be interpreted as[].Maybe I should clarify what we are doing for the sake of completeness but it is ok for me if you close the issue. The goal is to allow users to define webhooks with custom logic in LUA; their scripts should output a JSON response's body and sometimes the distinction between returning
{}or[]is important depending of the service calling the webhook.Thanks for your help !
davydog187 commentedon Jun 4, 2025
Thanks @anjou-low
I opened #84, does this help?
anjou-low commentedon Jun 4, 2025
I think the problem with this approach in our situation is that it requires the writer of the script to maintain the
empty_encoder, i.e they have to provide us a way to construct the function to use when wedeep_casttheir table since we don't know in advance the structure of their data.This is not something we want to do as this would make scripts too hard to maintain for the end users, especially considering nested keys, i.e. differentiating between
["a"]and["b", "a"]in the tableWhile this could be achieved by passing a list of
keysto theempty_encoder, it would quickly make it hard to deal with.We will stick with doing the JSON serialization on the LUA side for now. I suppose there could be a use for the
empty_encoderoption indeep_castbut we don't have one yet and if we are the only ones who might need it, I don't want to be responsible for any extra work this might cause you 😛So feel free to close the issue as is and again thanks for your help!
davydog187 commentedon Jun 22, 2025
@anjou-low I think this the wrong way to think about the problem. I agree that your users shouldn't be worried about serialization, and it is instead up to your application to decide how to address this problem, either through schemas, a convention, or some heuristic.
I will move forward with the ability for users to provide a function for deciding how to encode empty values, and leave it there.