Skip to content

Spike on chrysalis idea - #6

Closed
stevenproctor wants to merge 4 commits into
mainfrom
spike-on-chrysalis-idea
Closed

stevenproctor wants to merge 4 commits into
mainfrom
spike-on-chrysalis-idea

Conversation

@stevenproctor

Copy link
Copy Markdown
Contributor

No description provided.

@stevenproctor
stevenproctor changed the base branch from main to error-checking February 21, 2023 03:39
@stevenproctor

Copy link
Copy Markdown
Contributor Author

Targeting the error-checking branch cleans up some of the ideas here.

I will ultimately target main after error-checking goes into main.

(catch #?(:clj Throwable :cljs :default) err
(transition ctx err))))))
ctx)
f (or (stage ix) identity)]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed redundant let.

Comment thread src/lambda_toolshed/papillon/async.cljc Outdated
(ns lambda-toolshed.papillon.async)

(defprotocol Chrysalis
(eclose [this handler]))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eclose
intransitive verb
ēˈklōz
-ed/-ing/-s
of an insect
: to emerge from the eggshell or pupal case

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

linguistically it works, though does have me wondering if sticking too close to the entomological theme within the library, might cause/increase friction for non-native English speakers.

or cross pollination with finance industry terms like "e-close"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"or cross pollination with ..."
I see what you did there.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But isn't it the Clojure way to find an underutilized word, give a definition, and get it into more mainstream usage in the Clojure community? 😜

@1100001-jones 1100001-jones Feb 27, 2023 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, a lot of lib/repos do a similar thing. To be clear, Chrysalis is probably fine, my concern is more around eclose and how it might lead to human parsing errors. On a quick read can be read as close instead, leading them to thinking it's about closing or dismissing a thing, instead of realizing or completion of a thing.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am okay with a different/better name. I am even open to renaming the protocol.

I went with these to avoid conceptual collisions with other libraries' ideas of having an "Async Context" to let it stand independently.

I wanted to ensure they were distinct enough that someone didn't confuse this protocol with something else. 😉

@stevenproctor stevenproctor Mar 15, 2023 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any objections to emerge? Or is that too close to other functions like eclose is?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

emerge can work, it's a clear enough term that it'd be a lot less likely to cause language parsing errors.

if emerge doesn't quite click, the synonyms surface or materialize might be options.

yeah, finding the best term here is tough, when time allows I've been digging through CS literature to try and find the best answer (most things just use "async"). the problem of, how do we describe the context of asynchronous operations in a unique but also still related way.

something like allochronic (https://www.merriam-webster.com/dictionary/allochronic) might work in place of async, though from the definition is more about geologic time. possible that allochronic it's esoteric enough in this context that it would also cause confusion.

I'll keep looking just in case.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AsyncSystem might also work for the protocol
https://en.wikipedia.org/wiki/Asynchronous_system

Comment thread src/lambda_toolshed/papillon.cljc Outdated
([ixs]
(execute ixs {}))
([ixs ctx]
#?(:org.babashka/nbb nil

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't give the 2-arity to nbb as nbb doesn't support core.async

CLJ and CLJS get the nice behavior we see in the tests about being able to get async results on a channel.

Comment thread src/lambda_toolshed/papillon.cljc Outdated
result (leave (enter ctx))]
(if (satisfies? ReadPort result)
(present-async result)
(if (satisfies? Chrysalis result)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I like this...

Part of me says:

If using callback functions, we should always present via callback functions, even if it ends up being all synchronous.

The other part:

It is kinda nice to piggyback off the callback version, as that means we test that version when using the 2-arity version in our tests. Also 🤢🤮 of callback functions when something is completely synchronous.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still trying to think through the best paths for all this. Like, is there a way to reasonably represent these concepts as part of the context map through reserved/predefined namespaced keys? nbb reader conditional can still happen, could be two separate inside of present-async.

Taking the path of "sane defaults, overwritable in user land" is a thing, always triggering callback functions if present does make sense. Don't want that functionality, don't set them.

Also, if they're represented as keys on the context, it allows for dynamically removing or adding them as needed during execution.

With the base problem being how best to represent an agnostic interface for async mechanisms, it has me wondering what path reduces the most conceptual complexity.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The imposition of callback fns for completely synchronous work is a lot of incidental complexity that is not present in alternative interceptor libs.

Regarding the desire to support alternate implementations of deferred computations, has there been any investigation of an approach using requiring-resolve and multiple alternative impl namespaces (all implementing the Chrysalis protocol) that could be optionally added as an additional artifact? So require lambda-toolshed/papillon and lambda-toolshed/papillon-manifold. In l-t/execute you could provide an optional parameter to override the default (core.async) and bring in an alternate implementation.

Apologies if this has already been considered...

@1100001-jones 1100001-jones Feb 22, 2023 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting 🤔

Pretty sure we chatted about something similar, but weren't really sure about how to represent it. one example @stevenproctor was looking at was AsyncContext from sieppari.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow... that AsyncContext in sieppari is exactly what I had in mind, with the addition of separate artifacts to complement add-ons like manifold.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the term is "Same Same, but Different." 😉

I looked at it, and it was part of the reason I finally went with a distinct protocol.

I opted not to do catch and continue because of our error handling with core.async channels, and I went with only having eclose instead.

I put the core in async, although maybe if we decide to return it, we can move it to the main namespace. For extending ReadPort with the protocol, I put that in the lambda-toolshed.papillon.async.core_async namespace, which the tests require to load that functionality to get channels.

I left out manifold as I didn't want to put a dependency on manifold in the project, as those who want it could extend it themselves.

Given how we are currently handling the pure sync version, I was also unsure if we would want a blocking take similar to the AsyncContext of sieppari.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My endorsement of sieppari's approach was limited to the protocol + extension to "built-in" types + (presumably) external artifacts/customization to extend to other defereds. I did not mean to endorse catch/continue and TBH I have not dug deep enough into either sieppari's implementation or ours to render a useful opinion on the shape of the protocol.

(I will try to dig deeper into the Chrysalis implementation this weekend).

I strongly agree with leaving out any third party dependencies.

I agree that a blocking take "behind the scenes" in the sync path is to be avoided.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The imposition of callback fns for completely synchronous work is a lot of incidental complexity that is not present in alternative interceptor libs.

Yeah, that is what I was thinking, but it did feel off that the result could be given via callback or return value, and the person would have to know which case they are in.

The other options that came to mind were:

  1. don't piggyback the 2-arity and the 4-arity; if you use the 4-arity version, you always live in callback land. 😕
  2. to return the result and check if Chrysalis protocol, and have them call eclose with their handler (this was made cleaner by the transition from the error-handling branch, as before it took both the initial-context and the candidate context.
  3. always return a Chrysalis, and make the raw context object (either the current map or a new record type) be shared so that eclose returns itself
  4. return a Chrysalis as above, and give a blocking take that does identify if it was in synchronous execution.
  5. some other option that you all might think of

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, numbers 3/4 involve wrapping sync behavior in Chrysalis. That's a reasonable solution as long as we stick to plain maps for context (with magic type metadata) not some custom type and we don't transition to async and then "fake it" with a blocking take -that will present an unexpected debugging nightmare.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed some changes that refined some of this thinking.

I now think of present-async as an abstraction, and we can present-promise, present-channel, or some other function if given in the context under the key :lambda-toolshed.papillon/present-async.

present-promise is only available in ClojureScript.

present-channel is not available in nbb.

The present-async default, if not given, is:

  • present-promise if in nbb,
  • present-channel otherwise.

(is (empty? (::ix/queue res)))
(is (empty? (::ix/stack res)))
(is (= (ex-message the-exception) (ex-message (res ::error))))
(is (= expected-log (::ix/trace res))))))))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After some feedback on how it should behave, I will add some tests for the 4-arity version.

@stevenproctor

stevenproctor commented Feb 21, 2023 •

Copy link
Copy Markdown
Contributor Author

This is for #2 to think about supporting a Clojure runtime where core.async doesn't exist.

Also, it helps support the world where people might choose Manifold, Promesa, or something else they already have with their project.

Base automatically changed from error-checking to main February 28, 2023 01:56
Give ability to specify how to present an async result as the
return value of the function in async world
@stevenproctor
stevenproctor force-pushed the spike-on-chrysalis-idea branch from 27c3dee to 1c04dd6 Compare February 28, 2023 02:19
@stevenproctor stevenproctor mentioned this pull request Feb 28, 2023
@stevenproctor
stevenproctor marked this pull request as ready for review March 29, 2023 01:03
@stevenproctor

Copy link
Copy Markdown
Contributor Author

I think this should be ready for a more "final review"

@cch1 cch1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reasonable way to break this PR into pieces? I'm thinking one piece to generalize async support through Chrysalis and a PR to support babashka.

(update ctx :number #(* % 2)))})

;; Run an interceptor chain with one interceptor in it that is synchronous
;; Run an interceptor chain with two interceptor in it that is synchronous

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plural

goog.string.format))))
(:require #?@(:cljs ([goog.string :as gstring]
goog.string.format))
#?(:org.babashka/nbb []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we concluded bb does support core async...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nbb not bb. 😉

(emerge result #(present-async % presenter))
(presenter result)))

#?(:cljs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm bothered by the lack of generalization of the "present" concept that results in two diametric fn names. Can't present-channel and present-promise be unified in one fn whose implementation differs across platforms?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea was that it would allow someone else to give a different presenter (e.g. manifold or promesa), or pick if they want promises vs channels if in CLJS.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would present-as-promise and present-as-channel be better names?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a way to achieve both? unification of terms, while maintaining optionality/composability? possibly protocol driven?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, my belief/hope is that the terms can be unified. It's not a question of naming -it's the generalization (or lack thereof).

([ixs ctx]
(let [ixs (map-indexed namer ixs)
ctx (init-ctx ctx ixs)
present-async (::present-async ctx #?(:org.babashka/nbb present-promise :default present-channel))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See previous comment about unification of present.

@stevenproctor

Copy link
Copy Markdown
Contributor Author

I can look into breaking it into two.

The async generalization came from trying to support nbb, but I think I should be able to split the two in a way that makes sense.

@stevenproctor

Copy link
Copy Markdown
Contributor Author

Is there a reasonable way to break this PR into pieces? I'm thinking one piece to generalize async support through Chrysalis and a PR to support babashka.

PR #7 opened.

Also, the presentation is for nbb (not babashka? node babashka?), not babashka itself per se.

The presentation also would help some with the tests that were updated in #7 on the CLJS side, which had to do a p->c call to get the Promise back to a channel for sake of testing.

@cch1 cch1 closed this Oct 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants