You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
"Because the traits are all interfaces we can't use operator | for error handling"
I realised that because @catch creates a temporary struct (the various Catch* record structs) that I could attach operator | to those types and make @catch work for K<F, A> types that are Fallible<F> or Fallible<E, F>.
This took me down a massive rabbit hole! So this release has quite a few changes. If you're using v5 then you'll need to pay attention. And, if you're using runtimes, you'll really need to pay attention!
@catch
As, many of you know, in v4 we can @catch errors raised in the Eff<RT, A> and Eff<A> types by using the | operator like a coalescing operator. For example:
publicstaticEff<RT,Unit> main =>from _1 in timeout(60*seconds,longRunning)| @catch(Errors.TimedOut, unit)from _2 inConsole<Eff<RT>,RT>.writeLine("done")
select unit;
This imposes a time-limit on the longRunning operation, which throws a TimedOut error if it doesn't finish in time. It then catches the timeout and continues safely by returning a default value of unit.
There were a number of types that @catch (depending on the overload) could create:
CatchValue<A> - for returning default values (as above)
CatchValue<E, A> - for returning default values (with generic error type)
CatchError - for returning an alternative error
CatchError<E> - for returning an alternative error (with generic error type)
CatchIO<A> - for returning and lifting an IO<A> as the result
CatchIO<E, A> - for returning and lifting an IO<A> as the result (with generic error type)
CatchM<M, A> - for returning and lifting an K<M, A> as the result
CatchM<E, M, A> - for returning and lifting an K<M, A> as the result (with generic error type)
Each one carries a predicate function and an action function. If the predicate returns true for the error raised then the action is run, otherwise the result it left alone. This means a chain of | catch(...) operations can effectively pattern match the errors raised.
Most importantly: the arguments to @catch can make the inference of the generic parameters automatic, so we don't have to manually write @catch<Error, Eff, int>(...) -- this makes catch usable.
Back to the idea that we have a Fallible<E, F> (and Fallible<F> which is equivalent to Fallible<Error, F>). Because, operator declarations can't have generic parameters, all generic parameters must come from the type.
To be able to leverage the Fallible<E, F> trait then we need F (the trait type), E (the error type), and A (the bound value type):
Add support only to CatchM and leave the other Catch* types as non-Fallible supporting
Remove all of the other Catch* types that can't support Fallible
Option 1 would mean that some usages of @catchwould work with Eff<A> but not K<Eff, A>. This felt unsatisfactory.
Option 2 would mean that some of the convenience @catch overrides would have to be removed. So, you couldn't write this anymore:
@catch(Errors.TimedOut, unit)
You'd have to write (one of):
@catch(Errors.TimedOut, SuccessEff(unit))
@catch(Errors.TimedOut,pure<Eff,Unit>(unit))
@catch(Errors.TimedOut, unitEff)// unitEff is a static readonly of SuccessEff
Option 2 is the option I've gone with. The reasons for this are primarily for consistency between the concrete types (Eff<A>) and their abstract pairs (K<Eff, A>), but also...
Every single Fallible type gets to use @catch!
So, previously, @catch only worked for Eff<RT, A>, Eff<A>, and IO<A>. It now works for:
IO<A>
Eff<RT, A>
Eff<A>
Either<L, R>
EitherT<L, M, R>
Fin<A>
FinT<M, A> - more on this later
Option<A>
OptionT<M, A>
Try<A>
TryT<A>
Validation<F, A>
ValidationT<F, M, A>
So now all Fallible types get to use @catch and they all get to use the same set (well, some are specifically for the Error type, like @expected and @exceptional, but other than that they're all the same).
Things to note about this change:
Because @catch is now entirely generic and based around Fallible types, the | operator can only return K<M, A>, so you may need to use .As() if you need to get back to the concrete type.
For catch-all situations, it's better to not use @catch at all, unless you need access to the error value.
MonadIO refactor
The generalisation of catching any errors from Fallible led to me doing some refactoring of the Eff<RT, A> and Eff<A> types. I realised not all errors were being caught. It appeared to be to do with how the IO monad was lifted into the Eff types. In the Monad<M> trait was a function: WithRunInIO which is directly taken from the equivalent function in Haskell's IO.Unlift package.
I decided that was too complicated to use. Every time I used it, it was turning my head inside out, and if it's like that for me then it's probably unusable for others who are not fully aware of unlifting and what it's about. So, I removed it, and UnliftIO (which depended on it).
I have now moved all lifting and unlifting functions to MonadIO:
Monad<M> inherits MonadIO<M>, which isn't how it should be, but because of the limitations of C#'s type system we have all monads expose the MonadIO functionality (otherwise monad-transformers won't work). I'm still thinking through alternative approaches, but I'm a little stumped at the moment. So, for now, there are default implementations for LiftIO and ToIO that throw exceptions. You only implement them if your type supports IO.
LiftIO as most will know, will lift an IO<A> into your monad-transformer.
ToIO is the opposite and will unpack the monad-transformer until it gets to the IO monad and will then return that as the bound value.
For example, this is the implementation for ReaderT:
publicstaticToIO<A>(K<ReaderT<Env, M>, A> ma)=>newReaderT<Env,M,IO<A>>(env => ma.As().runReader(env).ToIO());
So, we run the reader function with the env environment-value, it will return a K<M, A> which we then call ToIO() on to pass it down the transformer stack. Eventually it reaches the IO monad that just returns itself. This means we run the outer shell of the stack and not the inner IO.
That allows methods like MapIO to operate on the IO<A> monad, rather than the <A> within it:
M.ToIO(ma).Bind(io => M.LiftIO(f(io)));
What does this mean?
It means you can call .MapIO(...) on any monad that has an IO monad within it (as long as ToIO has been implemented for the whole stack)
Once we can map the IO we can generalise all of the IO behaviours...
Generalised IO behaviours
The IO<A> monad has many behaviours attached to it:
Local - for creating a local cancellation environment
Post - to make the IO computation run on the SynchronizationContext that was captured at the start of the IO operation
Fork - to make an IO computation run on its own thread
Await - for awaiting a forked IO operation's completion
Timeout - to timeout an IO operation if it takes too long
Bracket - to automatically track resource usage and clean it up when done
Repeat, RepeatWhile, RepeatUntil - to repeat an IO operation until conditions cause the loop to end
Retry, RetryWhile, RetryUntil - to retry an IO operation until successful or conditions cause the loop to end
Fold, FoldWhile, FoldUntil - to repeatedly run an IO operation and aggregating a result until conditions cause the loop to end
Zip - the ability to run multiple IO effects in parallel and join them in a tuppled result.
Many of the above had multiple overrides, meaning a few thousand lines of code. But, then we put our IO monad inside monad-transformers, or encapsulate them inside types like Eff<A> and suddenly those functions above are not available to us at all. We can't get at the IO<A> monad within to pass as arguments to the IO behaviours.
That's where MapIO comes in. Any monadic type or transformer type that has implemented ToIO (and has an IO<A> monad encapsulated within) can now directly invoke those IO behaviours. And not only that, they can be fully generalised:
So, to call the IO<A>.Timeout function for the IO<A> monad buried within K<M, A> we simply call MapIO to get the io monad an then use it to invoke our IO behaviour. It then automatically gets wrapped back up inside a K<M, A> monad.
This means every single IO behaviour is now available to you as soon as you make a type that encapsulates the IO monad. Again, as long as ToIO is implemented.
To avoid name-clashes with existing methods and functions, these are the generic behaviours:
LocalIO and Prelude.localIO
PostIO and Prelude.postIO
ForkIO and Prelude.forkIO
TimeoutIO and Prelude.timeoutIO
BracketO and Prelude.bracketIO
RepeatIO, RepeatWhileIO, RepeatUntilIO and Prelude.repeatIO, repeatWhileIO, repeatUntilIO
RetryIO, RetryWhileIO, RetryUntilIO and Prelude.retryIO, retryWhileIO, retryUntilIO
FoldIO, FoldWhile, FoldUntil and Prelude.foldIO, foldWhileIO, foldUntilIO
ZipIO and Prelude.zipIO
Deleted Eff extensions
All of the above means that Eff<RT, A> and Eff<A> don't nee their own Fork, Repeat, Retry, Zip, etc. extensions or prelude functions. So, they've been deleted. But note, that will have the following fallout:
The new extensions and functions all end with "IO" and so you'll need to fix up any errors
The new extensions and functions all return the generic K<Eff, A> and K<Eff<RT>, A>, so you may need strategic use of .As() to get back to a concrete type.
Eff gone back to ReaderT backed rather than StateT
When refactoring the Eff monads for v5 I decided to switch to use the StateT<RT, IO, A> transformer-stack as the underlying implementation. The problem with this is that the lifted IO monad isn't an IO<A> it's an IO<(A, S)> (because we need to return the updated state). Unfortunately, that means we can't implement ToIO for StateT because we'd lose the updated state if we mapped the resulting IO<(A, S)> to IO<A>; breaking the soundness of the StateT monad and probably bringing in other unexpected side-effects.
So if Eff<RT, A> and Eff<A> are going to be able to leverage these new generalised IO behaviours (from the last section), then they have to be implemented with ReaderT. If IO<A> is lifted into a ReaderT then it can yield an IO<A> in any ToIO implementation, which makes it sound.
Statefullness in runtimes (next part of the rabbit hole...)
The reason I decided to make Eff<RT, A> use a StateT transformer before was because I wanted the runtimes (RT) to allow for stateful behaviour. And so, going back to being a ReaderT meant that the Reads and Mutates traits could no longer work (because they both depended on Stateful, which is the generalised state mutation trait).
The following refactorings have happened:
The Has trait has now gone static
The Trait property is now called Ask
That means updating your runtimes to change .Trait to .Ask and to make the implementations static
The Reads trait has now been deleted, it ended up being the same as Has
The Mutates trait now derives from Has and provides a Mutable property that exposes an Atom<InnerEnv>
The idea being that your 'runtime' environment contains mutable values that can be atomically updated via Mutates
But, you can also access an atomic snapshot via Ask
This, I think, is the best way to allow for mutation in a 'readable' environment
A new trait called Local
This formalises the local-scope changes to the runtime that a Readable type can do via Readable.local(f, ma)
It also derives from Has
Which means you can Local.with(f, ma) to create a localised runtime and Has.ask to access the current value
The one piece of code in all of language-ext that needed this local-state was Activity<M, RT>. So, here's some of it, so you can see what's changed:
We're constraining the runtime, RT, to Has<M, ActivitySourceIO> and Local<M, ActivityEnv>
That means we can access the ActivitySourceIO interface by using: Has<M, RT, ActivitySourceIO>.ask
We can also access the current ActivityEnv by using Has<M, RT, ActivityEnv>.ask
And it means we can create a locally scoped runtime with Local.with<M, RT, ActivityEnv, A>(f, operation)
Note that Has<M, VALUE> is not the same as Has<M, Env, VALUE>
The first one is the trait. We use that to say there's a property, called Ask, that will return a K<M, VALUE> value.
This allows for access to the traits and configuration values (from the runtime)
However, because you will likely implement multiple traits and use multiple traits to refine your RT types in generalised code, it's not possible to call RT.Ask without there being type-system ambiguities.
So, the second Has type: Has<M, Env, VALUE>, has been added to resolve those ambiguities
This is the implementation of Has<M, Env, VALUE> implementation:
Because it constrains to only a single trait (Has<M, VALUE>) it can call Env.Ask and have it resolve unambiguously. This has the added benefit that the K<M, VALUE> value, that it reads from your runtime, will be cached. That will minimise allocations in effectful code.
So, this whole system now completely generalises the idea of environment access and scoping as well as mutation. It also means you don't need to carry around either the Stateful or Readable traits in your generalised effectful code.
Here's an example, before and after, from the Newsletter sample project:
Has been refactored to use these new traits and constraints.
FinT - new monad transformer
There's a new monad transformer, FinT, which is the transformer version of Fin. It was only of the last of the monadic types not to have a transformer pair, so that's now done.
Conclusion
This just confirms why keeping the project in a beta state for now makes sense. My real-world usage of the beta is bringing up these areas of improvement and allows me to bring them in without causing multiple migration problems! Any questions feel free to discuss below...
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
In the last release I wrote this:
I realised that because
@catch
creates a temporarystruct
(the variousCatch*
record structs) that I could attachoperator |
to those types and make@catch
work forK<F, A>
types that areFallible<F>
orFallible<E, F>
.@catch
As, many of you know, in
v4
we can@catch
errors raised in theEff<RT, A>
andEff<A>
types by using the|
operator like a coalescing operator. For example:This imposes a time-limit on the
longRunning
operation, which throws aTimedOut
error if it doesn't finish in time. It then catches the timeout and continues safely by returning a default value ofunit
.There were a number of types that
@catch
(depending on the overload) could create:CatchValue<A>
- for returning default values (as above)CatchValue<E, A>
- for returning default values (with generic error type)CatchError
- for returning an alternative errorCatchError<E>
- for returning an alternative error (with generic error type)CatchIO<A>
- for returning and lifting anIO<A>
as the resultCatchIO<E, A>
- for returning and lifting anIO<A>
as the result (with generic error type)CatchM<M, A>
- for returning and lifting anK<M, A>
as the resultCatchM<E, M, A>
- for returning and lifting anK<M, A>
as the result (with generic error type)Each one carries a predicate function and an action function. If the predicate returns
true
for the error raised then the action is run, otherwise the result it left alone. This means a chain of| catch(...)
operations can effectively pattern match the errors raised.Back to the idea that we have a
Fallible<E, F>
(andFallible<F>
which is equivalent toFallible<Error, F>
). Because, operator declarations can't have generic parameters, all generic parameters must come from the type.To be able to leverage the
Fallible<E, F>
trait then we needF
(the trait type),E
(the error type), andA
(the bound value type):Only one of the
Catch*
record structs has all of those generics:CatchM<E, M, A>
- for returning and lifting anK<M, A>
as the result (with generic error type)So, that's the only type that can support an
operator |
that can work withFallible<E, M>
:So, I had a couple of options:
CatchM
and leave the otherCatch*
types as non-Fallible supportingCatch*
types that can't support FallibleOption 1 would mean that some usages of
@catch
would work withEff<A>
but notK<Eff, A>
. This felt unsatisfactory.Option 2 would mean that some of the convenience
@catch
overrides would have to be removed. So, you couldn't write this anymore:You'd have to write (one of):
Option 2 is the option I've gone with. The reasons for this are primarily for consistency between the concrete types (
Eff<A>
) and their abstract pairs (K<Eff, A>
), but also...Every single
Fallible
type gets to use@catch
!So, previously,
@catch
only worked forEff<RT, A>
,Eff<A>
, andIO<A>
. It now works for:IO<A>
Eff<RT, A>
Eff<A>
Either<L, R>
EitherT<L, M, R>
Fin<A>
FinT<M, A>
- more on this laterOption<A>
OptionT<M, A>
Try<A>
TryT<A>
Validation<F, A>
ValidationT<F, M, A>
So now all
Fallible
types get to use@catch
and they all get to use the same set (well, some are specifically for theError
type, like@expected
and@exceptional
, but other than that they're all the same).Things to note about this change:
@catch
is now entirely generic and based aroundFallible
types, the|
operator can only returnK<M, A>
, so you may need to use.As()
if you need to get back to the concrete type.@catch
at all, unless you need access to the error value.MonadIO
refactorThe generalisation of catching any errors from
Fallible
led to me doing some refactoring of theEff<RT, A>
andEff<A>
types. I realised not all errors were being caught. It appeared to be to do with how theIO
monad was lifted into theEff
types. In theMonad<M>
trait was a function:WithRunInIO
which is directly taken from the equivalent function in Haskell'sIO.Unlift
package.I decided that was too complicated to use. Every time I used it, it was turning my head inside out, and if it's like that for me then it's probably unusable for others who are not fully aware of unlifting and what it's about. So, I removed it, and
UnliftIO
(which depended on it).I have now moved all lifting and unlifting functions to
MonadIO
:LiftIO
as most will know, will lift anIO<A>
into your monad-transformer.ToIO
is the opposite and will unpack the monad-transformer until it gets to theIO
monad and will then return that as the bound value.For example, this is the implementation for
ReaderT
:So, we run the reader function with the
env
environment-value, it will return aK<M, A>
which we then callToIO()
on to pass it down the transformer stack. Eventually it reaches theIO
monad that just returns itself. This means we run the outer shell of the stack and not the innerIO
.That allows methods like
MapIO
to operate on theIO<A>
monad, rather than the<A>
within it:What does this mean?
.MapIO(...)
on any monad that has anIO
monad within it (as long asToIO
has been implemented for the whole stack)Generalised IO behaviours
The
IO<A>
monad has many behaviours attached to it:Local
- for creating a local cancellation environmentPost
- to make the IO computation run on theSynchronizationContext
that was captured at the start of the IO operationFork
- to make an IO computation run on its own threadAwait
- for awaiting a forked IO operation's completionTimeout
- to timeout an IO operation if it takes too longBracket
- to automatically track resource usage and clean it up when doneRepeat
,RepeatWhile
,RepeatUntil
- to repeat an IO operation until conditions cause the loop to endRetry
,RetryWhile
,RetryUntil
- to retry an IO operation until successful or conditions cause the loop to endFold
,FoldWhile
,FoldUntil
- to repeatedly run an IO operation and aggregating a result until conditions cause the loop to endZip
- the ability to run multiple IO effects in parallel and join them in a tuppled result.Many of the above had multiple overrides, meaning a few thousand lines of code. But, then we put our
IO
monad inside monad-transformers, or encapsulate them inside types likeEff<A>
and suddenly those functions above are not available to us at all. We can't get at theIO<A>
monad within to pass as arguments to the IO behaviours.That's where
MapIO
comes in. Any monadic type or transformer type that has implementedToIO
(and has anIO<A>
monad encapsulated within) can now directly invoke those IO behaviours. And not only that, they can be fully generalised:So, to call the
IO<A>.Timeout
function for theIO<A>
monad buried withinK<M, A>
we simply callMapIO
to get theio
monad an then use it to invoke our IO behaviour. It then automatically gets wrapped back up inside aK<M, A>
monad.This means every single IO behaviour is now available to you as soon as you make a type that encapsulates the IO monad. Again, as long as
ToIO
is implemented.To avoid name-clashes with existing methods and functions, these are the generic behaviours:
LocalIO
andPrelude.localIO
PostIO
andPrelude.postIO
ForkIO
andPrelude.forkIO
TimeoutIO
andPrelude.timeoutIO
BracketO
andPrelude.bracketIO
RepeatIO
,RepeatWhileIO
,RepeatUntilIO
andPrelude.repeatIO
,repeatWhileIO
,repeatUntilIO
RetryIO
,RetryWhileIO
,RetryUntilIO
andPrelude.retryIO
,retryWhileIO
,retryUntilIO
FoldIO
,FoldWhile
,FoldUntil
andPrelude.foldIO
,foldWhileIO
,foldUntilIO
ZipIO
andPrelude.zipIO
Deleted
Eff
extensionsAll of the above means that
Eff<RT, A>
andEff<A>
don't nee their ownFork
,Repeat
,Retry
,Zip
, etc. extensions or prelude functions. So, they've been deleted. But note, that will have the following fallout:K<Eff, A>
andK<Eff<RT>, A>
, so you may need strategic use of.As()
to get back to a concrete type.Eff
gone back toReaderT
backed rather thanStateT
When refactoring the
Eff
monads forv5
I decided to switch to use theStateT<RT, IO, A>
transformer-stack as the underlying implementation. The problem with this is that the liftedIO
monad isn't anIO<A>
it's anIO<(A, S)>
(because we need to return the updated state). Unfortunately, that means we can't implementToIO
forStateT
because we'd lose the updated state if we mapped the resultingIO<(A, S)>
toIO<A>
; breaking the soundness of theStateT
monad and probably bringing in other unexpected side-effects.So if
Eff<RT, A>
andEff<A>
are going to be able to leverage these new generalised IO behaviours (from the last section), then they have to be implemented withReaderT
. IfIO<A>
is lifted into aReaderT
then it can yield anIO<A>
in anyToIO
implementation, which makes it sound.Statefullness in runtimes (next part of the rabbit hole...)
The reason I decided to make
Eff<RT, A>
use aStateT
transformer before was because I wanted the runtimes (RT
) to allow for stateful behaviour. And so, going back to being aReaderT
meant that theReads
andMutates
traits could no longer work (because they both depended onStateful
, which is the generalised state mutation trait).The following refactorings have happened:
Has
trait has now gonestatic
Trait
property is now calledAsk
.Trait
to.Ask
and to make the implementationsstatic
Reads
trait has now been deleted, it ended up being the same asHas
Mutates
trait now derives fromHas
and provides aMutable
property that exposes anAtom<InnerEnv>
Mutates
Ask
Local
Readable
type can do viaReadable.local(f, ma)
Has
Local.with(f, ma)
to create a localised runtime andHas.ask
to access the current valueThe one piece of code in all of language-ext that needed this local-state was
Activity<M, RT>
. So, here's some of it, so you can see what's changed:Things to note:
RT
, toHas<M, ActivitySourceIO>
andLocal<M, ActivityEnv>
ActivitySourceIO
interface by using:Has<M, RT, ActivitySourceIO>.ask
ActivityEnv
by usingHas<M, RT, ActivityEnv>.ask
Local.with<M, RT, ActivityEnv, A>(f, operation)
Has<M, VALUE>
is not the same asHas<M, Env, VALUE>
Ask
, that will return aK<M, VALUE>
value.RT
types in generalised code, it's not possible to callRT.Ask
without there being type-system ambiguities.Has
type:Has<M, Env, VALUE>
, has been added to resolve those ambiguitiesThis is the implementation of
Has<M, Env, VALUE>
implementation:Because it constrains to only a single trait (
Has<M, VALUE>
) it can callEnv.Ask
and have it resolve unambiguously. This has the added benefit that theK<M, VALUE>
value, that it reads from your runtime, will be cached. That will minimise allocations in effectful code.So, this whole system now completely generalises the idea of environment access and scoping as well as mutation. It also means you don't need to carry around either the
Stateful
orReadable
traits in your generalised effectful code.Here's an example, before and after, from the
Newsletter
sample project:Before
After
LanguageExt.Sys
Has been refactored to use these new traits and constraints.
FinT
- new monad transformerThere's a new monad transformer,
FinT
, which is the transformer version ofFin
. It was only of the last of the monadic types not to have a transformer pair, so that's now done.Conclusion
This just confirms why keeping the project in a beta state for now makes sense. My real-world usage of the beta is bringing up these areas of improvement and allows me to bring them in without causing multiple migration problems! Any questions feel free to discuss below...
This discussion was created from the release IO and effects refactoring.
Beta Was this translation helpful? Give feedback.
All reactions