run a step inside a go routine

[af-md]

closes #90

Summary

Adds support for running steps inside goroutines with deterministic step ID generation.

This is by no means the final solution, it's a PR to get feedback.

Problem

Currently, running steps inside goroutines causes non-deterministic step ID generation due to race conditions:

Solution

Pre-generate step IDs before launching goroutines:

• Added Go() function that pre-generates step IDs and runs steps in goroutines
• Added WithNextStepID option to pass pre-generated IDs to RunAsStep
• Modified RunAsStep to use pre-generated IDs when provided

    result, err := dbos.Go(ctx, func(ctx context.Context) (string, error) {
        return processItem(ctx, item)
    })

Open Questions

• Does this satisfy the UX you had in mind @maxdml ? I am a bit unsure as to what you had in mind but gave it a go anyway.
• Might be worth giving me fuller example of how you think the UX would work inside the workflow

ToDos

• I still need to add some docs
• Still need to write tests for it as well

[maxdml]

@af-md thanks for the PR!

The approach has the right fundamentals, i.e., dbos.Go generates a step ID and calls RunAsStep in a goroutine.

I think we can make this simpler by simply having a package level Go() that calls the package level RunAsStep inside a goroutine. That way we don't have to repeat all the logic from the package level RunAsStep into Go.

With respect to what dbos.Go returns to the user: it should be non blocking and return to the caller a channel (typed with the output type of the step). That way I can write:

chans = make([]chan int, 3)
for := range 10 {
    resChan1, err := dbos.Go(ctx, StepFnClosure)
    // handle err
}

// Read from each channel here

The res channel can hold types similar to the workflow outcome chan:

type stepOutcome[R any] struct {
	result R
	err    error
}

[af-md]

@maxdml does this feature have any conflict with what @apoliakov said about pre generating stepIDs: https://discord.com/channels/1156433345631232100/1166779411920597002/1413954852618244267

It makes sense to run steps inside Go routines - as they tend to be better performant compared to standard execution - however the users should be advised to write their code to wait for a step to complete (committed into DB) and then move onto the next step? probably that's what you were thinking of anyway...

[apoliakov]

@maxdml does this feature have any conflict with what @apoliakov said about pre generating stepIDs: https://discord.com/channels/1156433345631232100/1166779411920597002/1413954852618244267

It makes sense to run steps inside Go routines - as they tend to be better performant compared to standard execution - however the users should be told to write their code to wait for a step to complete (committed into DB) and then move onto the next step? probably that's what you were thinking of anyway...

Ah... what I said was a comment on how Python works. Here we may have an opportunity to make it act differently. But Max or Peter will need to opine on that

[maxdml]

@maxdml @apoliakov there is a small misunderstanding here. The problem we are solving with this PR is the non deterministic generation of stepIDs, resulting from the execution of steps in goroutines.

What this PR will do is to serialize the generation of step IDs from within the workflow. That way, step IDs will be generated deterministically, before the RunAsStep code executes, and regardless of the order in which they complete, they'll always have the same stepID.