Data Structure & Better APIs

[alanpeabody]

Overview

This library has been great for building and maintaining large (API) applications in Elixir. However a few patterns have emerged that could be handled better.

Typical current use cases identified are:

1. A simple create/update/delete of something.
2. 1 + Afterward syncing to a socket.
3. 1 (or 2) plus calling some async interactor in the background.
4. Use multis for more complicated work flows

Current pain points are:

• After callbacks often really only relevant to success cases, but have to define both
• After callbacks sometimes need more then just the results
• No good way of hooking interactors together
• Lack of strict data structure gives me feeling of malaise.

Some Ideas

Lets look at Plug.Conn, Plug.Builder and Plug.Router for inspiration!

%Interaction{
  assigns: %{},
  results: nil,
  success: false,
}

defmodule MyApp.CreateComment do
  use Interactor.Builder

  interaction MyApp.Auth.authorize, :create_comment
  interaction :comment_changeset
  interaction :insert, assign_as: :comment #built in! # but how does it know what to insert is it always from "result"?
  interaction :sync_to_socket, async: true
  interaction MyApp.UpdatePostCount, async: true
  
  def comment_changeset(%Interaction{assigns: %{attributes: attrs}}}) do
    %Comment{}
    |> cast(attrs, [:title, :body])
    |> Comment.validate # if non Interaction is returned it is put into results?
  end

  # Comment now both is result and is added to assigns.
  def sync_to_socket(int = %{Interaction{results: %Comment{}, assigns: %Comment{}}}) do
    # Push to socket.
  end
end

defmodule CommentController do
  # ...
  def create(conn, params) do
    case Interactor.call(CreateComment, %{attributes: params["comment"]}) do
      %{success: true, results: comment} -> render(conn, :show, data: comment)
      %{success: false, results: changeset} -> render(conn, :errors, data: changeset)
    end
  end
  # ...
end

Breakdown of example

Idea is the same sort of pattern as plug. In particular you have an %Interaction{} data structure, then each Interactor really is just a function or module with call/2 function that accept the interaction and the opts and returns the Interaction.

All interaction chains are started with Interactor.call/2 (/3?) which creates the %Interaction{} and calls call/2.

With Interactor.Builder you get some convenience macros similar to plug builder. In particular you have the interaction macro which runs interactions in order as long as the previous interaction did not fail. In addition it gives you some options such as :assign_as and :async which provide simple patterns for common behavior. In addition when an interaction is not returned the return value becomes the result in the interaction.

Built in interaction functions :insert, :update and :transaction also handle standard Repo calls and updating the interaction for you.

[alanpeabody]

Just an idea I had today. /cc @beerlington @totallymike

[totallymike]

I like this 👍

It seems that assigns ends up being the arguments of the first call, if I'm correct. From then on it is modified as it's passed through interactors.

The success is useful as well, though one could argue for the standard :ok/:error tuple. What do you think?

Another factor I'm considering is non-Ecto interactors. I see interactors being tremendously useful for chaining a sequence of API calls. I propose moving the :insert/:update/:transaction interactors into either a separate module to be used, or parameterizing their inclusion. That said, they do no harm so long as Ecto isn't a requirement.

Finally, what are the semantics around failures? I'm interested in the rollback story.

[alanpeabody]

@totallymike, I think if you return from an {:ok, thing} or {:error, thing} it would set the status and return value appropriately.

I don't think there needs to be an Ecto dependency, repo should maybe just be passed into the interaction as an opt.

I don't really have a great rollback/failure story besides that once an %Interaction{success: false} we stop calling the next on, like halt on conn. What are your needs/requirements for it?

[alanpeabody]

Alternative:

every step adds to the assigns the key of the function name or the :assign_as value. Things like built in update/insert would then take a :source opt or something.

eg:

interaction :comment_changeset
interaction :insert, source: :comment_changeset, assign_as: :comment
interaction :sync_to_websocket, async: true

The only problem with this would be not having a standard result field to access in controllers etc unless you standardized on something like assign_as: :data.

[totallymike]

I like the :assign_as feature. I agree that the difference between result and assigns, once the full interaction has finished, is a bit vague, especially around interactor return semantics. Probably just declaring that the return value of the final step is sufficient.

It does get a bit muddled, however, when you consider ancillary use cases for an after_call, say, logging. Though we could steal ideas from Phoenix.Endpoint and add instrumentation to Interactor as well, and then mount instrumentors for logging/stats. To clarify, something like this:

defmodule MyInteractor do
  use Interactor.Builder

  # In Phoenix, these are configured via config/config.exs, but I'll put it here for demonstration
  @instrumentors [MyInstrumentor]

  interactor :post_comment

  # ...
end

defmodule MyInstrumentor do
  def post_comment(:start, _meta, %Interaction{} = interaction), do: #...
  def post_comment(:stop, time_delta, _interaction) do
    ExStatsD.timer(time_delta, "post_comment.duration")
  end
end

[totallymike]

As for failure semantics, it would nice to teach interaction steps how to rollback if a further step fails.

Going back to chaining API calls, imagine that you have to create a number of entities in an API in sequence. If the second or third record fails because it's invalid or some such, you'd want to clean up after the early records. It would be burdensome to have to unpack the result and figure out where the failure happened to clean up appropriately, so steps could learn how to clean up for themselves, and they'd play in rewind.

#12 has some facility for this, but I struggled a bit with what arguments to pass around during the rollback step, and it's inconsistent.

[totallymike]

For an example of prior art on the rollback stuff, see https://github.com/collectiveidea/interactor#rollback, from which I stole the basic idea as to what it should look like.

[totallymike]

One idea would be to let the interaction macro look like this:

interaction :post_comment, rollback: :delete_comment

The assigns field could hold the post created by :post_comment, and :delete_comments could pluck it and use the data to send the appropriate DELETE request

If steps omit a rollback argument, we either use a default passthrough, or just don't call a rollback for that step.

[alanpeabody]

The rollback thing seems okay, RE instrumentation those seem like they could just be done with additional interaction steps and not need something special.

[beerlington]

I like where this is going. Couple questions/comments:

1. In your example you've got interaction MyApp.Auth.authorize, :create_comment. Would this somehow have access to the conn or current user?
2. It seems like inserting and updating records would be really common. What do you think of combining the changeset + insert/update interactions? Something like interaction :insert, with: :comment_changeset, assign_as: :comment. The with option is stolen from cast_assoc/3
3. The rollback thing is interesting, probably would have made some of the stripe stuff we were doing recently a little more robust.

[alanpeabody]

Responses:

1. The interaction could have access to the conn if you passed it in/assigned it. You would have to do so explicitly and look for :conn in the assigns in the authorize interactor.
2. Maybe? I like the idea of things being independent and simple to start, but we could build it up as it makes sense.
3. Rollback is another thing I think we could build up after we have a first pass. One challenge will be remembering exactly what interactions were called in order to roll them back, some kind of stack/list in the struct probably.

[totallymike]

In #12 I recurse through the interactors, and each one looks at the tagged tuple for an error, then calls the rollback. My implementation is a bit garbage because I was being hasty with my return values. Using the more formal %Interaction{} would definitely help there.