Workflows can encourage bad system design

In this article I'll discuss my thoughts on a potential design flaw relating to systems (primarily microservices) which use a workflow engine.  This issue was discussed in Sam Newman's excellent "Building Microservices" book.  I've lived through this and felt compelled to explain my take on it!


TLDR: Using a workflow engine runs the risk that other services called by the workflow are pushed to be data only with CRUD APIs and no behaviour.  Centralised workflow engines that perform all behaviour should be avoided.  Domain driven design should be considered first for your services.  Only then consider small embedded workflow engines as a way of writing the internal code.


Why Use a Workflow engine?

Workflow engines are a fantastic design choice when the process you are modelling is complex and involves many sequential or parallel tasks.  They can show:

  • non technical users the actual workflow representing the process that can be easy to follow and never drifts from reality (unlike diagrams that have a habit of going out of date)
  • what happened to a specific process (great for support)
  • what could happen to a process (great for discussing future design changes)

There are several downsides to using workflow engines, I'll explain the rest in another blog. For now - let's focus on this potential design flaw.  


Having a central orchestrator (workflow engine) can make your other services dumb

Let's consider a hypothetical business process - an online car sales platform (I know nothing about this domain by the way!).  Let's also consider that each task is a call to another microservice within the organisation.





At first glance this looks quite logical.  We could imagine the following components:

  • Orchestrator running the workflow
  • Validation service - Check that the order is valid (e.g. we have the car in stock)
  • Customer service - For handling customer data
  • Price service  - For pricing car orders
  • Order service - Saving orders with a price that are valid.

The problem is the order service.  With an Orchestrator calling it's API in the above way: 

  • The order service is dumb and can only ever be called by the orchestrator.  
  • Another client calling the Order service might not validate the order and save something incorrectly.
  • Everything stored by the Order service must be received from the orchestrator.  This means the orchestrator is highly coupled to the Order Service.

With the design above, the order service is basically just a data store exposed over http.

 

Here is what I prefer - embedded workflows

 



The diagram looks almost the same, but one crucial difference.  Now the order service has a workflow which governs its internal flow.  It's now a service with data and behaviour.  Also:

  • It can store whatever it likes internally and doesn't need to over expose so that an orchestrator is able to store what it needs.
  • We haven't made a service which can only be called by the orchestrator.


Your services should handle data and behaviour

When I learnt OO design, I was taught it's generally good design to have objects with data and behaviour.  Otherwise you have data objects in one place and procedural code in another.  The same thing goes for microservices.

I believe workflows suffer from this because they focus solely on one aspect - behaviour.  In other words, you handle the behaviour in one place, your going to handle the data in another.  But - workflow engines (such as Flowable) can be embedded within your app.


Conclusion - Domain Driven Design 1st, Workflows 2nd

When designing systems, consider the domain first.  Make sure you are clear on the business domain your microservuce handles.  if your service just so happens to have a lot of behaviour then consider embedding a workflow engine.  Just think of them as another way of writing code, to be kept internal and not something which influences other system APIs.


 

Comments

  1. This comment has been removed by the author.

    ReplyDelete
  2. Interesting read.

    One confusion on my part - what exactly is intrinsic to a workflow engine that encourages the problem you've identified? It appears to me that what makes the order service 'dumb' in your illustration has more to do with a developers approach to a microservice architecture.

    Consider the following comment you've made

    "Another client calling the Order service might not validate the order and save something incorrectly.... This means the orchestrator is highly coupled to the Order Service."

    In the same vein, the order service in your improved design with an embedded workflow still suffers from high coupling with the 'Price service' (can you price something not validated?) despite the supposed cause (the workflow engine) being eliminated. It may be that I'm critiquing the particular hypothetical used to harshly and missing wider implication, but hopefully the contention is at least clear.

    Looking forward to a reply.

    ReplyDelete
    Replies
    1. Hi Ashwin - thanks so much for taking the time to comment and sorry I took a long to reply.

      I think what makes this problem intrinsic to workflows is the fact that they are a great place to implement behaviour. This means that when faced with a choice on where to put behaviour, the decision will likely go to the service with the workflow. Perhaps even more so if there's only one around.

      However, to your point, developers could (and should) resist this! But the fear is (as Sam Newman said in his microservice book).... "If logic has a place where it can be centralized, it will become centralized!"

      Hope this makes sense and thanks again.

      Delete

Post a Comment

Popular posts from this blog

Lessons learned from a connection leak in production

How to connect your docker container to a service on the parent host

Client Side vs Server Side Session