Design patterns: Circuit Breaker

Today I want to introduce a CircuitBreaker – one of the reactive design patterns, especially usefull in areas such as web services interop. To get you better understand on it’s concepts, lets consider following scenario:

You’ve built a successful web service, making use of another external services in order to handle user requests. During the big churn of end users requesting your site, one of them became overloaded, starting to respond with increasing delays. Ultimately while trying to satisfy all incomming requests, it has exhausted all resources and gone down, entailing your service with it.

I think it’s a well-known example and a good one to show a nature of the problem. Because of RPC nature, it’s characteristics differ from in-proc calls:

• They operate on separate resources pool and might be used by more than one remote caller at the time.
• Their internal state, available resources and performance may be hard to predict. In most cases, they neither can be controlled nor monitored by the caller.
• Their life cycle is not bound to your local host. They may be reset/shut down while your service is still running and serving requests.

What reactive applications have to do with that? One of the key principles governing the Reactive Manifesto stands for quick responsiveness of the system, no matter if this response is possitive or not. Nowadays users are impatient. When they take action, they want a response, and they want it now.

This is when Circuit Breakers kicks in. They main role is to act as a decorator around your code to ensure, that you can quickly respond on any reliability problems:

• First, Circuit Breaker defines a safe time delay given to the action to respond. Service which won’t fit specified timeout is considered to be unresponsive or working incorrectly. It’s better to inform your user about possible problems after a reasonable delay than to show him/her a smoke screen and spinner for the next 2 minutes.
• Secondly it’s able to cut out the service in any sign of problems and give it a time to self repair. If you already know, that external service is not working properly, what is the point of calling it any way?

Most of the circuit breaker are realized as a state machines. Picture below presents a simple one:

where:

• Closed – this is initial state. While closed, Circuit Breaker will simply pass any request to underlying function. However it also checks if response will met a specified timeout criteria or ended with failure i.e. due to external service overload or crash. Any of these will trigger Circuit Breaker to come into open state.
• Open – while in this state, any request send through Circuit Breaker will follow fail fast semantic. Instead of forwarding request to the underlying function, CB will immediately throw an exception informing that it’s closed. Open state is not permanent – CB should automatically switch to half-open state after a specified time delay.
• HalfOpen – this is a kind of the probing state, signing that CB is checking if undrelying function is responsive again. While in it, CB will pass first request call as if it was in Closed state, to eventually get the response. All subsequent calls will fail fast just like in Open state. If request failed again or response didn’t came in specified timeout, CB switches back to Open state. If response was successful, CB becomes Closed.

I’ve created a simplistic implementation of the CircruitBreaker pattern in C# (source code). It allows to perform multiple asynchronous calls through one CB instance in non-blocking, thread-safe manner. It has also attached a complete test suite describing it’s behavior. Feel free to use it and to modify as you wish.

Example usage:

    // this service could be a singleton, and could be called concurrently
    public class CurrencyExchangeProxy : ICurrencyExchangeProxy
    {
      private readonly CircuitBreaker<ExchangeCurrencies, ExchangeRate> exchangeRates;

      public CurrencyExchangeProxy(Configuration config)
      {
          exchangeRates = new CircuitBreaker<ExchangeCurrencies, ExchangeRate>(
              async req => HttpGetAsync<ExchangeRate>(config.ExchangeServiceUrl, req),    // async function, CB should take care of
              config.GetServiceTimeout(),         // time given to function to finish
              config.OpenTimeout());              // time given to function dependent component to restore in case of failure
      }

      public async Task<ExchangeRate> GetExchangeRate(Currency from, Currency to)
      {
          return await exchangeRates.Call(new ExchangeCurrencies(from, to));
      }
    }

    public class InvoiceController : Controller
    {
      private readonly ICurrencyExchangeProxy currencyExchange;

     [HandleError(ExceptionType=typeof(CircuitBreakerOpenException), View="UnresponsiveExternalService")]
      public async Task<ActionView> DisplayInvoice(int id)
      {
          ...
          var exchangeRate = await currencyExchange.GetExchangeRate(invoice.Balance.Currency, localization.Currency);
          ...
      }