FailMode

In the distribution architecture such SOA or micro service, you can not guarantee services always work well. Maybe the server is down or network is slow or broken. So you need to handle invocation failures.

rpcx has three fail modes to handle invocation failures and you can set it when create XClient.

FailMode is meaningful only for sync call (XClient.Call).

Failfast

Example:failfast

In this mode, rpcx will finished this call if it gets a failure. The failure is not business failure and business failures are normal responses. It may be caused by server crashing or network issues.

Failover

Example:failover

In this mode, rpcx will try another nodes if it gets a failure. it will try retries until the service returns normal response. The retries is defined in Option and it is 3 in the defaultOption.

Failtry

Example:failtry

In this mode, rpcx still try current selected nodes if it gets a failure. it will try retries until the service returns normal response. The retries is defined in Option and it is 3 in the defaultOption.

Failbackup

Example:failbackup

In this mode, rpcx will send another request to one of servers if the first request has not returned in a given time. rpcx uses the the fastest returned response of the two requests. The given time is configured in Option.BackupLatency.

If you want to learn more details about backup request pattern, you can read Jeff Dean's Achieving Rapid Response Times in Large Online Services

From http://highscalability.com/blog/2012/6/18/google-on-latency-tolerant-systems-making-a-predictable-whol.htmlBackup requests are the idea of sending requests out to multiple replicas, but in a particular way. Here’s the example for a read operation for a distributed file system client:send request to first replicawait 2 ms, and send to second replicaservers cancel request on other replica when starting readA request could wait in a queue stuck behind an expensive query or a packet could be dropped, so if a reply is not returned quickly other replicas are tried. Responses come back faster if requests sit in multiple queues.

By smallnest updated 2018-12-04 11:47:26