“A Frontend Server, Front to Back” by Zach Tellman

Story of a front-end server aka a gateway server, a routing server

For Fitbit, needed because a monolith needed to be broken up [i.e. the way microservices need routing] frontend_server_monolith_break.png

Fitbit Frontend Server Responsibilities

4 Important Properties (in order of importance-ish)

System Design: Looking at the Extremes

Consider:
- What extremes should the system handle e.g. peak req/sec
- What should the system do when going above/under the extremes

Frontend Uses Async: Issues

Adding Transparency: State Machine & Passports

frontend_fsm_diagram.PNG

frontend_fsm_passport.PNG

Requests in Queues and Handling: Latency, Throughput, Resource Usage

Somewhat conflicting possible optimizations: - resource usage - make sure no resource is idle, min resources - latency per request - all requests get served immediately - throughput - volume but not necessarily low latency though

Load Balancing Techniques

Goal: - Minimize latency - Distribute work - Deal with workers that are unpredictably slow

frontend_rr_least_in_flight_perf.PNG

Important Metrics

"You improve what you measure"
Or
"Everything you ignore get worse"
  1. p999 Overhead
    • introduced by frontend server e.g. GC, computing routing
  2. request rate
  3. error rate

Lessons Learned