Core client: finite-state machine (FSM) structure
The core client can perform many activities (file transfers, computations, RPCs to scheduling servers) in parallel. To manage this parallelism, the core client is structured as a number of finite-state machines (FSM). For example, an HTTP transaction is represented by an FSM whose states might include:
- Waiting for connection establishment.
- Waiting to send request header.
- Waiting to send send request body.
- Waiting for reply header.
- Waiting for reply body.
FSMs of a particular type are managed by an FSM container. Each FSM container manages a set of FSMs, and provides a poll() function for detecting and performing state transitions. These functions are nonblocking; at the lowest level, they must use non-blocking network sockets, accessed using select().
The core client uses the following FSM types:
- NET_XFER (container: NET_XFER_SET). Each instance represents a network connection, for which data is being transferred to/from memory or a disk file. The poll() function uses select() to manage the FSM without blocking.
- HTTP_OP (container: HTTP_OP_SET). Each instance represents an HTTP operation (GET, PUT or POST).
- FILE_XFER (container: FILE_XFER_SET). Each instance represents a file transfer (upload or download) in progress.
- PERS_FILE_XFER (container: PERS_FILE_XFER_SET). Each instance represents a 'persistent file transfer', which recovers from server failures and disconnections, and implements retry and give-up policies.
- SCHEDULER_OP. There is only one instance. It encapsulates communication with scheduling servers, including backoff and retry policies.
- ACTIVE_TASK (container: ACTIVE_TASK_SET). Each instance represents a running application.
An FSM may be implemented using other FSMs; for example, FILE_XFER is implemented using HTTP_OP, which in turn is implemented using NET_XFER.