Transparent Process Migration: Design Alternatives and the Sprite Implementation

Douglis91

Process migration (PM): move a process’s execution from a source to destination machine of the same architecture.

Common Migrations: exec for a resource-intensive computation (efficient because address space is initially null; no need to transfer pages); eviction when process is transferred back to a home machine when remote machine is no longer idle.

Transparency: process executes on remote machine just as it would have on home machine with no appreciable difference to users, or even the process itself.

Spite process migration only one to be practically used (2 years); most common application is pmake—parallel make. Common speed up: 2 to 3x. Possible to achive 5 to 6x. 30% of work in entire system done by remotely migrated processes.

Four main trade-offs: transparency, residual dependencies (stuff the process needs is still on the home machine even after migration), performance, complexity.

Sprite environment: high-network integration; diskless hosts each running a host kernel that relies heavily on RPC to access high-performance, shared file servers, as well as communicate with other host kernels. Idle hosts plentiful (66-78% of hosts idle on average). User’s “own” their workstations (they expect that full resources of workstation are theirs when they are using it). Sprite uses kernel calls (as opposed to message passing, which would make process migration easier since messages could be forwarded; for kernel calls, process is relying mostly on remote machine; must transfer state to execute these kernel calls correctly on remote machine). Sprite already provide high degree of network support (native remote access to files and devices, all process ids are global, kernel to kernel RPC natively supported).

Why not just use rsh? rsh does not have transparency (cd, env.vars., etc all different on remote host), eviction, performance, or automatic selection of hosts.

Justification for PM: 1) flexibility (*any* process can be migrated; 2) PM is only moderatly more complicated than remote invocation

Big Problem: Managing State

1) Virtual memory: all pages need to be transferred. This takes the most amount of time for non newly exec’ed processes.

2) Open Files: internal identifier, current access position, possibly cached blocks need to be transferred.

3) Message Channels: (applicable to Mach, V) buffered messages

4) Execution State: registers and condition codes

5) Other kernel state: pids, uids, current working directory, signal masks/handlers,...

One of three things can be done:

1) transfer the state (higher transparency and complexity),

2) forward (less performance: ie may be cheaper to execute kernel call on remote machine than to forward to home machine, residual dependencies, less transparency), or

3) ignore it (less transparency, more complexity; can’t transfer frame buffer – so they punt on migrating such processes)

In some cases, we don’t want transparency—get_free_memory() should return amount of memory on remote machine; not home machine.

Migration Mechanics:

1) Virtual Memory: Approaches

a. Transfer all at once (disadv – process frozen for seconds, wasted work due to not all pages probably being used)

b. Pre-Copying (allow process to continue execution on home machine while transfer takes place; adv – reduces freeze time; disadv – some pages need to be copied twice due to modification after intial transfer)

c. Lazy Copying (leave pages on home machine until needed on remote machine; adv – pages that are never used are never copied at all; disadv—process has many short stalls on remote machine as page faults occur + residual dependencies on many machines on which process previously ran)

d. Sprite—Lazy-Copying w/ Sprite Network File System Support. Backing storage for virtual memory is on network file server. Sprite freezes process, flushes dirty pages to file server, discards address space on home machine. Migrated process on remote machine pages faults and retrieves pages from file server. (adv—fast because no writes to disk typically occur on page flush; pages simply transferred to file server cache, only dirty pages incur overhead at migration time; adv—eviction is relatively quick; adv—because pages are flushed to FS, never need to worry about residual pages; disadv—will do more work than lazy copying since pages may need to be copied twice: HM -> FS -> RM.) Also, Sprite uses exec very often in which no VM to transfer.

One complication: if process to be migrated shares VM pages with another process, makes this expensive, no Sprite disallows MP when use shared VM pages.

2) Open Files (Hard to migrate): transferred state instead of using forwarding due to high frequency for file related kernel calls.

a. File Reference: move ref from source to target w/o closing file (else someone could delete the file in the midst of the transfer)

b. Caching Info: source flushes modified blocks to file server. (just like VM page management)

c. Access Position: can be shared between multiple processes reading/writing from/to the file. (or occurs when a process forks—child gets copy of all open file descriptors) make file server store access position, and have it serialize reads/updates

(LOCUS approach different – passes around “access position” tokens to manage who is allowed to read/write file at any given time.

3) Process Control Block: keep small part of PCB on home with process id, etc. but store most process block info on remote machine.