User Modeling of Parallel Workloads

The goal of workload modeling is to predict a
computer s workload well enough to design it
correctly. A poor model will lead to degraded
performance and user satisfaction. Analyzing logs
from multiple real parallel computers uncovers
several statistical features locality of sampling,
daily cycles, weekly cycles, self similarity and
flurries that are missing from current workload
models. Their practical importance is demonstrated
by two new kinds of scheduling algorithms adaptive
scheduling and shortest job backfill first
scheduling which achieve an average 10% bottom-
line performance gain and 35% stability gain on the
production workloads.

The second part of this book presents a user based
workload model. It identifies four stable user types
and five stable session types. It then deduces model
parameters, the distributions of the arrival and
activity patterns for both users and sessions, their
dependencies and temporal structure. The methodology
and statistical toolset is explained to make it
easier to reuse in other domains.

The insights and practical advice provided will be
of use to anyone building or operating a large-scale
parallel computer.

Autorentext

David Talby, Ph.D., MBA: Studied Computer Science and Business Administration at the Hebrew University of Jerusalem. Practitioner, researcher and teacher in the areas of software engineering, agile methodologies and parallel computing. Currently a Sr. Manager of Software Development at Amazon.com.

Klappentext

The goal of workload modeling is to predict a computer's workload well enough to design it correctly. A poor model will lead to degraded performance and user satisfaction. Analyzing logs from multiple real parallel computers uncovers several statistical features - locality of sampling, daily cycles, weekly cycles, self similarity and flurries - that are missing from current workload models. Their practical importance is demonstrated by two new kinds of scheduling algorithms - adaptive scheduling and shortest job backfill first scheduling - which achieve an average 10% bottom-line performance gain and 35% stability gain on the production workloads.The second part of this book presents a user based workload model. It identifies four stable user types and five stable session types. It then deduces model parameters, the distributions of the arrival and activity patterns for both users and sessions, their dependencies and temporal structure. The methodology and statistical toolset is explained to make it easier to reuse in other domains.The insights and practical advice provided will be of use to anyone building or operating a large-scale parallel computer.