NWS

PageBox

NWS

Table of content

Principle
Architecture
Name server process
CPU Sensor
Network Sensor
Persistent state process
Forecaster process
C API
More information

Principle

Network Weather Service (NWS) was developed for use by schedulers in a networked computational environment.

NWS measures CPU availability and network performance.

It can also forecast the network performance and available CPU percentage for each machine that it monitors.

NWS addresses a hard problem: to measure how much we have we must consume a part of what we measure. It is like uncertainty principle of Heisenberg. NWS design is simple and comprehensive. We can expect that any similar monitoring tool apply the same methods. When we design such tools it is also useful to be aware about the NWS design to avoid reinventing the wheel or frantically designing a sort of perpetual motion machine.

Architecture

NWS uses four component processes:

Persistent State process: stores and retrieves measurements from persistent storage
Name server process: allows binding process and data names with TCP address and port
Sensor process: gathers performance measurement from a specified resource
Forecaster process: produces a predicted value of deliverable performance during a specified time frame for a specified resource

A monitored host only needs Sensor processes. There are two kinds of sensor process:

CPU sensor: measures the CPU availability
Network sensor: measures throughput, round-trip time and TCP connect-disconnect time

Sensors are the only Operating System specific part of NWS. They run on Unix and Linux. Sensor processes send their measurements to Persistent state processors, which can be remote.

Sensors attach time stamps to their measurements but they don’t try to correlate their measurements. This latter task is typically performed by a forecaster process.

The forecaster process acts as a proxy for the NWS client.

NWS processes are stateless. They are written in C and communicate using TCP/IP.

Name server process

The address of the NWS Name server is the only well-known address used by the system.

All other NWS processes register their name-location binding with the Name Server.

These bindings time out according to a time-to-live parameter that must accompany each registration. Therefore other processes renew their registration periodically.

CPU Sensor

The CPU Sensor combines information from uptime and vmstat with periodic active occupancy tests to measure the CPU availability.

uptime reports the average number of processes in the run queue over

vmstat gives the idle time, the user and the system time.

An active occupancy test consists of running a compute-intensive "probe" program and calculating the CPU availability as the ratio of its observed CPU occupancy time to the wall-clock time of its execution. The CPU Sensor adaptively adjusts the frequency with which probes are conducted.

Network Sensor

The NWS Network Sensor can only rely on probes when determining network load. It makes three kinds of measurement:

Small message round-trip time measured by timing a small message probe sent from a source Sensor to a destination Sensor and back
Large message throughput measured by timing the transfer of a large message (typically 64KB) using TCP and the acknowledgment of its receipt
TCP connection and disconnection time

To avoid introducing too much network load, network Sensors are organized as a hierarchy of Sensor sets called cliques. This word probably comes from graph theory. In that domain a clique is a region of a graph where each node is connected to the others. For more details you can look at our graph introduction and at our algorithm page.

Each Sensor participating to a clique conducts inter-machine experiments with every other clique members but not with Sensor outside the clique. It is possible to define different cliques at each level of the hierarchy and to promote one representative sensor from each clique to also participate in the clique at the next higher level. A clique can map a sub-network or a region.

To reduce contention within a clique, only a single clique member conducts experiments at a given time. This policy is implemented by passing a clique token among member Sensors.

Persistent state process

A persistent state process provides a text-string storage and retrieval service and keeps the data in a file managed as a circular queue.

Forecaster process

A forecaster process requires measurements to Persistent state processes.

Then the forecaster process orders the measurements by time stamp and calls different prediction modules.

The forecaster process keeps track of which prediction module gives the lowest aggregate error measure over time and reports the forecast returned by that module.

C API

The C API implements two functions:

InitForcaster connects to the Forecaster process and passes a list of requested forecasts
RequestForecasts returns a list of forecasts

More information

You can download NWS from ftp://nws.cs.utk.edu/pub/nws.

You can find NWS papers at http://nws.cs.ucsb.edu/publications.html.

Java PHP .NET
Reservation Controls Java controls
Polaris Grid Coordinator Grid V2
Distribution Installation NWS