PageBox: presentation deployment using .NET PageBox

for
FAQ Dev site .NET version Customization Install Grid Active Naming Grid API

Grid API for .NET

Principle

The Grid API is asymmetric:

It has been made so to support multithreading in an optimal way:

The involved controlling PageBox subscribe to a repository.

To participate to a PageBox Grid, a client thread creates a Grid object.

The client thread is typically but not necessarily created in an Application Server process.

In the case of the .NET flavor of the Grid API it must use a .NET language (VB.NET, C#, J#, JavaScript) and requires the .NET runtime environment.

Constructor

The Grid constructor takes five types of parameters:

The Grid constructor creates listener threads when they don’t exist. Otherwise it just registers queues to the listeners.

The Grid constructor uses the ActiveNaming Web service:

  1. To register itself on the controlling PageBox with SetKeyRange

  2. To get the list of the other Grid participants with GetCandidate

Because the Grid constructor has registered, participants created later will get it in their participant list.

The Grid constructor must also notify existing participants. To do that it simply uses the Scatter method of the Grid API.

Destination

With the list returned by GetCandidate the constructor builds an array of destinations.

A destination is defined by a GridEntry object. The GridEntry contains this information:

class GridEntry {

string UDPAddress;

int UDPPort;

string mailAddress;

int gridNb;

string activeNamingUrl;

}

A GridEntry fully describes how to send a message

It also fully identifies a Grid destination. Two Grid objects in the same process can share the same mail address and UDP address and port. A destination is identified by (UDPAddress, UDPPort, mailAddress, gridNb) where gridNb is an occurrence number.

activeNamingURL is the URL where the destination is registered. activeNamingURL allows removing destinations.

The index of a destination in the GridEntry array is the destination number used in the Send and Receive operations. Collective operations, Scatter and Gather apply to the destination array.

Send, Receive and message handler

Send checks if the destination is in another thread of the same process. If it is the case Send queues the message object passed as parameter on a receive queue of the target Grid object.

Otherwise Send serializes the message object passed as parameter using a BinaryFormatter.

Then depending on the transport mode, UDP or SMTP, Send sends a datagram or a mail to the recipient defined in the GridEntry object.

UDP and mail messages are received by a listener.

The UDP listener receives datagrams.

The POP3 listener connects to the mail server using the POP 3 protocol to get the message list and receive the messages.

The listener maintains a clients Hashtable that contains an entry for each Grid object/consumer thread that share the same listener.

The key of clients entries is the gridNb sequence number and the value is a GridQueue object. The GridQueue class has the following fields:

class GridQueue {

Queue clientQueue = new Queue();

Grid client;

bool receivePending = false;

}

The listener uses a BinaryFormatter to unserialize the message and retrieve the sequence number.

Then it retrieves the corresponding GridQueue object from the clients Hashtable and queues a message wrapper around the unserialized object on clientQueue when the client doesn’t have a message handler. If the client has a message handler the listener calls its Notify method.

The actual Receive first tries to dequeue objects queued by another thread.

If there is no message from another thread, Receive dequeues the message wrapper from clientQueue.

The message wrapper depends on the protocol. Beside the payload object it contains:

In case of an SMTP message, the mail address and the sequence number of the sender

In case of an UDP message, the IP address, port and sequence number of the sender

This information allows Receive to identify the sender in the destinations array and to build a Message object (the Message class is described in the Receive section).

We still have to explain the meaning of receivePending.

Sometimes there is no message to dequeue. In that case Receive set receivePending to true and waits up to timeout expiration. Then when a message arrives the listener ends the wait with a Monitor.Pulse(client).

Single message Scatter and Gather

These collective operations use the same principle as Send / Receive. However they allow optimisations.

Because the same object is sent to all recipients:

Multiple message Scatter and Gather

These collective operations use the same principle as Send / Receive. They allow less optimisation than single message Scatter and Gather.

In order to send only one datagram per UDP address and port and only one mail per mail address we send an array of messages wrappers. Once it has unserialized the wrapper array the listener dispatches message object on the appropriate GridQueue.

When applicable use Scatter and Gather rather than Send an Receive in order to minimize the number of mails and datagrams actually sent.

Classes

DatagramWrapper

DatagramWrapper is the object wrapper used in UDP and SMTP.

To send an object the PageBox Grid API

  1. Wraps the object in a DatagramWrapper or MulticastWrapper (see below)

  2. Serializes the object

The main function of a Wrapper is to store the target and the source sequence number along with the object to:

  1. Allow the listener to queue the object on the right

  2. Allow the target Grid object to identify the object source

DatagramWrapper contains the following data:

class DatagramWrapper {

object o = null;

int gridNbFrom = -1;

int gridNb = -1;

string objectClass = null;

}

For troubleshooting purpose we store the object class in DatagramWrapper.

When gridNb = -2 the message should be delivered to all Grid object sharing the same mail address or UDP address and port.

MulticastWrapper

MultcastWrapper is the object wrapper used in multicast UDP. It contains the following data:

class MulticastWrapper {

object o = null;

int gridNbFrom = -1;

string objectClass = null;

IPAddress addrFrom;

int portFrom;

}

For troubleshooting purpose we store the object class in DatagramWrapper.

We don’t need to specify the target sequence number because it is always all Grid objects sharing the same UDP address and port.

All Grid objects in the same group (same repository and communicator) have the same multicast address. Therefore to identify the source we include its UDP address and port.

GridCommand

As we explained in the previous section, we use the Grid API to distribute destination adding and removing. We use a special message object of GridCommand type.

GridCommand contains the following data:

class GridCommand {

Command cmd;

string UDPAddress;

IPAddress addr;

int UDPPort;

string mailAddress;

int gridNb;

string activeNamingUrl;

}

cmd describes the action to perform. Command is an enumeration defined like this:

enum Command {

Remove,

Add

}

addr is an IPAddress built on the sender side with IPAddress.Parse(UDPAddress).

activeNamingUrl is the URL of the sender controlling PageBox.

GridCommand has two methods used on the target side:

  1. Matches: compare a GridCommand object with a GridEntry and return true if the contain the same data

  2. Update: execute the operation specified in cmd

Log

Log has a single method, write allowing writing log entries on a file.

POPListener

POPListener is a listener that reads mails using the POP3 protocol.

GetListener

POPListener implements a factory method, GetListener whose signature is:

static POPListener GetListener(string POPServer, string POPUser, string POPPassword, Log log, double timeout, Grid grid);

where:

GetListener tries to find an existing listener for (POPServer, POPUser).

If there is no matching listener, GetListener:

  1. Creates a new POPListener object

  2. Creates a new thread with the ThreadRun method of the new POPListener object

  3. Starts the thread

GetListener adds a GridQueue object for the calling Grid in the clients Hashtable and returns the selected or created POPListener.

ThreadRun

ThreadRun is coded like this:

void ThreadRun() {

while(!toStop) {

if (Connect()) {

GetMessages();

Disconnect();

}

lock(this) {

Monitor.Wait(this, timeout);

}}}

At each loop it connects to the POP server with the Connect method, processes the messages with GetMessages and disconnects with the Disconnect method.

Connect and Disconnect just implement the RFC 1725. You can look at the papers of Agus Kurniawan and Bill Dean for more information.

GetMessages

GetMessages also implements RFC 1725.

It issues the STAT command to get the message list and calls ProcessMessage to process the messages.

ProcessMessage

ProcessMessage also implements RFC 1725.

It issues the RETR command to get a message and the DELE command to remove it from the POP server.

It calls QueueMessage with the message body and the mail origin.

QueueMessage

QueueMessage deserializes the message into an array of DatagramWrapper.

For each element of the array:

GridQueue

GridQueue manages a Queue called clientQueue.

It implements two methods:

  1. Enqueue to queue an SMTPMessage

  2. Dequeue to dequeue an SMTPMessage

SMTPMessage

SMTPMessage is a wrapper around a message object.

SMTPMessage contains the following data:

class SMTPMessage {

object o;

string mailAddress;

int gridNb;

}

Where

UDPListener

UDPListener is a listener that receives UDP datagrams.

It is designed like POPListener.

GetListener

UDPListener implements a factory method, GetListener whose signature is:

static UDPListener GetListener(string UDPAddress, int UDPPort, Log log, Grid grid);

where:

GetListener tries to find an existing listener for (UDPAddress, UDPPort).

If there is no matching listener, GetListener:

  1. Creates a new UDPListener object

  2. Creates a new thread with the ThreadRun method of the new UDPListener object

  3. Starts the thread

GetListener adds a UDPQueue object for the calling Grid in the clients Hashtable and returns the selected or created UDPListener.

ThreadRun

ThreadRun is coded like this:

void ThreadRun() {

IPHostEntry ihe = Dns.GetHostByName(UDPAddress);

UdpClient uc = new UdpClient(new IPEndPoint(ihe.AddressList[0], UDPPort));

IPEndPoint remoteEP = null;

while(!toStop) {

byte[] buf = uc.Receive(ref remoteEP);

QueueMessage(remoteEP.Address, remoteEP.Port, buf);

}}

Each time it receives a message, ThreadRun calls QueueMessage with the message body and the IP address and port of the origin.

QueueMessage

QueueMessage deserializes the message into an array of DatagramWrapper.

For each element of the array:

MulticastListener

MulticastListener is a listener that receives multicast datagrams.

It is designed like UDPListener.

GetListener

MulticastListener implements a factory method, GetListener whose signature is:

static MulticastListener GetListener(string UDPAddress, int UDPPort, int ttl, Log log, Grid grid);

where:

GetListener tries to find an existing listener for (UDPAddress, UDPPort).

If there is no matching listener, GetListener:

  1. Creates a new MulticastListener object

  2. Creates a new thread with the ThreadRun method of the new MulticastListener object

  3. Starts the thread

GetListener adds a UDPQueue object for the calling Grid in the clients Hashtable and returns the selected or created MulticastListener.

ThreadRun

ThreadRun is coded like this:

void ThreadRun() {

UdpClient uc = new UdpClient(UDPPort);

uc.JoinMulticastGroup(IPAddress.Parse(UDPAddress), ttl);

IPEndPoint remoteEP = null;

while(!toStop) {

byte[] buf = uc.Receive(ref remoteEP);

QueueMessage(buf);

}}

Each time it receives a message, ThreadRun calls QueueMessage with the message body and the IP address and port of the origin.

QueueMessage

QueueMessage deserializes the message into an array of DatagramWrapper.

For each element of the array:

UDPQueue

UDPQueue manages a Queue called clientQueue.

It implements two methods:

  1. Enqueue to queue an UDPMessage

  2. Dequeue to dequeue an UDPMessage

UDPMessage

SMTPMessage is a wrapper around a message object.

SMTPMessage contains the following data:

class UDPMessage {

object o;

IPAddress addr;

int port;

int gridNb;

}

Where

API

Constructor

Function:

Create a Grid environment and, if needed, listener threads.

Signature:

Grid(string PageBoxUrl, string repositoryUrl, string communicator, string UDPAddress, int UDPPort, string multicastAddress, int multicastPort, int ttl, string SMTPServer, string MailAddress, string POPServer, string POPUser, string POPPassword, string parm, string LogFile, double timeout);

Parameters:

PageBoxUrl: URL of the controlling PageBox.

repositoryUrl: URL of the communicator repository.

communicator: Name of the communicator.

UDPAddress: UDP address.

UDPPort: UDP Port.

multicastAddress: UDP multicast address. This address must be the same for all Grid objects with the same (repository, Web archive, communicator name).

multicastPort: UDP multicast port. This port must be the same for all Grid objects with the same (repository, Web archive, communicator name).

ttl: Time to live (number of hops) used with multicast. This time to live should be big enough to include all Grid objects with the same (repository, Web archive, communicator name).

SMTPServer: SMTP server used to send messages.

MailAddress: Mail address of this object.

POPServer: IP address of the POP3 server to use to read messages for the mail address above.

POPUser: POP3 user name to use to read messages for the mail address above.

POPPassword: POP3 password to use to read messages for the mail address above.

parm: parm is an extra string parameter the Grid API can use for any usage. The other Grid participants can retrieve this parameter with GetParms or GetParm. It can be for instance an XML string, an URL or a serialized object.

LogFile: File used to log Grid errors.

timeout: Period between two POP3 read or between two receive attempts in minutes.

Implementation:

The Grid constructor:

  1. Creates the Log object

  2. Creates an instance of the ActiveNaming proxy

  3. Calls POPListener.GetListener to get a POP3 listener

  4. Calls UDPListener.GetListener to get a UDP listener

  5. Calls MulticastListener.GetListener to get a multicast listener

  6. Uses the ActiveNaming proxy to get the partner lists in the retrievePartners method. retrievePartners also populates the destination array and notifies the alive partners

  7. Uses the ActiveNaming proxy to record its presence in the register method

  8. Populates the mailDests and UDPDests Hashtables

Send

Function:

Send a message to a destination.

Signature:

int Send(object o, int dest, Transport transport_mode);

Parameters:

o: Object to send.

dest: Destination number.

transport_mode: Transport.UDP or Transport.SMTP.

Returned value:

-1 in case of error.

Implementation:

Send first checks if the destination is another thread of the same process.

In this case, Send calls MemorySend.

Otherwise, depending of the transport mode, Send calls UDPSend or SMTPSend.

MemorySend builds a Message object wrapping the message object, queues this Message on the target Grid queue and if needed wakes up the target thread.

UDPSend builds a DatagramWrapper wrapping the message object, serializes the DatagramWrapper object and sends it in UDP.

SMTPSend builds a DatagramWrapper wrapping the message object, serializes the DatagramWrapper object. SMTPSend put the serialized DatagramWrapper in the body of a MailMessage and sends it in SMTP.

Receive

Function:

Receive a message.

Signature:

Message Receive(Transport transport_mode);

Parameter:

transport_mode: Transport.UDP or Transport.SMTP.

Returned value:

A Message object.

The Message class has this definition:

class Message {

object recv;

int from;

Transport transport_mode;

}

Where:

Implementation:

Depending on the transport mode, Receive calls UDPReceive or SMTPReceive.

UDPReceive:

  1. Checks if it received no message from another thread in the same process and if there is a message dequeues this message

  2. Dequeues the first message in the UDPListener object with the UDPListener.Receive method

  3. If there was no message on the UDPListener queue, dequeues the first message in the MulticastListener object with MulticastListener.Receive method

  4. If there was no message on the MulticastListener queue, waits up to the message arrival or to timeout and repeat the process

SMTPReceive:

  1. Checks if it received no message from another thread in the same process and if there is a message dequeues this message

  2. Dequeues the first message in the POPListener object with the POPListener.Receive method

  3. If there was no message on the POPListener queue, waits up to the message arrival or to timeout and repeat the process

ReceiveNonBlock

Function:

Receive a message.

Signature:

Message Receive(Transport transport_mode);

Parameter:

transport_mode: Transport.UDP or Transport.SMTP.

Returned value:

A Message object.

Implementation:

Depending on the transport mode, ReceiveNonBlock calls UDPReceive or SMTPReceive.

UDPReceive:

  1. Checks if it received no message from another thread in the same process and if there is a message dequeues this message

  2. Dequeues the first message in the UDPListener object with the UDPListener.Receive method

  3. If there was no message on the UDPListener queue, dequeues the first message in the MulticastListener object with MulticastListener.Receive method

SMTPReceive:

  1. Checks if it received no message from another thread in the same process and if there is a message dequeues this message

  2. Dequeues the first message in the POPListener object with the POPListener.Receive method

Single message Scatter

Function:

Send a message to all destinations.

It can be used to update all Grid objects in a communicator or for cache update.

Signature:

int Scatter(object o, Transport transport_mode);

Parameters:

o: Object to send.

transport_mode: Transport.UDP or Transport.SMTP.

Returned value:

-1 in case of error.

Implementation:

Depending on the transport mode, Scatter calls UDPScatter or SMTPScatter.

UDPScatter builds a DatagramWrapper wrapping the message object, serializes the DatagramWrapper object with a –2 (all threads) destination and sends it in multicast if possible or to each UDP address.

SMTPScatter builds a DatagramWrapper wrapping the message object, serializes the DatagramWrapper object with a –2 (all threads) destination. SMTPScatter put the serialized DatagramWrapper in the body of a MailMessage and sends it to all destination mail addresses.

Multiple message Scatter

Function:

Distributes a set of messages to destinations.

If there are fewer messages than destinations, Scatter only sends messages to the first destinations.

If there are more messages than destinations, Scatter sends more than one message to destinations.

Signature:

int[] Scatter(object[] os, Transport transport_mode);

Parameters:

os: Array of objects to send.

transport_mode: Transport.UDP or Transport.SMTP.

Returned value:

Array of destination status.

If a destination i has been sent two messages then status[i] = 2.

If Scatter failed to send a message to destination i then status[i] = -1.

Implementation:

Depending on the transport mode, Scatter calls UDPScatter or SMTPScatter.

For each UDP address and port, UDPScatter builds an array of DatagramWrapper whose size equals the number of threads on this UDP address and port, serializes this DatagramWrapper array and sends it to this UDP address and port.

If there are fewer objects than destinations then UDPScatter sends datagrams to the n first UDP addresses and port, n being the number of objects. If there are more objects than destinations then UDPScatter sends more than one datagram to at least some addresses.

For each mail address, SMTPScatter builds an array of DatagramWrapper whose size equals the number of threads on this mail address, serializes this DatagramWrapper array, set it in the body of a MailMessage and sends the MailMessage to this mail address.

If there are fewer objects than destinations then SMTPScatter sends mails to the n first mail addresses, n being the number of objects. If there are more objects than destinations then SMTPScatter sends more than one mail to at least some addresses.

Gather

Function:

Collects responses from a set of destinations.

Signature:

Message[] Gather(ref int[] errors, Transport transport_mode);

Parameters:

errors:

If errors = null then Gather expects a message from each destination. This mode can be used to collect response for a single message Scatter.

Otherwise errors is a status array typically returned by a former multiple message Scatter or by a GetDestinations. If errors[i] = 0 or –1 then no response is expected. If errors[i] = 2 two messages are expected from the destination i.

transport_mode: Transport.UDP or Transport.SMTP.

Returned value:

An array of Message objects. This array contains all messages received by the Grid object in their order of reception.

errors is updated. When all expected messages have been received, error entries equal 0 or -1. If an error entry > 0 then an expected message was not received.

Implementation:

Depending on the transport mode, Gather calls UDPGather or SMTPGather.

UDPGather loops on message receive up to the time the queues are empty.

UDPGather checks the source of each received message and decrements the corresponding errors entry.

If at least one errors entry > 0, UDPGather waits up to the message arrival or to timeout and repeat the process.

SMTPGather loops on message receive up to the time the queues are empty.

SMTPGather checks the source of each received message and decrements the corresponding errors entry.

If at least one errors entry > 0, SMTPGather waits up to the message arrival or to timeout and repeat the process.

GetDestinations

Function:

Convenience method to build an error array to use in Gather.

Signature:

int[] GetDestinations(int val);

Parameter:

val: value of status entries.

Returned value:

Array of destination status.

Implementation:

int[] errors = new int[destinations.Length];

for (int i = 0; i < errors.Length; ++i)

errors[i] = val;

return errors;

RemoveDestination

Function:

When it detects that a Grid object is no longer available, the Grid API removes it.

However only the API user can detect logical errors. In such cases it is up to the API user to inform the Grid infrastructure that a destination is no longer available, with RemoveDestination.

Signature:

bool RemoveDestination(int dest);

Parameter:

dest: destination number to remove.

Returned value:

false if RemoveDestination failed to remove the destination.

Possible causes:

Implementation:

RemoveDestination:

  1. Removes the destination from the destination array, from mailDests and from UDPDests

  2. Creates a deletion GridCommand object

  3. Broadcast the GridCommand object with Scatter

  4. Uses the ActiveNaming Web service to remove the destination from the keyRange object

DestNumber

DestNumber is a read-only property that returns the number of destinations.

GetParms

Function:

Returns the parm parameters of other Grid destinations.

Signature:

string[] GetParms();

Returned value:

An array of parm parameters.

The index of a parm in the parm array is the destination number (rank) of the participant.

GetParm

Function:

Returns the parm parameter of the specified destination.

Signature:

string GetParm(int dest);

Parameter:

dest: destination number whose parm must be returned.

Returned value:

The parm parameter of the specified destination.

GetDestFromParm

Function:

Returns the numbers of the destinations that have a given parm.

Signature:

int[] GetDestFromParm(string parm);

Parameter:

parm: parm whose destinations must be returned.

Returned value:

An array of destination numbers.

Subscribe

Function:

Registers a message handler.

Signature:

void Subscribe(GridCallback gc);

Parameter:

gc: message handler.

The message handler must implement the GridCallback interface:

public interface GridCallback

{

void Notify(Transport transport_mode, int from, object o);

}

Listener threads call the Notify method of the message handler when they receive a message. They set

Note:

Contact:support@pagebox.net
©2001-2004 Alexis Grandemange. Last modified .