Managing a network of nodes and data streams in one thread

From: Jim Peters (jim_at_uazu.net)
Date: 2002-03-21 15:17:57


Here is a plan, roughly expressed in Java. (For Dave, in Java every
mention of a classname is a pointer -- so these are pointers not
in-line objects):

class Node {
InPort in[];
OutPort out[];
void process() {};
};

class InPort {
OutPort link;
int cnt;
};

class OutPort {
double val;
int cnt;
};

It works like this: Every object that is interested in sending data
out or receiving data in inherits from Node. This allows it to set up
a list of its input and output ports. It, or some other code, can
then connect the input ports to the output ports of other Nodes by
setting the 'link' member of the InPort.

Node-supervisor code somewhere can keep a list of all Nodes and scan
them to see which depend on which others through their InPort 'link'
members. It can then sort them into an order that makes sure that
nothing gets run before data is available for it. So long as we are
not permitting feedback loops, this is easy.

The procedure for outputting data goes like this:

- Put a new value into out[nn].val
- Increment out[nn].cnt

That way anyone watching knows that a new value is available.

The Node-supervisor code can check to see if the OutPort 'cnt' value
differs from the in[nn].cnt value for any InPort pointing to it
through its in[nn].link. If it differs, then there is new data for
that InPort, which means that the Node process() function should be
called. The Node handling code can update the in[nn].cnt values to
match the linked-to out[nn].cnt value (i.e. in[nn].link.cnt) after the
Node process() call has returned.

The Node-supervisor code only ever has to run through its list of
Nodes in the sorted order it worked out initially, because this
guarantees that it will pick up all the Nodes that need to run in the
correct order for them to make data available for later Nodes.

For a simple Node that only handles a set of InPorts all at a single
rate, all it has to do in process() is read the in[nn].link.val value
for all its inputs, without worrying about in[nn].cnt, since the
Node-supervisor code will have made sure that everything is ready.

However, if the Node accepts several inputs that might be at different
rates (e.g. EEG and temperature), then it needs to check that
in[nn].cnt != in[nn].link.cnt before reading the value.

That's it, pretty well.

Unlike Dave's system, this is only useful for interconnecting things
within memory, and within the same thread. In that sense it is less
flexible, but faster. It also assumes that data streams are split up
into individual streams of floating point values.

This is all just the basic level -- we could extend this to allow a
Node to give short labels to its input and output ports so that they
are easier to identify using names rather than numbers, which might
allow the interconnections to be setup more easily from a config file.
Also the Nodes themselves could be given short labels when they are
created.

I guess someone will want to write lots of wrapper functions with long
names to encode what I just outlined. It seems really unnecessary to
me, but if that's what makes people happy ...

So, does this fit in with what other people were planning ?

[ I know Dave is away until next week sometime, but I thought I'd
write this anyway whilst I had it in my head. ]

Jim

-- 
Jim Peters (_)/=\~/_(_) jim_at_uazu.net
(_) /=\ ~/_ (_)
Uazú (_) /=\ ~/_ (_) http://
B'ham, UK (_) ____ /=\ ____ ~/_ ____ (_) uazu.net


This archive was generated by hypermail 2.1.4 : 2002-07-27 12:28:43 BST