From: Jim Peters (jim_at_uazu.net)
Date: 2002-03-27 23:35:35
Dave wrote:
> Hey Jim--I have several questions concerning your "byte stream" replacement
> data model.
>
> >class Node {
> > InPort in[];
> > OutPort out[];
> > void process() {};
> >};
> >
> >class InPort {
> > OutPort link;
> > int cnt;
> >};
> >
> >class OutPort {
> > double val;
> > int cnt;
> >};
>
> In OutPort, you have one single 'val' and a 'cnt'. What does 'cnt'
> refer to? Since 'val' is not an array or pointer, I am not sure how
> 'cnt' is being used.
Okay, I'm working on the idea that there is no buffering between the
parts of the system. So we only ever need to store a single value --
the current value. Since different data streams might go at different
rates, we also need some way to indicate whether the 'val' value is a
new one, or the same one we saw last time around. This is why I'm
having a 'cnt' counter value, which changes every time a new value
goes into 'val'.
Anyone watching an OutPort keeps a copy of 'cnt' from the last time
they looked. At some later point, if 'cnt' has changed since the last
time they looked, then there must be a new value there to process.
> What is 'nn' -- the next spot on the stack to push a sample value?
No, there is no stack. Each 'nn' is a different output stream. The
whole idea was that each 'Node' would have a set of input streams, and
a set of output streams.
> Also, am I right in assuming that the Sensor class would inherit
> Node since these are the classes interested in sending data?
I really don't know about the best way to set up an inheritance graph.
I would guess that you can say "a Sensor is a Node", so that probably
means it should inherit ... !?
> If that is the case, is it necessary for every Node to have an
> InPort link, or can that be null to indicate that there is no input
> link?
Well, maybe this needs to be expressed differently in C++. The idea
was that you could have an array of zero InPorts if you don't actually
need any. I don't know about how to do arrays 'nicely' in C++, but in
C, it could all be done like this:
typedef struct InPort InPort;
typedef struct OutPort OutPort;
typedef struct Node Node;
struct InPort {
OutPort *link;
int cnt;
};
struct OutPort {
double val;
int cnt;
};
struct Node {
int n_in; // Number of input streams, or 0
InPort *in; // Pointer to allocated array of 'n_in' InPort structures
int n_out; // Number of output streams, or 0
OutPort *out; // Pointer to allocated array of 'n_out' OutPort structures
void (*process)();
};
> Can you give some concrete examples of what that input/output
> linkage would look like using sensor data and/or dsp processing?
> Could you give one or more real life case examples to help me ground
> my understanding?
Okay, let's say we have a Sensor that is generating two channels of
data, a FilterBank that converts a single input stream into a set of
output streams (16 in this example), and a 'Christmas tree' display
module which accepts two sets of 16 inputs to display a double
bar-graph up the screen. I'll write this in pseudo-C, because my C++
knowledge is not too strong.
Sensor *dev= sensor_new("/dev/ttyS1", "prograph/9600", ... etc ...);
FBank *fb1= fbank_new(16, 1.0, 16.0); // 16 filters between 1Hz and 16Hz
FBank *fb2= fbank_new(16, 1.0, 16.0); // 16 filters between 1Hz and 16Hz
XmasDisp *xmas= xmas_new(16, ... display details, whatever ...);
// Connect filterbank inputs to the first two 'dev' output channels
fb1->in[0].link= &dev->out[0];
fb2->in[0].link= &dev->out[1];
// Connect display inputs to filterbank outputs -- pretend that
// ordering of display inputs is top-bottom, L/R/L/R, i.e. interleaved
// and upside-down compared to the filterbank outputs
for (int a= 0; a<32; a++) {
FBank *fb= (a&1) ? fb2 : fb1;
int n= 15 - a/2;
xmas->in[a].link= &fb->out[n];
}
That's it. When this network comes to be executed, the Node handling
code would do this sequence of calls for every input sample:
dev->process();
fb1->process();
fb2->process();
xmas->process();
In this case everything is simple, because all the data rates are the
same, so nothing really has to care about the 'cnt' values. However,
if channel 'B' (dev->out[1], say) only outputs data at 16Hz compared
to 256Hz for channel 'A' (dev->out[0]), then fb2->process() would only
get called once in every 16 calls to fb1->process(), and
xmas->process() would have to be aware that most of the time it is
only getting new values on half of its inputs and not on all of them
(although for display purposes, it isn't much of a problem if it keeps
on using the values without checking).
For display Nodes, I suggest that they keep a data structure somewhere
containing the current data to display, and on the ->process() call,
they update that data and then set a flag for some other thread to
actually do the redraw. That way it doesn't matter if the front-end
can only update for 1 in every 20 samples. With a little bit of care,
this makes the whole thing rock-solid, so it can work 100% reliably
even when there are large redraws or whatever to slow down the
front-end.
> >Node-supervisor code somewhere can keep a list of all Nodes and scan
> >them to see which depend on which others through their InPort 'link'
>
> Would the Node-supervisor be a part of BioDevice, in that it sits
> waiting for data from the device, and then scans the node list to
> see who gets what?
I'm seeing the Node supervisor code as part of the Node class,
represented by a few class-global functions and variables (I can't
remember whether C++ has this -- it must do, surely. Are they
'static' ?).
I know what you're saying -- how does the network get kicked off in
the first place ? This is another question, separate in some ways
from the question of how to connect up the network itself.
Here is one approach: we could say that anything that has outputs but
no inputs must obviously create data from some external source, in
which case it always gets called to trigger off a new cycle. Nodes
that have no inputs could be written in such a way that they block
until they have some data for us. Once the ->process() call returns,
then the rest of the network runs through with the new data, until
execution again returns to input device ->process() call to wait for
some more.
This is just one way we could approach it -- we could accomodate more
complex scenarios as well. This really depends on how we might choose
to arrange our threads and so on.
> >The procedure for outputting data goes like this:
> >
> >- Put a new value into out[nn].val
> >- Increment out[nn].cnt
> >
> >That way anyone watching knows that a new value is available.
>
> How does one watch for new values? Is this done by viture of the
> fact that they have added themselves as a Node to the node list with
> their own process() routine? Is this similar to the MVC pattern
> which registers several callback views via an update() routine for a
> given resource? Is this watching triggered by a wait state on
> select()? ??
In my plan each Node is known about by the Node-supervisor code --
most probably it would be linked into a list maintained by that code,
or stored in an array maintained by that code. That way, the
node-supervisor code knows when a Node has new data waiting by
observing the counter values at either end of the link. If the 'cnt'
values are the same for an InPort, then there is no new data,
otherwise there is.
'select()' is of no use, as we are not using different threads, nor
UNIX pipes or anything like that. Also, we are not registering any
callbacks with anything, because an individual Node knows nothing
about which other Nodes have ->link pointers pointing to its outputs.
However, the Node-supervisor code knows about all the active Nodes, so
it can watch over the whole thing easily without needing to use a
callback mechanism.
> So, in other words, at some point the Node-supervisor does a
> dependency check for all output ports that have linked input ports
> to make sure that waiting output data has actually been sent through
> the InPort link mechanism?
Yes.
> Also, just what does the process() method do? Is this a virtual
> method dependent on just what the Node needs to do with the data
> (such as display it, do some mathematical processing on it, such as
> FFT filtering, or somesuch)? Or is process() a function that simply
> moves the floating piont value from out[nn].val to in[nn].link.val
> and update the 'cnt's?
The ->process() method does the actual processing that the Node is
there for -- as you say, reading the device, filtering data streams,
data display, or whatever.
There are no values stored in the InPort arrays -- in[nn].link.val (or
in[nn].link->val in C) is actually a value from the 'out' array of
another Node (the .link member is an OutPort pointer).
> I'll be better able to speak to this to see after I have a better
> grasp of your proposal, and see if there is a way to use the
> process() method as a means to transfer data in other ways, perhaps
> as a callback function to a socket or somesuch. I like what you
> outline as it does have a lot less overhead than the
> block-and-unblock method of threads and other sharing mechanisms.
Certainly a process() method could be used to send output to a pipe or
socket, so long as there isn't much chance of it blocking (or else it
would hold other things up). If we were going to allow several Nodes
to read from different input devices, e.g. several sockets and/or
serial streams, then we would need some additional code that does a
big select(), before triggering whichever Node has waiting data.
> >I guess someone will want to write lots of wrapper functions with long
> >names to encode what I just outlined. It seems really unnecessary to
> >me, but if that's what makes people happy ...
>
> Well, gotta tell you. Those names to encode what you outlined sure
> would help this visually oriented person. We simply have different
> learning and comprehension styles. Personally, I marvel at someone
> capable of purely abstract and logical thought. I tend to think in
> pictures and descriptions, and while abstract logic is
> part-and-parcel of our work, it is not "first nature" for me. So
> I'll be the first to write a few wrapper functions... :)
Okay, no problem. It just seems a shame to hide such a simple
mechanism underneath an abstraction layer. People coming to a class
interface often have no idea what is really going on underneath. It
could be some incredibly inefficient mechanism, or as in this case,
something that could compile down to a couple of inlined instructions
per call.
Personally, when I come across interface documentation, I really
appreciate it when people outline the underlying mechanism. I feel
uncomfortable using any interface where I don't know (or can't easily
guess) the underlying mechanism. How am I supposed to make good
coding decisions when I don't know the implications of different
approaches ?
Well, that's just my personal frustration with 'object-oriented'
coding (or perhaps 'wrapper-oriented' coding), but never mind.
Anyway, I'm not going to argue about this (apart from grumbling a
bit), and I will accept the use of wrapper functions if it will make
things more conventional and predictable for C++ programmers.
Jim
-- Jim Peters (_)/=\~/_(_) jim_at_uazu.net (_) /=\ ~/_ (_) Uazú (_) /=\ ~/_ (_) http:// B'ham, UK (_) ____ /=\ ____ ~/_ ____ (_) uazu.net
This archive was generated by hypermail 2.1.4 : 2002-07-27 12:28:43 BST