From: Jim Peters (jim_at_uazu.net)
Date: 2002-04-02 11:17:40
Dave wrote:
> And then just have this value wrap back to zero when the integer
> maxes out (INT_MAX, or UINT_MAX if unsigned)?
Yes, it will wrap all by itself (using the ++ operator), so we don't
need to worry about this so long as we are only doing straight == or
!= comparisons.
> I am a bit concerned about doing this unbuffered, and would want to
> throw together some tests to see what happens when we scale it
> upwards by adding more and more process() calls. As long as the
> combined process() calls are faster than the rate at which data is
> streaming, all would be fine. I suppose that would hold true for
> any amount of processing we did, so buffered or unbuffered, we are
> going to have to handle the data faster than it comes in. I dunno;
> it will take a few tests to find out if this is an issue or not.
The only problem I can see is if people decide they want to do a big
FFT every 16 samples, say, which could mess up the whole thing because
a big FFT might take more than the time available within one sample
period. Slower operations like this could be moved over into another
Network in a lower priority thread (see ideas of a Bridge lower down).
> What I think you have, then, is a variation on the MVC paradigm. In
> this case, the "model" is your Node-supervisory code which knows
> about the data and all the "views" attached to the data. The
> attached observer views and their update() calls would be akin to
> your linked Nodes with their process() calls. However, MVC uses a
> notification system to the views, telling them when new data is
> available. And, as I understand it, it is up to each view to then
> to call the model to actually get the updated data. This is
> different from your idea where the model (i.e., node-supervisor)
> would be respondible for calling the update() views directly.
I guess it is all the same, one way or another.
> I don't know.... because if we say that, then the definition of a
> Sensor changes in that a sensor object now also takes on the meaning
> and function that is in the Device class. Perhaps it is more
> accurate to say that any object which produces or receives real time
> sample data inherits the Node class. Thus, the BioDevice class
> would inherit Node. It would be up to BioDevice to add the correct
> number of out[nn] data streams.
Okay, fine. I think we have two choices -- either you inherit from
Node, or you create a helper class which inherits from 'Node' that you
then embed in your class instance. For example, I'm thinking about a
'bridge' that has two Nodes, one in each of two different networks
(probably in two different threads) that allows data streams to pass
between different threads via a bunch of circular buffers. Obviously
this can't inherit from Node twice (at least, not so far as I know in
C++), so I think I'll need two helper classes, with instances of these
classes embedded in the 'bridge' class.
> I think that the above should not be a "Sensor," but instead
> "Device" -- or more pointedly, BioDevice. This might simply be a
> difference in the use in terminology, but I think it is important to
> clear up. The physical correlate of a BioSensor is an
> electrode/wire. Thus, a BioDevice will have one more more
> BioSensors. When you say above that "a Sensor is generating two
> channels of data," I think that shoud really read as "a BioDevice is
> generating two channels of data." This makes a difference in the
> way we use our BioDevice and BioSensor objects internally. It also
> helps keep behavior and function separate based on what we want to
> interact with -- a BioDevice or a BioSensor.
Okay, fine. A "BioDevice" typically has a whole bunch of
"BioSensors", then.
> What I like about your plan is the patch-chord kind of arrangment of
> the objects, where you can daisy-chain input and outputs together in
> a variety of configurations. This could keep data flows very
> flexible based on what may be needed later on down the line in terms
> of biofeedback protocols.
I think this is what everyone has been asking for all this time.
> Ok... except dev, fb1, fb2, and xmas would really be pointers kept
> in a list which would then be controlled by the node-supervisory
> code, right? The more I think about this, the more it seems like
> there should be a Network class to handle traversal of the nodes as
> well as adding or removing nodes from the list
Yes, that is a good idea, and I'm working with that idea now.
> What I would like to see is it more situated within the
> BioDevice/BioSensor framework. This would alleviate the need for
> node-supervisory code being in the node code (as globals or
> whatever), and keep the flow more natural in terms of the system.
I'm working on a test implementation now that doesn't use globals --
instead every Node belongs to a Network. That means we can have
several Networks running in different threads, or even in the same
thread. Even if we don't actually need multiple Networks, it is a
cleaner way to do it.
> For example, we've already touched on the issue of what kicks the
> whole network into motion. The natural starting place for that
> kick-off would be when data is received from a sensor. This is
> controlled by the BioDevice object, which reads the raw data stream
> in from the port/device and is able to separate it out into
> individual sensor streams for our use (or, plug one byte at a time
> into a receiving node as per your plan -- I am using the word
> "stream" here loosely). Otherwise, where (or who) would start the
> traversal of the node/port tree? If I am understanding correctly,
> that traversal will only occur when there is data ready from the
> external device for a particular port in the node list. Are there
> are other situations I'm not thinking of?
The only possibility is if there is more than one input stream we're
feeding into the network through different Nodes. Anyway, I'm going
with your idea of letting the BioDevice code control the network (or
at least have some 'main loop' alternate between calling the BioDevice
and running the network of nodes). As you say, this is all flexible
enough that we can add to and improve this later if necessary.
> If we do need to add some synchronization mechanisms, things like
> semaphores offer low overhead for this.
I think that if we are using a Network class to maintain the Nodes,
then we don't need the globals, and we can also do without semaphores
by using circular lock-free buffers for passing data between threads.
(I've used these before).
> Based on prior discussions, it seemed like others were not in favor
> of building in support for multiple devices, thus I am not sure
> under what conditions there would be a reason to have more than one
> producer node in the network. I would not mind having the
> flexibility of adding such capability, but am thinking that this
> will effect the answer to the question of who kicks off the
> traversal of the network when new data arrives.
I was thinking of work I did previously where I was handling a general
message-passing network, where messages might arrive from all kinds of
places. This required message generator 'nodes' to register the
filehandles they were waiting on with supervisor code which did the
select() call for all of them at once. We probably don't need
anything this complex, though, right ?
Actually, it was not only generator nodes that required select()
support, but also nodes that were waiting to write on blocking output
streams. Hopefully if we allow generous buffers between threads, the
buffers will only overflow in true processor-overload situations.
> Right now the way I see it is that the whole process will block
> until data is received from the external device (in BioDevice). I
> can see that there would be ways to extend this design later on if I
> or someone else really wants multiple device support.
Agreed.
As I've mentioned, I'm working on a test implementation of this. I'm
coding in simple C++, and using C for strings and so on.
The basic structure is there, but I've been trying to think of ways to
optimise it even further. However, I haven't found anything as
flexible and simple as what I've already suggested. With more
information provided by individual Nodes, certain other optimisations
could be made, but only at the expense of additional complexity for
both Node-writers and the Node/Network classes. So I'm sticking with
the simpler implementation.
The whole idea is to make something fairly robust, efficient and
flexible so that we can just forget about it and get on with using it
to build more interesting things.
Jim
-- Jim Peters (_)/=\~/_(_) jim_at_uazu.net (_) /=\ ~/_ (_) Uazú (_) /=\ ~/_ (_) http:// B'ham, UK (_) ____ /=\ ____ ~/_ ____ (_) uazu.net
This archive was generated by hypermail 2.1.4 : 2002-07-27 12:28:43 BST