From: Jim Peters (jim_at_uazu.net)
Date: 2002-04-03 13:39:41
Dave wrote:
> That's an interesting solution, having two networks and bridge nodes
> between them. What is going to get tricky is "patch-chording"
> between the networks. You would not be able to have any nodes in
> NetworkA (the fast network) be dependent on the output from anything
> in NetworkB (the slow network).
Yes -- using a setup like this, it would have to be one-way
communication only: from network A (fast) to network B (slow), but not
back again. Really this will all be a lot easier to judge when we
know the limitations of some of the processing nodes we might want to
be running.
> This could get complicated. John's post has me thinking that
> perhaps it would simplify things to keep data acquisition and data
> processing in two separate threads, rather than try to separate slow
> process()'s out piecemeal. It might make the whole system more
> managable?
You might be right there -- put the serial port handler in a
high-priority thread by itself so that processing delays won't affect
it, and then a big FFT every 16 samples might not be such a big deal.
In any case, we have the flexibility to choose either solution closer
to the time when we know how this might actually need to work.
> To change it in the way we are talking about above, I guess I would
> have to pass the BioNodeNetwork object into BioDevice when the
> BioDevice object is created.
Agreed -- you need to tell it which Network it belongs to.
> Does this mean that the instantiated Network object is going to need
> to be global? Or should it be a part of BioDevice itself? The
> latter only makes sense if we are committed to there being only one
> input source per network. But if not, then a Network is more
> autonomous...
I think whatever code sets up the BioDevice also sets up the rest of
the Network. The Network pointer doesn't have to be a global -- it
just gets passed to all the Nodes created in that part of the program.
After that point, they know which Network they belong to (assuming
they stored the pointer somewhere).
I think this will become clearer as other parts develop.
> Could you explain a little bit more of what you mean by circular
> lock-free buffers? Do you mean having something like a two-way pipe
> stream between the threads?
No, I'm talking about a one-way pipe that is very quick and efficient
and doesn't require any form of locking, so neither thread has to wait
even for a moment (unless the buffer is full or empty, of course).
There might be a complicated name for this kind of thing, but I don't
know what it is. Here is a simple example implementation:
struct Buffer {
int rd; // Offset of read-pointer into buf[]
int wr; // Offset of write-pointer into buf[]
double buf[256];
};
void
buffer_init(Buffer *bb) {
bb->rd= bb->wr= 0;
}
// Returns 0: no data available, 1: success, data returned in *rvp
int
buffer_read(Buffer *bb, double *rvp) {
if (bb->rd == bb->wr) return 0;
*rvp= bb->buf[bb->rd];
bb->rd= 255 & (bb->rd+1);
return 1;
}
// Returns 0: can't write, buffer full; 1: success, data written
int
buffer_write(Buffer *bb, double val) {
int new_wr= 255 & (bb->wr+1);
if (new_wr == bb->rd) return 0;
bb->buf[bb->wr]= val;
bb->wr= new_wr;
return 1;
}
One thread calls only buffer_read(), and the other calls only
buffer_write(). There are no race conditions because only the first
thread updates the ->wr member, and only the second thread updates the
->rd member.
This is all based on the assumption that a write on an 'int' memory
location is an atomic operation -- i.e. it is not possible for the
other thread to read the ->rd or ->wr member and get a scrambled
value. I checked one of the POSIX libraries once, and writing an
'int' to memory is atomic on every processor in common use today, so
this is safe, I believe. I can check again if anyone needs proof.
> NetworkA and NetworkB could have one set of nodes for their output.
> Their process() method could simply trigger an event in
> BioDataNetwork which wakes it up to then process its network. I
> dunno... it is certainly more straightforward to have one input
> source per Network, but I like to keep my options open, too, for
> future needs.
Let's just see how it goes.
> That, or make them non-blocking if we don't want to hang up our
> network waiting on a write(). I think we will get into some
> portability issues with that, though.
Sorry -- you do need to set them non-blocking in any case. What I
mean is that when you've written everything you can to an output
stream, and it can't take any more for the moment, then on UNIX you
have to use a select() to wait for the output stream to clear a bit
before you can write some more. That means that both input and output
on UNIX file-descriptor streams requires the use of select(), which
means that both input and output nodes need to register their
file-descriptors in the kind of system I was talking about. They get
notified when there is either new data (input) or new space to write
to (output).
If output file descriptors can't be set non-blocking, then their
output code has to be moved to another thread unless you're willing to
have your network stall every time it has to wait for the output
stream. Anyway, I think perhaps this is getting a bit ahead of what
we're actually discussing.
> >As I've mentioned, I'm working on a test implementation of this.
> >I'm coding in simple C++, and using C for strings and so on.
>
> That's great. I had a feeling that you might be doing something
> like this also. It will be great to compare notes.
Yes, this could be interesting ...
Actually, it's good that you're working with it too, because you are
bringing in new ideas and approaches that I hadn't thought about.
Jim
-- Jim Peters (_)/=\~/_(_) jim_at_uazu.net (_) /=\ ~/_ (_) Uazú (_) /=\ ~/_ (_) http:// B'ham, UK (_) ____ /=\ ____ ~/_ ____ (_) uazu.net
This archive was generated by hypermail 2.1.4 : 2002-07-27 12:28:43 BST