From: Sar Saloth (sarsaloth_at_yahoo.com)
Date: 2002-03-14 03:43:28
At 05:57 PM 2002-03-13 -0500, you wrote:
>On Wed, 13 Mar 2002 16:28:16 -0500, Sar Saloth wrote:
>
> >As far as the data file goes, I would continue writing the data, including
> >the junk and bad samples to get a complete character and not get the data
> >out of synch, and tag it with the suggested "Bad Data Marker" bit map
> >stream. That would fit in with your current logic flow, it wouldn't impact
> >much else, and if the BAD DATA Marker were at the end of the record, you
> >could tag it afterwards. That way your EDF would always line up right.
>
>Actually, by doing the above I am thinking that it does not impact the logic
>flow at all since it would be up to the "FileStorage" class to handle the
>error
>bit (and thus padding out the rest of the data record with zeros if that
>is the
>way we want to go). Unless.... getting out of sync would also disturb other
>processes, like FFT filters. Jim-P, I think I recall you saying that it is
>worse to have missing data rather than zero-value samples representing that
>missing data for the FFT filters. Is this correct? Shoot--I wish I knew how
>other packages handled this and how big an issue it really is.
I suspect that it would be better to create one zero sample than cause an
unknown phase shift due to jitter. One zero "Spike" should smear evenly
across the spectrum of an FFT, depending on where it occurred. How often
are these things supposed to happen?
Reasoning (so Joerg can correct me if my understanding of the math is
inadequate). A Zero (space-stuffing) impacts the FFT the same as a single
sample spike of amplitude negative to the lost sample. In real-life a
single spike with no ringing etc. has a very wide frequency
spectrum. Hopefully, if it is spread enough, a lot of it would get buried.
Another reason why padding with zeroes would be accuracy. If you lost sync
you know that there was data that was lost, so there was at least one
record before you picked up synch again. If you just ignore that data
record, you are guaranteeing being out by that one data record. If the PC
was in la-la land for a long long time, maybe you missed the next synch
too, but even then creating a padded record is one closer than obliterating
the bad data. Also, then how are you going to mark anything bad?
If these events are rare, is it worth worrying about? Is it worth trying
to reconstruct data around the error when it is rare? It might be better
just to know it happened.
> >This brings up an annoyance with EDF - they organize the channels
> >contiguously in one record, so that if your record size was larger (due to
> >the desire to keep the low data rate signals low) then you are up to the
> >latency of a record. -
> >explanation:
> >Every data record has identical size and channel grouping, so the slowest
> >rate signal must be present in every data record. Either one pads and
> >wastes space (by duplicating slow channels) or one accepts the latency is
> >at least as bad as the low data rate. (for comm purposes).
> >Does that throw EDF out of the window for future CheapEEG communication
> >formats? (this is separate from the actual file format issue).
>
>Actually, I'm not sure this is even an issue with the EEG devices being
>developed here, because the sample rate will be the same for all the
>channels.
>I think the only time you would run into this problem is with a device that
>transmits multiple modalities at differing rates and/or the mulitple device
>scenario that I brought up earlier.
Sounds good. So latency is a non-issue when everything is at one rate. Yes!
Also, if everything is at one rate, then a record could be one sample so a
record would be the SAME format as in "OpenEEG-1.0".
> >>I like doing it this way because I am able to send the data out
> immediately,
> >>and I have to think that it would be rare to receive a corrupt packet
> >>set. But
> >>what do you (or others) think? Would it be better and/or indifferent
> to send
> >>data out in 8 sample bursts (for EEG) and 3 sample bursts (for non-EEG)
> data
> >>for this device? Is there a situation where it would matter?
> >
> >Wouldn't the only real difference be latency? Would such latency be
> adequate?
>
>I think so, which would be more of an issue for feedback/stimulation.
>Personally, it just seems cleaner to have as continuous and consistent stream
>rather than sudden bursts, even if those "bursts" are only approximately 1/10
>of a second each. (I mistyped above, the "8" should be "24" for EEG sample
>data. Thus, there are about 10.66 EEG sample sets every second to get 256
>samples/sec.)
As long as the record doesn't need to be completely assembled before
transmission starts, would there be a difference between "bursts" and
continuous? That is the only problem that I see with EDF for data
transmission - since all of the samples for one channel are grouped
together, you can't send out the samples until you essentially have all of
the samples in the record. As you state, if everything is at the same
rate, 1 sample could be one record and we are all the way there, we don't
need to assemble the record before starting to send it.
> >>Does this effect data storage using the methods we have been talking about
> >>since we are storing "chunks" and if I suddenly realize that I am no longer
> >>synced on correct data boundaries? Should I just discard whatever I
> have of
> >>the current set, or save it even though this might mean that I only have
> >>say, 6
> >>EEG samples (and no non-EEG samples) that I can reasonably be sure of
> at that
> >>point for the current packet set?
> >
> >As along as you save complete EDF records and mark them as bad then your
> >current scheme should be OK, is that right?
>
>Theoretically, I think so. It might show up as odd for EDF viewers that did
>not recognize the error channel, but I don't know how important that is to
>this
>project anyway.
It shouldn't matter, in fact it is more important to anything that is
algorithmically derived, in which case bad-data markers would be
important. Also, a viewer might be able to view the bad data marker, since
zeroes means that the data is good.
<snip>
> DSP and photo/magstim operate at 8 millisec; *brainstem* speeds :)
<snip>
>That seems fast enough. :)
>Dave.
8 ms on feedback. YEEHA! I am not playing a video-game against that person!
I guess anyone into such a low latency wouldn't mind running everything at
the same rate to reduce latency anyway!! As you said, that is not going to
be an issue then.
Sar
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
This archive was generated by hypermail 2.1.4 : 2002-07-27 12:28:40 BST