Re: File validity stuff - Re: [buildcheapeeg] EDF and DDF file formats

From: Sar Saloth (sarsaloth_at_yahoo.com)
Date: 2002-03-14 16:33:47


At 11:05 PM 2002-03-13 +0000, you wrote:
>Sar Saloth wrote:
> > You could, as long as we all agreed up front. requirements - I want
> > something that gives checksum or CRC, maybe has error bits in the
> > same thing, and I also want to sequence number the records (to check
> > for missing or doubled records). I can see a reason for keeping the
> > bad data marker separate in that user application level software
> > needs to see the bad data, but it might be an outer layer that does
> > CRC checking and sequence number checking. I might be completely
> > wrong here - this is where my lack of experience is putting me out
> > of my element. comments?
>
>The trouble is that we're talking about two completely different
>tasks.

Yes, I was just hoping to use one set of code for both.

>You want to use EDF for transmitting data along a serial cable
>from your hardware, whereas we are also discussing using EDF (or
>something based on it) for storing data on the hard drive, and as a
>common format for converting other data to.

Yes.

>For instance, I would say we could keep whatever serial data format
>the hardware is sending right now, for example, and then convert to a
>common storage format later.

Yes, of course we wouldn't ask anyone to rewrite the firmware in their
working devices. However, if we agree on a flexible communications format
then it should be a simple and nicely self-contained bit of code with well
defined interfaces to do the conversion. Is that a silly statement due to
my lack of modern programming experience? - it might be. Hopefully future
products could use the flexible method of communication.

Yes, I am guilty in the above paragraph of trying to use an communications
protocol as an interface specification between bits of code. Is that a bad
thing? (that is a real question, not a rhetorical one).

Maybe the important thing is a common interface in the library?

>I think these two formats -- the serial transfer format and the
>storage format need to be separate. There are things like cycling
>numbers that you might want in your serial format that you don't want
>in the stored format, and there are things like annotation streams
>(taking an EDF example) that you might want in the stored file that
>you definitely don't want to be wasting bytes transferring over the
>serial link from the hardware device.

For reasons of dealing with FDA and regulators etc. I will personally keep
that CRC and sequence junk etc. in my data file, but I understand why you
don't want it (because functionally it is useless). Personally I think
that a bit of disk space isn't much of an issue these days. Since I was
thinking that the extra junk would be optional, it isn't really a pro or a
con for keeping the data formats the same (as long as we understand that
some translation may be required when going from data stream to the data
file to change for example the number of channels).

As far as the annotation stream, if the device had no annotations to send,
it couldn't send it if it wanted. If it was an event, maybe the event
codes would be more efficient? Unfortunately, the event code date-time
stamp still looks like it consumes a lot of characters. It is especially
bad if we make one record be one sample to minimize latency as the
annotations (or event stream or anything) will then take as much space as
one channel.

These things are all optional.

If the device has no annotations to write this is not an issue. If there
are annotations, then the data rate taken in EDF is a function of the PEAK
information rate (how quickly your annotations can come).

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com



This archive was generated by hypermail 2.1.4 : 2002-07-27 12:28:41 BST