Re: File validity stuff - Re: [buildcheapeeg] EDF and DDF file formats

From: Jim Peters (jim_at_uazu.net)
Date: 2002-03-13 23:05:57


Sar Saloth wrote:
> You could, as long as we all agreed up front. requirements - I want
> something that gives checksum or CRC, maybe has error bits in the
> same thing, and I also want to sequence number the records (to check
> for missing or doubled records). I can see a reason for keeping the
> bad data marker separate in that user application level software
> needs to see the bad data, but it might be an outer layer that does
> CRC checking and sequence number checking. I might be completely
> wrong here - this is where my lack of experience is putting me out
> of my element. comments?

The trouble is that we're talking about two completely different
tasks. You want to use EDF for transmitting data along a serial cable
from your hardware, whereas we are also discussing using EDF (or
something based on it) for storing data on the hard drive, and as a
common format for converting other data to.

For instance, I would say we could keep whatever serial data format
the hardware is sending right now, for example, and then convert to a
common storage format later.

I think these two formats -- the serial transfer format and the
storage format need to be separate. There are things like cycling
numbers that you might want in your serial format that you don't want
in the stored format, and there are things like annotation streams
(taking an EDF example) that you might want in the stored file that
you definitely don't want to be wasting bytes transferring over the
serial link from the hardware device.

Also, it is errors in the serial transfer that we are planning to
detect to put in the error bits, so what is the point in sending these
initially-zero error-bits along the serial line from the hardware ?

Serial transfer format needs things like this:

- Sync bytes (or bits)
- Cycling counters (maybe)
- CRC checks (maybe)

Stored format needs things like this:

- Notes of any errors in recording process
- Space to put lots of textual data
- Facilities for markers + annotations

Maybe both of these can be based on EDF, but I don't think you would
want to use exactly the same channels and formats for both parts of
the process.

> I was thinking of some kind of labelling scheme for these weird
> channels so that it wouldn't show up, even if we had something that
> didn't properly handle it. Or else we could come up with a new
> quantity or something?

In EDF you can give it a label, so you can label it "ERROR_BITS+CRC",
or something like that, and that way people will know to ignore it.

> The beauty of it is that we can just use EDF as it is and add this
> in when and if the time came as long as we agreed on a couple little
> things. That means nothing gets held up by this, as long as there
> is no shortcoming in EDF for our purpose.

As I said in the other posting, I think we could do better than EDF,
especially if people have lots of extra data to store, but EDF would
be usable if we're willing to give up that kind of flexibility. I
don't really know what the requirements are of people for the storage
of accompanying textual information.

Is EDF good enough, or do we want more flexibility ?

Jim

-- 
Jim Peters (_)/=\~/_(_) jim_at_uazu.net
(_) /=\ ~/_ (_)
Uazú (_) /=\ ~/_ (_) http://
B'ham, UK (_) ____ /=\ ____ ~/_ ____ (_) uazu.net


This archive was generated by hypermail 2.1.4 : 2002-07-27 12:28:40 BST