Skip to content

Conversation

@szz-dvl
Copy link
Contributor

@szz-dvl szz-dvl commented Mar 12, 2025

Reference issue (if any)

Feature #8976.

What does this implement/fix?

Added support for file-like objects for EDF/BDF/GDF files.

Additional information

Until today, mne was checking for the file extension to validate file types, we may not have a file name from a binary object, so, if a binary object (file-like) is provided we trust the user (depending on what API function he is using: read_raw_edf, read_raw_gdf, read_raw_bdf). A new enumeration has been declared to propagate the file type to underlying methods:

class FileType(Enum):
    """Enumeration to differentiate files when the extension is not known."""

    GDF = 1
    EDF = 2
    BDF = 3

In the function _read_edf_header a new exception is raised if the header can be properly parsed, informing the user that the file provided is not valid:

try:
     header_nbytes = int(_edf_str(fid.read(8)))
except ValueError:
    raise Exception(
        f"Bad {'EDF' if file_type is FileType.EDF else 'BDF'} file provided."
    )

In the function _read_gdf_header, a new exception is raised when the version of the document can't be properly parsed, informing the user that the file provided is not valid:

try:
    version = fid.read(8).decode()
    edf_info["type"] = edf_info["subtype"] = version[:3]
    edf_info["number"] = float(version[4:])
except ValueError:
    raise Exception("Bad GDF file provided.")        

To accommodate this changes the class RawEDF class has been duplicated (RawBDF) to differentiate between BDF and EDF files when the extension is not known. To avoid changes in the file mne-python/mne/io/tests/test_raw.py.

Additionally 4 new tests have been added to the file mne-python/mne/io/edf/tests/test_edf.py and 2 new tests have been added to the file mne-python/mne/io/edf/tests/test_gdf.py, those tests will open a valid file and check for the channels, and open an invalid file and expect an error. For GDF I'm using an empty file (I don't know where to get bad GDF files).

I've created a helper function to open GDF and EDF files under mne-python/mne/_edf/open.py.

To support in memory file-like objects I'm using the workaround mentioned here, the function has been placed in "fixes.py". The calls to numpy.fromfile have then been replaced by calls to the new function read_from_file_or_buffer. The implementation is as follows:

def read_from_file_or_buffer(
    file: str | bytes | os.PathLike | io.IOBase, dtype: numpy.typing.DTypeLike = float, count: int = -1
):
    """numpy.fromfile() wrapper, handling io.BytesIO file-like streams.

    Numpy requires open files to be actual files on disk, i.e., must support
    file.fileno(), so it fails with file-like streams such as io.BytesIO().

    If numpy.fromfile() fails due to no file.fileno() support, this wrapper
    reads the required bytes from file and redirects the call to
    numpy.frombuffer().

    See https://github.com/numpy/numpy/issues/2230
    """
    try:
        return numpy.fromfile(file, dtype=dtype, count=count)
    except io.UnsupportedOperation as e:
        if not (e.args and e.args[0] == "fileno" and isinstance(file, io.IOBase)):
            raise  # Nothing I can do about it
        dtype = numpy.dtype(dtype)
        buffer = file.read(dtype.itemsize * count)
        return numpy.frombuffer(buffer, dtype=dtype, count=count)
        

@welcome
Copy link

welcome bot commented Mar 12, 2025

Hello! 👋 Thanks for opening your first pull request here! ❤️ We will try to get back to you soon. 🚴

@szz-dvl szz-dvl marked this pull request as draft March 20, 2025 18:28
@szz-dvl szz-dvl marked this pull request as ready for review March 20, 2025 20:30
szz-dvl and others added 17 commits April 24, 2025 21:13
* upstream/main: (85 commits)
  FIX: Fix ICA.apply when fitted including marked bad channels (mne-tools#13478)
  FIX: Correctly set the calibration factor in Nihon Kohden reader (mne-tools#13468)
  [pre-commit.ci] pre-commit autoupdate (mne-tools#13479)
  MAINT: Update code credit (mne-tools#13477)
  Fix `versionadded` directive formatting (mne-tools#13471)
  typo in mailmap (mne-tools#13475)
  FIX: Fix _plot_topomap channel names plotting when using a mask (mne-tools#13470)
  FIX: Handle an Eyelink File with blank lines injected throughout file (mne-tools#13469)
  ENH: adds annotation filtering to raw and ica source figures (mne-tools#13460)
  MAINT: Restore VTK nightly wheel on Linux and bump changelog checker (mne-tools#13436)
  [pre-commit.ci] pre-commit autoupdate (mne-tools#13465)
  FIX: Fix add_reference_channels for passing two channels names (mne-tools#13466)
  ENH: Add on_missing for combine_channels (mne-tools#13463)
  Bump the actions group with 2 updates (mne-tools#13464)
  Move development dependencies into a dependency group (no more extra) (mne-tools#13452)
  ENH: add on_missing for rename_channels (mne-tools#13456)
  add advisory board to website (mne-tools#13462)
  ENH: Support Nihon Kohden EEG-1200A V01.00 (mne-tools#13448)
  MAINT: Update dependency specifiers (mne-tools#13459)
  ENH: Add encoding parameter to Nihon Kohden reader (mne-tools#13458)
  ...
@larsoner
Copy link
Member

larsoner commented Nov 5, 2025

Sorry we forgot about this one @szz-dvl -- feel free to ping us if it's time for a (re-)review and it has been a couple of weeks! I pushed a merge with upstream/main and a couple of tiny cosmetic changes, otherwise looks like you addressed the comments above so I'll mark for merge-when-green. Thanks in advance @szz-dvl !

@larsoner larsoner enabled auto-merge (squash) November 5, 2025 18:40
@larsoner larsoner added this to the 1.11 milestone Nov 5, 2025
@szz-dvl
Copy link
Contributor Author

szz-dvl commented Nov 5, 2025

Hey @larsoner I completely forgot about this one, Please give me tomorrow morning to review everything and I think we will be ready to go.

Thanks to you!

@szz-dvl
Copy link
Contributor Author

szz-dvl commented Nov 6, 2025

@larsoner I'm not able to see what is going wrong with docs and lincheck, any clue?

Thanks

@larsoner
Copy link
Member

larsoner commented Nov 6, 2025

It has to do with CircleCI build permissions... I pushed an empty commit that should come back green. I'll mark for merge-when-green, thanks in advance @szz-dvl !

@larsoner larsoner enabled auto-merge (squash) November 6, 2025 14:47
@larsoner larsoner merged commit 6f42037 into mne-tools:main Nov 6, 2025
32 checks passed
@welcome
Copy link

welcome bot commented Nov 6, 2025

🎉 Congrats on merging your first pull request! 🥳 Looking forward to seeing more from you in the future! 💪

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants