Enable AxonaIO to read spiketimes and waveforms #986

JuliaSprenger · 2021-04-30T12:53:59Z

This PR implements the required rawio methods for reading spiketimes and waveforms. It also restructures the IO to be able to support multi-file datasets (.bin & unit files).

This PR depends on the Axona unit test data that will be added to the ephy_testing_data: https://gin.g-node.org/NeuralEnsemble/ephy_testing_data/pulls/49/files

pep8speaks · 2021-04-30T12:54:02Z

Hello @JuliaSprenger! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-06-18 12:16:53 UTC

sbuergers

Very nicely done. I mostly have nit-picky comments. I will send you the sample data so we can run proper tests with neo (should then also include the filenames in the rawio and io tests of axona).

neo/rawio/axonarawio.py

JuliaSprenger · 2021-05-06T11:36:26Z

@samuelgarcia Do you have any comments for this PR? I already rebased to include the latest changes in the test suite.

JuliaSprenger · 2021-05-07T14:40:25Z

I added more testfiles to gin (https://gin.g-node.org/NeuralEnsemble/ephy_testing_data/pulls/54). @samuelgarcia can you check these and merge both PRs if you are happy with the current state?

JuliaSprenger · 2021-05-11T09:05:48Z

Hi @samuelgarcia it seems like the new test data files are not automatically downloaded from gin. Can we do a manual reset of the cache?

samuelgarcia · 2021-05-11T11:38:16Z

Weird. I will have a look.
Even with the cache the file should be retreive if not present.
What happen on local mahcine ?

JuliaSprenger · 2021-05-11T11:41:02Z

I observed the same effect locally. The new files were not automatically downloaded and I had to delete the complete ephys_testing_data folder to trigger a new download.

samuelgarcia · 2021-05-11T11:46:13Z

STrange. I will have a look.

sbuergers

I think with this patch it should be good now (besides the gin issues mentioned above):

JuliaSprenger#1

sbuergers · 2021-05-11T22:12:33Z

neo/rawio/axonarawio.py

+        # spike times are repeated for each contact -> use only first contact
+        unit_spikes = raw_spikes['spiketimes'][::4]
+
+        # slice spike times only if needed


Currently, when t_start and t_stop are given the whole timeseries is returned.

Only specific segments are returned when either t_start or t_stop is None.

I made a local patch for this, and will still be working with it today. If it works fine I will either just send it to you or put it in the catalystneuro fork.

JuliaSprenger · 2021-05-17T17:12:33Z

@samuelgarcia Any news regarding the download of the new test data?

JuliaSprenger · 2021-05-21T14:38:30Z

@samuelgarcia I cherry-picked your datalad fix also in this branch, so now all files are downloaded properly and tests pass.
Ready for final review / merge from my side.

JuliaSprenger · 2021-06-11T17:31:08Z

Hi @samuelgarcia with the fetching of the testfiles fixed and this PR being in a completish state for some time already it's up to your final approval for this to be merged!

neo/rawio/axonarawio.py

…bin file

…analogsignal_chunk when no .bin file

…arawio (rather than depend on spikeextractors)

This reverts commit 09520c5.

This reverts commit 0340f4f.

This reverts commit 5b1b934.

… to axonarawio (rather than depend on spikeextractors)" This reverts commit cbed9f3.

…for get_analogsignal_chunk when no .bin file" This reverts commit cccb739.

…loat

JuliaSprenger · 2021-06-18T13:24:05Z

I had to rebase on master to resolve conflicts with base, but now all comments are addressed. @samuelgarcia You can either provide more feedback or merge.

samuelgarcia · 2021-06-18T13:35:19Z

This looks good.
Do you want I merge now ?
What conflit you want to fix ?

JuliaSprenger · 2021-06-18T14:37:59Z

I am ready to merge. @samuelgarcia go ahead ;)

sbuergers suggested changes May 2, 2021

View reviewed changes

JuliaSprenger marked this pull request as ready for review May 6, 2021 10:50

JuliaSprenger force-pushed the enh/axona_spikes branch from 235efb2 to a073f6b Compare May 6, 2021 11:11

apdavison modified the milestone: 0.10.0 May 10, 2021

apdavison added the IO label May 10, 2021

apdavison assigned samuelgarcia May 10, 2021

JuliaSprenger mentioned this pull request May 11, 2021

Axonarawio #958

Merged

This was referenced May 13, 2021

Enh/axona spikes patch indexing JuliaSprenger/python-neo#1

Merged

made channel_names annotation 1 indexed in line with axona file format #974

Merged

Axonaunitextractor SpikeInterface/spikeextractors#637

Merged

sbuergers suggested changes May 17, 2021

View reviewed changes

sbuergers mentioned this pull request May 31, 2021

Fix/allow missing bin file (axonarawio) #993

Closed

JuliaSprenger mentioned this pull request Jun 11, 2021

filename needs to have .set or .bin extension #972

Closed

JuliaSprenger requested a review from samuelgarcia June 11, 2021 17:29

samuelgarcia reviewed Jun 15, 2021

View reviewed changes

neo/rawio/axonarawio.py Outdated Show resolved Hide resolved

samuelgarcia reviewed Jun 15, 2021

View reviewed changes

neo/rawio/axonarawio.py Outdated Show resolved Hide resolved

samuelgarcia reviewed Jun 15, 2021

View reviewed changes

neo/rawio/axonarawio.py Outdated Show resolved Hide resolved

samuelgarcia reviewed Jun 15, 2021

View reviewed changes

neo/rawio/axonarawio.py Show resolved Hide resolved

sbuergers and others added 19 commits June 18, 2021 13:32

ensure signal_streams and signal_channels are added when there is no …

362f48b

…bin file

estimate total number of samples from setfile sr and duration if no bin

95e52cf

cast duration as float, not int (can be decimal)

5e0d978

signal size has to be int!

b5d93b0

ok, final fix for returning signal size

a222ddf

pep8 changes

01a38c1

use spikeextractors.axonaunitrecordingextractor.get_traces() for get_…

4b89142

…analogsignal_chunk when no .bin file

add functions for creating mock cont signal when no .bin file to axon…

cb5a877

…arawio (rather than depend on spikeextractors)

pep8 changes

4ddf198

remove reference to get_traces in get_tetrode_channel_table

a78bbd8

linting change

0825012

Revert "remove reference to get_traces in get_tetrode_channel_table"

c58651e

This reverts commit 09520c5.

Revert "pep8 changes"

6064275

This reverts commit 0340f4f.

Revert "linting change"

26c3cc6

This reverts commit 5b1b934.

Revert "add functions for creating mock cont signal when no .bin file…

41075bf

… to axonarawio (rather than depend on spikeextractors)" This reverts commit cbed9f3.

Revert "use spikeextractors.axonaunitrecordingextractor.get_traces() …

6f76bbd

…for get_analogsignal_chunk when no .bin file" This reverts commit cccb739.

only call _get_signal_streams_header() if .bin file is avilable

b94dd1d

use signal length 0 if no signals is present

1663de2

[neuralynx] readd comment

43e5c65

JuliaSprenger force-pushed the enh/axona_spikes branch from 4e38c0e to 43e5c65 Compare June 18, 2021 11:33

sprenger added 5 commits June 18, 2021 13:41

[axona] clarify tetrode-unit correspondence

cc3c832

[axona] code clarification

2ee76d0

[axona] make spiketime/waveform handling 2D

7cfb8b8

[axona] cleanup

5f3a7c0

[axona] change default dtype for header item conversion from int to f…

5fe92d5

…loat

samuelgarcia merged commit 3d8b139 into NeuralEnsemble:master Jun 21, 2021

JuliaSprenger deleted the enh/axona_spikes branch April 3, 2023 13:58

Enable AxonaIO to read spiketimes and waveforms #986

Enable AxonaIO to read spiketimes and waveforms #986

Uh oh!

Conversation

JuliaSprenger commented Apr 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pep8speaks commented Apr 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment last updated at 2021-06-18 12:16:53 UTC

Uh oh!

sbuergers left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JuliaSprenger commented May 6, 2021

Uh oh!

JuliaSprenger commented May 7, 2021

Uh oh!

JuliaSprenger commented May 11, 2021

Uh oh!

samuelgarcia commented May 11, 2021

Uh oh!

JuliaSprenger commented May 11, 2021

Uh oh!

samuelgarcia commented May 11, 2021

Uh oh!

sbuergers left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sbuergers May 11, 2021

Choose a reason for hiding this comment

Uh oh!

JuliaSprenger commented May 17, 2021

Uh oh!

JuliaSprenger commented May 21, 2021

Uh oh!

JuliaSprenger commented Jun 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JuliaSprenger commented Jun 18, 2021

Uh oh!

samuelgarcia commented Jun 18, 2021

Uh oh!

JuliaSprenger commented Jun 18, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

JuliaSprenger commented Apr 30, 2021 •

edited

Loading

pep8speaks commented Apr 30, 2021 •

edited

Loading

sbuergers left a comment •

edited

Loading

JuliaSprenger commented Jun 11, 2021 •

edited

Loading