Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
932bdde
Preliminary commit for Avro backend.
Nov 19, 2015
b2091f4
Adding test for avro to iterator
Nov 19, 2015
4cbe99f
Enabling read and write, adding append methods.
Nov 25, 2015
e07cfa7
Adding dependencies to build files
Nov 30, 2015
6cba475
Adding a convert edge from iterator back to avro
Dec 3, 2015
9ded2e1
Ensuring avro backend is registered on import
Dec 3, 2015
d7496ba
Fixing version problem with toolz
Dec 3, 2015
2adb2df
Adding support for array types
Dec 18, 2015
31ed159
Merge branch 'master' into avro
Dec 18, 2015
491c4d1
Attempting to fix travis build
Dec 18, 2015
2eeba0a
Reformatting test data
Dec 18, 2015
471becc
Adding conda selector for avro, since it is not Python 3 compatible
Dec 18, 2015
e05d1e1
Small fixes from code review.
Dec 19, 2015
31821d2
Converting tests to pytest idiom.
Dec 19, 2015
3773348
Changing import order to ensure test skip if avro not installed
Dec 19, 2015
2c0246a
Final small changes requested from code review
Dec 19, 2015
0d35449
Merge branch 'master' into avro
Dec 19, 2015
01b67a4
Use multipledispatch for schema discovery
Dec 22, 2015
a583e55
Bug fix for boolean types
Dec 22, 2015
eb004c7
Cleanup
Dec 29, 2015
58d4e6e
Adding dshape to avro schema generation, with doctest
Dec 30, 2015
64753e2
Bumping datashape version to fix build
Jan 4, 2016
d575ead
Merge branch 'master' into avro
Jan 7, 2016
5c2d082
Attempt to fix build for python3
Jan 29, 2016
78a591f
Whitespace error.
Jan 29, 2016
ee5e6f3
Python 3 support
Jan 29, 2016
ff32e9e
One more python3 bug I missed.
Jan 29, 2016
cbec204
Changing doctest not to be format sensitive.
Jan 29, 2016
4ebbf9b
Merge branch 'master' into avro
Feb 2, 2016
060c2b4
Updating avro type mappings
Feb 2, 2016
32900b9
Changing AVRO.uri attribute to AVRO.path, and allowing codec keyword …
Feb 2, 2016
95724cf
Dropping requirement for schema to be defined for new AVRO resource o…
Feb 2, 2016
76f2a72
Merge branch 'master' into avro
Jun 27, 2016
264fca2
Adding fastavro to meta.yaml
Jun 27, 2016
f47d8a8
Merge branch 'master' into ahasha-avro
kwmsmith Aug 17, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,11 @@ install:

# Install various deps
- conda uninstall toolz
- pip install -U toolz sas7bdat psycopg2 dill 'pymongo<3'
- pip install -U toolz sas7bdat psycopg2 dill 'pymongo<3' fastavro
- pip install git+git://github.com/blaze/dask.git#egg=dask-dev[complete]
- if [ -n "$PANDAS_VERSION" ]; then pip install $PANDAS_VERSION; fi
- if [[ $TRAVIS_PYTHON_VERSION == '3.4' || $TRAVIS_PYTHON_VERSION == '3.5' ]]; then pip install -U avro-python3; fi
- if [[ $TRAVIS_PYTHON_VERSION == '2.7' ]]; then pip install -U avro; fi

# install pyspark
- if [[ $TRAVIS_PYTHON_VERSION == '2.7' || $TRAVIS_PYTHON_VERSION == '3.4' ]]; then conda install spark=$SPARK_VERSION -c blaze -c https://conda.binstar.org/blaze/channel/dev -c anaconda-cluster; fi
Expand Down
3 changes: 3 additions & 0 deletions conda.recipe/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ requirements:

test:
requires:
- avro # [py27]
- avro-python3 # [not py27]
- fastavro
- pytest
- h5py
- pytables >=3.0.0
Expand Down
2 changes: 2 additions & 0 deletions extra-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,5 @@ sas7bdat
paramiko
pywebhdfs
boto
avro
fastavro
3 changes: 3 additions & 0 deletions odo/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,13 @@
from .backends.sparksql import SparkDataFrame
with ignoring(ImportError):
from .backends.url import URL
with ignoring(ImportError):
from .backends.avro import AVRO
with ignoring(ImportError):
from .backends.dask import dask



restart_ordering() # Restart multipledispatch ordering and do ordering


Expand Down
Loading