Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
931a6e2
got rid of default year in nfl
Jul 3, 2016
9b4c32d
got rid of default year in nba
Jul 3, 2016
580ef79
updated cache dir name
Jul 3, 2016
6241ddc
fixed problems from the PFR redesign for nfl.players
Jul 11, 2016
857f16b
fixed problems from the PFR redesign for nfl.teams
Jul 11, 2016
dc09951
changed from requests to selenium, added parseInfoTable utils function
Jul 24, 2016
f9c48fe
added ascii encoding in cacheHTML decorator
Jul 24, 2016
375a0d8
fixed problems from the PFR redesign for nfl.boxscores and nfl.pbp
Jul 24, 2016
62df006
added year param to nfl.teams functions, other small changes
Jul 25, 2016
2747398
updated, refactored constants and imports in nfl.finders
Aug 6, 2016
aa9b4b9
more refactoring, fixed up finders
Aug 6, 2016
03b253a
small fixes, lots of updates to nfl.teams
Aug 6, 2016
97ea7f5
encode UTF8 fix in cacheHTML
Aug 6, 2016
9836890
updated cacheValid function for PFR
Aug 17, 2016
949883b
added hashing to cacheHTML and small change to memoized
Aug 17, 2016
b54d79f
updated import style to relative imports
Aug 17, 2016
0dc38f3
a bunch of implementations
Aug 17, 2016
f210c73
added plays functions to nfl.Player
Aug 17, 2016
26e1f5c
implemented nfl.Player splits, some refactoring
Aug 18, 2016
15d7574
advanced splits in nfl.Player, changed sportsref.nfl to underscore_case
Aug 21, 2016
79c724f
made sportsref.nba underscore_case
Aug 21, 2016
633da79
got rid of ncaaf and ncaab
Aug 21, 2016
e572ec3
changing module to (mostly) standardized underscore_case for methods
Aug 21, 2016
7db63bf
simple time-based caching for now. cache functions are complicated
Aug 31, 2016
c69fa25
added str and repr, changed bsID to boxscoreID, fixed bug in nfl.Team…
Aug 31, 2016
2652901
updated README
Aug 31, 2016
539fee6
Merge branch 'develop' for next release (0.7)
Aug 31, 2016
b790ff3
updated setup.py for 0.7.0
Aug 31, 2016
e99a3ca
added self parameter
Sep 27, 2016
16b577d
version 0.7.1
Sep 27, 2016
2087481
applied autopep8
Oct 25, 2016
af29d9c
Merge branch 'master' into develop
Oct 25, 2016
e856420
added nba.players
Nov 22, 2016
0637289
complying with PEP8
Nov 22, 2016
c63a10a
fixed pep8 stuff, fixed imports, some restructuring. fixed bugs in PSF
Nov 22, 2016
a2e9b71
enhanced parse_table
Nov 22, 2016
9e9ff21
added leagues/... to rel_url_to_id
Nov 22, 2016
baa238f
updated kind_rpb
Nov 22, 2016
c610d5d
added reg/playoffs to NBA player stats functions
Nov 22, 2016
73973c9
updated nba.boxscores, nba.seasons, and added an option to parse a ta…
Nov 24, 2016
82642be
changed playerID to player_id, continued changing to underscores
Nov 25, 2016
5421bf1
updates to nba.seasons
Nov 25, 2016
d932299
Merge branch 'develop'
Nov 27, 2016
5c102bb
version 0.7.2
Nov 27, 2016
1d2a379
changed naming in nba.Player
Nov 27, 2016
fc40d56
Changed names in nba.Player
Nov 27, 2016
f6622a6
version 0.7.3
Nov 27, 2016
6f002ec
better variable name in get_html
Dec 13, 2016
7802159
converted boxscoreID to boxscore_id
Dec 14, 2016
27f0fd7
took out set_index in nba.Season
Dec 14, 2016
0c1e5c4
version 0.7.4
Dec 14, 2016
f2b22a0
Merge branch 'develop' for version 0.7.4
Dec 14, 2016
8be32c3
got rid of set_index calls (at least for now)
Dec 14, 2016
c6ac0ab
version 0.7.5
Dec 14, 2016
ae7f100
one more set_index
Dec 14, 2016
f6d4eef
version 0.7.6
Dec 14, 2016
50d1f36
cache all html for now
Dec 14, 2016
66562d0
version 0.7.7
Dec 14, 2016
33be22b
fixed bug on KG's page
Dec 15, 2016
689bb36
version 0.7.9
Dec 15, 2016
5e4b5f2
version 0.7.10
Dec 15, 2016
b12b44e
fixed some outdated parts of nfl subpackage
Dec 19, 2016
d04b7e9
Version 0.7.11
Dec 19, 2016
4ae5a68
Revert back to requests, uncommenting tables embedded in HTML comments
Dec 21, 2016
0828b89
changed version to 0.7.12
Dec 21, 2016
3d3225d
version 0.7.12 - switched back to requests
Dec 21, 2016
4dd14c8
added actual uncommenting in get_html for requests
Dec 21, 2016
de80cb2
version 0.7.13
Dec 21, 2016
6978dfe
fixed unicode problems in cache_html, version 0.7.14
Dec 21, 2016
f42b1db
Version 0.7.14 - fixed unicode problems in cache_html
Dec 21, 2016
82ca3c6
changed license to GNU GPLv3
Dec 21, 2016
85b0d93
Merge branch 'develop'
Dec 21, 2016
0c77589
got rid of license
Dec 21, 2016
cfc871b
Make sure develop branch is updated
Dec 21, 2016
02ddba6
fixed 404 error handling
Dec 21, 2016
dc61114
NOW fixed unicode problems (I hope)
Dec 21, 2016
f8d25df
added conditions to only change column names if flatten=True in parse…
Dec 21, 2016
74dda6d
fixed outdated parts of nba.BoxScore
Dec 21, 2016
febd951
added sphinx for documentation
Dec 21, 2016
6983394
moved doc directory to project root
Dec 22, 2016
f5ef3de
changed 'memoized' to 'memoize'
Dec 25, 2016
e52e07c
added class_memoize function for memoizing class instances
Dec 25, 2016
467f3f9
edited gitignore, added docs folder
Dec 25, 2016
4ed37ee
finally got class memoization right, using the mementos package (and …
Dec 25, 2016
6da7b28
got rid of doc folder, changed it to docs earlier
Dec 25, 2016
4e1b41d
updated docs folder to have source and build as separate folders
Dec 25, 2016
0c7759d
added gh-pages make target
Dec 25, 2016
6bfa6af
Fixed memoizing issues, adding some dependencies. Added documentation.
Dec 25, 2016
f26833f
version 0.7.15
Dec 25, 2016
ef0c173
Updated the players.py file to parse correctly. Note: this branch uti…
matthewstirling Jan 15, 2017
4e76953
Added methods to get team roster, a list of teams in ncaa, and a list…
matthewstirling Jan 16, 2017
a2435fc
Edited nfl.Player.name
Jan 17, 2017
068af98
initial merge of master into ncaaf
matthewstirling Jan 23, 2017
4901ab6
fixed broken utils references.
matthewstirling Jan 23, 2017
60134ba
changed the position return to a list.
matthewstirling Jan 24, 2017
afd1994
made fixes per the refactor or utils and also the remaining pull requ…
matthewstirling Jan 25, 2017
838b2da
Merge remote-tracking branch 'origin/ncaaf' into ncaaf
matthewstirling Jan 25, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ coverage.xml
*.log

# Sphinx documentation
docs/_build/
docs/build/

# PyBuilder
target/
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,6 @@
# sportsref
Scraping sports data from sports-reference.com and related sites

NOTE: Very much still a WIP. Feel free to use, just bear in mind that the API
is subject to change. Documentation is on the to-do list, once the API is a bit
more rigid.
35 changes: 35 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = sportsref
SOURCEDIR = source
BUILDDIR = build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

GH_PAGES_SOURCES = docs/Makefile docs/source

gh-pages:
git checkout gh-pages
rm -rf build _sources _static
git checkout master $(GH_PAGES_SOURCES)
git reset HEAD
make html
mv -fv build/html/* ./
rm -rf $(GH_PAGES_SOURCES) build
git add -A
git commit -m "Generated gh-pages for " \
"`git log master -1 --oneline --abbrev-commit`" && git push origin gh-pages
git checkout master
36 changes: 36 additions & 0 deletions docs/make.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
@ECHO OFF

pushd %~dp0

REM Command file for Sphinx documentation

if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=source
set BUILDDIR=build
set SPHINXPROJ=sportsref

if "%1" == "" goto help

%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
)

%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
goto end

:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%

:end
popd
162 changes: 162 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# -*- coding: utf-8 -*-
#
# sportsref documentation build configuration file, created by
# sphinx-quickstart on Sun Dec 25 04:56:24 2016.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))


# -- General configuration ------------------------------------------------

# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx.ext.autodoc',
'sphinx.ext.intersphinx',
'sphinx.ext.viewcode',
'sphinx.ext.githubpages']

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'

# The master toctree document.
master_doc = 'index'

# General information about the project.
project = u'sportsref'
copyright = u'2016, Matt Goldberg'
author = u'Matt Goldberg'

# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = u''
# The full version, including alpha/beta/rc tags.
release = u''

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = []

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'

# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False


# -- Options for HTML output ----------------------------------------------

# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'alabaster'

# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#
# html_theme_options = {}

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']


# -- Options for HTMLHelp output ------------------------------------------

# Output file base name for HTML help builder.
htmlhelp_basename = 'sportsrefdoc'


# -- Options for LaTeX output ---------------------------------------------

latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',

# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',

# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',

# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}

# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'sportsref.tex', u'sportsref Documentation',
u'Matt Goldberg', 'manual'),
]


# -- Options for manual page output ---------------------------------------

# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'sportsref', u'sportsref Documentation',
[author], 1)
]


# -- Options for Texinfo output -------------------------------------------

# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'sportsref', u'sportsref Documentation',
author, 'sportsref', 'One line description of project.',
'Miscellaneous'),
]




# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {'https://docs.python.org/': None}
20 changes: 20 additions & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
.. sportsref documentation master file, created by
sphinx-quickstart on Sun Dec 25 04:56:24 2016.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.

Welcome to sportsref's documentation!
=====================================

.. toctree::
:maxdepth: 2
:caption: Contents:



Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
8 changes: 8 additions & 0 deletions docs/source/modules.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
sportsref
=========

.. toctree::
:maxdepth: 4

setup
sportsref
7 changes: 7 additions & 0 deletions docs/source/setup.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
setup module
============

.. automodule:: setup
:members:
:undoc-members:
:show-inheritance:
54 changes: 54 additions & 0 deletions docs/source/sportsref.nba.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
sportsref.nba package
=====================

Submodules
----------

sportsref.nba.boxscores module
------------------------------

.. automodule:: sportsref.nba.boxscores
:members:
:undoc-members:
:show-inheritance:

sportsref.nba.pbp module
------------------------

.. automodule:: sportsref.nba.pbp
:members:
:undoc-members:
:show-inheritance:

sportsref.nba.players module
----------------------------

.. automodule:: sportsref.nba.players
:members:
:undoc-members:
:show-inheritance:

sportsref.nba.seasons module
----------------------------

.. automodule:: sportsref.nba.seasons
:members:
:undoc-members:
:show-inheritance:

sportsref.nba.teams module
--------------------------

.. automodule:: sportsref.nba.teams
:members:
:undoc-members:
:show-inheritance:


Module contents
---------------

.. automodule:: sportsref.nba
:members:
:undoc-members:
:show-inheritance:
30 changes: 30 additions & 0 deletions docs/source/sportsref.nfl.finders.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
sportsref.nfl.finders package
=============================

Submodules
----------

sportsref.nfl.finders.GPF module
--------------------------------

.. automodule:: sportsref.nfl.finders.GPF
:members:
:undoc-members:
:show-inheritance:

sportsref.nfl.finders.PSF module
--------------------------------

.. automodule:: sportsref.nfl.finders.PSF
:members:
:undoc-members:
:show-inheritance:


Module contents
---------------

.. automodule:: sportsref.nfl.finders
:members:
:undoc-members:
:show-inheritance:
Loading