Scheduled weekly dependency update for week 01

[pyup-bot]

Update biopython from 1.72 to 1.73.

The bot wasn't able to find a changelog for this release. Got an idea?

Links

• PyPI: https://pypi.org/project/biopython
• Changelog: https://pyup.io/changelogs/biopython/
• Homepage: https://biopython.org/

Update click-plugins from 1.0.3 to 1.0.4.

Changelog

1.0.4

------------------

- Preemptive fix for a breaking change in Click v7.  CLI command names generated from functions with underscores will have dashes instead of underscores.  See https://github.com/click-contrib/click-plugins/issues/19.

Links

• PyPI: https://pypi.org/project/click-plugins
• Changelog: https://pyup.io/changelogs/click-plugins/
• Repo: https://github.com/click-contrib/click-plugins

Update click from 6.7 to 7.0.

Changelog

7.0

-----------

Released 2018-09-25

-   Drop support for Python 2.6 and 3.3. (`967`_, `976`_)
-   Wrap ``click.Choice``'s missing message. (`202`_, `1000`_)
-   Add native ZSH autocompletion support. (`323`_, `865`_)
-   Document that ANSI color info isn't parsed from bytearrays in
 Python 2. (`334`_)
-   Document byte-stripping behavior of ``CliRunner``. (`334`_,
 `1010`_)
-   Usage errors now hint at the ``--help`` option. (`393`_, `557`_)
-   Implement streaming pager. (`409`_, `889`_)
-   Extract bar formatting to its own method. (`414`_)
-   Add ``DateTime`` type for converting input in given date time
 formats. (`423`_)
-   ``secho``'s first argument can now be ``None``, like in ``echo``.
 (`424`_)
-   Fixes a ``ZeroDivisionError`` in ``ProgressBar.make_step``, when the
 arg passed to the first call of ``ProgressBar.update`` is 0.
 (`447`_, `1012`_)
-   Show progressbar only if total execution time is visible. (`487`_)
-   Added the ability to hide commands and options from help. (`500`_)
-   Document that options can be ``required=True``. (`514`_, `1022`_)
-   Non-standalone calls to ``Context.exit`` return the exit code,
 rather than calling ``sys.exit``. (`533`_, `667`_, `1098`_)
-   ``click.getchar()`` returns Unicode in Python 3 on Windows,
 consistent with other platforms. (`537`_, `821`_, `822`_,
 `1088`_, `1108`_)
-   Added ``FloatRange`` type. (`538`_, `553`_)
-   Added support for bash completion of ``type=click.Choice`` for
 ``Options`` and ``Arguments``. (`535`_, `681`_)
-   Only allow one positional arg for ``Argument`` parameter
 declaration. (`568`_, `574`_, `1014`_)
-   Add ``case_sensitive=False`` as an option to Choice. (`569`_)
-   ``click.getchar()`` correctly raises ``KeyboardInterrupt`` on "^C"
 and ``EOFError`` on "^D" on Linux. (`583`_, `1115`_)
-   Fix encoding issue with ``click.getchar(echo=True)`` on Linux.
 (`1115`_)
-   ``param_hint`` in errors now derived from param itself. (`598`_,
 `704`_, `709`_)
-   Add a test that ensures that when an argument is formatted into a
 usage error, its metavar is used, not its name. (`612`_)
-   Allow setting ``prog_name`` as extra in ``CliRunner.invoke``.
 (`616`_, `999`_)
-   Help text taken from docstrings truncates at the ``\f`` form feed
 character, useful for hiding Sphinx-style parameter documentation.
 (`629`_, `1091`_)
-   ``launch`` now works properly under Cygwin. (`650`_)
-   Update progress after iteration. (`651`_, `706`_)
-   ``CliRunner.invoke`` now may receive ``args`` as a string
 representing a Unix shell command. (`664`_)
-   Make ``Argument.make_metavar()`` default to type metavar. (`675`_)
-   Add documentation for ``ignore_unknown_options``. (`684`_)
-   Add bright colors support for ``click.style`` and fix the reset
 option for parameters ``fg`` and ``bg``. (`703`_, `809`_)
-   Add ``show_envvar`` for showing environment variables in help.
 (`710`_)
-   Avoid ``BrokenPipeError`` during interpreter shutdown when stdout or
 stderr is a closed pipe. (`712`_, `1106`_)
-   Document customizing option names. (`725`_, `1016`_)
-   Disable ``sys._getframes()`` on Python interpreters that don't
 support it. (`728`_)
-   Fix bug in test runner when calling ``sys.exit`` with ``None``.
 (`739`_)
-   Clarify documentation on command line options. (`741`_, `1003`_)
-   Fix crash on Windows console. (`744`_)
-   Fix bug that caused bash completion to give improper completions on
 chained commands. (`754`_, `774`_)
-   Added support for dynamic bash completion from a user-supplied
 callback. (`755`_)
-   Added support for bash completions containing spaces. (`773`_)
-   Allow autocompletion function to determine whether or not to return
 completions that start with the incomplete argument. (`790`_,
 `806`_)
-   Fix option naming routine to match documentation and be
 deterministic. (`793`_, `794`_)
-   Fix path validation bug. (`795`_, `1020`_)
-   Add test and documentation for ``Option`` naming: functionality.
 (`799`_)
-   Update doc to match arg name for ``path_type``. (`801`_)
-   Raw strings added so correct escaping occurs. (`807`_)
-   Fix 16k character limit of ``click.echo`` on Windows. (`816`_,
 `819`_)
-   Overcome 64k character limit when writing to binary stream on
 Windows 7. (`825`_, `830`_)
-   Add bool conversion for "t" and "f". (`842`_)
-   ``NoSuchOption`` errors take ``ctx`` so that ``--help`` hint gets
 printed in error output. (`860`_)
-   Fixed the behavior of Click error messages with regards to Unicode
 on 2.x and 3.x. Message is now always Unicode and the str and
 Unicode special methods work as you expect on that platform.
 (`862`_)
-   Progress bar now uses stderr by default. (`863`_)
-   Add support for auto-completion documentation. (`866`_, `869`_)
-   Allow ``CliRunner`` to separate stdout and stderr. (`868`_)
-   Fix variable precedence. (`873`_, `874`_)
-   Fix invalid escape sequences. (`877`_)
-   Fix ``ResourceWarning`` that occurs during some tests. (`878`_)
-   When detecting a misconfigured locale, don't fail if the ``locale``
 command fails. (`880`_)
-   Add ``case_sensitive=False`` as an option to ``Choice`` types.
 (`887`_)
-   Force stdout/stderr writable. This works around issues with badly
 patched standard streams like those from Jupyter. (`918`_)
-   Fix completion of subcommand options after last argument (`919`_,
 `930`_)
-   ``_AtomicFile`` now uses the ``realpath`` of the original filename
 so that changing the working directory does not affect it.
 (`920`_)
-   Fix incorrect completions when defaults are present (`925`_,
 `930`_)
-   Add copy option attrs so that custom classes can be re-used.
 (`926`_, `994`_)
-   "x" and "a" file modes now use stdout when file is ``"-"``.
 (`929`_)
-   Fix missing comma in ``__all__`` list. (`935`_)
-   Clarify how parameters are named. (`949`_, `1009`_)
-   Stdout is now automatically set to non blocking. (`954`_)
-   Do not set options twice. (`962`_)
-   Move ``fcntl`` import. (`965`_)
-   Fix Google App Engine ``ImportError``. (`995`_)
-   Better handling of help text for dynamic default option values.
 (`996`_)
-   Fix ``get_winter_size()`` so it correctly returns ``(0,0)``.
 (`997`_)
-   Add test case checking for custom param type. (`1001`_)
-   Allow short width to address cmd formatting. (`1002`_)
-   Add details about Python version support. (`1004`_)
-   Added deprecation flag to commands. (`1005`_)
-   Fixed issues where ``fd`` was undefined. (`1007`_)
-   Fix formatting for short help. (`1008`_)
-   Document how ``auto_envvar_prefix`` works with command groups.
 (`1011`_)
-   Don't add newlines by default for progress bars. (`1013`_)
-   Use Python sorting order for ZSH completions. (`1047`_, `1059`_)
-   Document that parameter names are converted to lowercase by default.
 (`1055`_)
-   Subcommands that are named by the function now automatically have
 the underscore replaced with a dash. If you register a function
 named ``my_command`` it becomes ``my-command`` in the command line
 interface.
-   Hide hidden commands and options from completion. (`1058`_,
 `1061`_)
-   Fix absolute import blocking Click from being vendored into a
 project on Windows. (`1068`_, `1069`_)
-   Fix issue where a lowercase ``auto_envvar_prefix`` would not be
 converted to uppercase. (`1105`_)

.. _202: https://github.com/pallets/click/issues/202
.. _323: https://github.com/pallets/click/issues/323
.. _334: https://github.com/pallets/click/issues/334
.. _393: https://github.com/pallets/click/issues/393
.. _409: https://github.com/pallets/click/issues/409
.. _414: https://github.com/pallets/click/pull/414
.. _423: https://github.com/pallets/click/pull/423
.. _424: https://github.com/pallets/click/pull/424
.. _447: https://github.com/pallets/click/issues/447
.. _487: https://github.com/pallets/click/pull/487
.. _500: https://github.com/pallets/click/pull/500
.. _514: https://github.com/pallets/click/issues/514
.. _533: https://github.com/pallets/click/pull/533
.. _535: https://github.com/pallets/click/issues/535
.. _537: https://github.com/pallets/click/issues/537
.. _538: https://github.com/pallets/click/pull/538
.. _553: https://github.com/pallets/click/pull/553
.. _557: https://github.com/pallets/click/pull/557
.. _568: https://github.com/pallets/click/issues/568
.. _569: https://github.com/pallets/click/issues/569
.. _574: https://github.com/pallets/click/issues/574
.. _583: https://github.com/pallets/click/issues/583
.. _598: https://github.com/pallets/click/issues/598
.. _612: https://github.com/pallets/click/pull/612
.. _616: https://github.com/pallets/click/issues/616
.. _629: https://github.com/pallets/click/pull/629
.. _650: https://github.com/pallets/click/pull/650
.. _651: https://github.com/pallets/click/issues/651
.. _664: https://github.com/pallets/click/pull/664
.. _667: https://github.com/pallets/click/issues/667
.. _675: https://github.com/pallets/click/pull/675
.. _681: https://github.com/pallets/click/pull/681
.. _684: https://github.com/pallets/click/pull/684
.. _703: https://github.com/pallets/click/issues/703
.. _704: https://github.com/pallets/click/issues/704
.. _706: https://github.com/pallets/click/pull/706
.. _709: https://github.com/pallets/click/pull/709
.. _710: https://github.com/pallets/click/pull/710
.. _712: https://github.com/pallets/click/pull/712
.. _719: https://github.com/pallets/click/issues/719
.. _725: https://github.com/pallets/click/issues/725
.. _728: https://github.com/pallets/click/pull/728
.. _739: https://github.com/pallets/click/pull/739
.. _741: https://github.com/pallets/click/issues/741
.. _744: https://github.com/pallets/click/issues/744
.. _754: https://github.com/pallets/click/issues/754
.. _755: https://github.com/pallets/click/pull/755
.. _773: https://github.com/pallets/click/pull/773
.. _774: https://github.com/pallets/click/pull/774
.. _790: https://github.com/pallets/click/issues/790
.. _793: https://github.com/pallets/click/issues/793
.. _794: https://github.com/pallets/click/pull/794
.. _795: https://github.com/pallets/click/issues/795
.. _799: https://github.com/pallets/click/pull/799
.. _801: https://github.com/pallets/click/pull/801
.. _806: https://github.com/pallets/click/pull/806
.. _807: https://github.com/pallets/click/pull/807
.. _809: https://github.com/pallets/click/pull/809
.. _816: https://github.com/pallets/click/pull/816
.. _819: https://github.com/pallets/click/pull/819
.. _821: https://github.com/pallets/click/issues/821
.. _822: https://github.com/pallets/click/issues/822
.. _825: https://github.com/pallets/click/issues/825
.. _830: https://github.com/pallets/click/pull/830
.. _842: https://github.com/pallets/click/pull/842
.. _860: https://github.com/pallets/click/issues/860
.. _862: https://github.com/pallets/click/issues/862
.. _863: https://github.com/pallets/click/pull/863
.. _865: https://github.com/pallets/click/pull/865
.. _866: https://github.com/pallets/click/issues/866
.. _868: https://github.com/pallets/click/pull/868
.. _869: https://github.com/pallets/click/pull/869
.. _873: https://github.com/pallets/click/issues/873
.. _874: https://github.com/pallets/click/pull/874
.. _877: https://github.com/pallets/click/pull/877
.. _878: https://github.com/pallets/click/pull/878
.. _880: https://github.com/pallets/click/pull/880
.. _883: https://github.com/pallets/click/pull/883
.. _887: https://github.com/pallets/click/pull/887
.. _889: https://github.com/pallets/click/pull/889
.. _918: https://github.com/pallets/click/pull/918
.. _919: https://github.com/pallets/click/issues/919
.. _920: https://github.com/pallets/click/pull/920
.. _925: https://github.com/pallets/click/issues/925
.. _926: https://github.com/pallets/click/issues/926
.. _929: https://github.com/pallets/click/pull/929
.. _930: https://github.com/pallets/click/pull/930
.. _935: https://github.com/pallets/click/pull/935
.. _949: https://github.com/pallets/click/issues/949
.. _954: https://github.com/pallets/click/pull/954
.. _962: https://github.com/pallets/click/pull/962
.. _965: https://github.com/pallets/click/pull/965
.. _967: https://github.com/pallets/click/pull/967
.. _976: https://github.com/pallets/click/pull/976
.. _990: https://github.com/pallets/click/pull/990
.. _991: https://github.com/pallets/click/pull/991
.. _993: https://github.com/pallets/click/pull/993
.. _994: https://github.com/pallets/click/pull/994
.. _995: https://github.com/pallets/click/pull/995
.. _996: https://github.com/pallets/click/pull/996
.. _997: https://github.com/pallets/click/pull/997
.. _999: https://github.com/pallets/click/pull/999
.. _1000: https://github.com/pallets/click/pull/1000
.. _1001: https://github.com/pallets/click/pull/1001
.. _1002: https://github.com/pallets/click/pull/1002
.. _1003: https://github.com/pallets/click/pull/1003
.. _1004: https://github.com/pallets/click/pull/1004
.. _1005: https://github.com/pallets/click/pull/1005
.. _1007: https://github.com/pallets/click/pull/1007
.. _1008: https://github.com/pallets/click/pull/1008
.. _1009: https://github.com/pallets/click/pull/1009
.. _1010: https://github.com/pallets/click/pull/1010
.. _1011: https://github.com/pallets/click/pull/1011
.. _1012: https://github.com/pallets/click/pull/1012
.. _1013: https://github.com/pallets/click/pull/1013
.. _1014: https://github.com/pallets/click/pull/1014
.. _1016: https://github.com/pallets/click/pull/1016
.. _1020: https://github.com/pallets/click/pull/1020
.. _1022: https://github.com/pallets/click/pull/1022
.. _1027: https://github.com/pallets/click/pull/1027
.. _1047: https://github.com/pallets/click/pull/1047
.. _1055: https://github.com/pallets/click/pull/1055
.. _1058: https://github.com/pallets/click/pull/1058
.. _1059: https://github.com/pallets/click/pull/1059
.. _1061: https://github.com/pallets/click/pull/1061
.. _1068: https://github.com/pallets/click/issues/1068
.. _1069: https://github.com/pallets/click/pull/1069
.. _1088: https://github.com/pallets/click/issues/1088
.. _1091: https://github.com/pallets/click/pull/1091
.. _1098: https://github.com/pallets/click/pull/1098
.. _1105: https://github.com/pallets/click/pull/1105
.. _1106: https://github.com/pallets/click/pull/1106
.. _1108: https://github.com/pallets/click/pull/1108
.. _1115: https://github.com/pallets/click/pull/1115

Links

• PyPI: https://pypi.org/project/click
• Changelog: https://pyup.io/changelogs/click/
• Homepage: https://palletsprojects.com/p/click/

Update coverage from 4.5.1 to 4.5.2.

The bot wasn't able to find a changelog for this release. Got an idea?

Links

• PyPI: https://pypi.org/project/coverage
• Changelog: https://pyup.io/changelogs/coverage/
• Repo: https://bitbucket.org/ned/coveragepy

Update matplotlib from 3.0.0 to 3.0.2.

Changelog

Links

• PyPI: https://pypi.org/project/matplotlib
• Changelog: https://pyup.io/changelogs/matplotlib/
• Homepage: http://matplotlib.org

Update numpy from 1.15.2 to 1.15.4.

Changelog

1.15.4

==========================

This is a bugfix release for bugs and regressions reported following the 1.15.3
release.  The Python versions supported by this release are 2.7, 3.4-3.7. The
wheels are linked with OpenBLAS v0.3.0, which should fix some of the linalg
problems reported for NumPy 1.14.

Compatibility Note
==================

The NumPy 1.15.x OS X wheels released on PyPI no longer contain 32-bit
binaries.  That will also be the case in future releases. See
`11625 <https://github.com/numpy/numpy/issues/11625>`__ for the related
discussion.  Those needing 32-bit support should look elsewhere or build
from source.

Contributors
============

A total of 4 people contributed to this release.  People with a "+" by their
names contributed a patch for the first time.

* Charles Harris
* Matti Picus
* Sebastian Berg
* bbbbbbbbba +

Pull requests merged
====================

A total of 4 pull requests were merged for this release.

* `12296 <https://github.com/numpy/numpy/pull/12296>`__: BUG: Dealloc cached buffer info
* `12297 <https://github.com/numpy/numpy/pull/12297>`__: BUG: Fix fill value in masked array '==' and '!=' ops.
* `12307 <https://github.com/numpy/numpy/pull/12307>`__: DOC: Correct the default value of `optimize` in `numpy.einsum`
* `12320 <https://github.com/numpy/numpy/pull/12320>`__: REL: Prepare for the NumPy 1.15.4 release


==========================

1.15.3

==========================

This is a bugfix release for bugs and regressions reported following the 1.15.2
release.  The Python versions supported by this release are 2.7, 3.4-3.7. The
wheels are linked with OpenBLAS v0.3.0, which should fix some of the linalg
problems reported for NumPy 1.14.

Compatibility Note
==================

The NumPy 1.15.x OS X wheels released on PyPI no longer contain 32-bit
binaries.  That will also be the case in future releases. See
`11625 <https://github.com/numpy/numpy/issues/11625>`__ for the related
discussion.  Those needing 32-bit support should look elsewhere or build
from source.

Contributors
============

A total of 7 people contributed to this release.  People with a "+" by their
names contributed a patch for the first time.

* Allan Haldane
* Charles Harris
* Jeroen Demeyer
* Kevin Sheppard
* Matthew Bowden +
* Matti Picus
* Tyler Reddy

Pull requests merged
====================

A total of 12 pull requests were merged for this release.

* `12080 <https://github.com/numpy/numpy/pull/12080>`__: MAINT: Blacklist some MSVC complex functions.
* `12083 <https://github.com/numpy/numpy/pull/12083>`__: TST: Add azure CI testing to 1.15.x branch.
* `12084 <https://github.com/numpy/numpy/pull/12084>`__: BUG: test_path() now uses Path.resolve()
* `12085 <https://github.com/numpy/numpy/pull/12085>`__: TST, MAINT: Fix some failing tests on azure-pipelines mac and...
* `12187 <https://github.com/numpy/numpy/pull/12187>`__: BUG: Fix memory leak in mapping.c
* `12188 <https://github.com/numpy/numpy/pull/12188>`__: BUG: Allow boolean subtract in histogram
* `12189 <https://github.com/numpy/numpy/pull/12189>`__: BUG: Fix in-place permutation
* `12190 <https://github.com/numpy/numpy/pull/12190>`__: BUG: limit default for get_num_build_jobs() to 8
* `12191 <https://github.com/numpy/numpy/pull/12191>`__: BUG: OBJECT_to_* should check for errors
* `12192 <https://github.com/numpy/numpy/pull/12192>`__: DOC: Prepare for NumPy 1.15.3 release.
* `12237 <https://github.com/numpy/numpy/pull/12237>`__: BUG: Fix MaskedArray fill_value type conversion.
* `12238 <https://github.com/numpy/numpy/pull/12238>`__: TST: Backport azure-pipeline testing fixes for Mac


==========================

Links

• PyPI: https://pypi.org/project/numpy
• Changelog: https://pyup.io/changelogs/numpy/
• Homepage: http://www.numpy.org

Update pyfaidx from 0.5.4.2 to 0.5.5.2.

Changelog

0.5.5.1

- Added better exceptions for files that do not contain any valid sequences (144)

0.5.5

- Added proper BGZF support, fixing 131. Note: this fix requires code present in Biopython 1.73, which is not yet released. If you need this functionality in the mean time, install the biopython development version
- Enhancement to `faidx` script: allow -v to be specified alongside -g (142). Thanks daler!

Links

• PyPI: https://pypi.org/project/pyfaidx
• Changelog: https://pyup.io/changelogs/pyfaidx/
• Homepage: http://mattshirley.com
• Docs: https://pythonhosted.org/pyfaidx/

Update pyparsing from 2.2.1 to 2.3.0.

Changelog

2.3.0

-----------------------------
- NEW SUPPORT FOR UNICODE CHARACTER RANGES
This release introduces the pyparsing_unicode namespace class, defining
a series of language character sets to simplify the definition of alphas,
nums, alphanums, and printables in the following language sets:
. Arabic
. Chinese
. Cyrillic
. Devanagari
. Greek
. Hebrew
. Japanese (including Kanji, Katakana, and Hirigana subsets)
. Korean
. Latin1 (includes 7 and 8-bit Latin characters)
. Thai
. CJK (combination of Chinese, Japanese, and Korean sets)

For example, your code can define words using:

 korean_word = Word(pyparsing_unicode.Korean.alphas)

See their use in the updated examples greetingInGreek.py and
greetingInKorean.py.

This namespace class also offers access to these sets using their
unicode identifiers.

- POSSIBLE API CHANGE: Fixed bug where a parse action that explicitly 
returned the input ParseResults could add another nesting level in
the results if the current expression had a results name.

     vals = pp.OneOrMore(pp.pyparsing_common.integer)("int_values")
     
     def add_total(tokens):
         tokens['total'] = sum(tokens)
         return tokens   this line can be removed

     vals.addParseAction(add_total)
     print(vals.parseString("244 23 13 2343").dump())

Before the fix, this code would print (note the extra nesting level):

 [244, 23, 13, 2343]
 - int_values: [244, 23, 13, 2343]
   - int_values: [244, 23, 13, 2343]
   - total: 2623
 - total: 2623

With the fix, this code now prints:

 [244, 23, 13, 2343]
 - int_values: [244, 23, 13, 2343]
 - total: 2623

This fix will change the structure of ParseResults returned if a 
program defines a parse action that returns the tokens that were 
sent in. This is not necessary, and statements like "return tokens" 
in the example above can be safely deleted prior to upgrading to 
this release, in order to avoid the bug and get the new behavior.

Reported by seron in Issue 22, nice catch!

- POSSIBLE API CHANGE: Fixed a related bug where a results name 
erroneously created a second level of hierarchy in the returned 
ParseResults. The intent for accumulating results names into ParseResults
is that, in the absence of Group'ing, all names get merged into a
common namespace. This allows us to write:

    key_value_expr = (Word(alphas)("key") + '=' + Word(nums)("value"))
    result = key_value_expr.parseString("a = 100") 
 
and have result structured as {"key": "a", "value": "100"} 
instead of [{"key": "a"}, {"value": "100"}].

However, if a named expression is used in a higher-level non-Group 
expression that *also* has a name, a false sub-level would be created 
in the namespace:

     num = pp.Word(pp.nums)
     num_pair = ("[" + (num("A") + num("B"))("values") + "]")
     U = num_pair.parseString("[ 10 20 ]")
     print(U.dump())

Since there is no grouping, "A", "B", and "values" should all appear
at the same level in the results, as:

     ['[', '10', '20', ']']
     - A: '10'
     - B: '20'
     - values: ['10', '20']

Instead, an extra level of "A" and "B" show up under "values":

     ['[', '10', '20', ']']
     - A: '10'
     - B: '20'
     - values: ['10', '20']
       - A: '10'
       - B: '20'

This bug has been fixed. Now, if this hierarchy is desired, then a
Group should be added:

     num_pair = ("[" + pp.Group(num("A") + num("B"))("values") + "]")

Giving:

     ['[', ['10', '20'], ']']
     - values: ['10', '20']
       - A: '10'
       - B: '20'

But in no case should "A" and "B" appear in multiple levels. This bug-fix
fixes that.

If you have current code which relies on this behavior, then add or remove
Groups as necessary to get your intended results structure.

Reported by Athanasios Anastasiou.

- IndexError's raised in parse actions will get explicitly reraised 
as ParseExceptions that wrap the original IndexError. Since 
IndexError sometimes occurs as part of pyparsing's normal parsing 
logic, IndexErrors that are raised during a parse action may have
gotten silently reinterpreted as parsing errors. To retain the 
information from the IndexError, these exceptions will now be 
raised as ParseExceptions that reference the original IndexError. 
This wrapping will only be visible when run under Python3, since it
emulates "raise ... from ..." syntax. 

Addresses Issue 4, reported by guswns0528.

- Added Char class to simplify defining expressions of a single
character. (Char("abc") is equivalent to Word("abc", exact=1))

- Added class PrecededBy to perform lookbehind tests. PrecededBy is 
used in the same way as FollowedBy, passing in an expression that
must occur just prior to the current parse location.

For fixed-length expressions like a Literal, Keyword, Char, or a 
Word with an `exact` or `maxLen` length given, `PrecededBy(expr)` 
is sufficient. For varying length expressions like a Word with no 
given maximum length, `PrecededBy` must be constructed with an 
integer `retreat` argument, as in 
`PrecededBy(Word(alphas, nums), retreat=10)`, to specify the maximum 
number of characters pyparsing must look backward to make a match. 
pyparsing will check all the values from 1 up to retreat characters 
back from the current parse location.

When stepping backwards through the input string, PrecededBy does 
*not* skip over whitespace.

PrecededBy can be created with a results name so that, even though
it always returns an empty parse result, the result *can* include
named results.

Idea first suggested in Issue 30 by Freakwill.

- Updated FollowedBy to accept expressions that contain named results,
so that results names defined in the lookahead expression will be 
returned, even though FollowedBy always returns an empty list.
Inspired by the same feature implemented in PrecededBy.

2.2.2

-------------------------------
- Fixed bug in SkipTo, if a SkipTo expression that was skipping to
an expression that returned a list (such as an And), and the 
SkipTo was saved as a named result, the named result could be 
saved as a ParseResults - should always be saved as a string.
Issue 28, reported by seron.

- Added simple_unit_tests.py, as a collection of easy-to-follow unit 
tests for various classes and features of the pyparsing library. 
Primary intent is more to be instructional than actually rigorous 
testing. Complex tests can still be added in the unitTests.py file.

- New features added to the Regex class:
- optional asGroupList parameter, returns all the capture groups as
 a list
- optional asMatch parameter, returns the raw re.match result
- new sub(repl) method, which adds a parse action calling
 re.sub(pattern, repl, parsed_result). Simplifies creating 
 Regex expressions to be used with transformString. Like re.sub,
 repl may be an ordinary string (similar to using pyparsing's 
 replaceWith), or may contain references to capture groups by group 
 number, or may be a callable that takes an re match group and 
 returns a string.
 
 For instance:
     expr = pp.Regex(r"([Hh]\d):\s*(.*)").sub(r"<\1>\2</\1>")
     expr.transformString("h1: This is the title")

 will return
     <h1>This is the title</h1>

- Fixed omission of LICENSE file in source tarball, also added 
CODE_OF_CONDUCT.md per GitHub community standards.

Links

• PyPI: https://pypi.org/project/pyparsing
• Changelog: https://pyup.io/changelogs/pyparsing/
• Repo: https://github.com/pyparsing/pyparsing/
• Docs: https://pythonhosted.org/pyparsing/

Update python-dateutil from 2.7.3 to 2.7.5.

Changelog

2.7.4

==========================

Data updates
------------

- Updated tzdata version to 2018f.

Links

• PyPI: https://pypi.org/project/python-dateutil
• Changelog: https://pyup.io/changelogs/python-dateutil/
• Docs: https://dateutil.readthedocs.io

Update pytz from 2018.5 to 2018.9.

The bot wasn't able to find a changelog for this release. Got an idea?

Links

• PyPI: https://pypi.org/project/pytz
• Homepage: http://pythonhosted.org/pytz
• Docs: https://pythonhosted.org/pytz/

Update six from 1.11.0 to 1.12.0.

Changelog

1.12.0

------

- Issue 259, pull request 260: `six.add_metaclass` now preserves
`__qualname__` from the original class.

- Pull request 204: Add `six.ensure_binary`, `six.ensure_text`, and
`six.ensure_str`.

Links

• PyPI: https://pypi.org/project/six
• Changelog: https://pyup.io/changelogs/six/
• Repo: https://github.com/benjaminp/six
• Docs: https://pythonhosted.org/six/

Update click_plugins from 1.0.4 to 1.0.4.

Changelog

1.0.4

------------------

- Preemptive fix for a breaking change in Click v7.  CLI command names generated from functions with underscores will have dashes instead of underscores.  See https://github.com/click-contrib/click-plugins/issues/19.

1.0.3

------------------

- Include tests in MANIFEST.in - See further discussion in 8

1.0.2

------------------

- General packaging and Travis-CI improvements.
- Don't include tests in MANIFEST.in - 8

1.0.1

------------------

- Fixed a typo in an error message - 5

1.0

----------------

- Initial release.

Links

• PyPI: https://pypi.org/project/click-plugins
• Changelog: https://pyup.io/changelogs/click-plugins/
• Repo: https://github.com/click-contrib/click-plugins

Update pandas from 0.23.4 to 0.23.4.

Changelog

0.23.4

------------------------

This is a minor bug-fix release in the 0.23.x series and includes some small regression fixes
and bug fixes. We recommend that all users upgrade to this version.

.. warning::

Starting January 1, 2019, pandas feature releases will support Python 3 only.
See :ref:`install.dropping-27` for more.

.. contents:: What's new in v0.23.4
 :local:
 :backlinks: none

.. _whatsnew_0234.fixed_regressions:

Fixed Regressions
~~~~~~~~~~~~~~~~~

- Python 3.7 with Windows gave all missing values for rolling variance calculations (:issue:`21813`)

.. _whatsnew_0234.bug_fixes:

Bug Fixes
~~~~~~~~~

**Groupby/Resample/Rolling**

- Bug where calling :func:`DataFrameGroupBy.agg` with a list of functions including ``ohlc`` as the non-initial element would raise a ``ValueError`` (:issue:`21716`)
- Bug in ``roll_quantile`` caused a memory leak when calling ``.rolling(...).quantile(q)`` with ``q`` in (0,1) (:issue:`21965`)

**Missing**

- Bug in :func:`Series.clip` and :func:`DataFrame.clip` cannot accept list-like threshold containing ``NaN`` (:issue:`19992`)

0.23.3

----------------------

This release fixes a build issue with the sdist for Python 3.7 (:issue:`21785`)
There are no other changes.


.. _whatsnew_0211:

0.23.2

----------------------

This is a minor bug-fix release in the 0.23.x series and includes some small regression fixes
and bug fixes. We recommend that all users upgrade to this version.

.. note::

Pandas 0.23.2 is first pandas release that's compatible with
Python 3.7 (:issue:`20552`)

.. warning::

Starting January 1, 2019, pandas feature releases will support Python 3 only.
See :ref:`install.dropping-27` for more.

.. contents:: What's new in v0.23.2
 :local:
 :backlinks: none

.. _whatsnew_0232.enhancements:

Logical Reductions over Entire DataFrame
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

:meth:`DataFrame.all` and :meth:`DataFrame.any` now accept ``axis=None`` to reduce over all axes to a scalar (:issue:`19976`)

.. ipython:: python

df = pd.DataFrame({"A": [1, 2], "B": [True, False]})
df.all(axis=None)


This also provides compatibility with NumPy 1.15, which now dispatches to ``DataFrame.all``.
With NumPy 1.15 and pandas 0.23.1 or earlier, :func:`numpy.all` will no longer reduce over every axis:

.. code-block:: python

>>>  NumPy 1.15, pandas 0.23.1
>>> np.any(pd.DataFrame({"A": [False], "B": [False]}))
A    False
B    False
dtype: bool

With pandas 0.23.2, that will correctly return False, as it did with NumPy < 1.15.

.. ipython:: python

np.any(pd.DataFrame({"A": [False], "B": [False]}))


.. _whatsnew_0232.fixed_regressions:

Fixed Regressions
~~~~~~~~~~~~~~~~~

- Fixed regression in :meth:`to_csv` when handling file-like object incorrectly (:issue:`21471`)
- Re-allowed duplicate level names of a ``MultiIndex``. Accessing a level that has a duplicate name by name still raises an error (:issue:`19029`).
- Bug in both :meth:`DataFrame.first_valid_index` and :meth:`Series.first_valid_index` raised for a row index having duplicate values (:issue:`21441`)
- Fixed printing of DataFrames with hierarchical columns with long names (:issue:`21180`)
- Fixed regression in :meth:`~DataFrame.reindex` and :meth:`~DataFrame.groupby`
with a MultiIndex or multiple keys that contains categorical datetime-like values (:issue:`21390`).
- Fixed regression in unary negative operations with object dtype (:issue:`21380`)
- Bug in :meth:`Timestamp.ceil` and :meth:`Timestamp.floor` when timestamp is a multiple of the rounding frequency (:issue:`21262`)
- Fixed regression in :func:`to_clipboard` that defaulted to copying dataframes with space delimited instead of tab delimited (:issue:`21104`)


Build Changes
~~~~~~~~~~~~~

- The source and binary distributions no longer include test data files, resulting in smaller download sizes. Tests relying on these data files will be skipped when using ``pandas.test()``. (:issue:`19320`)

.. _whatsnew_0232.bug_fixes:

Bug Fixes
~~~~~~~~~

**Conversion**

- Bug in constructing :class:`Index` with an iterator or generator (:issue:`21470`)
- Bug in :meth:`Series.nlargest` for signed and unsigned integer dtypes when the minimum value is present (:issue:`21426`)

**Indexing**

- Bug in :meth:`Index.get_indexer_non_unique` with categorical key (:issue:`21448`)
- Bug in comparison operations for :class:`MultiIndex` where error was raised on equality / inequality comparison involving a MultiIndex with ``nlevels == 1`` (:issue:`21149`)
- Bug in :meth:`DataFrame.drop` behaviour is not consistent for unique and non-unique indexes (:issue:`21494`)
- Bug in :func:`DataFrame.duplicated` with a large number of columns causing a 'maximum recursion depth exceeded' (:issue:`21524`).

**I/O**

- Bug in :func:`read_csv` that caused it to incorrectly raise an error when ``nrows=0``, ``low_memory=True``, and ``index_col`` was not ``None`` (:issue:`21141`)
- Bug in :func:`json_normalize` when formatting the ``record_prefix`` with integer columns (:issue:`21536`)

**Categorical**

- Bug in rendering :class:`Series` with ``Categorical`` dtype in rare conditions under Python 2.7 (:issue:`21002`)

**Timezones**

- Bug in :class:`Timestamp` and :class:`DatetimeIndex` where passing a :class:`Timestamp` localized after a DST transition would return a datetime before the DST transition (:issue:`20854`)
- Bug in comparing :class:`DataFrame`s with tz-aware :class:`DatetimeIndex` columns with a DST transition that raised a ``KeyError`` (:issue:`19970`)

**Timedelta**

- Bug in :class:`Timedelta` where non-zero timedeltas shorter than 1 microsecond were considered False (:issue:`21484`)


.. _whatsnew_0170:

0.23.1

-----------------------

This is a minor bug-fix release in the 0.23.x series and includes some small regression fixes
and bug fixes. We recommend that all users upgrade to this version.

.. warning::

Starting January 1, 2019, pandas feature releases will support Python 3 only.
See :ref:`install.dropping-27` for more.

.. contents:: What's new in v0.23.1
 :local:
 :backlinks: none

.. _whatsnew_0231.fixed_regressions:

Fixed Regressions
~~~~~~~~~~~~~~~~~

**Comparing Series with datetime.date**

We've reverted a 0.23.0 change to comparing a :class:`Series` holding datetimes and a ``datetime.date`` object (:issue:`21152`).
In pandas 0.22 and earlier, comparing a Series holding datetimes and ``datetime.date`` objects would coerce the ``datetime.date`` to a datetime before comapring.
This was inconsistent with Python, NumPy, and :class:`DatetimeIndex`, which never consider a datetime and ``datetime.date`` equal.

In 0.23.0, we unified operations between DatetimeIndex and Series, and in the process changed comparisons between a Series of datetimes and ``datetime.date`` without warning.

We've temporarily restored the 0.22.0 behavior, so datetimes and dates may again compare equal, but restore the 0.23.0 behavior in a future release.

To summarize, here's the behavior in 0.22.0, 0.23.0, 0.23.1:

.. code-block:: python

 0.22.0... Silently coerce the datetime.date
>>> Series(pd.date_range('2017', periods=2)) == datetime.date(2017, 1, 1)
0     True
1    False
dtype: bool

 0.23.0... Do not coerce the datetime.date
>>> Series(pd.date_range('2017', periods=2)) == datetime.date(2017, 1, 1)
0    False
1    False
dtype: bool

 0.23.1... Coerce the datetime.date with a warning
>>> Series(pd.date_range('2017', periods=2)) == datetime.date(2017, 1, 1)
/bin/python:1: FutureWarning: Comparing Series of datetimes with 'datetime.date'.  Currently, the
'datetime.date' is coerced to a datetime. In the future pandas will
not coerce, and the values not compare equal to the 'datetime.date'.
To retain the current behavior, convert the 'datetime.date' to a
datetime with 'pd.Timestamp'.
  !/bin/python3
0     True
1    False
dtype: bool

In addition, ordering comparisons will raise a ``TypeError`` in the future.

**Other Fixes**

- Reverted the ability of :func:`~DataFrame.to_sql` to perform multivalue
inserts as this caused regression in certain cases (:issue:`21103`).
In the future this will be made configurable.
- Fixed regression in the :attr:`DatetimeIndex.date` and :attr:`DatetimeIndex.time`
attributes in case of timezone-aware data: :attr:`DatetimeIndex.time` returned
a tz-aware time instead of tz-naive (:issue:`21267`) and :attr:`DatetimeIndex.date`
returned incorrect date when the input date has a non-UTC timezone (:issue:`21230`).
- Fixed regression in :meth:`pandas.io.json.json_normalize` when called with ``None`` values
in nested levels in JSON, and to not drop keys with value as `None` (:issue:`21158`, :issue:`21356`).
- Bug in :meth:`~DataFrame.to_csv` causes encoding error when compression and encoding are specified (:issue:`21241`, :issue:`21118`)
- Bug preventing pandas from being importable with -OO optimization (:issue:`21071`)
- Bug in :meth:`Categorical.fillna` incorrectly raising a ``TypeError`` when `value` the individual categories are iterable and `value` is an iterable (:issue:`21097`, :issue:`19788`)
- Fixed regression in constructors coercing NA values like ``None`` to strings when passing ``dtype=str`` (:issue:`21083`)
- Regression in :func:`pivot_table` where an ordered ``Categorical`` with missing
values for the pivot's ``index`` would give a mis-aligned result (:issue:`21133`)
- Fixed regression in merging on boolean index/columns (:issue:`21119`).

.. _whatsnew_0231.performance:

Performance Improvements
~~~~~~~~~~~~~~~~~~~~~~~~

- Improved performance of :meth:`CategoricalIndex.is_monotonic_increasing`, :meth:`CategoricalIndex.is_monotonic_decreasing` and :meth:`CategoricalIndex.is_monotonic` (:issue:`21025`)
- Improved performance of :meth:`CategoricalIndex.is_unique` (:issue:`21107`)


.. _whatsnew_0231.bug_fixes:

Bug Fixes
~~~~~~~~~

**Groupby/Resample/Rolling**

- Bug in :func:`DataFrame.agg` where applying multiple aggregation functions to a :class:`DataFrame` with duplicated column names would cause a stack overflow (:issue:`21063`)
- Bug in :func:`pandas.core.groupby.GroupBy.ffill` and :func:`pandas.core.groupby.GroupBy.bfill` where the fill within a grouping would not always be applied as intended due to the implementations' use of a non-stable sort (:issue:`21207`)
- Bug in :func:`pandas.core.groupby.GroupBy.rank` where results did not scale to 100% when specifying ``method='dense'`` and ``pct=True``
- Bug in :func:`pandas.DataFrame.rolling` and :func:`pandas.Series.rolling` which incorrectly accepted a 0 window size rather than raising (:issue:`21286`)

**Data-type specific**

- Bug in :meth:`Series.str.replace()` where the method throws `TypeError` on Python 3.5.2 (:issue:`21078`)
- Bug in :class:`Timedelta` where passing a float with a unit would prematurely round the float precision (:issue:`14156`)
- Bug in :func:`pandas.testing.assert_index_equal` which raised ``AssertionError`` incorrectly, when comparing two :class:`CategoricalIndex` objects with param ``check_categorical=False`` (:issue:`19776`)

**Sparse**

- Bug in :attr:`SparseArray.shape` which previously only returned the shape :attr:`SparseArray.sp_values` (:issue:`21126`)

**Indexing**

- Bug in :meth:`Series.reset_index` where appropriate error was not raised with an invalid level name (:issue:`20925`)
- Bug in :func:`interval_range` when ``start``/``periods`` or ``end``/``periods`` are specified with float ``start`` or ``end`` (:issue:`21161`)
- Bug in :meth:`MultiIndex.set_names` where error raised for a ``MultiIndex`` with ``nlevels == 1`` (:issue:`21149`)
- Bug in :class:`IntervalIndex` constructors where creating an ``IntervalIndex`` from categorical data was not fully supported (:issue:`21243`, :issue:`21253`)
- Bug in :meth:`MultiIndex.sort_index` which was not guaranteed to sort correctly with ``level=1``; this was also causing data misalignment in particular :meth:`DataFrame.stack` operations (:issue:`20994`, :issue:`20945`, :issue:`21052`)

**Plotting**

- New keywords (sharex, sharey) to turn on/off sharing of x/y-axis by subplots generated with pandas.DataFrame().groupby().boxplot() (:issue:`20968`)

**I/O**

- Bug in IO methods specifying ``compression='zip'`` which produced uncompressed zip archives (:issue:`17778`, :issue:`21144`)
- Bug in :meth:`DataFrame.to_stata` which prevented exporting DataFrames to buffers and most file-like objects (:issue:`21041`)
- Bug in :meth:`read_stata` and :class:`StataReader` which did not correctly decode utf-8 strings on Python 3 from Stata 14 files (dta version 118) (:issue:`21244`)
- Bug in IO JSON :func:`read_json` reading empty JSON schema with ``orient='table'`` back to :class:`DataFrame` caused an error (:issue:`21287`)

**Reshaping**

- Bug in :func:`concat` where error was raised in concatenating :class:`Series` with numpy scalar and tuple names (:issue:`21015`)
- Bug in :func:`concat` warning message providing the wrong guidance for future behavior (:issue:`21101`)

**Other**

- Tab completion on :class:`Index` in IPython no longer outputs deprecation warnings (:issue:`21125`)
- Bug preventing pandas being used on Windows without C++ redistributable installed (:issue:`21106`)



.. _whatsnew_050:

0.23.0

----------------------

This is a major release from 0.22.0 and includes a number of API changes,
deprecations, new features, enhancements, and performance improvements along
with a large number of bug fixes. We recommend that all users upgrade to this
version.

Highlights include:

- :ref:`Round-trippable JSON format with 'table' orient <whatsnew_0230.enhancements.round-trippable_json>`.
- :ref:`Instantiation from dicts respects order for Python 3.6+ <whatsnew_0230.api_breaking.dict_insertion_order>`.
- :ref:`Dependent column arguments for assign <whatsnew_0230.enhancements.assign_dependent>`.
- :ref:`Merging / sorting on a combination of columns and index levels <whatsnew_0230.enhancements.merge_on_columns_and_levels>`.
- :ref:`Extending Pandas with custom types <whatsnew_023.enhancements.extension>`.
- :ref:`Excluding unobserved categories from groupby <whatsnew_0230.enhancements.categorical_grouping>`.
- :ref:`Changes to make output shape of DataFrame.apply consistent <whatsnew_0230.api_breaking.apply>`.

Check the :ref:`API Changes <whatsnew_0230.api_breaking>` and :ref:`deprecations <whatsnew_0230.deprecations>` before updating.

.. warning::

Starting January 1, 2019, pandas feature releases will support Python 3 only.
See :ref:`install.dropping-27` for more.

.. contents:: What's new in v0.23.0
 :local:
 :backlinks: none
 :depth: 2

.. _whatsnew_0230.enhancements:

New features
~~~~~~~~~~~~

.. _whatsnew_0230.enhancements.round-trippable_json:

JSON read/write round-trippable with ``orient='table'``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A ``DataFrame`` can now be written to and subsequently read back via JSON while preserving metadata through usage of the ``orient='table'`` argument (see :issue:`18912` and :issue:`9146`). Previously, none of the available ``orient`` values guaranteed the preservation of dtypes and index names, amongst other metadata.

.. ipython:: python

df = pd.DataFrame({'foo': [1, 2, 3, 4],
 	      'bar': ['a', 'b', 'c', 'd'],
 	      'baz': pd.date_range('2018-01-01', freq='d', periods=4),
 	      'qux': pd.Categorical(['a', 'b', 'c', 'c'])
 	      }, index=pd.Index(range(4), name='idx'))
df
df.dtypes
df.to_json('test.json', orient='table')
new_df = pd.read_json('test.json', orient='table')
new_df
new_df.dtypes

Please note that the string `index` is not supported with the round trip format, as it is used by default in ``write_json`` to indicate a missing index name.

.. ipython:: python
:okwarning:

df.index.name = 'index'

df.to_json('test.json', orient='table')
new_df = pd.read_json('test.json', orient='table')
new_df
new_df.dtypes

.. ipython:: python
:suppress:

import os
os.remove('test.json')


.. _whatsnew_0230.enhancements.assign_dependent:


``.assign()`` accepts dependent arguments
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The :func:`DataFrame.assign` now accepts dependent keyword arguments for python version later than 3.6 (see also `PEP 468
<https://www.python.org/dev/peps/pep-0468/>`_). Later keyword arguments may now refer to earlier ones if the argument is a callable. See the
:ref:`documentation here <dsintro.chained_assignment>` (:issue:`14207`)

.. ipython:: python

 df = pd.DataFrame({'A': [1, 2, 3]})
 df
 df.assign(B=df.A, C=lambda x:x['A']+ x['B'])

.. warning::

This may subtly change the behavior of your code when you're
using ``.assign()`` to update an existing column. Previously, callables
referring to other variables being updated would get the "old" values

Previous Behavior:

.. code-block:: ipython

   In [2]: df = pd.DataFrame({"A": [1, 2, 3]})

   In [3]: df.assign(A=lambda df: df.A + 1, C=lambda df: df.A * -1)
   Out[3]:
      A  C
   0  2 -1
   1  3 -2
   2  4 -3

New Behavior:

.. ipython:: python

   df.assign(A=df.A+1, C= lambda df: df.A* -1)



.. _whatsnew_0230.enhancements.merge_on_columns_and_levels:

Merging on a combination of columns and index levels
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Strings passed to :meth:`DataFrame.merge` as the ``on``, ``left_on``, and ``right_on``
parameters may now refer to either column names or index level names.
This enables merging ``DataFrame`` instances on a combination of index levels
and columns without resetting indexes. See the :ref:`Merge on columns and
levels <merging.merge_on_columns_and_levels>` documentation section.
(:issue:`14355`)

.. ipython:: python

left_index = pd.Index(['K0', 'K0', 'K1', 'K2'], name='key1')

left = pd.DataFrame({'A': ['A0', 'A1', 'A2', 'A3'],
                     'B': ['B0', 'B1', 'B2', 'B3'],
                     'key2': ['K0', 'K1', 'K0', 'K1']},
                    index=left_index)

right_index = pd.Index(['K0', 'K1', 'K2', 'K2'], name='key1')

right = pd.DataFrame({'C': ['C0', 'C1', 'C2', 'C3'],
                      'D': ['D0', 'D1', 'D2', 'D3'],
                      'key2': ['K0', 'K0', 'K0', 'K1']},
                     index=right_index)

left.merge(right, on=['key1', 'key2'])

.. _whatsnew_0230.enhancements.sort_by_columns_and_levels:

Sorting by a combination of columns and index levels
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Strings passed to :meth:`DataFrame.sort_values` as the ``by`` parameter may
now refer to either column names or index level names.  This enables sorting
``DataFrame`` instances by a combination of index levels and columns without
resetting indexes. See the :ref:`Sorting by Indexes and Values
<basics.sort_indexes_and_values>` documentation section.
(:issue:`14353`)

.. ipython:: python

 Build MultiIndex
idx = pd.MultiIndex.from_tuples([('a', 1), ('a', 2), ('a', 2),
                                 ('b', 2), ('b', 1), ('b', 1)])
idx.names = ['first', 'second']

 Build DataFrame
df_multi = pd.DataFrame({'A': np.arange(6, 0, -1)},
                        index=idx)
df_multi

 Sort by 'second' (index) and 'A' (column)
df_multi.sort_values(by=['second', 'A'])


.. _whatsnew_023.enhancements.extension:

Extending Pandas with Custom Types (Experimental)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Pandas now supports storing array-like objects that aren't necessarily 1-D NumPy
arrays as columns in a DataFrame or values in a Series. This allows third-party
libraries to implement extensions to NumPy's types, similar to how pandas
implemented categoricals, datetimes with timezones, periods, and intervals.

As a demonstration, we'll use cyberpandas_, which provides an ``IPArray`` type
for storing ip addresses.

.. code-block:: ipython

In [1]: from cyberpandas import IPArray

In [2]: values = IPArray([
   ...:     0,
   ...:     3232235777,
   ...:     42540766452641154071740215577757643572
   ...: ])
   ...:
   ...:

``IPArray`` isn't a normal 1-D NumPy array, but because it's a pandas
:class:`~pandas.api.extensions.ExtensionArray`, it can be stored properly inside pandas' containers.

.. code-block:: ipython

In [3]: ser = pd.Series(values)

In [4]: ser
Out[4]:
0                         0.0.0.0
1                     192.168.1.1
2    2001:db8:85a3::8a2e:370:7334
dtype: ip

Notice that the dtype is ``ip``. The missing value semantics of the underlying
array are respected:

.. code-block:: ipython

In [5]: ser.isna()
Out[5]:
0     True
1    False
2    False
dtype: bool

For more, see the :ref:`extension types <extending.extension-types>`
documentation. If you build an extension array, publicize it on our
:ref:`ecosystem page <ecosystem.extensions>`.

.. _cyberpandas: https://cyberpandas.readthedocs.io/en/latest/


.. _whatsnew_0230.enhancements.categorical_grouping:

New ``observed`` keyword for excluding unobserved categories in ``groupby``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Grouping by a categorical includes the unobserved categories in the output.
When grouping by multiple categorical columns, this means you get the cartesian product of all the
categories, including combinations where there are no observations, which can result in a large
number of groups. We have added a keyword ``observed`` to control this behavior, it defaults to
``observed=False`` for backward-compatibility. (:issue:`14942`, :issue:`8138`, :issue:`15217`, :issue:`17594`, :issue:`8669`, :issue:`20583`, :issue:`20902`)

.. ipython:: python

cat1 = pd.Categorical(["a", "a", "b", "b"],
                      categories=["a", "b", "z"], ordered=True)
cat2 = pd.Categorical(["c", "d", "c", "d"],
                      categories=["c", "d", "y"], ordered=True)
df = pd.DataFrame({"A": cat1, "B": cat2, "values": [1, 2, 3, 4]})
df['C'] = ['foo', 'bar'] * 2
df

To show all values, the previous behavior:

.. ipython:: python

df.groupby(['A', 'B', 'C'], observed=False).count()


To show only observed values:

.. ipython:: python

df.groupby(['A', 'B', 'C'], observed=True).count()

For pivotting operations, this behavior is *already* controlled by the ``dropna`` keyword:

.. ipython:: python

cat1 = pd.Categorical(["a", "a", "b", "b"],
                      categories=["a", "b", "z"], ordered=True)
cat2 = pd.Categorical(["c", "d", "c", "d"],
                      categories=["c", "d", "y"], ordered=True)
df = DataFrame({"A": cat1, "B": cat2, "values": [1, 2, 3, 4]})
df

.. ipython:: python

pd.pivot_table(df, values='values', index=['A', 'B'],
               dropna=True)
pd.pivot_table(df, values='values', index=['A', 'B'],
               dropna=False)


.. _whatsnew_0230.enhancements.window_raw:

Rolling/Expanding.apply() accepts ``raw=False`` to pass a ``Series`` to the function
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

:func:`Series.rolling().apply() <pandas.core.window.Rolling.apply>`, :func:`DataFrame.rolling().apply() <pandas.core.window.Rolling.apply>`,
:func:`Series.expanding().apply() <pandas.core.window.Expanding.apply>`, and :func:`DataFrame.expanding().apply() <pandas.core.window.Expanding.apply>` have gained a ``raw=None`` parameter.
This is similar to :func:`DataFame.apply`. This parameter, if ``True`` allows one to send a ``np.ndarray`` to the applied function. If ``False`` a ``Series`` will be passed. The
default is ``None``, which preserves backward compatibility, so this will default to ``True``, sending an ``np.ndarray``.
In a future version the default will be changed to ``False``, sending a ``Series``. (:issue:`5071`, :issue:`20584`)

.. ipython:: python

s = pd.Series(np.arange(5), np.arange(5) + 1)
s

Pass a ``Series``:

.. ipython:: python

s.rolling(2, min_periods=1).apply(lambda x: x.iloc[-1], raw=False)

Mimic the original behavior of passing a ndarray:

.. ipython:: python

s.rolling(2, min_periods=1).apply(lambda x: x[-1], raw=True)


.. _whatsnew_0210.enhancements.limit_area:

``DataFrame.interpolate`` has gained the ``limit_area`` kwarg
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

:meth:`DataFrame.interpolate` has gained a ``limit_area`` parameter to allow further control of which ``NaN`` s are replaced.
Use ``limit_area='inside'`` to fill only NaNs surrounded by valid values or use ``limit_area='outside'`` to fill only ``NaN`` s
outside the existing valid values while preserving those inside.  (:issue:`16284`) See the :ref:`full documentation here <missing_data.interp_limits>`.


.. ipython:: python

ser = pd.Series([np.nan, np.nan, 5, np.nan, np.nan, np.nan, 13, np.nan, np.nan])
ser

Fill one consecutive inside value in both directions

.. ipython:: python

ser.interpolate(limit_direction='both', limit_area='inside', limit=1)

Fill all consecutive outside values backward

.. ipython:: python

ser.interpolate(limit_direction='backward', limit_area='outside')

Fill all consecutive outside values in both directions

.. ipython:: python

ser.interpolate(limit_direction='both', limit_area='outside')

.. _whatsnew_0210.enhancements.get_dummies_dtype:

``get_dummies`` now supports ``dtype`` argument
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The :func:`get_dummies` now accepts a ``dtype`` argument, which specifies a dtype for the new columns. The default remains uint8. (:issue:`18330`)

.. ipython:: python

df = pd.DataFrame({'a': [1, 2], 'b': [3, 4], 'c': [5, 6]})
pd.get_dummies(df, columns=['c']).dtypes
pd.get_dummies(df, columns=['c'], dtype=bool).dtypes


.. _whatsnew_0230.enhancements.timedelta_mod:

Timedelta mod method
^^^^^^^^^^^^^^^^^^^^

``mod`` (%) and ``divmod`` operations are now defined on ``Timedelta`` objects
when operating with either timedelta-like or with numeric arguments.
See the :ref:`documentation here <timedeltas.mod_divmod>`. (:issue:`19365`)

.. ipython:: python

 td = pd.Timedelta(hours=37)
 td % pd.Timedelta(minutes=45)

.. _whatsnew_0230.enhancements.ran_inf:

``.rank()`` handles ``inf`` values when ``NaN`` are present
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In previous versions, ``.rank()`` would assign ``inf`` elements ``NaN`` as their ranks. Now ranks are calculated properly. (:issue:`6945`)

.. ipython:: python

 s = pd.Series([-np.inf, 0, 1, np.nan, np.inf])
 s

Previous Behavior:

.. code-block:: ipython

 In [11]: s.rank()
 Out[11]:
 0    1.0
 1    2.0
 2    3.0
 3    NaN
 4    NaN
 dtype: float64

Current Behavior:

.. ipython:: python

 s.rank()

Furthermore, previously if you rank ``inf`` or ``-inf`` values together with ``NaN`` values, the calculation won't distinguish ``NaN`` from infinity when using 'top' or 'bottom' argument.

.. ipython:: python

 s = pd.Series([np.nan, np.nan, -np.inf, -np.inf])
 s

Previous Behavior:

.. code-block:: ipython

 In [15]: s.rank(na_option='top')
 Out[15]:
 0    2.5
 1    2.5
 2    2.5
 3    2.5
 dtype: float64

Current Behavior:

.. ipython:: python

 s.rank(na_option='top')

These bugs were squashed:

- Bug in :meth:`DataFrame.rank` and :meth:`Series.rank` when ``method='dense'`` and ``pct=True`` in which percentile ranks were not being used with the number of distinct observations (:issue:`15630`)
- Bug in :meth:`Series.rank` and :meth:`DataFrame.rank` when ``ascending='False'`` failed to return correct ranks for infinity if ``NaN`` were present (:issue:`19538`)
- Bug in :func:`DataFrameGroupBy.rank` where ranks were incorrect when both infinity and ``NaN`` were present (:issue:`20561`)


.. _whatsnew_0230.enhancements.str_cat_align:

``Series.str.cat`` has gained the ``join`` kwarg
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Previously, :meth:`Series.str.cat` did not -- in contrast to most of ``pandas`` -- align :class:`Series` on their index before concatenation (see :issue:`18657`).
The method has now gained a keyword ``join`` to control the manner of alignment, see examples below and :ref:`here <text.concatenate>`.

In v.0.23 `join` will default to None (meaning no alignment), but this default will change to ``'left'`` in a future version of pandas.

.. ipython:: python
:okwarning:

 s = pd.Series(['a', 'b', 'c', 'd'])
 t = pd.Series(['b', 'd', 'e', 'c'], index=[1, 3, 4, 2])
 s.str.cat(t)
 s.str.cat(t, join='left', na_rep='-')

Furthermore, :meth:`Series.str.cat` now works for ``CategoricalIndex`` as well (previously raised a ``ValueError``; see :issue:`20842`).

.. _whatsnew_0230.enhancements.astype_category:

``DataFrame.astype`` performs column-wise conversion to ``Categorical``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

:meth:`DataFrame.astype` can now perform column-wise conversion to ``Categorical`` by supplying the string ``'category'`` or
a :class:`~pandas.api.types.CategoricalDtype`. Previously, attempting this would raise a ``NotImplementedError``. See the
:ref:`categorical.objectcreation` section of the documentation for more details and examples. (:issue:`12860`, :issue:`18099`)

Supplying the string ``'category'`` performs column-wise conversion, with only labels appearing in a given column set as categories:

.. ipython:: python

 df = pd.DataFrame({'A': list('abca'), 'B': list('bccd')})
 df = df.astype('category')
 df['A'].dtype
 df['B'].dtype


Supplying a ``CategoricalDtype`` will make the categories in each column consistent with the supplied dtype:

.. ipython:: python

 from pandas.api.types import CategoricalDtype
 df = pd.DataFrame({'A': list('abca'), 'B': list('bccd')})
 cdt = CategoricalDtype(categories=list('abcd'), ordered=True)
 df = df.astype(cdt)
 df['A'].dtype
 df['B'].dtype


.. _whatsnew_0230.enhancements.other:

Other Enhancements
^^^^^^^^^^^^^^^^^^

- Unary ``+`` now permitted for ``Series`` and ``DataFrame`` as  numeric operator (:issue:`16073`)
- Better support for :meth:`~pandas.io.formats.style.Styler.to_excel` output 

[pyup-bot]

Closing this in favor of #18