Python 3 rules do not correctly generate platform independent rules

[ewhauser]

Command line:

function update() {
     BASE_DIR="third_party/pypi"
     WHEELS_DIR_NAME="wheels"
     RULES_FILE_NAME="rules.bzl"

     reqs_file="$1"
     output_dir="$2"
     python_path="$3"

     bazel build @com_bluecore_rules_pyz//pypi:pip_generate_wrapper && \
     bazel-bin/external/com_bluecore_rules_pyz/pypi/pip_generate_wrapper \
         -requirements "$BASE_DIR/${reqs_file}" \
         -outputDir "$BASE_DIR/$output_dir" \
         -wheelDir "$WHEELS_DIR_NAME" \
         -wheelURLPrefix "https://storage.googleapis.com/dronedeploy-pypi-public/" \
         -outputBzlFileName "$RULES_FILE_NAME" \
         -pythonPath "$python_path" \
         -workspacePrefix "${output_dir}_"
 }

 update py2_requirements.txt py2 python2
 update py3_requirements.txt py3 python3

Python 2:

     pyz_library(
         name = "cryptography",
         deps = [
             "asn1crypto",
             "cffi",
             "enum34",
             "idna",
             "ipaddress",
             "six",
         ] + select({
             "@com_bluecore_rules_pyz//rules_python_zip:linux": ["@py2_cryptography__linux//:lib"],
             "@com_bluecore_rules_pyz//rules_python_zip:osx": ["@py2_cryptography__osx//:lib"],
         }),
         licenses = ["notice"],
         visibility = ["//visibility:public"],
     )

...

    if "py2_cryptography__linux" not in existing_rules:
         http_archive(
             name = "py2_cryptography__linux",
             url =
                 "https://files.pythonhosted.org/packages/87/e6/915a482dbfef98bbdce6be1e31825f591fc67038d4ee09864c1d2c3db371/cryptography-2.3.1-cp27-cp27mu-manylinux1_x86_64.   whl",
             sha256 =
                 "31db8febfc768e4b4bd826750a70c79c99ea423f4697d1dab764eb9f9f849519",
             build_file_content = _BUILD_FILE_CONTENT,
             type = "zip",
         )
     if "py2_cryptography__osx" not in existing_rules:
         http_archive(
             name = "py2_cryptography__osx",
             url =
                 "https://files.pythonhosted.org/packages/5d/b1/9863611b121ee524135bc0068533e6d238cc837337170e722224fe940e2d/cryptography-2.3.1-cp27-cp27m-macosx_10_6_intel.    whl",
             sha256 =
                 "17db09db9d7c5de130023657be42689d1a5f60502a14f6f745f6f65a6b8195c0",
             build_file_content = _BUILD_FILE_CONTENT,
             type = "zip",
         )

Python 3:

     pyz_library(
         name = "cryptography",
         deps = [
             "asn1crypto",
             "cffi",
             "enum34",
             "idna",
             "ipaddress",
             "six",
         ] + ["@py3_cryptography//:lib"],
         licenses = ["notice"],
         visibility = ["//visibility:public"],
     )

...

    if "py3_cryptography" not in existing_rules:
         http_archive(
             name = "py3_cryptography",
             url =
                 "https://files.pythonhosted.org/packages/98/0b/a6f293e5f10095dd8657a1b125c1ba6995c59d39cd8e20355475c8f760d0/cryptography-2.3.1-cp34-abi3-macosx_10_6_intel.     whl",
             sha256 =
                 "dc2d3f3b1548f4d11786616cf0f4415e25b0fbecb8a1d2cd8c07568f13fdde38",
             build_file_content = _BUILD_FILE_CONTENT,
             type = "zip",
         )

[evanj]

Ah, right this is a great point. I am not sure what the best way of managing a project that needs to use both Python2 and Python3 is. Its going to be ... messy probably given that the dependency resolution can't easily happen in Bazel itself. My current thinking is maybe it would be best to make sure pip_generate can generate the correct dependencies, and then use two separate pypi_rules.bzl files, one for Python2 and one for Python3; would that work for your use case?

[joshburkart]

I think the issue you're pointing out @ewhauser is that the Python 3 version only finds a Mac .whl file? No idea why this would be...! Might just be a PyPI problem?

[gorset]

I'm seeing the same issue. Got a repository with python 3 only. Trying to adopt pyz rules, but pip_generate doesn't add all platforms for python3 dependencies. For our case, that's grpcio, numpy, pandas etc.