a

Author: hutaobo

README.md

@@ -1,65 +1,64 @@
 # pyXenium
 
-A toy Python package for analyzing 10x Xenium data.
+Utilities for loading and analyzing 10x Genomics Xenium exports in Python.
 
-## Installation
+## Quickstart
+
+Install:
 
 ```bash
-pip install -U "pyXenium>=0.2.0"
-pip install "git+https://github.com/hutaobo/pyXenium.git@main"
+pip install pyXenium
 ```
 
-## Quickstart
-
-### Load a partial Xenium dataset from Hugging Face
-
-The snippet below uses the **public demo dataset** and the **v2 loader** that supports `base_url`.
+Load a dataset hosted online (e.g. Hugging Face). **By default, `load_anndata_from_partial` looks for the 10x MEX triplet under `<base>/cell_feature_matrix/`**:
 
-<!-- START load_partial_example -->
 ```python
 from pyXenium.io.partial_xenium_loader import load_anndata_from_partial
 
 BASE = "https://huggingface.co/datasets/hutaobo/pyxenium-gsm9116572/resolve/main"
 
 adata = load_anndata_from_partial(
     base_url=BASE,
-    analysis_name="analysis.zarr.zip",
-    cells_name="cells.zarr.zip",
-    transcripts_name="transcripts.zarr.zip",
-    # Optional: if you uploaded a 10x MEX triplet under BASE/mex/
-    # mex_dir=BASE + "/mex",
-    # mex_matrix_name="matrix.mtx.gz",
-    # mex_features_name="features.tsv.gz",
-    # mex_barcodes_name="barcodes.tsv.gz",
-    build_counts_if_missing=True,
+    analysis_name="analysis.zarr.zip",     # optional, attaches clusters if present
+    cells_name="cells.zarr.zip",           # optional, attaches spatial centroids if present
+    # By default it will read MEX from: BASE + "/cell_feature_matrix/"
 )
 print(adata)
 ```
-<!-- END load_partial_example -->
-
-> **Note:** Requires `pyXenium>=0.2.0`.
-> The demo dataset is hosted at:
-> - Hugging Face Datasets: [hutaobo/pyxenium-gsm9116572](https://huggingface.co/datasets/hutaobo/pyxenium-gsm9116572)
-
----
 
-## Development
+**What gets loaded:**
+- **Counts**: from MEX (`cell_feature_matrix/{matrix.mtx.gz, features.tsv.gz, barcodes.tsv.gz}`).
+- **Clusters** (optional): from `analysis.zarr[.zip]` if provided.
+- **Spatial centroids** (optional): from `cells.zarr[.zip]` if provided.
 
-To install with development dependencies (testing, docs, etc.):
+If MEX is missing:
+- The function returns an **empty-gene AnnData** (rows=cells if we can infer cell IDs; otherwise empty).
+- Clusters/spatial are still attached when possible.
+- To get real counts, upload MEX to `<base>/cell_feature_matrix/` or pass `mex_dir=...` explicitly.
 
-```bash
-pip install -e ".[dev]"
-pytest
+### Override the MEX location (optional)
+```python
+adata = load_anndata_from_partial(
+    base_url=BASE,
+    mex_dir=BASE + "/cell_feature_matrix",   # explicit
+    analysis_name="analysis.zarr.zip",
+    cells_name="cells.zarr.zip",
+)
 ```
 
----
-
-## Links
-
-- 📦 PyPI: [pyXenium](https://pypi.org/project/pyXenium/)
-- 📖 Documentation: [Read the Docs](https://pyxenium.readthedocs.io/en/latest/)
-- 💻 Source code: [GitHub](https://github.com/hutaobo/pyXenium)
-
-## License
+### Local folder example
+```python
+adata = load_anndata_from_partial(
+    base_dir="/path/to/xenium_export",
+    analysis_name="analysis.zarr",
+    cells_name="cells.zarr",
+    # will look for /path/to/xenium_export/cell_feature_matrix/
+)
+```
 
-MIT
+### Troubleshooting
+- **FileNotFoundError: MEX missing files** → Ensure the three files exist in `cell_feature_matrix/`:
+  `matrix.mtx.gz`, `features.tsv.gz`, `barcodes.tsv.gz`.
+- **Different obs names** → We honor 10x barcodes (from MEX). If your Zarr stores numeric
+  cell IDs, we normalize them to strings internally but prefer the barcodes from MEX.
+- **Large downloads** → Remote MEX is downloaded once into a temp dir per session run.

docs/usage/load_partial.md

@@ -1,11 +1,62 @@
-# Load a partial Xenium dataset from Hugging Face
+# Partial Loading: counts + clusters + spatial
 
-This example uses the **public demo dataset** and the **v2 loader** that supports `base_url`.
+`load_anndata_from_partial` reconstructs an `AnnData` using any subset of Xenium outputs:
 
-```{include} ../_includes/README.md
-:start-after: <!-- START load_partial_example -->
-:end-before: <!-- END load_partial_example -->
+- **Counts** (required for expression): **MEX triplet** from `cell_feature_matrix/`
+- **Clusters** (optional): `analysis.zarr` / `analysis.zarr.zip`
+- **Spatial centroids** (optional): `cells.zarr` / `cells.zarr.zip`
+
+## Default behavior
+
+- If `mex_dir` is **not** provided, the loader **automatically** looks for
+  `<base>/cell_feature_matrix/{matrix.mtx.gz, features.tsv.gz, barcodes.tsv.gz}`.
+- If found ⇒ counts are loaded from MEX (fast & robust).
+- If not found ⇒ returns an **empty-gene AnnData** but still attaches clusters/spatial if available.
+
+## Examples
+
+### Remote (Hugging Face)
+
+```python
+from pyXenium.io.partial_xenium_loader import load_anndata_from_partial
+
+BASE = "https://huggingface.co/datasets/hutaobo/pyxenium-gsm9116572/resolve/main"
+
+adata = load_anndata_from_partial(
+    base_url=BASE,
+    analysis_name="analysis.zarr.zip",
+    cells_name="cells.zarr.zip",
+)
+print(adata)
+```
+
+### Local folder
+
+```python
+adata = load_anndata_from_partial(
+    base_dir="/data/xenium_export",
+    analysis_name="analysis.zarr",
+    cells_name="cells.zarr",
+)
+```
+
+### Explicit MEX path (optional)
+
+```python
+adata = load_anndata_from_partial(
+    base_url=BASE,
+    mex_dir=BASE + "/cell_feature_matrix",
+)
 ```
 
-> **Requires** `pyXenium >= 0.2.0`.  
-> Demo dataset: `hutaobo/pyxenium-gsm9116572` on Hugging Face.
+## What gets attached
+
+- **Counts**: MEX → `.X` (CSR) and `.layers["counts"]`
+- **Clusters** (if `analysis.zarr*` present): `adata.obs["Cluster"]`
+- **Spatial** (if `cells.zarr*` present): `adata.obsm["spatial"]` (or `spatial3d`)
+
+## Notes & FAQ
+
+- We prioritize **10x barcodes** (MEX) as `obs.index`.
+- If Zarr stores numeric `cell_id` as `(N,2)` integers, we normalize internally; alignment prefers barcodes.
+- Want counts but don’t have MEX? Upload `cell_feature_matrix/` or export MEX from 10x Xenium output.

mkdocs.yml

@@ -2,3 +2,5 @@ site_name: pyXenium
 nav:
   - Home: index.md
   - Usage: usage.md
+  - Guide:
+      - Partial Loading: guide/partial-loading.md

src/pyXenium/io/partial_xenium_loader.py

@@ -15,6 +15,8 @@
 - Do NOT build counts from transcripts.zarr anymore. If MEX missing, returns an AnnData with
   empty gene dimension (but still attaches clusters/spatial when available).
 - Robust `cell_id` detection: include root-level "cell_id" and support `(N,2)` numeric ids -> "cell_{num}".
+- Fix: when downloading MEX from URLs, download all three files into the SAME temporary directory
+  to avoid "MEX missing files" errors.
 - Keep all previous public APIs and behaviors (backward compatible).
 
 Author: Taobo Hu (pyXenium project)
@@ -66,7 +68,7 @@ def _is_url(p: Optional[str | os.PathLike]) -> bool:
 
 
 def _fetch_to_temp(src: str, suffix: Optional[str] = None) -> Path:
-    """Download a URL to a temporary file and return its Path."""
+    """Download a single URL to a temporary file and return its Path."""
     logger.info(f"Downloading: {src}")
     r = requests.get(src, stream=True, timeout=120)
     r.raise_for_status()
@@ -80,6 +82,24 @@ def _fetch_to_temp(src: str, suffix: Optional[str] = None) -> Path:
     return dst
 
 
+def _download_many_to_same_temp(urls: Sequence[str]) -> Path:
+    """Download multiple URLs into the SAME temporary directory.
+    Returns the temp directory Path containing all files (base names preserved).
+    """
+    tmpdir = Path(tempfile.mkdtemp(prefix="pyxenium_"))
+    for u in urls:
+        logger.info(f"Downloading: {u}")
+        r = requests.get(u, stream=True, timeout=120)
+        r.raise_for_status()
+        name = Path(urllib.parse.urlparse(u).path).name or "tmp"
+        dst = tmpdir / name
+        with open(dst, "wb") as f:
+            for chunk in r.iter_content(chunk_size=1024 * 1024):
+                if chunk:
+                    f.write(chunk)
+    return tmpdir
+
+
 def _p(x: Optional[os.PathLike | str]) -> Optional[Path]:
     return None if x is None else Path(x).expanduser().resolve()
 
@@ -384,12 +404,14 @@ def load_anndata_from_partial(
     # 1) If explicit mex_dir is provided
     if mex_dir is not None:
         if _is_url(mex_dir):
-            # download three files into a temp dir
+            # download three files into the SAME temp dir
             base_url_mex = str(mex_dir).rstrip("/")
-            m_p = _fetch_to_temp(base_url_mex + "/" + mex_matrix_name)
-            _ = _fetch_to_temp(base_url_mex + "/" + mex_features_name)
-            _ = _fetch_to_temp(base_url_mex + "/" + mex_barcodes_name)
-            mex_dir_p = m_p.parent
+            tmpdir = _download_many_to_same_temp([
+                base_url_mex + "/" + mex_matrix_name,
+                base_url_mex + "/" + mex_features_name,
+                base_url_mex + "/" + mex_barcodes_name,
+            ])
+            mex_dir_p = tmpdir
         else:
             mex_dir_p = _p(mex_dir)
 
@@ -402,13 +424,15 @@ def load_anndata_from_partial(
                 mex_dir_p = cand
 
         if mex_dir_p is None and base_url is not None:
-            # download three files from base_url/cell_feature_matrix/
+            # download three files from base_url/cell_feature_matrix/ into SAME temp dir
             root_url = base_url.rstrip("/") + "/" + mex_default_subdir
             try:
-                m_p = _fetch_to_temp(root_url + "/" + mex_matrix_name)
-                _ = _fetch_to_temp(root_url + "/" + mex_features_name)
-                _ = _fetch_to_temp(root_url + "/" + mex_barcodes_name)
-                mex_dir_p = m_p.parent
+                tmpdir = _download_many_to_same_temp([
+                    root_url + "/" + mex_matrix_name,
+                    root_url + "/" + mex_features_name,
+                    root_url + "/" + mex_barcodes_name,
+                ])
+                mex_dir_p = tmpdir
             except Exception as e:
                 logger.warning(f"Failed to fetch MEX from {root_url}: {e}")