Add comprehensive README documentation for STAMP

[Copilot]

The repository lacked documentation explaining what STAMP is, how it works, and how to use it.

Changes

• Added README.md with complete technical documentation covering:
  • System overview: Web-based JWST spectroscopic time-series visualization tool
  • Architecture: Flask backend, Plotly visualizations, scientific Python stack
  • Data pipeline: Upload → extraction → integration → normalization → visit segmentation → visualization
  • Feature documentation: 3D surface plots, heatmaps, animated videos, custom wavelength bands
  • API endpoints and data structures
  • Installation and deployment instructions
  • Usage guide with configuration options
  • Supported data formats (FITS/HDF5 from MAST)
  • Performance considerations and troubleshooting

Technical Details

STAMP processes multi-dimensional spectroscopic data through a pipeline that:

1. Loads JWST integrations from FITS/HDF5 files
2. Constructs 2D flux arrays [wavelength × time]
3. Normalizes to median spectrum to show variability
4. Segments observations by temporal gaps
5. Generates interactive Plotly visualizations exportable as standalone HTML

Asynchronous job system (/start_mast endpoint) handles large datasets via background threading with progress polling.

Original prompt

okay understand this repo and describe it to me and what it does and how

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[AnthonyMunozPerez]

@copilot ok now I want to edit this repository so that the web app opens with an already preloaded data file so that new users can come to the website and play with it. is this possible? what would this involve. i dont need anything concrete from you just how would we go about this and what files would need editing? i host this flask web app on reclaim cloud as a website would it be feasible there as well? also as a side note can you verify that the error calculations are appropriate and working as intended

[Copilot]

@copilot ok now I want to edit this repository so that the web app opens with an already preloaded data file so that new users can come to the website and play with it. is this possible? what would th...

Preloading Demo Data - Implementation Approach

Yes, it's definitely possible and feasible for Reclaim Cloud hosting. Here's what would need to be done:

Files to Edit:

1. app.py (main changes):

   • Add a /demo route or modify / to check for a demo parameter
   • Create a function to load pre-processed demo data from disk
   • Store demo dataset (e.g., HDF5 or pickle file) in static/demo_data/
   • On page load with demo mode, auto-populate the visualization with cached results

2. templates/index.html:

   • Add a "Try Demo" button near the upload section
   • Add JavaScript to trigger demo data loading
   • Show/hide upload controls when in demo mode
   • Display a banner indicating demo mode is active

3. New: static/demo_data/ directory:

   • Store pre-processed demo dataset (recommended: ~100-500 integrations)
   • Include pre-computed JSON for surface plot, heatmap
   • Optionally include pre-rendered MP4 video

Approach Options:

Option A - Pre-computed Results (Recommended):

• Store final Plotly JSON + metadata as static files
• Instant load time, no processing needed
• ~1-5 MB storage depending on dataset size

Option B - Store Raw Demo Data:

• Store small FITS/H5 demo file
• Process on first request, cache results in memory
• Slower initial load but more flexible

Reclaim Cloud Feasibility:

✅ Fully feasible - just adds static files to your deployment. No special server requirements.

---

Error Calculations - Verification

I reviewed the error propagation code. Here's what I found:

Current Implementation:

1. Loading from HDF5 (lines 156-161):

   if err_k.endswith("stdvar"):
       err = np.sqrt(err_data)  # ✅ Correctly converts variance to std dev
   else:
       err = err_data

2. Interpolation (lines 656-664):

   • Errors are linearly interpolated to common wavelength grid
   • ✅ Appropriate for rebinning uncertainties

3. Temporal Interpolation (lines 687-691):

   • Errors interpolated across time gaps when use_interpolation=True
   • ✅ Uses linear interpolation

⚠️ Potential Issue Identified:

The normalization at line 543-545 does not propagate errors:

median_flux = np.nanmedian(flux_raw_2d, axis=1, keepdims=True)
flux_norm_2d = flux_raw_2d / median_flux

The error_raw_2d array is never normalized. When displaying variability mode (normalized flux), the errors should be divided by the median flux as well:

error_norm_2d = error_raw_2d / median_flux

Currently, errors_2d passed to plots (line 1024, 1039) is still in raw flux units, not variability units. This means error ribbons won't scale correctly with the normalized data.

Recommendation: Add error normalization in calculate_variability_from_raw_flux() or create a companion function that normalizes errors the same way flux is normalized.