Skip to content

joshualutkemuller/unstructured_signal_fusion

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 

Repository files navigation

SignalFusion

SignalFusion is a regime-aware decision intelligence framework that combines unstructured text and structured quantitative signals to infer latent system states and prioritize events under uncertainty.

The system is designed as a general platform, demonstrated across two domains:

  • Customer Support Operations
  • Financial News & Market Events

Key Ideas

  • Systems operate in latent regimes that are not directly observable
  • Text and quantitative signals jointly reveal those regimes
  • Event importance is conditional on regime, not absolute

Architecture Overview

  1. Data ingestion (structured + unstructured)
  2. Text embedding (Sentence Transformers)
  3. Time aggregation
  4. Feature alignment
  5. PCA for stability
  6. Hidden Markov Model (HMM)
  7. Post-hoc regime labeling
  8. Importance scoring
  9. Explainability & visualization

Repository Structure

signalfusion/ ├── config/ │ ├── run.yaml │ ├── importance.yaml │ ├── financial_importance.yaml │ ├── src/ │ ├── ingestion/ │ ├── features/ │ ├── nlp/ │ ├── models/ │ ├── decisioning/ │ ├── visuals/ │ └── utils/ │ ├── customer_ticket_runner.py ├── runner_financial.py ├── outputs/ └── README.md


Regime Inference

Latent regimes are inferred using an HMM trained on:

  • aggregated structured features
  • aggregated text embeddings

The model outputs probabilistic state membership over time.

State semantics are assigned after inference using domain-specific stress indicators.


Importance Overlay

Each event (ticket or headline) receives an importance score based on weighted components such as:

  • recency
  • magnitude / urgency
  • sentiment
  • entity involvement

Scores are bucketed (P0–P3) for operational use.


Topic Drift & Explainability

Within each regime:

  • embeddings are clustered (KMeans)
  • clusters are labeled using TF-IDF

This reveals sub-themes without modifying the core regime model.


Outputs

  • features_matrix.csv
  • state_probabilities.csv
  • bi_table.csv
  • state_summary.csv
  • state_top_terms.csv
  • state_topic_clusters_summary.csv
  • top_important_events.csv
  • regime probability plots
  • stress signal overlays

Disclaimer

This system is designed for decision support and analysis. It does not constitute investment advice or predictive guarantees.


Author

Built as a platform-oriented data science project emphasizing robustness, interpretability, and cross-domain applicability.

About

SignalFusion is a regime-aware decision intelligence framework that combines unstructured text and structured quantitative signals to infer latent system states and prioritize events under uncertainty. The system is designed as a general platform, demonstrated across two domains: Customer Support Operations Financial News & Market Events

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages