Skip to content

Gorvvb/Knowledge-Graph-Reasoner

Repository files navigation

Knowledge Graph Reasoner

It's a local knowledge graph engine that takes plain text, and extracts both relationships and entities. Feed it Wikipedia articles, and watch it find connections across everything it has read.


What does it do?

  • Ingests plain text, Wikipedia articles, research papers, notes, etc
  • Extracts entities (people, places, concepts, etc) and the relationships between them
  • Resolves similar entity names into a single node. ("Aristotle", "The philosopher Aristotle" into one node)
  • Reasons over the graph. Answers are grounded only in ingested data, so we avoid LLM hallucinations
  • Finds paths between any two entities across the graph
  • Detects contradictions in the knowledge base
  • Visualises as a fast interactive dark-themed HTML graph powered by WebGL.
  • Persists across sessions. The graph will grow every time you input something new

Setup

Requirements

  • Python 3.11+
  • A free Groq API key: console.groq.com
  • NVIDIA GPU recommended for embedding model (runs on CPU too, just slower)

Install

# 1. Clone and enter the project
git clone https://github.com/Gorvvb/Knowledge-Graph-Reasoner
cd Knowledge-Graph-Reasoner

# 2. Create and activate a virtual python environment
python -m venv venv
venv\Scripts\activate    # Windows
source venv/bin/activate # Linux / Mac

# 3. Install dependencies
pip install -r requirements.txt

# 4. Download the spaCy language model
python -m spacy download en_core_web_sm

# 5. Set your Groq API key
set GROQ_API_KEY=gsk_your_key_here          # Windows cmd
$env:GROQ_API_KEY = "gsk_your_key_here"     # PowerShell
export GROQ_API_KEY=gsk_your_key_here       # Linux / Mac
# Or set the varable in enviorment variables

#. 6. Run
python cli.py

Usage

Ingesting documents

You can ingest as many documents as you like. Each one adds to the same persistent graph, so the more you give it, the richer the connections become.

Fetch a Wikipedia artivle directly by name:

> ingest-wiki Aristotle
> ingest-wiki Thermodynamics

Ingest a batch of Wikipedia articles in one go:

> ingest-wiki-batch
  title: Evolution
  title: Genetics
  title: Natural selection
  [blank line to start ingesting]

Not sure of the exact article title? Search first:

> wiki-search thermodynamics

From a local file:

> ingest path/to/article.txt (Make sure it does not have empty lines)

By pasting text directly:

> ingest-text
[Paste the text (make sure there are no empty lines), then enter an empty line]
Then you will need to enter a source name.

See what you have already ingested:

> sources

Tip: Start with a few Wikipedia articles on related topics. Ingest Aristotle, then Alexander the Great, then Plato. Then run connect Aristotle :: Alexander the Great and watch it narrate the path across all three documents.


Querying

Ask a question (answered only from graph data):

> query Who influenced Nietzsche?
> query Why did the Roman Empire collapse?
> query What connects Darwin to Aristotle?

Look up everything known about an entity:

> lookup Aristotle
> lookup Roman Empire

Find the path between two entities:

> connect Aristotle :: modern science
> connect Charles Darwin :: Aristotle

Note the :: separator between the two entity names.

Scan for contradictions:

> contradictions

Visualization

> visualize                   - render the full graph
> visualize Aristotle         - render a subgraph around Aristotle (default depth is 2)
> visualize Aristotle 3       - render a subgraph with custom depth
> visualize Charles Darwin 1  - renders only direct neighbours

Open data/graph.html in any browser. Scroll to zoom, click a node to highlight its direct neighbourhood, and click empty space to reset. Hover over a node for source citations, and the nodes are also color coded according to type.


Other commands

> sources         - list all ingested documents
> stats           - node count, most connected nodes
> search plato    - find nodes by name substring
> reset           - wipe the graph entirely (will ask for confirmation)
> help            - show all commands
> quit            - exit

Good starting articles for a STEM graph

Evolution, Charles Darwin, Natural selection, Taxonomy, Scientific method, Scientific revolution, Mathematics, Physics, Chemistry, Genetics, Neuroscience, Thermodynamics, Calculus

Once you have a few of these in, try connect Mathematics :: Evolution or connect Thermodynamics :: Biology and see what it finds.

About

A local knowledge graph engine that can ingests Wikipedia articles and plain text, and extract entities and their relationships using spaCy and Groq LLMs. Has WebGL visualization, natural language querying, and path finding between concepts.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages