Deepkoro: Using Deep Language Models to Understand the Heart of Religion
Overview:
This document will provide a sense of the general motivation for the project. It will also explain, at a high level, the technologies used and techniques deployed, as well as potential takeaways and areas for further improvement/exploration.
Motivation:
Fascinating advancements in technology have resulted in dramatic changes to the way our computers are able to receive, process, and generate data. We are reaching new heights in information processing and generation with developments in machine learning and artificial intelligence. The implications are more vast and deeper than many realize, and the parallels to neuroscience and the human mind are astounding. The true nature of the human senses of vision, hearing, smell, taste, and feel are so amorphous and complex that it is difficult to imagine a world in which a computer can emulate with any degree of confidence such experiences. Artificial intelligence may hold the key to the advancement of knowledge and scientific discovery. Since written language is at the heart of knowledge transmission and human advancement, a natural domain for study arises when one considers some of the oldest and most widely read texts by cultures around the world: religious literature.
Interpretation:
Given the vast array of sacred scriptures from across the space and time of human existence, imagine if the true essence of the commonality therein could be honed to generate a unifying and general piece of religious writing. Such a realization could potentially unite the world's religions through the revelation of the ideas at the heart of spirituality. Our aim for the project is to harness the power of Deep Learning to begin undertaking precisely the task of synthesizing completely original written language embodying what is core and common to the foundational principles of the world's existing religious texts. We do so while deploying different popular deep language models for comparison.
Tools and Techniques:
Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) Design using TensorFlow and Keras in Python
Generative Pretrained Transformer (GPT-2) Implementation in Google Colaboratory
Potential Takeaways/Implications:
We have generated reasonably coherent text data informed by the limited training corpus, but more advancements are needed to reach the stated aim of Deepkoro.
Areas for Improvement and/or Future Exploration:
While this is a small step in the right direction with respect to understanding the heart of religion, much progress remains to be made. The dataset of religious texts, as presently populated, is in no way exhaustive. With an increased corpus, more rigorously validated translations, and oversampling corrections with respect to the historical development of world religions, our models would be better equipped to create in a way that is less biased and appropriately weights inputs. Additionally, by testing and incorporating emerging models and technologies (e.g., harnessing the improvements that GPT-3 offers over GPT-2) and by yielding to the expertise of religious experts with regard to the interpretation of results, we might better glean insights from Deepkoro.
Author(s):
Daniel S. Toohey
Acknowledgements/Links:
Project Gutenberg (http://www.gutenberg.org/)
OpenAI (https://openai.com/blog/better-language-models/)
Houston Smith (https://books.google.com/books/about/The_World_s_Religions.html?id=1G4eNRWYT6gC)
Ludwik Zamenhof (https://uea.org/)