Fast Hash

A fast solution to calculate hash for large files in Java

About The Project

Although most algorithms are efficient enough when calculating a hash for a single file, you will experience a noticeable performance impact when hashing multiple large files, the issue comes from the fact that they read the whole file at once
Fast Hash purpose is to obtain good speed compared to others by reusing any hashing algorithm you want but applying it only on specific chunks of the file

Fast Hash internally use MessageDigest to calculate the hash, you can use any algorithm supported by MessageDigest such as MD5, SHA1, SHA-256...

Fast Hash is inspired by Oshash but implemented differently, here's how it works:

• It reads up to a configurable chunk size from the beginning
• It reads the file size
• It reads up to a configurable chunk size from the end
• Digest the 3 parts and generate a unique hash for the file

You can use SHA1 algorithm and a chunk size of 64 * 1024 which should be good for most cases

Getting Started

This project is built with Maven

Prerequisites

JDK 11+ and Maven

Installation

1. Clone the repo

   git clone https://github.com/marwenlahmar/fast-hash.git

2. Install

   mvn clean install

(back to top)

Usage

To get an instance of Fast Hash

FastHash fastHash = FastHash.getInstance("SHA1", 64 * 1024);

Fast Hash instances are thread safe

To calculate the hash

String hash = fastHash.digest(seekableDataStream);

Fast Hash takes a SeekableDataStream interface as argument Fast Hash provides two implementations for SeekableDataStream: ByteArrayDataStream and FileDataStream

(back to top)

Contributing

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

License

Distributed under the MIT License. See LICENSE for more information.

(back to top)

Contact

Marwen Lahmar - @lahmar_marwen - marwen.lahmar@gmail.com

Project Link: https://github.com/marwenlahmar/fast-hash

(back to top)