it-claws

it-claws

A common-line tool that automates the process of find and download the latest common hardware drivers, and diagnostic tool.
Report Bug · Request Feature

About The Project

it-claws is a Python-based command-line tool for downloading the latest PC hardware drivers, diagnostic tools and more. Leveraging Selenium, it is able to automatically navigate official websites to locate and retrieve the most up-to-date versions of the target software.

This tool also serves as a companion to install-it. Refer to the Usage section for more information.

(back to top)

Built With

(back to top)

Getting Started

Prerequisites

• Python >= 3.12
• 7zip
• Google Chrome, Microsoft Edge, or Mozilla Firefox

Setup

Install Python Dependencies

pip install -r requirements.txt

Commands

• Run the script

  python src/main.py

• Display help message

  python src/main.py -h

(back to top)

Usage

Customising Scraping Configurations

Creating configuration

it-claws comes with a brunch of software scraping preset (see: src/config.py), which includes a curated list of common hardware drivers, diagnostic tools and common software.

Run the following command to configure your scraping list:

python src/main.py --configure

Providing your custom configuration

To use a custom configuration file, specify it with the -c or --claw-config option.

The module src/url.py provides helper methods to extract download URLs from various websites. You can leverage those when drafting your own scraping configuration.

it-claw accepts Python pickle files, Python source files, and JSON files. Details are as follow.

Python Pickle File

The expected type is Iterable[ClawPrize].

python src/main.py -c ./custom-config.pkl

Python Source File

it-claws will look for the CLAW_CONFIG variable defined in the specified .py file.

# custom-config.py

CLAW_CONFIG: Iterable[ClawPrize] = [
  # define your configuration here
]

python src/main.py -c ./custom-config.py

JSON File

Refer to the JSON Schema for guidance on constructing a valid scraping configuration.

python src/main.py -c ./custom-config.json

Specific the Browser for Scraping

it-claws required a web browser to scrape the download URL. You can use any one of Google Chrome, Microsoft Edge, or Mozilla Firefox (default).

To specific a browser, use -w or --web-driver with the name of the browser choice.

python src/main.py -w Chrome

Including Extra Files in the Archive

Use -i or --include-files to specify the file(s) or directory path(s) you want to include in the output archive. To include multiple paths, either separate them with spaces or provide the option multiple times.

python src/main.py -i foo/ bar/ -i README.md

The install-it/conf directory contains configuration files of the scraping preset for install-it. To use this tool to download drivers and utilities for install-it, include the directory:

python src/main.py -i ./install-it/conf

Then, you can import the output archive into install-it using its import function.

Extract RAR Files in Linux

7z or the binaries from 7-zip.org does not seem to support RAR format.

To extract RAR files, consider to use the package p7zip-full p7zip-rar instead of 7z.

Docker Deployment

it-claws provided container images for you to run it in containerised enviroments. Get the docker image details at Packages.

You can deploy the docker image via docker CLI with:

docker run \
  --name=it-claws \
  -v /path/to/rclone-config:/config/rclone \
  -v /path/to/config:/config/app \
  -v /path/to/downloads:/app/downloads \ `#optional, supply to presist downloads`
  ghcr.io/install-it/it-claws:latest

Configuration

If you want to use the built-in configurator to create the scraping configuration inside the container environment, you may use docker's interactive mode.

# run the container in an interactive terminal session with container auto remove enabled 
docker run -it --rm ghcr.io/install-it/it-claws:latest bash

python src/main.py --configure

# leave the interactive terminal session
exit

Customisation

Override the default CMD (how-to) to alternate how it-claws should be executed.

docker run --name=it-claws <other options> python src/main.py -d /foo/downloads -e ignore

A few scripts for automation are provided. See scripts for more information.

(back to top)