diff --git a/README.md b/README.md index ee849fd..00a41c5 100644 --- a/README.md +++ b/README.md @@ -1,124 +1,45 @@ # fetch -`fetch` is a modern, high-level HTTP(S) client for the command line. +A modern, high-level HTTP(S) client for the command line. ![Example of fetch with an image and JSON responses](./assets/example.jpg) -### Features include: +## Features -- **Response formatting**: automatically formats and colors output (json, html, xml, css, csv, msgpack, protobuf, etc.) -- **Image rendering**: render images directly in your terminal -- **Compression**: automatic gzip and zstd response body decompression -- **Authentication**: support for Basic Auth, Bearer Token, and AWS Signature V4 -- **Form body**: send multipart or urlencoded form bodies -- **gRPC support**: make gRPC calls with automatic JSON-to-protobuf conversion -- **Editor integration**: use an editor to modify the request body -- **Configuration**: global and per-host configuration -- _and more!_ +- **Response formatting** - Automatic formatting and syntax highlighting for JSON, XML, HTML, CSS, CSV, MessagePack, Protocol Buffers, and more +- **Image rendering** - Display images directly in your terminal +- **gRPC support** - Make gRPC calls with automatic JSON-to-protobuf conversion +- **Authentication** - Built-in support for Basic Auth, Bearer Token, AWS Signature V4, and mTLS +- **Compression** - Automatic gzip and zstd response body decompression +- **Configuration** - Global and per-host configuration file support ---- +## Quick Start -## Installation - -You can install `fetch` using an installation script, by compiling from source, -or from pre-built binaries. - -### Installation Script - -For macOS or Linux, download and run the [install.sh](./install.sh) script: +#### Install ```sh +# Install fetch from shell script curl -fsSL https://raw.githubusercontent.com/ryanfowler/fetch/main/install.sh | bash -``` - -### Building from Source -Ensure you have Go installed, then run: - -```sh +# Or install fetch with Go go install github.com/ryanfowler/fetch@latest -``` - -### Pre-built Binaries - -Visit the [GitHub releases page](https://github.com/ryanfowler/fetch/releases) -to download the binary for your operating system. -### Updating - -Once installed, you can update the fetch binary in-place by running: - -```sh -fetch --update -``` - -Or you can let the application auto-update by including the following setting in -your [configuration file](#Configuration): - -```ini -auto-update = true +# Make a request +fetch httpbin.org/json ``` ---- - -## Usage - -To make a GET request to a URL and print the status code to stderr and the response body to stdout: - -```sh -fetch example.com -``` - -
HTTP/1.1 200 OK
-
-{
-  "name": "example",
-  "value": 42
-}
-
- -For complete usage documentation including all available options and advanced features, see [USAGE.md](docs/USAGE.md). - ---- - -## Configuration - -`fetch` can be configured using a file with an ini-like format. Configuration files support both global settings and host-specific configurations. - -### File Locations - -`fetch` searches for configuration files in the following order: - -- the file location specified with the `-c` or `--config` flag -- on Windows at `%AppData%\fetch\config` -- on Unix-like systems at `$XDG_CONFIG_HOME/fetch/config` or `$HOME/.config/fetch/config` - -### Configuration Precedence - -Settings are applied in the following order of precedence: - -- CLI flags (highest priority) -- domain-specific configuration -- global configuration -- default values (lowest priority) - -### Basic Example - -```ini -# Global settings -color = on -format = on -timeout = 30 - -# Domain-specific settings -[api.example.com] -header = X-API-Key: your-api-key -timeout = 10 -``` - -For complete configuration documentation including all available options, file format details, and advanced examples, see [CONFIG.md](docs/CONFIG.md). - ---- +## Documentation + +- **[Getting Started](docs/getting-started.md)** - Installation, first steps, and basic concepts +- **[CLI Reference](docs/cli-reference.md)** - Complete reference for all command-line options +- **[Configuration](docs/configuration.md)** - Configuration file format and options +- **[Authentication](docs/authentication.md)** - Basic, Bearer, AWS SigV4, and mTLS +- **[Request Bodies](docs/request-bodies.md)** - JSON, XML, forms, multipart, and file uploads +- **[Output Formatting](docs/output-formatting.md)** - Supported content types and formatting options +- **[Image Rendering](docs/image-rendering.md)** - Terminal image protocols and formats +- **[gRPC](docs/grpc.md)** - Making gRPC requests with Protocol Buffers +- **[Advanced Features](docs/advanced-features.md)** - DNS, proxies, TLS, HTTP versions, and more +- **[Troubleshooting](docs/troubleshooting.md)** - Common issues, debugging, and exit codes ## License diff --git a/docs/USAGE.md b/docs/USAGE.md deleted file mode 100644 index fbbd1e8..0000000 --- a/docs/USAGE.md +++ /dev/null @@ -1,701 +0,0 @@ -# Usage Guide - -This guide provides comprehensive documentation for using `fetch`, a modern HTTP client for the command line. - -## Basic Usage - -To make a GET request to a URL: - -```sh -fetch example.com -``` - -### URL Schemes - -When no scheme is provided, `fetch` defaults to HTTPS: - -```sh -fetch example.com # Uses https://example.com -fetch 192.168.1.1:8080 # Uses https://192.168.1.1:8080 -``` - -Loopback addresses default to HTTP for local development convenience: - -```sh -fetch localhost:3000 # Uses http://localhost:3000 -fetch 127.0.0.1:8080 # Uses http://127.0.0.1:8080 -``` - -You can always specify the scheme explicitly: - -```sh -fetch http://example.com # Force HTTP -fetch https://localhost # Force HTTPS for localhost -``` - -## Authentication Options - -### AWS Signature V4 - -**Flag**: `--aws-sigv4 REGION/SERVICE` - -Sign the request using [AWS Signature V4](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html). - -Requires: `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables. - -```sh -fetch --aws-sigv4 us-east-1/s3 example.com -``` - -### Basic Authentication - -**Flag**: `--basic USER:PASS` - -Enable HTTP Basic Authentication. - -```sh -fetch --basic username:password example.com -``` - -### Bearer Token - -**Flag**: `--bearer TOKEN` - -Enable HTTP Bearer Token Authentication. - -```sh -fetch --bearer mysecrettoken example.com -``` - -## Request Body Options - -Body options generally accept values in the format: `[@]VALUE` - -- A value without `@` prefix is sent directly -- A value prefixed with `@` sends a file at the given path -- A value of `@-` sends data read from stdin - -### Raw Request Body - -**Flag**: `-d, --data [@]VALUE` - -Send a raw request body. - -```sh -fetch -d 'Hello, world!' -m PUT example.com -fetch -d @data.txt -m PUT example.com -fetch -d @- -m PUT example.com < data.txt -``` - -### JSON Request Body - -**Flag**: `-j, --json [@]VALUE` - -Send a JSON request body. Automatically sets the `Content-Type` header to `application/json`. - -```sh -fetch -j '{"hello":"world"}' -m PUT example.com -fetch -j @data.json -m PUT example.com -``` - -### XML Request Body - -**Flag**: `-x, --xml [@]VALUE` - -Send an XML request body. Automatically sets the `Content-Type` header to `application/xml`. - -```sh -fetch -x 'value' -m PUT example.com -fetch -x @data.xml -m PUT example.com -``` - -### URL-Encoded Form Body - -**Flag**: `-f, --form KEY=VALUE` - -Send a URL-encoded form body. Can be used multiple times to add multiple fields. - -```sh -fetch -f hello=world -f name=value -m POST example.com -``` - -### Multipart Form Body - -**Flag**: `-F, --multipart NAME=[@]VALUE` - -Send a multipart form body. Can be used multiple times to add multiple fields. - -```sh -fetch -F hello=world -F data=@/path/to/file.txt -m POST example.com -fetch -F "file=@image.png" -F "description=My image" -m POST example.com -``` - -### Editor Integration - -**Flag**: `-e, --edit` - -Edit the request body with an editor before sending. Uses `VISUAL` or `EDITOR` environment variables, or falls back to well-known editors. - -```sh -fetch --edit -m PUT example.com -``` - -## General Request Options - -### HTTP Method - -**Flag**: `-m, --method METHOD` (alias: `-X`) - -Specify the HTTP method to use. Default is GET. - -```sh -fetch -m POST example.com -fetch -X DELETE example.com -``` - -### Custom Headers - -**Flag**: `-H, --header NAME:VALUE` - -Set custom headers on the request. Can be used multiple times. - -```sh -fetch -H "x-custom-header: value" -H "Authorization: Bearer token" example.com -``` - -### Query Parameters - -**Flag**: `-q, --query KEY=VALUE` - -Append query parameters to the URL. Can be used multiple times. - -```sh -fetch -q hello=world -q page=2 example.com -``` - -### Range Requests - -**Flag**: `-r, --range RANGE` - -Set the [Range](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range) request header. Can be used multiple times for multiple ranges. - -```sh -fetch -r 0-1023 example.com -fetch -r 0-499 -r 1000-1499 example.com -``` - -### Redirects - -**Flag**: `--redirects NUM` - -Set the maximum allowed automatic redirects. Use `0` to disable redirects. - -```sh -fetch --redirects 0 example.com -fetch --redirects 10 example.com -``` - -### Timeout - -**Flag**: `-t, --timeout SECONDS` - -Set a timeout for the entire request in seconds. Accepts decimal values. - -```sh -fetch --timeout 30 example.com -fetch --timeout 2.5 example.com -``` - -### Custom DNS Server - -**Flag**: `--dns-server IP[:PORT]|URL` - -Use a custom DNS server. Can be either: - -- IP address with optional port for UDP DNS -- HTTPS URL for DNS-over-HTTPS - -```sh -fetch --dns-server 8.8.8.8 example.com -fetch --dns-server 1.1.1.1:53 example.com -fetch --dns-server https://1.1.1.1/dns-query example.com -``` - -### Proxy - -**Flag**: `--proxy PROXY` - -Route the request through the specified proxy. - -```sh -fetch --proxy http://localhost:8080 example.com -fetch --proxy socks5://localhost:1080 example.com -``` - -### Unix Socket - -**Flag**: `--unix PATH` - -Make the request over a Unix domain socket. Only available on Unix-like systems. - -```sh -fetch --unix /var/run/docker.sock http://unix/containers/json -fetch --unix /var/run/service.sock http://unix/api/status -``` - -### TLS Options - -**Flag**: `--insecure` - -Allow for invalid TLS certificates from the server. - -```sh -fetch --insecure https://self-signed.example.com -``` - -**Flag**: `--tls VERSION` - -Specify the minimum TLS version to use. Must be one of: `1.0`, `1.1`, `1.2`, `1.3`. - -```sh -fetch --tls 1.3 example.com -``` - -### HTTP Version - -**Flag**: `--http VERSION` - -Specify the exact HTTP version to use. Must be `1`, `2`, or `3`. -By default, HTTP/2 is preferred but will fallback to using HTTP/1.1. - -```sh -fetch --http 1 example.com -``` - -### Compression - -**Flag**: `--no-encode` - -Disable automatic gzip and zstd request/response compression. - -```sh -fetch --no-encode example.com -``` - -### Custom CA Certificate - -**Flag**: `--ca-cert` - -Use a custom CA certificate. - -```sh -fetch --ca-cert ca-cert.pem example.com -``` - -## Output Options - -### Output to File - -**Flag**: `-o, --output PATH` - -Write the response body to the specified file. Truncates existing files. - -```sh -fetch -o response.json example.com/api/data -fetch -o ~/downloads/file.zip example.com/file.zip -``` - -### Output to Current Directory - -**Flag**: `-O, --output-current-dir` - -Write the response body to the current directory using the filename from the URL. - -```sh -fetch -O example.com/path/to/file.txt -# Creates ./file.txt -``` - -### Colored Output - -**Flag**: `--color OPTION` (alias: `--colour`) - -Set whether output should be colored. Options: `auto`, `off`, `on`. - -```sh -fetch --color off example.com -fetch --colour on example.com -``` - -### Formatted Output - -**Flag**: `--format OPTION` - -Set whether output should be formatted. Options: `auto`, `off`, `on`. - -Supported formats for automatic formatting and syntax highlighting: - -- JSON (`application/json`) -- HTML (`text/html`) -- XML (`application/xml`, `text/xml`) -- CSS (`text/css`) -- CSV (`text/csv`) -- MessagePack (`application/msgpack`) -- NDJSON/JSONLines (`application/x-ndjson`) -- Protobuf (`application/x-protobuf`, `application/protobuf`) -- Server-Sent Events (`text/event-stream`) - -CSV output is automatically column-aligned. When output is too wide for the terminal, it switches to a vertical "record view" format where each row is displayed with field names as labels. - -```sh -fetch --format off example.com -fetch --format on example.com -``` - -### Image Rendering - -**Flag**: `--image OPTION` - -Set how images should be rendered in the terminal. Options: `auto`, `native`, `off`. - -- `auto`: Try optimal protocol, fallback to external tools -- `native`: Use only built-in decoders (jpeg, png, tiff, webp) -- `off`: Disable image rendering - -```sh -fetch --image native example.com/image.png -fetch --image off example.com/image.jpg -``` - -### Pager Control - -**Flag**: `--no-pager` - -Disable piping output to a pager like `less`. - -```sh -fetch --no-pager example.com -``` - -## Verbosity Options - -### Verbose Output - -**Flag**: `-v, --verbose` - -Increase verbosity of output to stderr. Can be used multiple times: - -- `-v`: Show response headers -- `-vv`: Show request and response headers -- `-vvv`: Show DNS and TLS details - -```sh -fetch -v example.com -fetch -vv example.com -fetch -vvv example.com -``` - -### Silent Mode - -**Flag**: `-s, --silent` - -Suppress verbose output. Only warnings and errors are written to stderr. - -```sh -fetch -s example.com -``` - -### Ignore HTTP Status - -**Flag**: `--ignore-status` - -Don't determine exit code from HTTP status. Always exit with code 0 instead of using 4xx/5xx status codes. - -```sh -fetch --ignore-status example.com -``` - -## Configuration Options - -### Config File - -**Flag**: `-c, --config PATH` - -Specify a custom configuration file path. - -```sh -fetch --config ~/.config/fetch/custom.conf example.com -``` - -## Utility Options - -### Help - -**Flag**: `-h, --help` - -Print help information. - -```sh -fetch --help -``` - -### Version Information - -**Flag**: `-V, --version` - -Print version information. - -```sh -fetch --version -``` - -### Build Information - -**Flag**: `--buildinfo` - -Print detailed build information including version, commit, and build date. - -```sh -fetch --buildinfo -``` - -### Update - -**Flag**: `--update` - -Update the fetch binary in place. - -```sh -fetch --update -``` - -### Shell Completion - -**Flag**: `--complete SHELL` - -Output shell completion scripts. Supported shells: `fish`, `zsh`. - -```sh -fetch --complete zsh > ~/.zshrc.d/fetch-completion.zsh -fetch --complete fish > ~/.config/fish/completions/fetch.fish -``` - -### Dry Run - -**Flag**: `--dry-run` - -Print request information without actually sending the request. - -```sh -fetch --dry-run -m POST -j '{"test": true}' example.com -``` - -## Value Formats - -### File References - -Many options support file references with the `@` prefix: - -- `@filename` - Read content from file -- `@-` - Read content from stdin -- `@~/path` - Home directory expansion supported - -### Environment Variables - -The following environment variables are recognized: - -- `AWS_ACCESS_KEY_ID` - For AWS Signature V4 authentication -- `AWS_SECRET_ACCESS_KEY` - For AWS Signature V4 authentication -- `VISUAL` or `EDITOR` - For editor integration -- `HTTPS_PROXY`, `HTTP_PROXY`, `NO_PROXY` - For proxy configuration - -### Special Characters - -When using special characters in values, proper shell escaping may be required: - -```sh -# Escape quotes in JSON -fetch -j '{"message": "Hello \"World\""}' example.com - -# Use single quotes to avoid shell interpretation -fetch -H 'Authorization: Bearer token-with-$pecial-chars' example.com -``` - -## gRPC Support - -`fetch` supports making gRPC calls with automatic protocol handling, JSON-to-protobuf conversion, and response formatting. - -### Basic gRPC Request - -**Flag**: `--grpc` - -Enable gRPC mode. This flag: - -- Uses the HTTP/2 protocol -- Sets the method to `POST` -- Adds gRPC headers (`Content-Type: application/grpc+proto`, `TE: trailers`, etc.) -- Applies gRPC framing to the request body -- Handles gRPC framing on the response - -The service and method are specified in the URL path in the format `/package.Service/Method`: - -```sh -fetch https://localhost:50051/mypackage.MyService/MyMethod --grpc --insecure -``` - -### Proto Schema Options - -To enable JSON-to-protobuf conversion for request bodies and rich protobuf formatting for responses, provide a proto schema using one of these options: - -**Flag**: `--proto-file PATH` - -Compile `.proto` file(s) using `protoc`. Requires `protoc` to be installed. Can be specified multiple times or with comma-separated paths. - -```sh -fetch https://localhost:50051/echo.EchoService/Echo \ - --grpc \ - --proto-file service.proto \ - -j '{"message": "hello", "count": 42}' \ - --insecure -``` - -**Flag**: `--proto-desc PATH` - -Use a pre-compiled descriptor set file. This is useful when `protoc` isn't available at runtime or to avoid recompilation. - -Generate a descriptor set with: - -```sh -protoc --descriptor_set_out=service.pb --include_imports service.proto -``` - -Then use it: - -```sh -fetch https://localhost:50051/echo.EchoService/Echo \ - --grpc \ - --proto-desc service.pb \ - -j '{"message": "hello", "count": 42}' \ - --insecure -``` - -**Flag**: `--proto-import PATH` - -Add import paths for proto compilation. Use with `--proto-file` when your proto files have imports. - -```sh -fetch https://localhost:50051/mypackage.MyService/MyMethod \ - --grpc \ - --proto-file service.proto \ - --proto-import ./proto \ - --proto-import /usr/local/include \ - -j '{"field": "value"}' \ - --insecure -``` - -### How It Works - -When `--grpc` is used with a proto schema (`--proto-file` or `--proto-desc`): - -1. The service and method are extracted from the URL path -2. The method's input/output message types are looked up in the schema -3. JSON request bodies are automatically converted to protobuf -4. Protobuf responses are formatted with field names from the schema - -Without a proto schema, `fetch` still handles gRPC framing but: - -- Request bodies must be raw protobuf (not JSON) -- Responses are formatted using generic protobuf parsing (field numbers instead of names) - -### Examples - -**Simple gRPC call with JSON body:** - -```sh -fetch https://api.example.com/package.Service/Method \ - --grpc \ - --proto-file service.proto \ - -j '{"name": "test", "value": 123}' -``` - -**gRPC call with verbose output:** - -```sh -fetch https://api.example.com/package.Service/Method \ - --grpc \ - --proto-file service.proto \ - -j '{"request": "data"}' \ - -vv -``` - -**gRPC call to local server with self-signed certificate:** - -```sh -fetch https://localhost:50051/echo.EchoService/Echo \ - --grpc \ - --proto-desc echo.pb \ - -j '{"message": "hello"}' \ - --insecure -``` - -**Dry run to inspect the request:** - -```sh -fetch https://api.example.com/package.Service/Method \ - --grpc \ - --proto-file service.proto \ - -j '{"field": "value"}' \ - --dry-run -``` - -**Use an editor to modify the request body:** - -```sh -fetch https://api.example.com/package.Service/Method \ - --grpc \ - --proto-file service.proto \ - -j '{"field": "value"}' \ - --edit -``` - -## Advanced Usage - -### Combining Options - -Options can be combined for complex requests: - -```sh -fetch \ - --method POST \ - --header "Content-Type: application/json" \ - --header "Authorization: Bearer token" \ - --json '{"user": "john", "action": "login"}' \ - --query "version=2" \ - --timeout 30 \ - --verbose \ - example.com/api/auth -``` - -### Using with Pipes - -```sh -# Send stdin as request body -echo '{"hello": "world"}' | fetch -j @- -m POST example.com - -# Save response to file and view -fetch example.com/large-response.json | jq . > formatted.json - -# Chain requests -fetch example.com/auth | jq -r '.token' | fetch --bearer @- example.com/protected -``` - -### Configuration Precedence - -Options are applied in the following order (highest to lowest precedence): - -1. Command line flags -2. Domain-specific configuration -3. Global configuration -4. Default values - -This allows for flexible configuration where you can set defaults globally and override them per-domain or per-command. diff --git a/docs/advanced-features.md b/docs/advanced-features.md new file mode 100644 index 0000000..2fbfe32 --- /dev/null +++ b/docs/advanced-features.md @@ -0,0 +1,414 @@ +# Advanced Features + +This guide covers advanced networking, protocol, and TLS options in `fetch`. + +## Custom DNS Resolution + +### `--dns-server` + +Use a custom DNS server instead of the system resolver. + +### UDP DNS + +Specify an IP address with optional port: + +```sh +# Google DNS +fetch --dns-server 8.8.8.8 example.com + +# Cloudflare DNS with custom port +fetch --dns-server 1.1.1.1:53 example.com + +# IPv6 DNS server +fetch --dns-server "[2001:4860:4860::8888]:53" example.com +``` + +### DNS-over-HTTPS (DoH) + +Use HTTPS URL for encrypted DNS queries: + +```sh +# Cloudflare DoH +fetch --dns-server https://1.1.1.1/dns-query example.com + +# Google DoH +fetch --dns-server https://dns.google/dns-query example.com + +# Quad9 DoH +fetch --dns-server https://dns.quad9.net/dns-query example.com +``` + +### Configuration File + +```ini +# Use Cloudflare DNS globally +dns-server = 1.1.1.1 + +# Use DoH for specific hosts +[secure.example.com] +dns-server = https://1.1.1.1/dns-query +``` + +## Proxy Configuration + +### `--proxy` + +Route requests through a proxy server. + +### HTTP Proxy + +```sh +fetch --proxy http://proxy.example.com:8080 example.com +``` + +### HTTPS Proxy + +```sh +fetch --proxy https://secure-proxy.example.com:8443 example.com +``` + +### SOCKS5 Proxy + +```sh +fetch --proxy socks5://localhost:1080 example.com +``` + +### Authenticated Proxy + +```sh +fetch --proxy http://user:password@proxy.example.com:8080 example.com +``` + +### Environment Variables + +`fetch` respects standard proxy environment variables: + +```sh +export HTTP_PROXY="http://proxy.example.com:8080" +export HTTPS_PROXY="http://proxy.example.com:8080" +export NO_PROXY="localhost,127.0.0.1,.internal.com" + +fetch example.com # Uses proxy from environment +``` + +### Configuration File + +```ini +# Global proxy +proxy = http://proxy.example.com:8080 + +# Host-specific proxy +[internal.example.com] +proxy = socks5://internal-proxy:1080 +``` + +## Unix Domain Sockets + +### `--unix` + +Connect via Unix domain socket instead of TCP. Available on Unix-like systems only. + +### Docker API + +```sh +fetch --unix /var/run/docker.sock http://localhost/containers/json +fetch --unix /var/run/docker.sock http://localhost/images/json +``` + +### Custom Services + +```sh +fetch --unix /var/run/myservice.sock http://localhost/api/status +fetch --unix ~/myapp.sock http://localhost/health +``` + +**Note**: The hostname in the URL is ignored when using Unix sockets; the socket path determines the destination. + +## HTTP Versions + +### `--http VERSION` + +Force a specific HTTP version. + +### HTTP/1.1 + +```sh +fetch --http 1 example.com +``` + +- Uses HTTP/1.1 protocol +- Single request per connection +- No header compression +- Useful for debugging or legacy servers + +### HTTP/2 + +```sh +fetch --http 2 example.com +``` + +- Default behavior (HTTP/2 preferred with fallback) +- Multiplexed streams +- Header compression (HPACK) +- Required for gRPC + +### HTTP/3 (QUIC) + +```sh +fetch --http 3 example.com +``` + +- Uses QUIC transport (UDP-based) +- Reduced latency +- Better handling of packet loss +- Not all servers support HTTP/3 + +### Version Detection + +By default, `fetch` negotiates the best available version: + +1. Attempts HTTP/2 via ALPN +2. Falls back to HTTP/1.1 if needed + +## TLS Configuration + +### Minimum TLS Version + +`--tls VERSION` sets the minimum acceptable TLS version: + +```sh +fetch --tls 1.2 example.com # Require TLS 1.2+ +fetch --tls 1.3 example.com # Require TLS 1.3 +``` + +| Value | Protocol | +| ----- | ----------------------------- | +| `1.0` | TLS 1.0 (legacy) | +| `1.1` | TLS 1.1 (deprecated) | +| `1.2` | TLS 1.2 (recommended minimum) | +| `1.3` | TLS 1.3 (most secure) | + +### Insecure Mode + +`--insecure` accepts invalid TLS certificates: + +```sh +fetch --insecure https://self-signed.example.com +``` + +**Warning**: Only use for development/testing. Never in production. + +### Custom CA Certificate + +`--ca-cert` specifies a custom CA certificate: + +```sh +fetch --ca-cert /path/to/ca.crt https://internal.example.com +``` + +Use cases: + +- Internal PKI with private CA +- Development with self-signed certificates +- Corporate environments with SSL inspection + +### Configuration File + +```ini +# Require TLS 1.2 minimum +tls = 1.2 + +# Internal server with private CA +[internal.company.com] +ca-cert = /etc/pki/internal-ca.crt + +# Development server (insecure) +[dev.localhost] +insecure = true +``` + +## Compression + +### `--no-encode` + +Disable automatic compression negotiation: + +```sh +fetch --no-encode example.com +``` + +By default, `fetch`: + +- Sends `Accept-Encoding: gzip, zstd` header +- Automatically decompresses responses + +Disabling is useful when: + +- Testing compression behavior +- Server has compression bugs +- You want to see raw compressed data + +## Range Requests + +### `-r, --range RANGE` + +Request specific byte ranges (partial content): + +```sh +# First 1KB +fetch -r 0-1023 example.com/file.bin + +# Last 500 bytes +fetch -r -500 example.com/file.bin + +# Skip first 1000 bytes +fetch -r 1000- example.com/file.bin +``` + +### Multiple Ranges + +```sh +fetch -r 0-499 -r 1000-1499 example.com/file.bin +``` + +This sets the header: + +``` +Range: bytes=0-499, 1000-1499 +``` + +### Use Cases + +- Resume interrupted downloads +- Download specific portions of large files +- Video seeking +- Parallel downloads + +## Redirect Control + +### `--redirects NUM` + +Set maximum number of automatic redirects: + +```sh +# Disable redirects +fetch --redirects 0 example.com + +# Allow up to 10 redirects +fetch --redirects 10 example.com +``` + +### Verbose Redirect Tracking + +```sh +fetch -v --redirects 5 example.com +``` + +Shows each redirect hop with status codes. + +## Request Timeout + +### `-t, --timeout SECONDS` + +Set a timeout for the entire request: + +```sh +fetch --timeout 30 example.com +fetch --timeout 2.5 example.com # Decimal seconds +``` + +The timeout covers: + +- DNS resolution +- Connection establishment +- TLS handshake +- Request/response transfer + +### Configuration File + +```ini +# Global timeout +timeout = 30 + +# Longer timeout for slow API +[slow-api.example.com] +timeout = 120 +``` + +## Combining Options + +Complex requests often combine multiple advanced options: + +```sh +fetch \ + --dns-server https://1.1.1.1/dns-query \ + --proxy socks5://localhost:9050 \ + --tls 1.3 \ + --timeout 60 \ + --http 2 \ + -v \ + https://example.onion/api +``` + +### Configuration File Example + +```ini +# Global settings +timeout = 30 +tls = 1.2 +dns-server = 8.8.8.8 + +# Internal services +[internal.company.com] +proxy = http://internal-proxy:8080 +ca-cert = /etc/pki/internal-ca.crt +insecure = false + +# Development environment +[localhost] +insecure = true +timeout = 5 + +# High-security API +[secure-api.example.com] +tls = 1.3 +timeout = 60 +``` + +## Debugging Network Issues + +### Verbose Output + +```sh +fetch -v example.com # Response headers +fetch -vv example.com # Request + response headers +fetch -vvv example.com # DNS + TLS details +``` + +### Dry Run + +Preview the request without sending: + +```sh +fetch --dry-run -m POST -j '{"test": true}' example.com +``` + +### Testing Connectivity + +```sh +# Test with specific DNS +fetch --dns-server 8.8.8.8 -v example.com + +# Test with explicit HTTP version +fetch --http 1 -v example.com + +# Test TLS configuration +fetch --tls 1.3 -vvv example.com +``` + +## See Also + +- [CLI Reference](cli-reference.md) - Complete option reference +- [Authentication](authentication.md) - mTLS and other auth methods +- [Configuration](configuration.md) - Configuration file options +- [Troubleshooting](troubleshooting.md) - Network debugging diff --git a/docs/authentication.md b/docs/authentication.md new file mode 100644 index 0000000..a937c7b --- /dev/null +++ b/docs/authentication.md @@ -0,0 +1,247 @@ +# Authentication + +`fetch` supports multiple authentication methods for accessing protected resources. + +## HTTP Basic Authentication + +Basic Authentication sends credentials as a base64-encoded `Authorization` header. + +### Command Line + +```sh +fetch --basic username:password example.com +``` + +### How It Works + +The `--basic` flag sets the `Authorization` header: + +``` +Authorization: Basic base64(username:password) +``` + +## Bearer Token Authentication + +Bearer tokens are commonly used with OAuth 2.0 and JWT-based authentication. + +### Command Line + +```sh +fetch --bearer mytoken123 example.com +``` + +### How It Works + +The `--bearer` flag sets the `Authorization` header: + +``` +Authorization: Bearer mytoken123 +``` + +### Using Environment Variables + +For security, avoid putting tokens directly in commands: + +```sh +fetch --bearer "$API_TOKEN" example.com +``` + +Or read from a file: + +```sh +fetch -H "Authorization: Bearer $(cat ~/.api-token)" example.com +``` + +## AWS Signature V4 + +Sign requests for AWS services using [AWS Signature V4](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html). + +### Prerequisites + +Set the required environment variables: + +```sh +export AWS_ACCESS_KEY_ID="your-access-key" +export AWS_SECRET_ACCESS_KEY="your-secret-key" +``` + +### Command Line + +```sh +fetch --aws-sigv4 REGION/SERVICE url +``` + +### Examples + +```sh +# S3 request +fetch --aws-sigv4 us-east-1/s3 https://my-bucket.s3.amazonaws.com/key + +# API Gateway +fetch --aws-sigv4 us-west-2/execute-api https://abc123.execute-api.us-west-2.amazonaws.com/prod/resource + +# Lambda function URL +fetch --aws-sigv4 eu-west-1/lambda https://xyz.lambda-url.eu-west-1.on.aws/ +``` + +### How It Works + +AWS SigV4 signs the request by: + +1. Creating a canonical request from the HTTP method, path, query string, headers, and body +2. Generating a signing key from your secret key, date, region, and service +3. Computing an HMAC-SHA256 signature +4. Adding the signature to the `Authorization` header + +## Mutual TLS (mTLS) + +mTLS provides two-way authentication where both client and server present certificates. + +### Basic Usage + +```sh +fetch --cert client.crt --key client.key example.com +``` + +### Combined Certificate and Key + +If your PEM file contains both the certificate and private key: + +```sh +fetch --cert client.pem example.com +``` + +### With Custom CA Certificate + +When the server uses a private CA: + +```sh +fetch --cert client.crt --key client.key --ca-cert ca.crt example.com +``` + +### Configuration File + +```ini +# Global mTLS settings +cert = /path/to/client.crt +key = /path/to/client.key + +# Host-specific mTLS +[api.secure.example.com] +cert = /path/to/api-client.crt +key = /path/to/api-client.key +ca-cert = /path/to/api-ca.crt +``` + +### Certificate Formats + +- Certificates and keys must be in PEM format +- Encrypted private keys are not supported +- Combined PEM files should have the certificate before the key + +### Example: Self-Signed Certificates + +Generate test certificates: + +```sh +# Generate CA +openssl genrsa -out ca.key 4096 +openssl req -new -x509 -days 365 -key ca.key -out ca.crt -subj "/CN=Test CA" + +# Generate client certificate +openssl genrsa -out client.key 4096 +openssl req -new -key client.key -out client.csr -subj "/CN=client" +openssl x509 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt +``` + +Use with fetch: + +```sh +fetch --cert client.crt --key client.key --ca-cert ca.crt https://mtls.example.com +``` + +## Custom Headers + +For authentication methods not directly supported, use custom headers: + +```sh +# API Key in header +fetch -H "X-API-Key: your-api-key" example.com + +# Custom token format +fetch -H "X-Auth-Token: custom-token" example.com + +# Multiple auth headers +fetch -H "X-API-Key: key" -H "X-Signature: sig" example.com +``` + +### Configuration File + +```ini +[api.example.com] +header = X-API-Key: your-api-key +header = X-Client-ID: client123 +``` + +## Authentication Precedence + +Authentication options are mutually exclusive. You cannot combine: + +- `--basic` +- `--bearer` +- `--aws-sigv4` + +If you need multiple authentication headers, use `-H` for additional headers. + +## Security Considerations + +1. **Avoid embedding secrets in scripts** - Use environment variables or secure vaults +2. **Protect configuration files** - Set appropriate file permissions (`chmod 600`) +3. **Use HTTPS** - Never send credentials over unencrypted HTTP +4. **Rotate credentials regularly** - Especially API keys and tokens + +### Secure Credential Handling + +```sh +# Using environment variables +export API_TOKEN="$(vault read -field=token secret/api)" +fetch --bearer "$API_TOKEN" example.com + +# Using password manager +fetch --basic "$(pass show api/credentials)" example.com + +# Reading from secure file +fetch --bearer "$(cat /run/secrets/api-token)" example.com +``` + +## Troubleshooting + +### 401 Unauthorized + +- Verify credentials are correct +- Check if the authentication method matches what the server expects +- Ensure tokens haven't expired + +### 403 Forbidden + +- Authentication succeeded but authorization failed +- Check if your credentials have the required permissions + +### Certificate Errors with mTLS + +- Verify certificate and key match: `openssl x509 -noout -modulus -in cert.crt | openssl md5` should match `openssl rsa -noout -modulus -in key.key | openssl md5` +- Check certificate expiration: `openssl x509 -noout -dates -in cert.crt` +- Ensure the CA certificate is correct for the server + +### AWS SigV4 Errors + +- Verify `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are set +- Check the region and service name are correct +- Ensure your credentials have the required IAM permissions +- Verify system clock is accurate (signatures are time-sensitive) + +## See Also + +- [CLI Reference](cli-reference.md) - All authentication flags +- [Configuration](configuration.md) - Setting up authentication in config files +- [Troubleshooting](troubleshooting.md) - Common issues and solutions diff --git a/docs/cli-reference.md b/docs/cli-reference.md new file mode 100644 index 0000000..5dd3775 --- /dev/null +++ b/docs/cli-reference.md @@ -0,0 +1,477 @@ +# CLI Reference + +Complete reference for all `fetch` command-line options. + +## Usage + +``` +fetch [OPTIONS] [URL] +``` + +## URL Handling + +When no scheme is provided, `fetch` defaults to HTTPS. Loopback addresses (`localhost`, `127.0.0.1`) default to HTTP. + +```sh +fetch example.com # https://example.com +fetch localhost:3000 # http://localhost:3000 +fetch http://example.com # Force HTTP +``` + +## HTTP Method + +### `-m, --method METHOD` + +Specify the HTTP method. Default: `GET`. + +**Alias**: `-X` + +```sh +fetch -m POST example.com +fetch -X DELETE example.com/resource/123 +``` + +## Headers and Query Parameters + +### `-H, --header NAME:VALUE` + +Set custom headers. Can be used multiple times. + +```sh +fetch -H "Authorization: Bearer token" example.com +fetch -H "X-Custom: value" -H "Accept: application/json" example.com +``` + +### `-q, --query KEY=VALUE` + +Append query parameters to the URL. Can be used multiple times. + +```sh +fetch -q page=1 -q limit=50 example.com +``` + +## Request Body Options + +Body options are mutually exclusive - only one can be used per request. + +### `-d, --data [@]VALUE` + +Send a raw request body. Content-Type is auto-detected when using file references. + +```sh +fetch -d 'Hello, world!' -m PUT example.com +fetch -d @data.txt -m PUT example.com +fetch -d @- -m PUT example.com < data.txt +``` + +### `-j, --json [@]VALUE` + +Send a JSON request body. Sets `Content-Type: application/json`. + +```sh +fetch -j '{"hello": "world"}' -m POST example.com +fetch -j @data.json -m POST example.com +``` + +### `-x, --xml [@]VALUE` + +Send an XML request body. Sets `Content-Type: application/xml`. + +```sh +fetch -x 'value' -m PUT example.com +fetch -x @data.xml -m PUT example.com +``` + +### `-f, --form KEY=VALUE` + +Send a URL-encoded form body. Can be used multiple times. + +```sh +fetch -f username=john -f password=secret -m POST example.com/login +``` + +### `-F, --multipart NAME=[@]VALUE` + +Send a multipart form body. Use `@` prefix for file uploads. Can be used multiple times. + +```sh +fetch -F hello=world -F file=@document.pdf -m POST example.com/upload +``` + +### `-e, --edit` + +Open an editor to modify the request body before sending. Uses `VISUAL` or `EDITOR` environment variables. + +```sh +fetch --edit -m PUT example.com +``` + +## Authentication + +Authentication options are mutually exclusive. + +### `--basic USER:PASS` + +HTTP Basic Authentication. + +```sh +fetch --basic username:password example.com +``` + +### `--bearer TOKEN` + +HTTP Bearer Token Authentication. + +```sh +fetch --bearer mysecrettoken example.com +``` + +### `--aws-sigv4 REGION/SERVICE` + +Sign requests with AWS Signature V4. Requires `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables. + +```sh +fetch --aws-sigv4 us-east-1/s3 s3.amazonaws.com/bucket/key +``` + +### `--cert PATH` + +Client certificate file for mTLS. PEM format. + +```sh +fetch --cert client.crt --key client.key example.com +``` + +### `--key PATH` + +Client private key file for mTLS. Required if `--cert` is a certificate-only file. + +```sh +fetch --cert client.crt --key client.key example.com +``` + +## Output Options + +### `-o, --output PATH` + +Write response body to a file. Use `-` for stdout (bypasses binary detection). + +```sh +fetch -o response.json example.com/api/data +fetch -o - example.com/file.bin > output.bin +``` + +### `-O, --remote-name` + +Write response body to current directory using the filename from the URL. + +**Alias**: `--output-current-dir` + +```sh +fetch -O example.com/path/to/file.txt # Creates ./file.txt +``` + +### `-J, --remote-header-name` + +Use filename from `Content-Disposition` header. Requires `-O`. + +```sh +fetch -O -J example.com/download +``` + +### `--clobber` + +Overwrite existing output file (default behavior is to fail if file exists). + +```sh +fetch -o output.json --clobber example.com/data +``` + +## Formatting Options + +### `--format OPTION` + +Control response formatting. Values: `auto`, `on`, `off`. + +```sh +fetch --format off example.com # Disable formatting +fetch --format on example.com # Force formatting +``` + +### `--color OPTION` + +Control colored output. Values: `auto`, `on`, `off`. + +**Alias**: `--colour` + +```sh +fetch --color off example.com +``` + +### `--image OPTION` + +Control image rendering. Values: `auto`, `native`, `off`. + +- `auto` - Try optimal protocol, fallback to external tools +- `native` - Use only built-in decoders (JPEG, PNG, TIFF, WebP) +- `off` - Disable image rendering + +```sh +fetch --image native example.com/image.png +fetch --image off example.com/photo.jpg +``` + +### `--no-pager` + +Disable piping output to a pager (`less`). + +```sh +fetch --no-pager example.com +``` + +## Network Options + +### `-t, --timeout SECONDS` + +Request timeout in seconds. Accepts decimal values. + +```sh +fetch --timeout 30 example.com +fetch --timeout 2.5 example.com +``` + +### `--redirects NUM` + +Maximum automatic redirects. Use `0` to disable. + +```sh +fetch --redirects 0 example.com # Don't follow redirects +fetch --redirects 10 example.com +``` + +### `--dns-server IP[:PORT]|URL` + +Use custom DNS server. Supports UDP DNS and DNS-over-HTTPS. + +```sh +fetch --dns-server 8.8.8.8 example.com +fetch --dns-server 1.1.1.1:53 example.com +fetch --dns-server https://1.1.1.1/dns-query example.com +``` + +### `--proxy PROXY` + +Route request through a proxy. + +```sh +fetch --proxy http://localhost:8080 example.com +fetch --proxy socks5://localhost:1080 example.com +``` + +### `--unix PATH` + +Make request over a Unix domain socket. Unix-like systems only. + +```sh +fetch --unix /var/run/docker.sock http://unix/containers/json +``` + +## TLS Options + +### `--tls VERSION` + +Minimum TLS version. Values: `1.0`, `1.1`, `1.2`, `1.3`. + +```sh +fetch --tls 1.3 example.com +``` + +### `--insecure` + +Accept invalid TLS certificates. Use with caution. + +```sh +fetch --insecure https://self-signed.example.com +``` + +### `--ca-cert PATH` + +Custom CA certificate file. + +```sh +fetch --ca-cert ca-cert.pem example.com +``` + +## HTTP Version + +### `--http VERSION` + +Force specific HTTP version. Values: `1`, `2`, `3`. + +- `1` - HTTP/1.1 +- `2` - HTTP/2 (default preference) +- `3` - HTTP/3 (QUIC) + +```sh +fetch --http 1 example.com +fetch --http 3 example.com +``` + +## Compression + +### `--no-encode` + +Disable automatic gzip/zstd compression. + +```sh +fetch --no-encode example.com +``` + +## Range Requests + +### `-r, --range RANGE` + +Request specific byte ranges. Can be used multiple times. + +```sh +fetch -r 0-1023 example.com/file.bin +fetch -r 0-499 -r 1000-1499 example.com/file.bin +``` + +## Verbosity + +### `-v, --verbose` + +Increase output verbosity. Can be stacked. + +- `-v` - Show response headers +- `-vv` - Show request and response headers +- `-vvv` - Show DNS and TLS details + +```sh +fetch -v example.com +fetch -vvv example.com +``` + +### `-s, --silent` + +Suppress verbose output. Only errors shown on stderr. + +```sh +fetch -s example.com +``` + +### `--ignore-status` + +Don't use HTTP status code for exit code. Always exit 0 on successful request. + +```sh +fetch --ignore-status example.com/not-found +``` + +## gRPC Options + +### `--grpc` + +Enable gRPC mode. Automatically sets HTTP/2, POST method, and gRPC headers. + +```sh +fetch --grpc https://localhost:50051/package.Service/Method +``` + +### `--proto-file PATH` + +Compile `.proto` file(s) for JSON-to-protobuf conversion. Requires `protoc`. Can specify multiple comma-separated paths. + +```sh +fetch --grpc --proto-file service.proto -j '{"field": "value"}' localhost:50051/pkg.Svc/Method +``` + +### `--proto-desc PATH` + +Use pre-compiled descriptor set file instead of `--proto-file`. + +```sh +# Generate descriptor: +protoc --descriptor_set_out=service.pb --include_imports service.proto + +# Use descriptor: +fetch --grpc --proto-desc service.pb -j '{"field": "value"}' localhost:50051/pkg.Svc/Method +``` + +### `--proto-import PATH` + +Add import paths for proto compilation. Use with `--proto-file`. + +```sh +fetch --grpc --proto-file service.proto --proto-import ./proto localhost:50051/pkg.Svc/Method +``` + +## Configuration + +### `-c, --config PATH` + +Specify configuration file path. + +```sh +fetch --config ~/.config/fetch/custom.conf example.com +``` + +## Utility Options + +### `-h, --help` + +Print help information. + +### `-V, --version` + +Print version. + +### `--buildinfo` + +Print detailed build information. + +### `--update` + +Update fetch binary in place. + +### `--complete SHELL` + +Output shell completion scripts. Values: `fish`, `zsh`. + +```sh +fetch --complete zsh > ~/.zshrc.d/_fetch +fetch --complete fish > ~/.config/fish/completions/fetch.fish +``` + +### `--dry-run` + +Print request information without sending. + +```sh +fetch --dry-run -m POST -j '{"test": true}' example.com +``` + +## Environment Variables + +| Variable | Description | +| ----------------------- | -------------------------------- | +| `AWS_ACCESS_KEY_ID` | AWS access key for `--aws-sigv4` | +| `AWS_SECRET_ACCESS_KEY` | AWS secret key for `--aws-sigv4` | +| `VISUAL` / `EDITOR` | Editor for `--edit` option | +| `HTTPS_PROXY` | HTTPS proxy URL | +| `HTTP_PROXY` | HTTP proxy URL | +| `NO_PROXY` | Hosts to bypass proxy | + +## File References + +Many options support file references with the `@` prefix: + +- `@filename` - Read content from file +- `@-` - Read content from stdin +- `@~/path` - Home directory expansion + +```sh +fetch -j @data.json -m POST example.com +echo '{"test": true}' | fetch -j @- -m POST example.com +``` diff --git a/docs/CONFIG.md b/docs/configuration.md similarity index 97% rename from docs/CONFIG.md rename to docs/configuration.md index 790a690..6ff4228 100644 --- a/docs/CONFIG.md +++ b/docs/configuration.md @@ -349,6 +349,7 @@ ca-cert = /path/to/api-ca.crt ``` **Notes:** + - If `cert` is provided without `key`, the tool will attempt to read the private key from the certificate file (combined PEM format) - If the private key cannot be found, an error will be displayed - Encrypted private keys are not supported @@ -513,13 +514,13 @@ header = X-Company-ID: company-identifier ## Configuration File Validation -`fetch` validates configuration files when loading them and will report errors with specific line numbers: +`fetch` validates configuration files when loading them and will report errors: ``` config file '/home/user/.config/fetch/config': line 15: invalid option: 'invalid-option' ``` -Common validation errors include: +Validation errors may include: - Invalid option names - Invalid values for specific options (e.g., `color = invalid`) @@ -535,3 +536,9 @@ Common validation errors include: 5. **Test configurations** with dry-run mode: `fetch --dry-run example.com` 6. **Use comments** to document complex configurations 7. **Enable auto-update** for security and feature updates + +## See Also + +- [CLI Reference](cli-reference.md) - All command-line options +- [Authentication](authentication.md) - Detailed authentication setup +- [Advanced Features](advanced-features.md) - DNS, proxies, and TLS configuration diff --git a/docs/getting-started.md b/docs/getting-started.md new file mode 100644 index 0000000..33543f5 --- /dev/null +++ b/docs/getting-started.md @@ -0,0 +1,141 @@ +# Getting Started + +This guide will help you install `fetch` and make your first HTTP request. + +## Installation + +### Installation Script (Recommended) + +For macOS or Linux, use the installation script: + +```sh +curl -fsSL https://raw.githubusercontent.com/ryanfowler/fetch/main/install.sh | bash +``` + +### Building from Source + +If you have Go installed: + +```sh +go install github.com/ryanfowler/fetch@latest +``` + +### Pre-built Binaries + +Download binaries for your operating system from the [GitHub releases page](https://github.com/ryanfowler/fetch/releases). + +### Verify Installation + +```sh +fetch --version +``` + +## Making Your First Request + +Make a GET request by providing a URL: + +```sh +fetch example.com +``` + +When no scheme is provided, `fetch` defaults to HTTPS: + +```sh +fetch example.com # Uses https://example.com +fetch 192.168.1.1:8080 # Uses https://192.168.1.1:8080 +``` + +Loopback addresses default to HTTP for local development: + +```sh +fetch localhost:3000 # Uses http://localhost:3000 +fetch 127.0.0.1:8080 # Uses http://127.0.0.1:8080 +``` + +You can always specify the scheme explicitly: + +```sh +fetch http://example.com # Force HTTP +fetch https://localhost # Force HTTPS for localhost +``` + +## Understanding the Output + +When you make a request, `fetch` displays: + +1. **Status line** (to stderr) - Shows the HTTP version, status code, and status text +2. **Response body** (to stdout) - Automatically formatted based on content type + +``` +HTTP/1.1 200 OK + +{ + "name": "example", + "value": 42 +} +``` + +The response body is automatically formatted and syntax-highlighted for supported content types like JSON, XML, HTML, and more. + +## Common Options + +### Change HTTP Method + +```sh +fetch -m POST example.com +fetch -X DELETE example.com/resource/123 +``` + +### Add Headers + +```sh +fetch -H "Authorization: Bearer token" example.com +fetch -H "X-Custom: value" -H "Accept: application/json" example.com +``` + +### Send JSON Data + +```sh +fetch -j '{"name": "test"}' -m POST example.com/api +``` + +### View Request and Response Headers + +```sh +fetch -v example.com # Show response headers +fetch -vv example.com # Show request and response headers +fetch -vvv example.com # Show DNS and TLS details +``` + +## Updating + +Update `fetch` to the latest version: + +```sh +fetch --update +``` + +Or enable automatic updates in your [configuration file](configuration.md): + +```ini +auto-update = true +``` + +## Shell Completions + +Generate shell completion scripts: + +```sh +# Zsh +fetch --complete zsh > ~/.zshrc.d/fetch-completion.zsh + +# Fish +fetch --complete fish > ~/.config/fish/completions/fetch.fish +``` + +## Next Steps + +- **[CLI Reference](cli-reference.md)** - Complete list of all command-line options +- **[Configuration](configuration.md)** - Set up a configuration file for persistent settings +- **[Authentication](authentication.md)** - Learn about authentication options +- **[Request Bodies](request-bodies.md)** - Send JSON, XML, forms, and files diff --git a/docs/grpc.md b/docs/grpc.md new file mode 100644 index 0000000..9c8e5dc --- /dev/null +++ b/docs/grpc.md @@ -0,0 +1,366 @@ +# gRPC Support + +`fetch` supports making gRPC calls with automatic protocol handling, JSON-to-protobuf conversion, and formatted responses. + +## Overview + +gRPC (gRPC Remote Procedure Calls) is a high-performance RPC framework that uses Protocol Buffers for serialization and HTTP/2 for transport. + +## Basic gRPC Request + +### `--grpc` + +Enable gRPC mode. This flag: + +- Forces HTTP/2 protocol +- Sets method to POST +- Adds gRPC headers (`Content-Type: application/grpc+proto`, `TE: trailers`) +- Applies gRPC message framing +- Handles gRPC response framing + +```sh +fetch --grpc https://localhost:50051/package.Service/Method +``` + +### URL Format + +The service and method are specified in the URL path: + +``` +https://host:port/package.ServiceName/MethodName +``` + +Example: + +```sh +# Call Echo method on EchoService in the echo package +fetch --grpc https://localhost:50051/echo.EchoService/Echo +``` + +## Proto Schema Options + +To enable JSON-to-protobuf conversion and rich response formatting, provide a proto schema. + +### `--proto-file PATH` + +Compile `.proto` files using `protoc` (must be installed). Supports multiple comma-separated paths. + +```sh +fetch --grpc --proto-file service.proto \ + -j '{"message": "hello"}' \ + https://localhost:50051/echo.EchoService/Echo +``` + +Multiple files: + +```sh +fetch --grpc --proto-file common.proto,service.proto \ + -j '{"request": "data"}' \ + https://localhost:50051/pkg.Service/Method +``` + +### `--proto-desc PATH` + +Use a pre-compiled descriptor set file. Useful when: + +- `protoc` isn't available at runtime +- You want faster startup (no compilation) +- Building CI/CD pipelines + +Generate a descriptor set: + +```sh +protoc --descriptor_set_out=service.pb --include_imports service.proto +``` + +Use the descriptor: + +```sh +fetch --grpc --proto-desc service.pb \ + -j '{"message": "hello"}' \ + https://localhost:50051/echo.EchoService/Echo +``` + +### `--proto-import PATH` + +Add import paths for proto compilation. Use with `--proto-file` when your protos have imports. + +```sh +fetch --grpc \ + --proto-file service.proto \ + --proto-import ./proto \ + --proto-import /usr/local/include \ + -j '{"field": "value"}' \ + https://localhost:50051/pkg.Service/Method +``` + +## How It Works + +### With Proto Schema + +1. The service and method are extracted from the URL path +2. `fetch` looks up the method's input/output message types in the schema +3. JSON request bodies are converted to protobuf wire format +4. The request is framed with gRPC length-prefix +5. Response protobuf is formatted as JSON with field names from the schema + +### Without Proto Schema + +When no schema is provided: + +- Request bodies must be raw protobuf (not JSON) +- Responses are formatted using generic protobuf parsing +- Field numbers are shown instead of names + +## Request Bodies + +### JSON to Protobuf + +With a proto schema, send JSON that matches your message structure: + +```sh +fetch --grpc --proto-file user.proto \ + -j '{ + "name": "John Doe", + "age": 30, + "email": "john@example.com" + }' \ + https://localhost:50051/users.UserService/CreateUser +``` + +The JSON is automatically converted to protobuf wire format. + +### Nested Messages + +```sh +fetch --grpc --proto-file order.proto \ + -j '{ + "customer": { + "id": 123, + "name": "Jane" + }, + "items": [ + {"product_id": 1, "quantity": 2}, + {"product_id": 5, "quantity": 1} + ] + }' \ + https://localhost:50051/orders.OrderService/CreateOrder +``` + +### Empty Requests + +For methods that take empty messages: + +```sh +fetch --grpc --proto-file service.proto \ + https://localhost:50051/health.HealthService/Check +``` + +Or with an empty JSON object: + +```sh +fetch --grpc --proto-file service.proto \ + -j '{}' \ + https://localhost:50051/health.HealthService/Check +``` + +### File-based JSON + +```sh +fetch --grpc --proto-file service.proto \ + -j @request.json \ + https://localhost:50051/pkg.Service/Method +``` + +## Response Formatting + +### With Schema + +Responses are formatted as JSON with field names: + +```json +{ + "name": "John Doe", + "age": 30, + "email": "john@example.com", + "created_at": { + "seconds": 1704067200, + "nanos": 0 + } +} +``` + +### Without Schema + +Generic protobuf parsing shows field numbers: + +``` +1: "John Doe" +2: 30 +3: "john@example.com" +4 { + 1: 1704067200 + 2: 0 +} +``` + +## TLS and Security + +### Self-Signed Certificates + +For development servers with self-signed certificates: + +```sh +fetch --grpc --insecure \ + --proto-file service.proto \ + -j '{"request": "data"}' \ + https://localhost:50051/pkg.Service/Method +``` + +### Custom CA Certificate + +```sh +fetch --grpc \ + --ca-cert ca.crt \ + --proto-file service.proto \ + -j '{"request": "data"}' \ + https://server.example.com:50051/pkg.Service/Method +``` + +### mTLS + +```sh +fetch --grpc \ + --cert client.crt \ + --key client.key \ + --ca-cert ca.crt \ + --proto-file service.proto \ + -j '{"request": "data"}' \ + https://secure.example.com:50051/pkg.Service/Method +``` + +## Debugging + +### Verbose Output + +See request and response headers: + +```sh +fetch --grpc --proto-file service.proto \ + -j '{"field": "value"}' \ + -vv \ + https://localhost:50051/pkg.Service/Method +``` + +### Dry Run + +Inspect the request without sending: + +```sh +fetch --grpc --proto-file service.proto \ + -j '{"field": "value"}' \ + --dry-run \ + https://localhost:50051/pkg.Service/Method +``` + +### Edit Before Sending + +Modify the JSON request body in an editor: + +```sh +fetch --grpc --proto-file service.proto \ + -j '{"template": "value"}' \ + --edit \ + https://localhost:50051/pkg.Service/Method +``` + +## Examples + +### Health Check + +```sh +fetch --grpc https://localhost:50051/grpc.health.v1.Health/Check +``` + +### Create Resource + +```sh +fetch --grpc --proto-file api.proto \ + -j '{ + "resource": { + "name": "my-resource", + "type": "TYPE_A", + "config": {"key": "value"} + } + }' \ + https://api.example.com/resources.ResourceService/Create +``` + +### List with Pagination + +```sh +fetch --grpc --proto-file api.proto \ + -j '{"page_size": 10, "page_token": ""}' \ + https://api.example.com/users.UserService/ListUsers +``` + +### With Authentication + +```sh +fetch --grpc --proto-file api.proto \ + -H "Authorization: Bearer $TOKEN" \ + -j '{"id": "123"}' \ + https://api.example.com/users.UserService/GetUser +``` + +## Troubleshooting + +### "protoc not found" + +Install Protocol Buffers compiler: + +```sh +# macOS +brew install protobuf + +# Ubuntu/Debian +apt install protobuf-compiler + +# Or use --proto-desc with pre-compiled descriptors +``` + +### "method not found in schema" + +- Verify the URL path matches `package.Service/Method` exactly +- Check that your proto file defines the service and method +- Ensure all required imports are included via `--proto-import` + +### "failed to parse JSON" + +- Verify JSON syntax is correct +- Check field names match proto definitions (use snake_case) +- Ensure types match (strings quoted, numbers not quoted) + +### Connection Errors + +- gRPC requires HTTP/2 - ensure server supports it +- Check port number (gRPC typically uses different ports than REST) +- For TLS issues, try `--insecure` for testing + +### Response Parsing Errors + +- If response is empty, check gRPC status in headers (`-vv`) +- Verify proto schema matches server's actual message format +- Try without schema to see raw wire format + +## Limitations + +- **Unary calls only**: Streaming RPCs are not supported +- **Single message**: Cannot send multiple messages in one request +- **gRPC-Web**: Standard gRPC protocol only, not gRPC-Web + +## See Also + +- [CLI Reference](cli-reference.md) - All gRPC options +- [Authentication](authentication.md) - mTLS setup +- [Troubleshooting](troubleshooting.md) - Common issues diff --git a/docs/image-rendering.md b/docs/image-rendering.md new file mode 100644 index 0000000..ee64327 --- /dev/null +++ b/docs/image-rendering.md @@ -0,0 +1,206 @@ +# Image Rendering + +`fetch` can render images directly in your terminal when fetching image URLs. + +## Image Control + +### `--image OPTION` + +Control how images are rendered: + +| Value | Description | +| -------- | ---------------------------------------------------------- | +| `auto` | Try optimal protocol, fallback to external tools (default) | +| `native` | Use only built-in decoders | +| `off` | Disable image rendering | + +```sh +fetch --image native example.com/photo.jpg +fetch --image off example.com/image.png +``` + +## Supported Image Formats + +### Built-in Decoders (Native) + +These formats are decoded without external tools: + +- **JPEG** - `.jpg`, `.jpeg` +- **PNG** - `.png` +- **TIFF** - `.tiff`, `.tif` +- **WebP** - `.webp` + +### With External Adapters + +When `--image auto` (default), these additional formats are supported if you have the required tools installed: + +- **AVIF** - `.avif` +- **HEIF/HEIC** - `.heif`, `.heic` +- **GIF** - `.gif` (static frame) +- **BMP** - `.bmp` +- **And many more...** + +## Terminal Protocols + +`fetch` automatically detects your terminal and uses the best available image protocol. + +### Kitty Graphics Protocol + +**Supported terminals**: Kitty, Ghostty, Konsole + +The highest quality protocol with full color support and efficient transmission. + +### iTerm2 Inline Images + +**Supported terminals**: iTerm2, WezTerm, Hyper, Mintty + +Base64-encoded PNG images displayed inline. + +### Unicode Block Characters + +**Supported terminals**: All terminals + +Fallback rendering using Unicode block characters (▀▄█). Works everywhere but with reduced resolution. + +## Terminal Detection + +`fetch` detects your terminal emulator through environment variables: + +| Terminal | Detection Method | +| ---------------- | ---------------------------------------------- | +| Kitty | `KITTY_PID` or `TERM=xterm-kitty` | +| Ghostty | `GHOSTTY_BIN_DIR` or `TERM=xterm-ghostty` | +| iTerm2 | `ITERM_SESSION_ID` or `TERM_PROGRAM=iTerm.app` | +| WezTerm | `WEZTERM_EXECUTABLE` or `TERM_PROGRAM=WezTerm` | +| Konsole | `KONSOLE_VERSION` | +| Hyper | `TERM_PROGRAM=Hyper` | +| Mintty | `TERM_PROGRAM=mintty` | +| VS Code | `TERM_PROGRAM=vscode` or `VSCODE_INJECTION` | +| Windows Terminal | `WT_SESSION` | +| Apple Terminal | `TERM_PROGRAM=Apple_Terminal` | +| Alacritty | `TERM=alacritty` | +| Zellij | `ZELLIJ` | +| tmux | `TERM_PROGRAM=tmux` | + +## External Adapters + +When native decoders can't handle an image format, `fetch` tries external tools in this order: + +### 1. VIPS (`vips`) + +Fast image processing library. + +```sh +# Install on macOS +brew install vips + +# Install on Ubuntu/Debian +apt install libvips-tools +``` + +### 2. ImageMagick (`magick`) + +Comprehensive image manipulation tool. + +```sh +# Install on macOS +brew install imagemagick + +# Install on Ubuntu/Debian +apt install imagemagick +``` + +### 3. FFmpeg (`ffmpeg`) + +Multimedia framework with image support. + +```sh +# Install on macOS +brew install ffmpeg + +# Install on Ubuntu/Debian +apt install ffmpeg +``` + +## Configuration + +Set image rendering preferences in your [configuration file](configuration.md): + +```ini +# Disable image rendering +image = off + +# Use only native decoders +image = native + +# Auto-detect (default) +image = auto +``` + +## Image Sizing + +Images are automatically resized to fit within 80% of the terminal height while maintaining aspect ratio. This ensures images don't overwhelm the terminal display. + +## Examples + +### View an Image + +```sh +fetch example.com/photo.jpg +``` + +### Download Instead of Display + +```sh +fetch -o photo.jpg example.com/photo.jpg +``` + +### Force Native Decoding + +```sh +fetch --image native example.com/image.png +``` + +### Disable Image Rendering + +```sh +fetch --image off example.com/image.jpg +``` + +## Troubleshooting + +### Image Not Displaying + +1. **Check terminal support**: Not all terminals support inline images +2. **Verify format**: Use `--image native` to test if it's a format issue +3. **Install adapters**: Install VIPS, ImageMagick, or FFmpeg for more formats +4. **Check terminal size**: Very small terminals may not render properly + +### Poor Quality + +1. **Use a native protocol terminal**: Kitty, Ghostty, or iTerm2 provide best quality +2. **Check image dimensions**: Very large images are resized +3. **Block character fallback**: Quality is limited with Unicode blocks + +### Colors Look Wrong + +1. **Terminal color support**: Ensure your terminal supports 24-bit color +2. **tmux/screen**: May reduce color depth +3. **Try native decoding**: `--image native` + +### Image Dimensions Too Large + +`fetch` limits images to 8192x8192 pixels maximum to prevent memory issues. Larger images will fail to decode with a "dimensions are too large" error. + +## Limitations + +- **Animated GIFs**: Only the first frame is displayed +- **Maximum size**: 8192x8192 pixels +- **Memory limit**: Images are loaded into memory for processing +- **Terminal multiplexers**: tmux and screen may interfere with image protocols + +## See Also + +- [CLI Reference](cli-reference.md) - Image option details +- [Output Formatting](output-formatting.md) - Other content type handling +- [Configuration](configuration.md) - Default image settings diff --git a/docs/output-formatting.md b/docs/output-formatting.md new file mode 100644 index 0000000..94a7f59 --- /dev/null +++ b/docs/output-formatting.md @@ -0,0 +1,368 @@ +# Output Formatting + +`fetch` automatically formats and syntax-highlights response bodies based on content type. + +## Format Control + +### `--format OPTION` + +Control response body formatting: + +| Value | Description | +| ------ | ------------------------------------------ | +| `auto` | Format when stdout is a terminal (default) | +| `on` | Always format output | +| `off` | Never format output | + +```sh +fetch --format off example.com/api # Raw output +fetch --format on example.com/api # Force formatting +``` + +### `--color OPTION` + +Control syntax highlighting: + +| Value | Description | +| ------ | ----------------------------------------- | +| `auto` | Color when stdout is a terminal (default) | +| `on` | Always use colors | +| `off` | Never use colors | + +```sh +fetch --color off example.com/api # No colors +fetch --color on example.com/api | less -R # Colors piped to less +``` + +## Supported Content Types + +### JSON + +**Content-Types**: `application/json`, `*/*+json`, `*/*-json` + +Features: + +- Pretty-printing with proper indentation +- Syntax highlighting for keys, strings, numbers, booleans, null + +```sh +fetch example.com/api/users +``` + +Output: + +```json +{ + "id": 1, + "name": "John Doe", + "email": "john@example.com", + "active": true +} +``` + +### XML + +**Content-Types**: `application/xml`, `text/xml`, `*/*+xml` + +Features: + +- Proper indentation +- Color-coded elements, attributes, and content + +```sh +fetch example.com/api/data.xml +``` + +Output: + +```xml + + + + John Doe + john@example.com + + +``` + +### HTML + +**Content-Type**: `text/html` + +Features: + +- Proper indentation of nested elements +- Syntax highlighting +- Embedded CSS handling + +```sh +fetch example.com +``` + +### CSS + +**Content-Type**: `text/css` + +Features: + +- Selector highlighting +- Property and value coloring +- Proper indentation + +```sh +fetch example.com/styles.css +``` + +### CSV + +**Content-Types**: `text/csv`, `application/csv` + +Features: + +- Column alignment for readability +- Vertical "record view" for wide data that doesn't fit terminal width + +```sh +fetch example.com/data.csv +``` + +Standard output (fits terminal): + +``` +name email age +John Doe john@example.com 30 +Jane Smith jane@example.com 25 +``` + +Vertical mode (wide data): + +``` +--- Record 1 --- +name: John Doe +email: john@example.com +age: 30 + +--- Record 2 --- +name: Jane Smith +email: jane@example.com +age: 25 +``` + +### MessagePack + +**Content-Types**: `application/msgpack`, `application/x-msgpack`, `application/vnd.msgpack` + +Features: + +- Automatic conversion to JSON format +- Same formatting as JSON responses + +```sh +fetch example.com/api/data.msgpack +``` + +### Protocol Buffers + +**Content-Types**: `application/protobuf`, `application/x-protobuf`, `application/x-google-protobuf`, `application/vnd.google.protobuf`, `*/*+proto` + +Features: + +- Wire format parsing (without schema) +- Field number display +- With gRPC schema: field names and proper types + +Without schema (generic parsing): + +``` +1: "John Doe" +2: 30 +3: "john@example.com" +``` + +With schema (via `--proto-file` or `--proto-desc`): + +```json +{ + "name": "John Doe", + "age": 30, + "email": "john@example.com" +} +``` + +See [gRPC documentation](grpc.md) for schema-aware formatting. + +### Server-Sent Events (SSE) + +**Content-Type**: `text/event-stream` + +Features: + +- Streaming output as events arrive +- Event type and data parsing + +```sh +fetch example.com/events +``` + +Output: + +``` +event: message +data: {"user": "john", "text": "Hello!"} + +event: message +data: {"user": "jane", "text": "Hi there!"} +``` + +### NDJSON / JSON Lines + +**Content-Types**: `application/x-ndjson`, `application/ndjson`, `application/x-jsonl`, `application/jsonl`, `application/x-jsonlines` + +Features: + +- Streaming output line by line +- Each line formatted as JSON + +```sh +fetch example.com/stream.ndjson +``` + +Output: + +```json +{"id": 1, "event": "start"} +{"id": 2, "event": "data", "value": 42} +{"id": 3, "event": "end"} +``` + +### Images + +**Content-Type**: `image/*` + +Images are rendered directly in the terminal. See [Image Rendering](image-rendering.md) for details. + +## Output to File + +### `-o, --output PATH` + +Write response body to a file: + +```sh +fetch -o response.json example.com/api/data +``` + +Formatting is disabled when writing to a file. + +### `-o -` (Stdout) + +Force output to stdout, bypassing binary detection: + +```sh +fetch -o - example.com/file.bin > output.bin +``` + +### `-O, --remote-name` + +Save to current directory using filename from URL: + +```sh +fetch -O example.com/files/document.pdf +# Creates ./document.pdf +``` + +### `-J, --remote-header-name` + +Use filename from `Content-Disposition` header: + +```sh +fetch -O -J example.com/download +# Uses server-provided filename +``` + +### `--clobber` + +Overwrite existing files: + +```sh +fetch -o output.json --clobber example.com/data +``` + +## Pager + +When stdout is a terminal and output is large, `fetch` pipes output through `less` for easier navigation. + +### Disable Pager + +```sh +fetch --no-pager example.com/large-response +``` + +### Pager Environment + +The pager uses these flags: `less -FIRX` + +- `-F` - Quit if output fits on screen +- `-I` - Case-insensitive search +- `-R` - Handle ANSI colors +- `-X` - Don't clear screen on exit + +## Binary Detection + +When stdout is a terminal, `fetch` checks if the response appears to be binary data. If so, it displays a warning instead of corrupting your terminal: + +``` +warning: the response body appears to be binary +``` + +To force output: + +```sh +fetch -o - example.com/binary.dat > file.dat +``` + +## Configuration + +Set defaults in your [configuration file](configuration.md): + +```ini +# Always format output +format = on + +# Disable colors +color = off + +# Disable pager +no-pager = true +``` + +## Examples + +### Pipe to jq + +```sh +fetch --format off example.com/api | jq '.users[0]' +``` + +### Save Pretty JSON + +```sh +fetch example.com/api | tee response.json +``` + +### Force Colors in Pipe + +```sh +fetch --color on example.com/api | less -R +``` + +### Raw Binary Download + +```sh +fetch -o archive.zip example.com/download.zip +``` + +## See Also + +- [CLI Reference](cli-reference.md) - All formatting options +- [Image Rendering](image-rendering.md) - Terminal image display +- [Configuration](configuration.md) - Default settings diff --git a/docs/request-bodies.md b/docs/request-bodies.md new file mode 100644 index 0000000..ae0c653 --- /dev/null +++ b/docs/request-bodies.md @@ -0,0 +1,333 @@ +# Request Bodies + +`fetch` provides multiple options for sending request bodies with different content types. + +## Overview + +Request body options are **mutually exclusive** - you can only use one per request: + +| Option | Content-Type | Use Case | +| ----------------- | ----------------------------------- | ---------------------- | +| `-d, --data` | Auto-detected | Raw data, binary files | +| `-j, --json` | `application/json` | JSON APIs | +| `-x, --xml` | `application/xml` | XML/SOAP APIs | +| `-f, --form` | `application/x-www-form-urlencoded` | Simple forms | +| `-F, --multipart` | `multipart/form-data` | File uploads | + +## Raw Data + +The `-d` or `--data` flag sends raw request body data. Content-Type is auto-detected when reading from files. + +### Inline Data + +```sh +fetch -d 'Hello, world!' -m PUT example.com/resource +``` + +### From File + +```sh +fetch -d @data.txt -m PUT example.com/resource +fetch -d @payload.bin -m POST example.com/upload +``` + +### From Stdin + +```sh +echo 'Request body' | fetch -d @- -m PUT example.com/resource +cat data.json | fetch -d @- -m POST example.com/api +``` + +### Content-Type Detection + +When using `@filename`, the Content-Type is detected from the file extension. +Some examples are: + +| Extension | Content-Type | +| --------- | -------------------------- | +| `.json` | `application/json` | +| `.xml` | `application/xml` | +| `.html` | `text/html` | +| `.txt` | `text/plain` | +| `.csv` | `text/csv` | +| Unknown | `application/octet-stream` | + +Override with a header: + +```sh +fetch -d @data.bin -H "Content-Type: application/custom" -m POST example.com +``` + +## JSON Bodies + +The `-j` or `--json` flag sends JSON data and sets `Content-Type: application/json`. + +### Inline JSON + +```sh +fetch -j '{"name": "test", "value": 42}' -m POST example.com/api +``` + +### From File + +```sh +fetch -j @payload.json -m POST example.com/api +``` + +### From Stdin + +```sh +echo '{"key": "value"}' | fetch -j @- -m POST example.com/api + +# Build JSON dynamically +jq -n '{name: "test", time: now}' | fetch -j @- -m POST example.com/api +``` + +### Nested JSON + +```sh +fetch -j '{ + "user": { + "name": "John", + "email": "john@example.com" + }, + "settings": { + "theme": "dark", + "notifications": true + } +}' -m POST example.com/api/users +``` + +## XML Bodies + +The `-x` or `--xml` flag sends XML data and sets `Content-Type: application/xml`. + +### Inline XML + +```sh +fetch -x 'John' -m POST example.com/api +``` + +### From File + +```sh +fetch -x @request.xml -m POST example.com/soap/endpoint +``` + +### SOAP Example + +```sh +fetch -x ' + + + + 123 + + +' -m POST example.com/soap +``` + +## URL-Encoded Forms + +The `-f` or `--form` flag sends URL-encoded form data. Use multiple times for multiple fields. + +### Basic Form + +```sh +fetch -f username=john -f password=secret -m POST example.com/login +``` + +### With Special Characters + +Values are automatically URL-encoded: + +```sh +fetch -f "message=Hello World!" -f "email=user@example.com" -m POST example.com/contact +``` + +### Generated Content-Type + +``` +Content-Type: application/x-www-form-urlencoded +``` + +Request body: + +``` +username=john&password=secret +``` + +## Multipart Forms + +The `-F` or `--multipart` flag sends multipart form data, typically used for file uploads. + +### Text Fields + +```sh +fetch -F name=John -F email=john@example.com -m POST example.com/users +``` + +### File Uploads + +Use `@` prefix to upload files: + +```sh +fetch -F file=@document.pdf -m POST example.com/upload +fetch -F avatar=@photo.jpg -F name=John -m POST example.com/profile +``` + +### Multiple Files + +```sh +fetch -F "files=@doc1.pdf" -F "files=@doc2.pdf" -m POST example.com/upload +``` + +### Mixed Content + +```sh +fetch \ + -F "title=My Document" \ + -F "description=A sample upload" \ + -F "file=@document.pdf" \ + -F "thumbnail=@preview.png" \ + -m POST example.com/documents +``` + +### Home Directory Expansion + +The `~` is expanded to your home directory: + +```sh +fetch -F config=@~/config.json -m POST example.com/settings +``` + +## Editor Integration + +The `-e` or `--edit` flag opens an editor to compose or modify the request body before sending. + +### Basic Usage + +```sh +fetch --edit -m PUT example.com/resource +``` + +### With Initial Content + +Combine with other body options to edit before sending: + +```sh +fetch -j '{"name": "template"}' --edit -m POST example.com/api +``` + +### Editor Selection + +The editor is selected in this order: + +1. `VISUAL` environment variable +2. `EDITOR` environment variable +3. Well-known editors (`vim`, `nano`, etc.) + +```sh +EDITOR=code fetch --edit -m POST example.com/api +``` + +## File Reference Syntax + +Body options that accept `[@]VALUE` support these formats: + +| Format | Description | +| ----------- | ------------------------ | +| `value` | Use the literal value | +| `@filename` | Read content from file | +| `@-` | Read content from stdin | +| `@~/path` | Read from home directory | + +### Examples + +```sh +# Literal value +fetch -j '{"inline": true}' -m POST example.com + +# From file +fetch -j @data.json -m POST example.com + +# From stdin +cat data.json | fetch -j @- -m POST example.com + +# Home directory +fetch -d @~/Documents/data.txt -m PUT example.com +``` + +## Method Inference + +When a request body is provided without specifying a method, the default is still GET. Always specify the method explicitly for clarity: + +```sh +# Explicit POST +fetch -j '{"data": true}' -m POST example.com + +# Don't rely on implicit behavior +fetch -j '{"data": true}' example.com # Still uses GET! +``` + +## Large Files + +For large file uploads, consider: + +1. **Streaming**: `fetch` streams file content rather than loading it all into memory +2. **Timeout**: Set appropriate timeouts with `--timeout` +3. **Progress**: Use `-v` to see request/response headers + +```sh +fetch -F "large=@bigfile.zip" --timeout 300 -v -m POST example.com/upload +``` + +## Examples + +### REST API Create + +```sh +fetch -j '{ + "title": "New Post", + "content": "Hello, World!", + "published": true +}' -m POST example.com/api/posts +``` + +### Form Login + +```sh +fetch -f username=admin -f password=secret -m POST example.com/login +``` + +### File Upload with Metadata + +```sh +fetch \ + -F "file=@report.pdf" \ + -F "title=Monthly Report" \ + -F "tags=finance,monthly" \ + -m POST example.com/documents +``` + +### GraphQL Query + +```sh +fetch -j '{ + "query": "{ user(id: 1) { name email } }" +}' -m POST example.com/graphql +``` + +### Webhook Payload + +```sh +fetch -j @webhook-payload.json \ + -H "X-Webhook-Secret: $WEBHOOK_SECRET" \ + -m POST example.com/webhooks/receive +``` + +## See Also + +- [CLI Reference](cli-reference.md) - All request body options +- [gRPC](grpc.md) - Sending Protocol Buffer messages +- [Output Formatting](output-formatting.md) - Response formatting diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md new file mode 100644 index 0000000..ace996b --- /dev/null +++ b/docs/troubleshooting.md @@ -0,0 +1,384 @@ +# Troubleshooting + +This guide helps diagnose and fix common issues with `fetch`. + +## Exit Codes + +`fetch` uses exit codes to indicate the result of a request: + +| Exit Code | Meaning | +| --------- | ---------------------------------- | +| 0 | Success (HTTP 2xx-3xx) | +| 4 | Client error (HTTP 4xx) | +| 5 | Server error (HTTP 5xx) | +| 6 | Other HTTP status or request error | + +### Ignore HTTP Status + +To always exit 0 regardless of HTTP status: + +```sh +fetch --ignore-status example.com/not-found +echo $? # Always 0 if request completed +``` + +## Debugging with Verbosity + +### `-v` - Response Headers + +```sh +fetch -v example.com +``` + +Shows: + +- HTTP version and status +- Response headers + +### `-vv` - Request and Response Headers + +```sh +fetch -vv example.com +``` + +Shows: + +- Request method and URL +- Request headers +- Response headers + +### `-vvv` - Full Debug Output + +```sh +fetch -vvv example.com +``` + +Shows: + +- DNS resolution details +- TLS handshake information +- All headers + +## Dry Run Mode + +Preview the request without sending: + +```sh +fetch --dry-run -m POST -j '{"test": true}' example.com +``` + +Shows: + +- Complete request that would be sent +- Headers and body +- Useful for debugging authentication and body formatting + +## Common Issues + +### TLS Certificate Errors + +**Symptom**: "certificate signed by unknown authority" or similar + +**Causes**: + +- Self-signed certificate +- Expired certificate +- Wrong hostname +- Missing intermediate certificates +- Corporate SSL inspection + +**Solutions**: + +1. **For development/testing only**: + + ```sh + fetch --insecure https://self-signed.example.com + ``` + +2. **Add custom CA certificate**: + + ```sh + fetch --ca-cert /path/to/ca.crt https://internal.example.com + ``` + +3. **Check certificate details**: + + ```sh + fetch -vvv https://example.com 2>&1 | grep -i tls + ``` + +4. **Verify certificate externally**: + ```sh + openssl s_client -connect example.com:443 -showcerts + ``` + +### Connection Timeouts + +**Symptom**: Request hangs or "context deadline exceeded" + +**Causes**: + +- Server unreachable +- Firewall blocking connection +- DNS resolution failure +- Slow server response + +**Solutions**: + +1. **Set explicit timeout**: + + ```sh + fetch --timeout 10 example.com + ``` + +2. **Test DNS resolution**: + + ```sh + fetch --dns-server 8.8.8.8 -vvv example.com + ``` + +3. **Test network connectivity**: + ```sh + ping example.com + curl -v example.com + ``` + +### Connection Refused + +**Symptom**: "connection refused" + +**Causes**: + +- Server not running +- Wrong port +- Firewall blocking + +**Solutions**: + +1. **Verify the URL and port** +2. **Check if service is running**: + ```sh + nc -zv example.com 443 + ``` + +### DNS Resolution Failures + +**Symptom**: "no such host" or DNS errors + +**Solutions**: + +1. **Try alternative DNS**: + + ```sh + fetch --dns-server 8.8.8.8 example.com + fetch --dns-server https://1.1.1.1/dns-query example.com + ``` + +2. **Verify hostname**: + ```sh + nslookup example.com + dig example.com + ``` + +### Proxy Issues + +**Symptom**: Connection errors when using proxy + +**Solutions**: + +1. **Verify proxy URL format**: + + ```sh + fetch --proxy http://proxy:8080 example.com + fetch --proxy socks5://proxy:1080 example.com + ``` + +2. **Test without proxy**: + + ```sh + unset HTTP_PROXY HTTPS_PROXY + fetch example.com + ``` + +3. **Check proxy authentication**: + ```sh + fetch --proxy http://user:pass@proxy:8080 example.com + ``` + +### HTTP/2 Issues + +**Symptom**: "http2: server sent GOAWAY" or protocol errors + +**Solutions**: + +1. **Force HTTP/1.1**: + + ```sh + fetch --http 1 example.com + ``` + +2. **Check server HTTP/2 support**: + ```sh + curl --http2 -v example.com 2>&1 | grep -i alpn + ``` + +### gRPC Errors + +**Symptom**: gRPC calls failing + +**Common issues**: + +1. **Method not found**: + - Check URL path format: `/package.Service/Method` + - Verify proto schema includes the method + +2. **Proto compilation fails**: + - Install `protoc`: `brew install protobuf` + - Check import paths: `--proto-import` + +3. **JSON conversion errors**: + - Verify JSON field names match proto (use snake_case) + - Check types: strings must be quoted + +```sh +# Debug gRPC request +fetch --grpc --proto-file service.proto \ + -j '{"field": "value"}' \ + --dry-run \ + https://localhost:50051/pkg.Service/Method +``` + +### Image Rendering Not Working + +**Symptom**: Images show as raw bytes or don't display + +**Solutions**: + +1. **Check terminal support**: + - Use Kitty, iTerm2, WezTerm, or Ghostty for best results + +2. **Force native decoding**: + + ```sh + fetch --image native example.com/image.png + ``` + +3. **Install image adapters**: + + ```sh + brew install vips imagemagick ffmpeg + ``` + +4. **Disable image rendering**: + ```sh + fetch --image off example.com/image.jpg + ``` + +### Binary Data Warning + +**Symptom**: "the response body appears to be binary" + +**Solutions**: + +1. **Save to file**: + + ```sh + fetch -o output.bin example.com/file.bin + ``` + +2. **Force output to stdout**: + ```sh + fetch -o - example.com/file.bin > output.bin + ``` + +### Large Response Truncated + +**Symptom**: Response appears cut off + +**Info**: Formatting is limited to 16MB of data + +**Solutions**: + +1. **Save to file** (no size limit): + + ```sh + fetch -o large-response.json example.com/large + ``` + +2. **Disable formatting**: + ```sh + fetch --format off example.com/large + ``` + +## Configuration Issues + +### Config File Not Loading + +1. **Check location**: + - Windows: `%AppData%\fetch\config` + - macOS/Linux: `~/.config/fetch/config` + +2. **Specify explicitly**: + + ```sh + fetch --config /path/to/config example.com + ``` + +3. **Validate syntax**: + - Check for error messages on stderr + - Verify INI format + +### Settings Not Applied + +**Precedence** (highest to lowest): + +1. Command-line flags +2. Host-specific config +3. Global config +4. Defaults + +Use dry-run to verify: + +```sh +fetch --dry-run example.com +``` + +## Getting More Help + +### Version and Build Info + +```sh +fetch --version +fetch --buildinfo +``` + +### Help Text + +```sh +fetch --help +``` + +### Report Issues + +If you encounter a bug: + +1. Gather debug output: + + ```sh + fetch -vvv example.com 2>&1 + ``` + +2. Note your environment: + + ```sh + fetch --buildinfo + uname -a + ``` + +3. Report at: https://github.com/ryanfowler/fetch/issues + +## See Also + +- [CLI Reference](cli-reference.md) - All options and flags +- [Configuration](configuration.md) - Config file format +- [Advanced Features](advanced-features.md) - Network options