@inferencesh/sdk

Official JavaScript/TypeScript SDK for inference.sh — Run AI models with a simple API.

Installation

npm install @inferencesh/sdk
# or
yarn add @inferencesh/sdk
# or
pnpm add @inferencesh/sdk

Getting an API Key

Get your API key from the inference.sh dashboard.

Quick Start

import { inference } from '@inferencesh/sdk';

const client = inference({ apiKey: 'your-api-key' });

// Run a task and wait for the result
const result = await client.run({
  app: 'your-app',
  input: {
    prompt: 'Hello, world!'
  }
});

console.log(result.output);

Usage

Basic Usage

import { inference } from '@inferencesh/sdk';

const client = inference({ apiKey: 'your-api-key' });

// Wait for result (default behavior)
const result = await client.run({
  app: 'my-app',
  input: { prompt: 'Generate something amazing' }
});

console.log('Output:', result.output);

With Setup Parameters

Setup parameters configure the app instance (e.g., model selection). Workers with matching setup are "warm" and skip setup:

const result = await client.run({
  app: 'my-app',
  setup: { model: 'schnell' },  // Setup parameters
  input: { prompt: 'hello' }
});

Fire and Forget

// Get task info immediately without waiting
const task = await client.run(
  { app: 'my-app', input: { prompt: 'hello' } },
  { wait: false }
);

console.log('Task ID:', task.id);
console.log('Status:', task.status);

Real-time Status Updates

const result = await client.run(
  { app: 'my-app', input: { prompt: 'hello' } },
  {
    onUpdate: (update) => {
      console.log('Status:', update.status);
      console.log('Progress:', update.logs);
    }
  }
);

Batch Processing

async function processImages(images: string[]) {
  const results = [];
  
  for (const image of images) {
    const result = await client.run({
      app: 'image-processor',
      input: { image }
    }, {
      onUpdate: (update) => console.log(`Processing: ${update.status}`)
    });
    
    results.push(result);
  }
  
  return results;
}

File Upload

// Upload from base64
const file = await client.uploadFile('data:image/png;base64,...', {
  filename: 'image.png',
  contentType: 'image/png'
});

// Use the uploaded file in a task
const result = await client.run({
  app: 'image-app',
  input: { image: file.uri }
});

Cancel a Task

const task = await client.run(
  { app: 'long-running-app', input: {} },
  { wait: false }
);

// Cancel if needed
await client.cancel(task.id);

Agent Chat

Chat with AI agents using client.agent().

Using a Template Agent

Use an existing agent from your workspace by its namespace/name@shortid:

import { inference } from '@inferencesh/sdk';

const client = inference({ apiKey: 'your-api-key' });

// Create agent from template
const agent = client.agent('my-org/assistant@abc123');

// Send a message with streaming
await agent.sendMessage('Hello!', {
  onMessage: (msg) => {
    if (msg.content) {
      for (const c of msg.content) {
        if (c.type === 'text' && c.text) {
          process.stdout.write(c.text);
        }
      }
    }
  }
});

// Clean up
agent.disconnect();

Creating an Ad-Hoc Agent

Create agents on-the-fly without saving to your workspace:

import { inference, tool, string } from '@inferencesh/sdk';

const client = inference({ apiKey: 'your-api-key' });

// Create ad-hoc agent
const agent = client.agent({
  coreApp: 'infsh/claude-sonnet-4@abc123',  // LLM to use
  systemPrompt: 'You are a helpful assistant.',
  tools: [
    tool('get_weather')
      .description('Get current weather')
      .params({ city: string('City name') })
      .handler(async (args) => {
        // Your tool logic here
        return JSON.stringify({ temp: 72, conditions: 'sunny' });
      })
      .build()
  ]
});

await agent.sendMessage('What is the weather in Paris?', {
  onMessage: (msg) => console.log(msg),
  onToolCall: async (call) => {
    // Tool handlers are auto-executed if defined
  }
});

Agent Methods

Method	Description
sendMessage(text, options?)	Send a message to the agent
getChat(chatId?)	Get chat history
stopChat(chatId?)	Stop current generation
submitToolResult(toolId, resultOrAction)	Submit result for a client tool (string or {action, form_data})
streamMessages(chatId?, options?)	Stream message updates
streamChat(chatId?, options?)	Stream chat updates
disconnect()	Clean up streams
reset()	Start a new conversation

API Reference

inference(config)

Creates a new inference client.

Parameter	Type	Required	Description
config.apiKey	string	Yes	Your inference.sh API key
config.baseUrl	string	No	Custom API URL (default: https://api.inference.sh)

client.run(params, options?)

Runs a task on inference.sh.

Parameters:

Parameter	Type	Required	Description
params.app	string	Yes	App identifier (e.g., 'username/app-name')
params.input	object	Yes	Input parameters for the app
params.setup	object	No	Setup parameters (affects worker warmth/scheduling)
params.infra	string	No	Infrastructure: 'cloud' or 'private'
params.variant	string	No	App variant to use

Options:

Option	Type	Default	Description
wait	boolean	true	Wait for task completion
onUpdate	function	-	Callback for status updates
autoReconnect	boolean	true	Auto-reconnect on connection loss
maxReconnects	number	5	Max reconnection attempts
reconnectDelayMs	number	1000	Delay between reconnects (ms)

client.cancel(taskId)

Cancels a running task.

client.uploadFile(data, options?)

Uploads a file to inference.sh.

Parameters:

Parameter	Type	Description
data	string | Blob	Base64 string, data URI, or Blob
options.filename	string	Filename
options.contentType	string	MIME type
options.public	boolean	Make file publicly accessible

Task Status Constants

import {
  TaskStatusQueued,
  TaskStatusRunning,
  TaskStatusCompleted,
  TaskStatusFailed,
  TaskStatusCancelled
} from '@inferencesh/sdk';

if (task.status === TaskStatusCompleted) {
  console.log('Done!');
}

TypeScript Support

This SDK is written in TypeScript and includes full type definitions. All types are exported:

import type { Task, ApiTaskRequest, RunOptions } from '@inferencesh/sdk';

Requirements

• Node.js 18.0.0 or higher
• Modern browsers with fetch support

License

MIT © inference.sh