Embedding Foundry

[anagri]

Is it possible to embed the foundry local binary, along with its dependencies, in an app and run it via sub-process? will be a good use case if we can embed the foundry local binary for a genai app, instead of depending on having it installed separately.

i can see the msix installer when unzipped contains all the files required. will having this be sufficient? or requires some extra setup. ollama works great with a standalone binary and dependencies in such use cases, looking for similar setup for foundry local.

kindly let me know.

[samuel100]

@anagri Thanks for raising an issue! Our plan is to make this part of the initialization process in the SDK. For example, in Python (JavaScript and C# also available):

from foundry_local import FoundryLocalManager

alias = "phi-3.5-mini"
manager = FoundryLocalManager(alias)

We'll include in the bootstrap Foundry Local binary acquisition on the end user's machine. Currently, as you point out we assume the binary is on the user's device and then go through a process of starting the process and model acquisition. The rationale for not including this at public preview launch is we need to put the Foundry Local binary on a diet before enabling it - reducing it from ~1GB on x64 to a few MB.

[anagri]

looking forward to a standalone binary, or standalone folder to deploy local foundry. as might have use cases when you are packaging it as part of edge app, or embedded in a docker image.

[HaoHoo]

The rationale for not including this at public preview launch is we need to put the Foundry Local binary on a diet before enabling it - reducing it from ~1GB on x64 to a few MB.

Hope is enhanced for continued transfer on a weak internet link across a firewall or VPN.