ModelFusion

The TypeScript library for building AI applications.

Machine LearningBot
vercel/modelfusionmodelfusion.dev

README

ModelFusion


The TypeScript library for building AI applications.


Introduction


ModelFusion is an abstraction layer for integrating AI models into JavaScript and TypeScript applications, unifying the API for common operations such as text streaming, object generation, and tool usage. It provides features to support production environments, including observability hooks, logging, and automatic retries. You can use ModelFusion to build AI applications, chatbots, and agents.

- Vendor-neutral: ModelFusion is a non-commercial open source project that is community-driven. You can use it with any supported provider.
- Multi-modal: ModelFusion supports a wide range of models including text generation, image generation, vision, text-to-speech, speech-to-text, and embedding models.
- Type inference and validation: ModelFusion infers TypeScript types wherever possible and validates model responses.
- Observability and logging: ModelFusion provides an observer framework and logging support.
- Resilience and robustness: ModelFusion ensures seamless operation through automatic retries, throttling, and error handling mechanisms.
- Built for production: ModelFusion is fully tree-shakeable, can be used in serverless environments, and only uses a minimal set of dependencies.

Quick Install


  1. ```sh
  2. npm install modelfusion
  3. ```

Or use a starter template:

- ModelFusion terminal app starter
- Next.js, Vercel AI SDK, Llama.cpp & ModelFusion starter
- Next.js, Vercel AI SDK, Ollama & ModelFusion starter

Usage Examples


[!TIP]

The basic examples are a great way to get started and to explore in parallel with the documentation. You can find them in the examples/basic folder.


You can provide API keys for the different integrations using environment variables (e.g.,OPENAI_API_KEY) or pass them into the model constructors as options.

Generate Text


Generate text using a language model and a prompt. You can stream the text if it is supported by the model. You can use images for multi-modal prompting if the model supports it (e.g. with llama.cpp).
You can use prompt styles to use text, instruction, or chat prompts.

generateText


  1. ```ts
  2. import { generateText, openai } from "modelfusion";

  4. const text = await generateText({
  5.   model: openai.CompletionTextGenerator({ model: "gpt-3.5-turbo-instruct" }),
  6.   prompt: "Write a short story about a robot learning to love:\n\n",
  7. });
  8. ```

streamText


  1. ```ts
  2. import { streamText, openai } from "modelfusion";

  4. const textStream = await streamText({
  5.   model: openai.CompletionTextGenerator({ model: "gpt-3.5-turbo-instruct" }),
  6.   prompt: "Write a short story about a robot learning to love:\n\n",
  7. });

  9. for await (const textPart of textStream) {
  10.   process.stdout.write(textPart);
  11. }
  12. ```

streamText with multi-modal prompt


Multi-modal vision models such as GPT 4 Vision can process images as part of the prompt.

  1. ```ts
  2. import { streamText, openai } from "modelfusion";
  3. import { readFileSync } from "fs";

  5. const image = readFileSync("./image.png");

  7. const textStream = await streamText({
  8.   model: openai
  9.     .ChatTextGenerator({ model: "gpt-4-vision-preview" })
  10.     .withInstructionPrompt(),

  12.   prompt: {
  13.     instruction: [
  14.       { type: "text", text: "Describe the image in detail." },
  15.       { type: "image", image, mimeType: "image/png" },
  16.     ],
  17.   },
  18. });

  20. for await (const textPart of textStream) {
  21.   process.stdout.write(textPart);
  22. }
  23. ```

Generate Object

Generate typed objects using a language model and a schema.