Skip to main content

BedrockChat

Amazon Bedrock is a fully managed service that makes Foundation Models (FMs) from leading AI startups and Amazon available via an API. You can choose from a wide range of FMs to find the model that is best suited for your use case.

Setup​

You'll need to install the @langchain/community package:

tip

See this section for general instructions on installing integration packages.

npm install @langchain/community

Then, you'll need to install a few official AWS packages as peer dependencies:

npm install @aws-crypto/sha256-js @aws-sdk/credential-provider-node @smithy/protocol-http @smithy/signature-v4 @smithy/eventstream-codec @smithy/util-utf8 @aws-sdk/types

You can also use BedrockChat in web environments such as Edge functions or Cloudflare Workers by omitting the @aws-sdk/credential-provider-node dependency and using the web entrypoint:

npm install @aws-crypto/sha256-js @smithy/protocol-http @smithy/signature-v4 @smithy/eventstream-codec @smithy/util-utf8 @aws-sdk/types

Usage​

Currently, only Anthropic, Cohere, and Mistral models are supported with the chat model integration. For foundation models from AI21 or Amazon, see the text generation Bedrock variant.

import { BedrockChat } from "@langchain/community/chat_models/bedrock";
// Or, from web environments:
// import { BedrockChat } from "@langchain/community/chat_models/bedrock/web";
import { HumanMessage } from "@langchain/core/messages";

// If no credentials are provided, the default credentials from
// @aws-sdk/credential-provider-node will be used.

// modelKwargs are additional parameters passed to the model when it
// is invoked.
const model = new BedrockChat({
  model: "anthropic.claude-3-sonnet-20240229-v1:0",
  region: "us-east-1",
  // endpointUrl: "custom.amazonaws.com",
  // credentials: {
  //   accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID!,
  //   secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY!,
  // },
  // modelKwargs: {
  //   anthropic_version: "bedrock-2023-05-31",
  // },
});

// Other model names include:
// "mistral.mistral-7b-instruct-v0:2"
// "mistral.mixtral-8x7b-instruct-v0:1"
//
// For a full list, see the Bedrock page in AWS.

const res = await model.invoke([
  new HumanMessage({ content: "Tell me a joke" }),
]);
console.log(res);

/*
  AIMessage {
    content: "Here's a silly joke for you:\n" +
      '\n' +
      "Why can't a bicycle stand up by itself?\n" +
      "Because it's two-tired!",
    name: undefined,
    additional_kwargs: { id: 'msg_01NYN7Rf39k4cgurqpZWYyDh' }
  }
*/

const stream = await model.stream([
  new HumanMessage({ content: "Tell me a joke" }),
]);

for await (const chunk of stream) {
  console.log(chunk.content);
}

/*
  Here
  's
  a
  silly
  joke
  for
  you
  :


  Why
  can
  't
  a
  bicycle
  stand
  up
  by
  itself
  ?

  Because
  it
  's
  two
  -
  tired
  !
*/

API Reference:

Multimodal inputs​

tip

Multimodal inputs are currently only supported by Anthropic Claude-3 models.

Anthropic Claude-3 models hosted on Bedrock have multimodal capabilities and can reason about images. Here's an example:

import * as fs from "node:fs/promises";

import { BedrockChat } from "@langchain/community/chat_models/bedrock";
// Or, from web environments:
// import { BedrockChat } from "@langchain/community/chat_models/bedrock/web";
import { HumanMessage } from "@langchain/core/messages";

// If no credentials are provided, the default credentials from
// @aws-sdk/credential-provider-node will be used.

// modelKwargs are additional parameters passed to the model when it
// is invoked.
const model = new BedrockChat({
  model: "anthropic.claude-3-sonnet-20240229-v1:0",
  region: "us-east-1",
  // endpointUrl: "custom.amazonaws.com",
  // credentials: {
  //   accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID!,
  //   secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY!,
  // },
  // modelKwargs: {
  //   anthropic_version: "bedrock-2023-05-31",
  // },
});

const imageData = await fs.readFile("./hotdog.jpg");

const res = await model.invoke([
  new HumanMessage({
    content: [
      {
        type: "text",
        text: "What's in this image?",
      },
      {
        type: "image_url",
        image_url: {
          url: `data:image/jpeg;base64,${imageData.toString("base64")}`,
        },
      },
    ],
  }),
]);
console.log(res);

/*
  AIMessage {
    content: 'The image shows a hot dog or frankfurter. It has a reddish-pink sausage filling encased in a light brown bread-like bun. The hot dog bun is split open, revealing the sausage inside. This classic fast food item is a popular snack or meal, often served at events like baseball games or cookouts. The hot dog appears to be against a plain white background, allowing the details and textures of the food item to be clearly visible.',
    name: undefined,
    additional_kwargs: { id: 'msg_01XrLPL9vCb82U3Wrrpza18p' }
  }
*/

API Reference:

Help us out by providing feedback on this documentation page: