Tools

In BrainSoup, a Tool is a set of functions that an agent can utilize to perform specific tasks or operations.

Tools can be enabled or disabled for each agent individually, or at the chat room level. This allows for fine-grained control over the capabilities of the agents.

These tools can be simple, such as a calculator for performing mathematical calculations, or more complex like HTTP tools for making network requests. But tools are not limited to what comes pre-packaged with the software.

Users have the ability to add new tools, extending the capabilities of their agents and enhancing their interaction with external environments. This could involve anything from running scripts on the local host, interacting with databases, managing files, to even integrating with other APIs or software.

There are 3 types of tools in BrainSoup: Built-in Tools, OpenAI Plugins and Custom Tools.

Built-in Tools

In the current version of BrainSoup, the following built-in tools are available:

  • FileSystemRead: Read files and directories in the sandboxed file system of the chat room.
  • FileSystemWrite: Write, modify or delete files and directories in the sandboxed file system of the chat room.
  • FileSystemExecute: Run a program in the sandboxed file system of the chat room and return its output.
  • FileArchiver: Archive or extract files in various formats.
  • Clock: This tool provides the agent with the current date, time, and timezone information.
  • Calculator: Evaluate mathematical expressions. Supports basic arithmetic operations, as well as more advanced functions like trigonometry, logarithms, and exponentials, etc. Can evaluate expressions in batch, allowing agents to perform multiple calculations in a single request, which is useful when dealing with tabular data.
  • DateTimeCalculator: Do date, calendar and time zone calculations.
  • GeoCalculator: Convert between addresses and coordinates and compute distances.
  • WebBrowser: Provides the functionality of a web browser. It allows agents to use a search engine, browse the web and retrieve information from it.
  • Http: Very versatile tool that enables agents to make HTTP requests. This could be used for interacting with web services, APIs, or any server that communicates over HTTP.
  • Python: This tool allows agents to execute Python scripts. Python is a versatile programming language that can be used for a wide range of tasks.
  • PowerShell: Execute PowerShell commands and get the results. PowerShell is a powerful scripting language that can interact with the Windows operating system and other applications.
  • CMD: Execute CMD commands and get the results. It provides access to the command prompt on Windows systems.
  • GuiAutomator: Interact with the Windows desktop. It can simulate mouse clicks, keyboard input, and other GUI events.
  • CsvReader: Read CSV files with SQL queries. It provides a powerful way to filter, sort, and aggregate data from CSV files.
  • XlsxReader: Read Excel files with SQL queries. It provides a powerful way to filter, sort, and aggregate data from Excel files.
  • Diagrams: Generates diagrams with Mermaid-js. It supports various diagram types like flowcharts, sequence diagrams, and UML diagrams.
  • TextToImageDallE2 & TextToImageDallE3: These tools allow agents to generate images from text using the OpenAI DALL-E 2 and DALL-E 3 models. The DALL-E 3 model provides better quality images but is slower and more expensive than the DALL-E 2 model.
  • ImageToText: Extract very detailled text descriptions from images using the GPT-4o model.
  • SpeechToText: Transcribe speech to text using the OpenAI Whistler model.
  • Camera: Capture images from a camera and extract information.
  • Screenshot: Capture screenshots and extract information.
  • ImageEditor: Edit and convert images and edit EXIF data.
  • AudioVideoEditor: Edit and convert audio and video files.
  • YouTube: Watch YouTube videos and extract information.
  • Discord: Send messages to Discord servers.
  • Slack: Send messages to Slack channels.
  • PushBullet: Send push notifications and SMS via PushBullet.
  • GitHub: Interact with GitHub repositories.

OpenAI Plugins

OpenAI Plugins are plugins created for the ChatGPT service from OpenAI. They serve as an equivalent to BrainSoup's tools. BrainSoup is designed to be compatible with these plugins, which means it can utilize a vast array of plugins already written for ChatGPT.

These plugins are registered as tools by placing their JSON manifest, or the URL to it, in the Tools/OpenAi folder. The name of the .json or .url file determines the name of the tool in BrainSoup. Once registered, OpenAI plugins will be immediately recognized by BrainSoup, and will be available for activation at the agent or chat room level, just like any other tool.

Custom Tools

In BrainSoup, Custom Tools are user-created tools that run commands on the local machine. These are defined by a file named manifest.json located in a subfolder of Tools/Custom . Once a custom tool is added, it will be immediately recognized by BrainSoup, and will be available for activation at the agent or chat room level.

Structure of a Custom Tool

The manifest.json file describes the tool and its functions. The first level of its structure is thus an array of functions, defined as follows:

  • Function properties:
    • name: The name of the function.
    • description: A description of the function.
    • parameters (optional): An array of parameters that the function accepts.
    • command: The command to be executed.
    • timeout (optional): The maximum time in seconds that the command is allowed to run before it is terminated. If not specified, the default timeout is 10 seconds.
    • workingDirectory (optional): The directory in which the command should be executed. If not specified, the command will be executed in the tool directory.
  • Parameter properties:
    • name: The name of the parameter.
    • description: A description of the parameter.
    • isRequired: Whether the parameter is required or optional.
    • defaultValue (optional): The default value of the parameter, if it is optional.

Passing Parameters to a Custom Tool

Parameters can be used in the command by enclosing them in double curly braces. For example, if a parameter named destination is defined, it can be used in the command as {{destination}} . The command will be executed with the parameters replaced by their values. Parameters are also injected into the environment variables of the created process. For example, if a parameter named FileContent is defined, it will be available in the ARGS_FILECONTENT environment variable.

Example of a Custom Tool

Here's an example of what a manifest.json might look like for a simple custom tool that sends a ping request:

{
  "functions": [
    {
      "name": "Send",
      "description": "Send ICMP ECHO_REQUEST datagram to host",
      "parameters": [
        {
          "name": "destination",
          "description": "IP address or host name",
          "isRequired": true
        }
      ],
      "command": "ping {{destination}}",
      "timeout": 25
    }
  ]
}

In this example, the tool has one function named Send , which sends an ICMP ECHO_REQUEST (ping) to a specified destination. The destination is provided as a parameter, which is required for the function to work.

This tool would use whatever command line interpreter is available on your system (such as cmd or bash) to execute the ping command with the provided destination.

Tips:

  • Name and description are crucial for the tool to be easily identifiable and understandable by the agents. They should be clear, concise and in English.
  • If your script requires configuration, a good practice is to place it in a separate config file (with the appropriate extension) next to the script.
  • If your command needs to be run in a directory other than the tool directory, you can set workingDirectory .
  • You can set defaultValue for a parameter if it is not required.
  • Arguments are also injected into the environment variables of the created process (useful for multi-line arguments). Example: FileContent argument will be available in ARGS_FILECONTENT environment variable.