This guide explains how to build and run flow-graphs within the HyperFlow AI development environment.
Flow-graphs are visual representations of AI applications or workflows, constructed by connecting nodes that perform specific tasks. See the chapter introducing flow-graphs for more background information.
The Flow-Graph Editor is the primary workspace for building and refining flow-graphs. It is located within the main HyperFlow Integrated Development Environment (IDE). Access it by selecting "Editor" in the main UI Mode panel.
Existing or installed flow-graphs are displayed in the "Flow-Graphs" tab within the directory pane on the left. Selecting a flow-graph from this list loads it into the editor.
New flow-graphs can be created in several ways:
Each method will open a dialog where the flow-graph can be named, described, tagged for organization within the directory, and set to public or private. These details can be edited later by selecting "Edit Details" from the "Flow-Graphs" drop-down menu.
Flow-graphs are built by connecting nodes with process flow lines and data flow lines. Nodes are selected from the "Node" tab in the directory pane, where they are organized into functional groups. The current HyperFlow release offers around 40 node types, each designed for a specific task related to AI application or workflow development.
Nodes are added to the flow-graph by dragging them from the node tab onto the flow-graph surface. For example, to create a simple chatbot, one might start with a User input node, which accepts an input prompt from the user, and an Call LLM node, which sends the prompt to an LLM and receives a response.
Nodes are connected into a flow-graph with two kinds of lines or connectors:
Process Flow Lines (Purple): These lines connect nodes in the order they should be executed. For example, connecting the User input node to the Call LLM node indicates that the user input should be gathered before the LLM is called.
Data Flow Lines (Blue): These lines define the data connections between nodes. For instance, connecting the User prompt output of the User input node to the User prompt input of the Call LLM node ensures that the user's input is passed to the LLM.
You create a connection by hovering over one of the process connector ports or data-flow connector ports, until you see a small “+” cross icon. Press and drag the end of the connector line over the port at the other end of the desired connection.
Nodes have various inputs and outputs representing their different capabilities. These will be explored further in later guides and are also documented in the nodes reference section of the beta documentation.
To create a looping chatbot, as in the example below, the process flow exit connector of the Call LLM node can be wired back to the entry connector of the User input node.
HyperFlow allows for the creation of directed cyclic graphs, enabling the construction of complex AI applications and workflows.
Not all nodes need to be wired into the process flow (with purple connectors). The Instructions node in the above flow-graph is such an example. These are called pure data nodes and usually supply data to a node in the process flow. They are run “on-demand” by the flow-graph engine, as nodes in the process flow that have such nodes connected to input ports are encountered. Such nodes themselves can have pure data nodes supplying their inputs, and so form classic demand-driven data-flow subgraphs within the main process-flow graph. This ability to mix process-driven and data-driven graphs within a single flow-graph is one of many distinctive capabilities of HyperFlow, in this case simplifying the wiring considerably.
Each node in HyperFlow has configurable parameters (also known as “hyper-parameters”) that control its behavior. These parameters can be viewed and modified by clicking on a node in the Editor view, which opens its parameter panel on the right side of the editor.
Most nodes have preset default values for their parameters to simplify initial setup. The User input node, for example, defaults to accepting text input but can be configured to also accept file uploads. The Call LLM node defaults to a chat-based call but can be configured to generate other media types, such as images, audio or video.
The Call LLM node requires the selection of an LLM service, as HyperFlow provides access to a large of such services that you can choose between. This selection can be done before running the flow-graph or deferred until runtime. If left until runtime, HyperFlow will pause execution at the node and prompt for a selection. Available services include major commercial services and an integrated Ollama server with various open-source models.
As you build and wire-up your flow-graph in the HyperFlow IDE Editor, it is common practice to click on each node to open its parameter panel and configure its parameters settings. This “configuration time” setup is not essential, you can defer setting parameters until runtime, but it is worth review the options available for each node, to understand its likely runtime behavior.
Deferring parameter setting
This option to defer parameter selection until runtime is a unique feature of HyperFlow, allowing developers to test, tune, and optimize their applications by presetting some parameters and adjusting others during execution.
Flow-graphs can be run in "developer mode" by clicking the "Run" button in the top-right control tool bar. This mode provides detailed insight into the flow-graph's execution.
When a flow-graph is run in developer mode:
The view zooms and positions to show the currently running node.
Two panels open on the right:
As the flow-graph executes, the run session log updates with information about each step. Details can be expanded or collapsed as needed. For example, the log in the above image shows the knowledge DB search results and search metadata, the composite prompt sent to the LLM with estimated token counts and source step number. The Call LLM is showing a spinning wheel status, indicating the LLM has been called and we are awaiting its generated response.
The developer can interact with the flow-graph during execution, providing input and observing the results in the session log. The flow-graph can at any time be put into a single-step pause mode by hitting the “Pause” button in the run-control toolbar, or exited completely by selecting "Editor" in the Mode buttons, allowing for further editing and parameter adjustments and further test runs.
Run logs are stored and can be accessed later by selecting "Run history" in the "Run" button dropdown.
This guide covers the basics of building and running flow-graphs in HyperFlow. Future guides will explore more advanced features, including single-stepping, breakpoints, locking and unlocking parameters, wire-able parameters, and history access in data flows.
This section of the guide delves deeper into building and running flow-graphs within HyperFlow, demonstrating the process through the construction of an agentic AI application.
Agentic applications typically involve multiple Large Language Models (LLMs), each assigned a specific role within a larger workflow. These LLMs effectively act as agents, performing their designated tasks. In this example, we will build a two-agent content generation flow. One LLM will generate an image based on a user's request, and the other will act as a reviewer, helping the user refine the image by suggesting improvements to the prompt. This improved prompt is then fed back to the image-generating LLM, creating a loop that continues until the user is satisfied with the result.
The first step is to create a new flow-graph and open it in the editor. We will begin by adding two Call LLM nodes. One will serve as the image generator, and the other will be the image reviewer. These nodes can be added by dragging them from the node tab listing or by using the editor's right-click context menu. We also need a User input node to capture the initial image request and an Instructions node to provide guidance to the reviewing agent.
The first node we added was a Call LLM node and the first node added is automatically designated as the start node. Since we want the User input node to be the starting point, we right-click on it and select "Set as Start Node."
Next, we configure the process flow by connecting the nodes with purple process flow lines. The order should be: User Input -> Image Generator LLM -> Reviewer LLM. To manage the iterative refinement, we introduce a Loop controller node. The process flow from the Reviewer LLM is connected to this loop controller. We set the "max loop count" parameter of the loop controller to a specific number, such as 3, to define the maximum number of iterations. The top exit connector of the loop controller is the "loop continue" exit, so it should be connected back to the entry connector of the Image Generator LLM, creating the iterative loop.
HyperFlow allows for the locking and unlocking of parameters. Unlocked parameters pause the flow-graph execution at their node, allowing for runtime adjustments. Locked parameters use the preset value and do not pause execution. For now, we will lock the "max loop count" parameter to allow the flow-graph to run without pausing at the loop controller. Parameters can be locked individually by clicking the lock icon that appears when hovering over the parameter, or all at once using the "Lock all" button at the top of the parameter panel.
By default, Call LLM nodes are preset for chat or text generation. We need to unlock the parameters of the Image Generator LLM and set its role to "Image generator". This will update the available LLM services to those that support image generation.
Now we configure the data flow. The User prompt output of the User input node is connected to the User prompt input of the Image Generator LLM. The "generated content" output of the Image Generator LLM is connected to the Content input of the Reviewer LLM. The Instructions output of the Instructions node is connected to the Instructions input of the Reviewer LLM. Finally, the Generated response output of the Reviewer LLM, which contains the improved prompt, needs to be connected back to the input of the Image Generator LLM.
However, in the current HyperFlow release, a data flow input connector can only accept one input source. To handle this, we introduce a Merge inputs node. This node will merge the original user prompt and the improved prompt from the Reviewer LLM, passing through the most recent data it receives. The User prompt output of the User input node is connected to Input 1 of the Merge inputs node, the Generated response output of the Reviewer LLM is connected to Input 2, and the output of the Merge inputs node is connected to the User prompt input of the Image Generator LLM.
You can see the final rig below. Although the automatic wiring layout in HyperFlow generally does a good job of making the wiring obvious, in some cases it is difficult and so wires can be hidden behind other nodes or appear to connect to ports they do not. If you want to see where an output port is connected, just click on the name of the output port, as has been done in the image below, which will highlight all the destinations and wiring in red.
The Instructions node plays a crucial role in guiding the behavior of the Reviewer LLM. This is where prompt engineering comes into play, and it is often considered one of the most critical and challenging aspects of building effective generative AI applications. The instructions provided here will directly influence the quality and relevance of the Reviewer LLM's output, which in turn affects the overall performance of the agentic system.
In this example, the Reviewer LLM is tasked with analyzing a generated image and suggesting improvements to the prompt that produced it. The instructions we provide dictate how it approaches this task. We might, for instance, instruct it with the following:
"Review the user prompt as a request for an image and analyze the supplied image relative to the user's original request prompt, generating suggested improvements in the form of a detailed prompt for DALL-E 3. Don't necessarily repeat the structure and content of the supplied image, try variations that might fit the original prompt better. You must output only the detailed DALL-E 3 prompt, no other comments or critiques."
Let's break down why this specific instruction set is crafted this way and the importance of careful wording:
The Importance and Challenges of Prompt Engineering
The effectiveness of an LLM-based application often hinges on the quality of the prompts used. Poorly crafted prompts can lead to irrelevant, inaccurate, or nonsensical outputs. Therefore, designing effective prompts, often referred to as "prompt engineering," is a crucial skill in developing generative AI applications.
Prompt engineering is often an iterative process that requires experimentation and fine-tuning. It involves understanding the nuances of how LLMs interpret language and how different phrasing or instructions can lead to vastly different results. Even subtle changes in wording can significantly impact the LLM's output.
Key Considerations in Prompt Engineering:
In the context of our agentic flow, the instructions provided to the Reviewer LLM will directly impact the quality of the improved prompts and, consequently, the quality of the iteratively generated images. It's likely that these instructions will require careful refinement through experimentation to achieve the best results. You might, for example, experiment with different ways to structure the instructions, provide examples of good image generations and how they relate to a request prompt, or ask the reviewing LLM to adopt a particular persona or expertise.
The process of prompt engineering often involves a significant amount of trial and error, observation, and adjustment. It is often the part of building an LLM-based AI application that requires the most care and experimentation. Mastering this skill is essential for building effective and reliable generative AI applications.
For now, we will lock these instructions, but it's important to remember that they will likely be revisited and refined as the application is developed and tested further.
Some nodes, like the Instructions node, are "pure data" nodes. They supply data and do not need to be wired into the process flow. They are automatically run on demand by the nodes they supply input data to. HyperFlow supports mixed data flows and directed process flows in the same graph, a unique capability of HyperFlow.
Before running the flow-graph, we review the parameters of each node. The Merge inputs node parameters can be locked, as they likely won't need runtime changes. For the Image Generator LLM, we select an appropriate LLM service (e.g., OpenAI) and image-generating model (e.g., DALL-E 3). We can also add tags to generated images for organization. For the Reviewer LLM, we select a multimodal model capable of analyzing images (e.g., OpenAI GPT-4.0) and lock the selection.
To be able to pause and inspect the revised image prompt on each loop, we add a breakpoint to the Image Generator LLM by right-clicking on it and selecting "Toggle breakpoint."
To run the flow-graph with a test chatbot interface, we select "Run in Chatbot" from the dropdown menu to the right of the Run button. This opens the HyperFlow IDE’s test chatbot window on the left side of the IDE. Note that even in test chatbot mode, developer mode features are active, displaying the parameter panel and run session log for detailed insight into the flow-graph's execution.
The parameter panel will prompt for the initial user input in the test chatbot. We enter an image request, such as, "a happy penguin wearing a red coat, skating happily on ice."
Due to the breakpoint, the flow-graph will pause at the Image Generator LLM. The step controller buttons will appear, and the session log will indicate "Waiting on next step." Pressing "Step" executes the current node and pauses at the next. Pressing "Continue" resumes execution until the next input, breakpoint, or the end of the flow-graph. The session log provides detailed information about each step. The output of the Merge inputs node can be observed, showing that it passes the most recent data.