🛠️Step 3: Selecting the LLM Model

Model Options

  1. OpenAI LLMs:

    • Pros: Strong function calling, multimodal capabilities, and robust performance.

    • Cons: Higher cost compared to some other models.

  2. Claude LLMs:

    • Pros: Excellent for storytelling and conversational tasks.

    • Cons: May not be as versatile for technical tasks.

  3. Open-source LLM (eg: Llama 3, Mixral 8x7B, Falcon, etc.)

    1. Pros: More control over training data and compute. Cheaper inferencing cost.

    2. Cons: Limited abilities, and reliability.

Configure model in code:

  • Example: Set model="gpt-3.5-turbo" in API calls

Design system prompt:

  • Include instructions for Thought/Action/Observation loop

  • Define available actions and their usage

  • Provide example interaction

system_prompt = """
You are an AI assistant capable of performing actions and reasoning about your responses.
Follow this format for your thoughts and actions:
Thought: Reason about the task and decide on the next action.
Action: <action_name>: <action_input>
Observation: The result of the action will be provided here.

Available actions:
1. wikipedia(query): Search Wikipedia for information.
2. calculate(expression): Evaluate mathematical expressions.
3. simon_blog_search(query): Search Simon's blog for relevant posts.

Example interaction:
Human: What's the capital of France and what's its population plus 1000?
AI: To answer this question, I'll need to perform two actions.
Thought: First, I need to find the capital of France.
Action: wikipedia: capital of France
Observation: Wikipedia result for 'capital of France': https://en.wikipedia.org/wiki/Paris
Thought: Now that I know the capital is Paris, I need to find its population and add 1000.
Action: wikipedia: population of Paris
Observation: Wikipedia result for 'population of Paris': https://en.wikipedia.org/wiki/Paris
Thought: The population of Paris is approximately 2,140,526 (as of 2019). Now I need to add 1000 to this number.
Action: calculate: 2140526 + 1000
Observation: Result of '2140526 + 1000': 2141526
"""

Considerations

  1. Performance Metrics: Evaluate models based on their performance metrics (e.g., accuracy, latency).

  2. Cost Efficiency: Consider the cost associated with each model and its fit within your budget.

  3. Feature Set: Ensure the model supports the features required for your specific use case.

Resources

PreviousStep 2: Choosing the FrameworkNextStep 4: Integrating External APIs and Tools

Last updated