Skip to content

Conversation

ArturGaspar
Copy link
Member

@ArturGaspar ArturGaspar commented Feb 28, 2025

Description

Adds an XBlock similar to the short answer one, but which can select between different instructions for the LLM depending on context.

Testing instructions

  1. Add "multiagent_ai_eval" and configure required settings
  2. Use the XBlock so that it will switch characters.
  3. Use the XBlock so that it will end the session and provide an evaluation report.

The below example with the default settings shows how to test this functionality.

image

Other information

Private-ref: https://tasks.opencraft.com/browse/BB-9325

@ArturGaspar ArturGaspar changed the title Artur/multiagent feat: multi-agent AI XBlock Feb 28, 2025
@ArturGaspar ArturGaspar force-pushed the artur/multiagent branch 3 times, most recently from 0a58220 to f6f0470 Compare March 10, 2025 09:07


DEFAULT_SUPERVISOR_PROMPT = textwrap.dedent("""
You are a supervisor managing an interaction between the following agents: Coach, Character.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ArturGaspar Was this prompt provided by WGU?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pkulkark Yes.

role: str


class CharacterData(pydantic.BaseModel):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These pydantic models seem to be used for validation. Any reason for not using django serializers?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pkulkark Is there any reason to prefer them?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ArturGaspar Just for consistency with the rest of edx-platform xblocks. Since the xblock uses Django models, using a Django serializer would be more idiomatic.

ValidationMessage.ERROR,
f"{self._get_field_display_name('agent_prompt')}: {e}",
))
else:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need else here? Couldn't we just continue without it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The part under the else checks that the template will render correctly for each character. The part above that it will render at all with the scenario data.

If the agent prompt has errors with the scenario data, it would be pointlessly noisy to repeat it for every character.

<textarea class="user-input" rows="1" placeholder="Type your answer here" maxlength="1000"></textarea>
<span id="submit-button">Submit <i class="fa fa-paper-plane"></i></span>
{% if has_finish_button %}
<span class="chat-button" id="finish-button">Submit for evaluation</span>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ArturGaspar It might be better to disable this button until there's a user response. What do you think?

<span class="chat-button" id="finish-button">Submit for evaluation</span>
{% endif %}
<textarea id="user-input" rows="1" placeholder="Type your answer here" maxlength="1000"></textarea>
<span class="chat-button" id="submit-button">Submit <i class="fa fa-paper-plane"></i></span>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

else:
message = self._get_agent_response(agent, user_input)
character_name = self._get_character_name(agent)
character_data = self._get_character_data(character_name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the character_name and character_data being fetched within _get_agent_response function too? Couldn't we just pass that info back instead of calling them again?

@ArturGaspar ArturGaspar marked this pull request as draft October 13, 2025 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants