Domain experts often collaborate with AI experts to realize their AI application ideas.
These client-AI expert collaborations have become increasingly widespread.
Client-AI expert collaboration begins with the client, who outlines their needs, expectations, and available resources through pre-collaboration planning.
The pre-collaboration plan forms the basis for the discussion with
AI experts during which the stakeholders collaboratively iterate the plan into a concrete and actionable one.
The experience during this iteration heavily depends on the quality of the pre-collaboration plan.
But because the client often lacks knowledge about AI and about the information needs of the AI expert
coming up with a well-defined pre-collaboration plan is often challenging for the client.
It can be mentally taxing and can even lead to misinterpretation of intents.
To address this issue, we introduced PlanTogether, a system that provides guidance tailored to
each client as they outline a pre-collaboration plan.
For an illustration of how the system can aid a client during pre-collaboration planning
let's imagine Sarah, a manager at a used car dealership
who wants to avoid overpaying when purchasing used cars.
She decides to hire an AI expert to solve her problem
and uses PlanTogether to develop and concretize her AI application idea before entering discussions with AI expert.
Sarah opens up PlanTogether and starts with the ‘Project Objective’ section.
She notices the overview panel on top, which starts empty.
The system first asks her about ‘Task Performed by AI Applications: “Please explain the task the AI application needs to perform.”
But she finds out she cannot answer the question without any tips and suggestions
so she looks at the questions that can help her with this question and moves to 'End Goal'.
Sarah now sees the question “Describe the end goal of your AI application.”
For this question, she types in “Make better decisions on used car purchases.”
Afterward, Sarah wants to confirm whether her answer meets the intentions of the question.
Sarah clicks on the AI chatbot and asks, “Would my answer for End Goal be helpful for AI experts in charge of AI application development?”
The chatbot responds, “Instead of simply stating ‘Make a better decision on the used car purchases,’
clarify what the better decision indicates to suggest how the AI application can improve the decision making.”
Based on the response, Sarah modifies her answer to
“Avoid overpaying when purchasing used cars from their previous owners.”
Then, she clicks on “Save & Continue” to move on.
Sarah eventually returns to the earlier question about 'Task Performed by AI Application'.
This time, she sees the tip for answering the question
“Focus on how the AI model will enhance decision-making in used car purchases by evaluating price accuracy and negotiation outcomes.”
with a blue highlight on the phrase: “enhance decision-making in used car purchases.”
Hovering over the phrase, she sees that this tip draws information from the previously answered ‘End Goal’
telling her the relevance of her previous answer to that question.
She clicks on the downward arrow button and obtains answer suggestions tailored to the current plan.
Reading the tip and the suggestion, she decides to accept the second suggestion
“Predict accurate used car prices to ensure used car buyers do not overpay by comparing historical sale data and negotiation outcomes.”
and she continues.
Sarah proceeds to the question about ‘Domain Metrics’.
She is unfamiliar with the notion of ‘domain metrics’ and initially lacks ideas about how to approach this question.
Sarah sees a tip for answering the question
“Focus on how you measure the used car dealer’s decision-making performance in used car purchases.”
Hovering over the highlighted phrase “used car dealer’s decision making”
Sarah sees a reference to her previous answer for ‘Target User Groups & Demographics’.
Similarly, she hovers over “used car purchases” to review her answer for ‘End Goal’.
She re-reads the question in the context of her previous answers.
After some thought, she decides to get concrete examples by clicking on the expand button and sees the suggestions
“Cost to Market” and “Profit Margin.”
Through the examples, Sarah understands how she should approach this problem.
Sarah decides to double-check her understanding through web search.
She finds the definition “custom metrics that are specific to your product”
and the importance of capturing user expectations and experience, which confirms her understanding.
Knowing that maximizing profit margins is the end goal of the target users
and since “price gap” aligns with the problem of overpayment, she accepts the suggested answer, “Profit Margin,” and moves on.
To highlight the improvement of client experience, let's look at the case of Rick
a used car dealership manager in the same situation as Sarah.
But instead of using PlanTogether, he uses a web survey without personalized tips and suggestions.
Rick also makes it to the question about ‘Domain Metrics’, unfamiliar with the notion of ‘domain metrics’.
He opens up a search engine and searches for “domain metric”
which eventually leads to the same definition that Sarah saw: “custom metrics that are specific to your product”
However, Rick has difficulty linking the information and his situation into a coherent answer
he wants to see some examples to get him started.
He searches for “domain metric examples” and finds examples in domains such as marketing and education.
Still, because the problem of used car purchases is specific and relatively uncommon
Rick finds it difficult to link his problem to the examples to formulate an answer.
Rick ends up putting in a lot of time and effort before arriving at an answer.
Now let’s take a look at how PlanTogether works.
PlanTogether represents information dependencies within the pre-collaboration plan as a Planning Information Graph.
It uses this graph to extract and link relevant questions and information to help the client clearly see and leverage connections between information.
Based on this Planning Information Graph
the guidance generator presents personalized tips and suggestions for each node question and overview summaries.
Through a user study, we confirm that the tips and suggestions as well as the progression overview included in PlanTogether
help clients navigate information dependencies and write concrete actionable plans reflecting their domain expertise.