Writing an Answer or Choosing One

A customer asks for help. Should the software write a reply, find a record, or choose who handles the request? Those are different jobs, with different checks. Some may not need a language model at all.

In this authored illustration, a fictional bike shop receives this message:

I was charged twice for the tune-up on my daughter's bike. Can you fix that? And is the bike ready yet?

Generation writes or explains something: a reply to the customer, or a plain summary of the refund policy. How does an AI answer get made? follows this job.

Retrieval finds relevant material: the refund policy, the order record, the repair ticket. A reply without them has nothing to stand on.

Ranking puts candidates in order. If a search returns 10 policy passages, ranking decides which to read first.

Bounded selection chooses from allowed outcomes. Does this message go to billing, to repairs, or to a person for review? A poem about tune-ups is not a fourth option. This message asks two things, a billing fix and a repair status, so a router allowed only one destination drops half of it. Split the message into its requests first, or let the choice return more than one label.

Whether the card was charged twice, and how much to return, are exact questions. Code can compare the 2 charges and subtract, with the same answer every time.

Issuing the refund is an action. Actions belong to a workflow with permissions, under rules about who approves each step. When a model decides some steps along the way, people often call the system an agent. An agent is a system built around models and tools, not a separate kind of model.

Likewise, an assistant application is more than the model inside it. The shop's assistant might include the inbox, the policy search, the payment connection, and the approval rules. The model is one part.

A general language model can also choose. Ask, "Is this message about billing, repairs, or review? Answer with 1 word," and it classifies. Ask it to order 10 passages, and it ranks. The answer still arrives as generated text, so software must check that it is one of the allowed words.

Other models are built for choosing. A classifier learns fixed labels from labeled examples. A decision model takes the material and the allowed answers and returns a choice. Some are built on a language model underneath. The name describes the job, not the construction.

Jev, from TypeSafe, is one current decision model. You supply the material to judge, which TypeSafe calls state: here, the message and the order record. You also supply the allowed answers: billing, repairs, review. According to TypeSafe's description of its question types, Jev returns one of those answers with a probability for each, never a value outside the list.

TypeSafe calls this class of model "System One," a name borrowed from a book about fast and slow human thinking. That is the vendor's label, not evidence that the model thinks like a person.

An answer in the right shape can still be wrong. TypeSafe's confidence documentation describes its confidence number as a summary of how concentrated the probabilities are, and advises testing thresholds on your own data. Check its choices against examples with known answers to see how often that confidence is justified.

Channel3, which builds a large catalog of products and sorts each into a category, wrote about trying Jev. Its first approach walked down the category tree one level at a time, keeping a few of the best-scoring paths. For a youth bicycle, the right branch dropped out early, and the product ended up as a motor vehicle.

Channel3's existing pipeline already retrieved 30 likely categories, and the company reports the right one was among them more than 99% of the time. With Jev making the final choice from those 30, it reported better accuracy than the open model it had used for that step, at about the same cost. These are its own figures on its own catalog, given without sample details.

If an early step throws away the right answer, no later choice can recover it. Check what each stage keeps, as well as what the last one picks.

Read finding before answering for the retrieval step, or checking an AI answer to test an answer against its sources.

For the technical version, should a model make this decision? compares code, classifiers, and decision models against an error limit. The Jev workshop lets you route requests and compare methods on the same cases.