Marking AI Uncertainty: Distinguishing Fact from Inference

Language models do not, by default, signal how confident they are in what they say. A conclusion drawn from a fixed rule and a guess based on linguistic feel are presented with the same wording. This makes it difficult to tell whether a statement about a bazi chart is directly verifiable, merely rule-based, or largely speculative. One practical remedy is to ask the model to label its own uncertainty before using its output.

模型默认不区分把握程度

In ordinary output, a language model does not mark the difference between a claim that follows from a given chart and a claim that does not. If the model says that a certain stem or branch occupies the year pillar, it states this in the same tone as a guess about a person’s temperament. The phrasing does not reveal whether the statement came from something fixed in the input, from a general rule, or from looser association. A reader may therefore treat all statements as equally reliable, which they are not.

This is not only a matter of style. The model can produce a statement that is directly checkable against the chart, such as the presence of a specific heavenly stem in the month pillar, next to a statement that depends on interpretive convention, such as whether that stem forms a particular relationship with another. It can also produce a statement that is neither directly checkable nor based on a stated rule, but is instead a plausible-sounding inference. Without labels, all three kinds are mixed together.

要求它分三类标注

A simple instruction can reduce this confusion. When asking the model to analyze a chart, the user can require it to mark every conclusion as one of three types. The first type is a fact from the given chart. The second type is an inference from a通行 rule. The third type is a speculation that is neither of the first two.

The instruction should be explicit. The model should be told that each conclusion must carry a label, and that the label must reflect the source of the claim. If the model is allowed to choose its own categories or to leave some conclusions unlabeled, the result is less useful. The three-part scheme forces a separation that the model would not otherwise make.

The categories are not grades of truth. They are grades of checkability. A first-type statement can be checked directly against the input. A second-type statement can be checked against a rule if the rule is stated. A third-type statement cannot be checked in either way and must be treated differently.

三类标注各自的含义

The first type is the most straightforward. It consists of statements that can be directly verified against the chart the user supplied. If the model says that a certain heavenly stem appears in the day pillar, the user can look at the chart and confirm whether that stem is there. The model’s citation of a stem or branch must be traceable to the given chart. If it cites a character that is not in the chart, the label is wrong.

The second type covers conclusions that follow from a named rule. The model should be able to say what the rule is. For example, if it says that the chart contains a 伤官, it should be able to explain the definition of 伤官 and show how the relevant stems or branches satisfy that definition. If it says that a certain earthly branch relationship is present, it should be able to state the conditions under which that relationship holds. The statement is not directly given by the chart, but it is derived from the chart through a rule whose content can be examined.

The third type is everything else. These are statements that are neither directly present in the chart nor derived from a stated rule. They may be impressions, analogies, or probabilistic guesses. They can be kept as reference material, but they should not be used as a basis for judgment. Their proper role is secondary.

标注之后怎么用

Once the output is labeled, the labels determine how each part should be handled. Statements in the first category can be used directly. They are the least problematic because the user can check them against the chart without relying on the model’s interpretation.

Statements in the second category should be used only after the rule is checked. The user should ask the model to state the rule, then verify that the rule is a recognized one and that it has been applied correctly. If the rule is stated clearly and the application fits the chart, the conclusion can then be used. If the rule is vague or the application does not follow, the conclusion should not be accepted.

Statements in the third category should be read but not used. They may suggest directions for further thought, but they do not carry enough weight to support a conclusion. Treating them as if they were on the same level as the first two categories defeats the purpose of the labeling.

The overall effect is a filter. The first category gives a stable base of chart facts. The second gives a middle layer of rule-based interpretation that can be audited. The third gives a fringe of speculation that is kept separate. The user then builds the reading from the first two layers and treats the third as optional commentary.

它拒绝标注时说明了什么

There is a simple failure mode to watch for. If the model labels everything as the first type, the labeling has not really been executed. A chart contains only a limited set of directly checkable facts: the stems and branches in the four pillars, their positions, and nothing more. Interpretive statements about relationships, spirits, or personality cannot all be first-type facts. If the model marks them as such, it is either not following the instruction or not distinguishing between a datum and an inference.

This kind of output should be redone. The user can repeat the request, stress that only statements whose cited stems and branches appear in the given chart may be labeled as the first type, and ask the model to reclassify everything else. If the model continues to label all statements as first-type, its output is not usable under this scheme.

The distinction matters because it changes how the reader treats the text. An unlabeled reading invites the reader to accept a mixture of fact, rule, and guess as a single body of conclusions. A labeled reading forces each claim to declare where it comes from. That does not make the reading correct, but it makes it easier to audit.

In presenting results, this site keeps the computational part separate from the interpretive part. The computational part is deterministic: given the same input, it produces the same stems and branches, and the result can be rechecked. The interpretive part is where judgment enters. By separating the two, the site makes clear which parts of a result can be verified mechanically and which parts require a further decision about rules and their application. The three-label scheme described here serves the same purpose when a language model is used to produce a reading.