Research From Apple And Epfl Explains Why AI Models Can’t Truly...

In this section, we introduce Theorem 2, an important theoretical extension showing that merely giving a Transformer additional unguided computing space does not overcome the "local reasoning barrier." In the context of a "agnostic scratchpad," the model is permitted to produce a polynomial-length sequence of intermediate tokens (a scratchpad) with no oversight over their content. This theorem generalizes the earlier finding (Theorem 1). The high-locality "cycle task," which requires the model to identify if three particular nodes in a graph belong to the same cycle, is still the task at hand.

Source: HackerNoon