What is path dependence?

Path dependence refers to the possibility that we might get different results if we train an AI with the same data multiple times. High path dependence means that we get significantly different results, whereas low path dependence means that we get similar results across multiple training runs with the same model on the same data.

An intuitive example would be an AI that has to put a fruit into a single one of multiple baskets. Low path dependence would mean that no matter how many time the AI is re-trained, it always chooses the same basket (e.g. always the right-most basket). High path dependence would mean that it chooses different baskets even after having been trained on the same data^[1].

Through this example, we can understand the potential implications of this topic for AI alignment. Namely, we need to deeply understand the training process in order to explore and make predictions about the behaviors that the final trained AI might exhibit. In the case of low path dependence, we can think of the AI as having a certain “preference” for that basket and we can explore why it has that preference. If it chooses different baskets, though, we can explore things like why it chose a particular basket this time, or at what point during the training it “decided” to choose that particular basket and what the influencing factors were. If the training process robustly rewards a certain “path” over others, then it robustly yields the same type of agent behavior repeatedly. This means that we can infer that the resulting behavior will arise out of the training process with a high likelihood.