Climbing Picks unreliable

Climbing picks stop doing the climbing things. After, I notice that they are no longing climbing it is a persistent problem, and I must reload game to get them to do the thing. It seems to be after I play a minor card but am not certain that is the trigger as I tend to go to the Fae tower first and play a new card not needing to climb a lot prior to that.