In order to study the role of metacognition and self-regulation in student learning, college students were asked to solve problems in mathematics using an ITS assisting metacognition at the forethought stage. Activity involved selecting relevant facts and strategies and connecting them by arrows. It is found that patterns of arrow drawing have significant correlates in student performance and cognitive states. (i) Forward chaining is significantly more predominant during problem solving than during initial exploration that is not goal-driven. (ii) Students scoring in the middle are more likely to enter convergent pairs of arrows compared to students who scored low or high (P<0.00005). These findings enable diagnosing student problem solving and imply constraints on selection of cognitive architectures for modeling student learning. While no significant effect of the intervention on student scores was observed compared to controls, many students reported that the tool helped them to learn how to solve problems.