Our learning environments are becoming more technology-rich and mediated leading to an increase in the amount of longitudinal learning traces generated and collected on a daily basis. Teachers, parents, and policymakers agree on the power of data to improve teaching and learning. However, teachers are usually the front line end users responsible of orchestrating technology for both inside and outside classrooms, and harnessing learning traces to improve learning outcomes.

Although advantageous learning technology brings forth numerous challenges for teachers. In online learning, a teacher lacks cues of the vital link of face-to-face human interactions such as, gestures, facial expressions, direct contact, dialogue and feedback from learners. Further, the different pacing and paths of learners online, as well as the articulation of both in-class and off-class online learning activities, make it challenging for teachers to monitor learners. How can a teacher develop awareness of the state of learning online? How can a teacher foster learners' engagement with online materials? How can a teacher keep an eye on learners' progression? How can a teacher identify struggling learners? Probing learning traces for value and mapping it into pedagogical utility is still a complex and laborious process. Teachers face barriers to using data in their workplaces, including a lack of time and data literacy training to analyze, interpret, understand, and use data.

Teachers' dashboards are a type of learning analytics dashboards where the aim is to summarize and present learners' learning traces to teachers, and ultimately to make the state of online learning visible for teachers so that they can engage in informed interventions to assist learners to do their best. Teachers use such artifacts in a variety of ways to make sense of data, to inform their decisions and practices, to support learners, and to engage in conversations with parents. Prior research on teachers’ dashboards emphasizes the need to investigate new probes, underscore a lack of guidance to help the design and validation of teachers’ dashboards, and need for more longitudinal studies to understand how teachers leverage dashboards over long periods of time and how dashboards shape their pedagogical practices.

In this dissertation, we present five studies. First, a literature review synthesizing related work by classifying teachers' dashboards design choices, data types, and data-informed practices. Next, a case study of using a new multimodal data probe in learning analytics namely, learners' emotions to support teachers' awareness of the emotional state of learners in computer-mediated learning environments, which led to the design of a multimodal digital prototype dashboard called Emoda. The results from in-house testing of Emoda with two teachers showed that, while relevant, representing multimodal data was too complex to explore and interpret. This makes it difficult for us to validate the impact of this new data probe on teachers' practices. We had to iteratively design a new prototype relying only on video stream as a source of data, which led to the design of Emodash which was used to conduct a two months-long, “in the wild”, field study with five professional teachers. Next, to better ground the design of dashboards for teachers, we conducted a design study based on interviews with seven teachers to understand their needs and we collaboratively iterated on design prototypes, which led to the design of Progdash, a dashboard to assist teachers in monitoring learners’ progression in an online grammar and spelling learning platform. We then conducted an “in the wild” longitudinal evaluation of a three months-long field deployment with 17 teachers to evaluate the impact of Progdash on teachers' practices. And finally, by reflecting on our personal experience on the design and evaluation of the three dashboards for teachers, we turned to pertinent research in HCI and InfoVis to conduct a methodical study, which led to the articulation of Side a process model to guide the design and evaluation of teachers' dashboards through four mutually informed stages.

This dissertation contributes to the advancement of learning analytics dashboards by extending our understanding of (1) how to make use of multimodal data probes in learning analytics, (2) how teachers leverage dashboards over long periods of time, (3) how such artifacts impact teachers' pedagogical practices, (4) how to unify the design and evaluation of teachers' dashboards. Based on our findings, we provide (5) design implications pointing toward better dashboards in bridging online and situated learning (e.g., in-class, virtual classroom), fostering teachers' and learners' self-reflection, and supporting teachers' diverse needs, ecosystems and workflows.