Education and
Public Engagement
at the Lunar and Planetary Institute

Evaluation

LPI conducts rigorous evaluation on all programs, which incorporates comparison pre- and post-assessments, audience surveys, focus groups, and more. External evaluators are employed for most projects. Following are some data regarding a subset of our programs. Additional evaluation details and tools for particular programs are available upon request; please contact education@lpi.usra.edu.

Educator Professional Development

Explore: Fun with Science in the Library
For each of eleven different Earth and space science topics, Explore provides step-by-step instructions for a selection of hands-on activities, as well as supporting facilitator resources. The activities are featured in periodic in-person and online trainings for library professionals. Over 800 individuals from 35 states participated in training through the Explore Program from 1998 to 2013.

Read the Evaluation Results and Main Findings

Evaluation results: Pre- and post-tests showed that, overall, participants gained statistically significantly in Earth and space science knowledge from pre- to post-training. Overall, the trainings increase the participants’ confidence, as well as their self-reported ability and intention to use the activities. A follow-up survey of the Explore community was conducted in spring 2013, and the 168 respondents indicated that

  • 86% are actively using Explore materials
  • 60% implement Explore activities on a regular basis

Compared to their perceptions from before participating in Explore:

  • 60% are very committed to providing science and engineering experiences for their visitors (compared to 16% before the training)
  • Over 75% are more likely to advocate for including science and engineering in the programs offered at their facilities

Close

Read Evaluation Results and Main Findings from an external evaluation conducted by NASA's Exploration Mission Directorate

Explore: To the Moon and Beyond: Evaluation Report
External evaluation was conducted after the training was held.  Evaluation results: Following the training, more than 85% of the participants felt prepared to share the LRO Mission with their patrons. Over 90% of the participants rated the likely appeal to their communities of the LRO Mission, NASA’s future lunar missions, and having humans live and work on the Moon as good to excellent. 96% of the participants agreed to strongly agreed that they were prepared to use the activities and resources in their library setting, and 93% planned to host programs using the LRO Modules by the fall of 2006.

Main findings:

  • A critical element was having the LPI and LRO team empower the participants to use the materials in a manner that suited their environment and their audience.
      • Discussions at the close of the trainings included determination of paths for the librarians to work together in training or presenting the information (e.g., state library meetings, district or regional trainings). 87% of the participants (40 of 46 respondents) anticipate staying in contact with library colleagues they met during the training.

Close

Explore Jupiter’s Family Secrets
LPI developed library program materials based on Jupiter and the Juno project, for library programs targeted for children ages 8-13. To facilitate development of these products, LPI hosted and conducted trainings including three trainings in 2012-2013.

Read the Evaluation Results and Main Findings

Evaluation results: An external evaluator designed pre- and post-assessments and surveys and measured changes for the 2012 trainings; LPI conducted the evaluations for the 2013 training due to scheduling issues.

  • Participants reported significant increases in their preparedness (from 2.8 to 4.3 out of 5), confidence (from 3.3 to 4.2 out of 5) and perceived ability (from 3.1 to 4.2 out of 5) to lead space science activities as a result of the training.
  • Participants demonstrated statistically significant increases in their content knowledge of Jupiter and space science, with the average pre-test of 56% and an average post-test score of 79%.
  • Ratings of session content and instruction were close to excellent (4.4 out of 5).
  • Ninety six percent of participants in final training indicated that they plan to implement most or all of what they learned.

Main findings:

  • These trainings added the role of a librarian mentor, enabling participants to assist with successive trainings. The creation of a mentor librarian position appears to be a successful addition; librarian mentors helped to present activities, shared how the materials worked in their libraries and the way they addressed various challenges, and invited participants to contact them for further assistance after the training. Generally, participants indicated they would be very likely to utilize the mentor.
  • Mentors are interested in continuing to communicate with the librarians in their states, in sharing their experiences, and even have asked permission to conduct their own trainings to share out Explore resources.

Close

Faculty Institute for Earth and Space Science Education
A collaboration of scientists and science education specialists offered Faculty Institutes for university and community college faculty involved in teacher preparation. The effort focused on: 1) immersing participants in best science education practices for use in teacher preparation with respect to space and Earth science; 2) utilizing appropriate curricular support resources for teacher educators leveraging NASA science content and data; 3) creating a network of teacher education faculty; and 4) creating, testing, and refining a professional development model for pre-service faculty institutes.

Read the Evaluation Results and Main Findings

Evaluation results: The implementation of the FINESSE Institutes started well and was observed to improve notably over time. Most Institutes earned high ratings on post-training evaluations, with those tending to increase as the Institutes were refined as the result of experience. The degree to which participants implemented the ideas of FINESSE Institutes – the use of real data and inquiry teaching strategies in courses that are for or include preservice teachers – is challenging to measure. However, if we take the response rate of the online survey as a lower limit (supposing that non-responders are also non-implementers), we see that nearly a third (~30%) of participants self-report that they have implemented one or more FINESSE ideas in their classrooms.

Main findings:

  • The faculty educating future STEM teachers is diverse; future efforts should consider focusing within this audience. One of the largest challenges of FINESSE has been meeting diverse audience needs. The science faculty has different needs and is interested in different resources than the science education faculty, who are very different from other education faculty (such as some of those who primarily focused on pre-school and early elementary education methods). These diverse needs may also have contributed to the FINESSE participants being less inclined to spend time networking, as they have different priorities in their fields. The FINESSE team attempted to overcome this challenge by limiting the participation in FINESSE to science and science education faculty working with secondary pre-service teachers, and focusing on workshops at science education conferences.

  • Faculty investment in any project is challenged by time, resources, and the juggling of numerous responsibilities; future professional development models should consider budgeting for longer-term participation with additional activities. An increase in expectations of the participants (e.g., participation in two follow-up institutes at science conferences, implementation of an increasing number of backwards-faded-scaffolded activities across three to five years, etc.) and the pairing of faculty colleagues from the same university in the institutes, – coupled with appropriate monetary resources - may result in a deeper, permanent change in teaching – at least of the individuals involved.

  • Some science education faculty and many future science teachers are not comfortable with inquiry and the process and nature of science; it is important for NASA SMD to consider if and how it can strategically address this need within its education portfolio. FINESSE offers one avenue for engaging faculty preparing future teachers in inquiry through two-day institutes. The tools created through FINESSE remain available and FINESSE II continues to offer institute opportunities. However, like many pre-service teachers, many faculty teaching science education have never participated in authentic research and have varying degrees of understanding and comfort with inquiry. NASA SMD could offer a variety of professional development offerings, research experiences, etc. to help prepare exemplary STEM educators who deeply understand the process and nature of science – and who are able to engage their students in authentic inquiry.

  • If NASA SMD continues to address the strengthening of STEM pre-service teacher preparation, it is important to consider identifying strategic partners for doing so to broaden reach and increase return on investment. FINESSE worked closely with ASTE and NACCTEP, as well as with science societies. Future efforts could leverage federally funded programs, universities, or university consortia, such as UTeach (http://uteach-institute.org/community). Collaborations could also be established with the pre-service teacher activities at different NASA centers.

Close

Shell Space Science Training Series
LPI, working with JSC’s Astromaterials education staff, delivered a series of trainings in 2008 and 2009 at Harris County Department of Education for approximately 190 middle and high school science teachers, on topics such as plate tectonics and the rock cycle, Earth’s structure, Earth and Moon comparisons, Sun and seasons, stars and galaxies, the Solar System, and astrobiology. 

Read the Evaluation Results and Main Findings

Evaluation results: The average participant rating of the quality of the professional development was 4.42 out of 5, where 1 was poor and 5 was excellent.  Comparisons of pre- and post-assessments showed significant increases in understanding for each training.

Main findings: Although participants were encouraged to build their knowledge by attending multiple sessions, most were not able to do so.  Week-long summer sessions were more successful at provided extended professional development.

Close

Sustainable Trainer Engagement Program (STEP)
STEP is a train-the-trainers professional development project for middle school science education specialists and lead teachers. The objective of STEP is to increase the Earth and space science knowledge and pedagogy, and skills and confidence in providing professional development of middle school science specialists and lead teachers in the Houston region.  Thirty days of STEP trainings have been conducted for two cohorts of participants.  In the past 18 months, STEP participants have presented or assisted in presenting trainings for approximately 1,250 teachers. This project is still on-going.

Read the Evaluation Results and Main Findings

Evaluation results: Evaluations of this program are longitudinal and being conducted by an external evaluator; significant pre- and post-assessment data as well as some survey data are not yet available.  Cohort 1’s average rating of the FY13 trainings is 4.7 out of 5, and Cohort 2’s average is 4.9, where 1 was poor and 5 was excellent.  Surveys showed significant increases in participants’ self-efficacy in teaching ESS, in confidence in training other teachers to be effective, and in self-reported skill level in preparing other teachers to teach ESS.

Main findings to date:

  • Most participants are comfortable with sharing activities and resources with teachers, but many are limited at the school and district level in their ability to conduct professional development in spite of their leadership positions.
  • Participants are unsure how to best work with scientist mentors, and multiple opportunities to interact and plan with scientist mentors are required.
  • Participants find it very difficult to set aside time for participating in short teleconferences.

Close

Evaluation results: Following the training, more than 85% of the participants felt prepared to share the LRO Mission with their patrons. Over 90% of the participants rated the likely appeal to their communities of the LRO Mission, NASA’s future lunar missions, and having humans live and work on the Moon as good to excellent. 96% of the participants agreed to strongly agreed that they were prepared to use the activities and resources in their library setting, and 93% planned to host programs using the LRO Modules by the fall of 2006.

Main findings:

  • A critical element was having the LPI and LRO team empower the participants to use the materials in a manner that suited their environment and their audience.
  • Discussions at the close of the trainings included determination of paths for the librarians to work together in training or presenting the information (e.g., state library meetings, district or regional trainings). 87% of the participants (40 of 46 respondents) anticipate staying in contact with library colleagues they met during the training.

Close

Unknown Moon Institutes
These weeklong summer high school science teacher institutes, conducted annually from 2010 to 2013, alternated in location between LPI and John Hopkins University Applied Physics Laboratory. Participants received in-depth content and participated in rich inquiry-based activities about lunar science, talked daily with lunar scientists, and toured research facilities.  A total of 80 teachers attended from across the US and as far away as Australia. 

Read the Evaluation Results and Main Findings

Evaluation results: The average participant rating of the quality of the professional development for each training ranged from 4.6 to 4.9 out of 5, where 1 was poor and 5 was excellent. Comparisons of pre- and post-assessments showed significant increases in understanding for each training, with training specific average increases ranging from 17% to 25%.

Main findings: Participants found the activities very engaging, and highly valued access to the lunar scientists and their research, as well as the tours of research facilities.  Participants were surveyed on their likelihood to use the activities, and most planned to use most of the materials they had experienced.  Facilitators adjusted the institutes’ implementation each year based on findings from the previous years.  Main findings included:

  • Expectations that participants would be able to create new quality activities and lessons after a week of content and activities were unrealistic; after 2010, participants were instead encouraged to develop implementation plans using the lessons already developed and shared.
  • Increasing participants’ understanding of best practices in addressing misconceptions and understanding of the nature of science required not only modeling these concepts but also explicitly discussing them and applying them throughout the week.
  • While participants did not appear to absorb significant science content or pedagogy from scientists’ presentations and tours of research facilities, some of the teachers valued these experiences more highly than any other aspect of the institutes.

Close

Student Programs

High School Lunar Research Projects Program, 2009–2013
The Center for Lunar Science and Exploration’s High School Lunar Research Projects program allowed high school students to actively participate in lunar science and learn about pathways into scientific careers. The objectives of the program were to enhance 1) student views of the nature of science; 2) student attitudes to-ward science and science careers; and 3) student knowledge of lunar science. From 2009–2013, approximately 232 students and 21 teachers from across the United States participated in the program.

Read the Evaluation Results and Main Findings

Evaluation results: Pre- and post- student surveys demonstrated an increased understanding of the lunar formation (Giant Impact Hypothesis) and their ability to identify and describe major geologic features on the Moon.  A strong majority of students enter the program with very positive attitudes toward science, and post survey responses show that these attitudes remain the same following the program. Just under half (41.5%) of the students reported that their experience in the program has contributed to their consideration of a career in science. Teachers were also surveyed; 100% of teachers rated the Moon 101 activity as “Excellent” or “Good.” Eighty three percent rated the research portion of the program as “Excellent” and 83% rated the entire experience as “Excellent.”

Main findings: Further modifications of the Attitudes towards Science Inventory and the Views of Nature of Science Questionnaire will be needed to better assess the impact on students, to focus on particular aspects of science that the program can best address.

Close

Public Engagement

MyMoon
The Lunar and Planetary Institute (LPI) developed a lunar education new-media portal, MyMoon, in partnership with lunar scientists, educators, artists – and the public – to populate it with science content, diverse media exhibits, and opportunities for involvement. Through MyMoon, the general public has been able to interact with lunar and planetary science content that informs them about NASA’s lunar science research and missions, and engages them in future plans for lunar exploration and eventual habitation.

Read the Evaluation Results and Main Findings

Evaluation results: While the website struggled to gain an audience, the MyMoon social media enjoyed more success, especially over the last year with a stronger marketing strategy and investment into developing these platforms for the project.  Twitter followers increased to 1300, Facebook fans increased to 894, Google+ followers to 292, and Instagram followers to 566.  During the same period, web traffic declined, from 2000 monthly visitors in early 2013 to about 800 visitors per month in December 2013.

Main findings: In its final years, MyMoon’s social media presence saw some good success when more attention and advertising were devoted to it. More interestingly, the social media success suggests much greater ability to effectively engage the target audience and build a “community of enthusiasts.” This observation should be carefully considered for future (similar) projects, as it indicates that a project such as MyMoon may be better served by investing its resources into social media platforms/presences rather than a traditional website. Many of the same functions may be achieved via the social media and the development effort could be reduced and refocused into more important areas such as the actual science content and strategies, as well as the marketing/promotion/advertising strategies.

Considerations for Future Programs

1. Recruit and build strategic partners inside of and outside of the NASA community in the earliest stages of the project (e.g., missions, private space organizations, citizen science programs, organizations currently reaching the target audience but unrelated to the content, etc.).
2. Involve numerous individuals from the intended target audience to serve as advisors and ambassadors – a virtual “street team.”

  • This group should inform the design and implementation of the project.
  • They should continue to be “authentic voices” to reach out to and build the intended online community using their own platforms (e.g., social media, art, blogging, music, etc.) and help the web presence be consistently active and vibrant.

3. Provide an avenue for visitors to become actively involved in science (citizen science) or further exploration of the topic(s). Provide a “call to action” or opportunity for involvement in a “cause” related to the topic to serve as a rally-point for an online community of enthusiasts. If budgets allow, create and leverage a gaming component; providing something engaging to “do” on the website versus simply things to read or watch or comment on.
4. Create, launch, and focus efforts around the relevant social media platforms from the earliest stages of the project. Focus less on a website and instead utilize the latest in social media and online community technology

  • Think outside of the “box;” launch and run the project via the social media instead of a website. This would require less start-up investment and focus could be maintained on creating content for regular social media posting and engagement of the target audience (one of MyMoon’s strengths).
  • Be the go-to” place for all of the latest & greatest info/resources related to the topic that are relevant to target audience.

5. Create a marketing, advertising, and recruiting plan. Ensure that the project has both the budget and the expertise to implement the plan during every aspect of the project.  Identify your participation goals and milestones and track your success in meeting them.
6. Remove barriers to participation. Make sure that the space is designed with this in mind. Make it as easy as possible for your target audience to engage in a way in which they are accustomed.
7. Leverage NASA Imagery and Video resources early and often. NASA Audio/Visual resources have some of the greatest potential to draw people in and engage them. This should be leveraged fully and tied into the relevant social media (example: Instagram). High-quality interactives on the website may also be used to accomplish this more effectively.
8. Learn from successful programs internal to and external to NASA. In the development stages, take a look at similar projects that have been successful and do an analysis of what makes them prosper. What are they trying to achieve? Are they successful? What strategies are they using? What do they recommend changing? Reach out to these examples of success and try to build a relationship with one of them so that they may
serve as a mentor to the project.
9. Capitalize on the audiences' personal connections to the topic. For example, MyMoon initiated a “What the Moon means to me” campaign. These can be interviews or submissions from the target audience. A personal connection and investment will help to acquire long-term community members.

Close