August 2021

Featured Article

Lessons Learned in Adopting the Smart City Research Experience for Undergraduates and Teachers for Remote Participation Due to COVID-19

By: Kofi Nyarko, Associate Professor, Morgan State University

 

The National Science Foundation (NSF) provides a significant amount of funding to support undergraduate student research in engineering and science through its Research Experience for Undergraduates (REU) program.  The REU program, initially established in 1987, increase access to research opportunities to underrepresented minority students and students coming from non research-focused undergraduate institutions[1]. Research experiences for undergraduate students have long been identified as a powerful tool to support and prepare participants to pursue graduate education in a science, technology, engineering or mathematics (STEM) discipline[2-4]. The earlier students are exposed to STEM research experiences the better their chances of succeeding in STEM related professional careers or in the pursuit of a STEM related graduate degree [5-9].  Like REU programs, NSF Research Experience for Teachers (RET) programs have existed for many years, and result in numerous documented positive outcomes for the participants[10]. RET programs can help teachers to keep abreast of changes in their fields, help expose their students to cutting edge research, and “bridge the gap” between K-12 classrooms and university research laboratories[11].  

 

The Smart City Research Experience for Undergraduates (REU) and Research Experience for Teachers (RET) (SCR2) Mega-Site program was formed in 2018 with funding from the National Science Foundation to address the low participation and graduation rates of post-secondary students belonging to underrepresented minority groups in the engineering field. It is implemented by a 15-member HBCU/HSI consortium consisting of Morgan State, Howard, Hampton, Norfolk State, Maryland Eastern Shore, University of DC, North Carolina A&T, Tennessee State, Florida A&M,  University of Texas at El Paso, Alabama A&M, Jackson State, Southern, Prairie View, and Tuskegee Institute.  While typical REU or RET sites are located at one university (with the exception of Engineering Research Centers), this program is built around the following research strengths embodied by the consortium institutions: IoT Security, Renewable Energy, Energy Storage, Smart Grid, Human Computer Interaction, and Advanced Materials. The program is organized around five primary host sites (which has been expanded to 6 for 2021), to provide quality research opportunities to a large number of underserved undergraduates and the local high school teachers who serve the same communities in which these institutions recruit from for their STEM programs.   By combining the REU and RET programs in a synergistic way, undergraduate students and teachers can apply their unique perspective to collaborative engineering problems in a way that enhances their individual experiences. While undergraduate students learn to apply classroom concepts with scientific reasoning to solve real world problems, teachers can leverage their exposure to this process to hone their research skills and develop the required lesson plans and associated activities to enhance their school’s science curriculum and laboratories. In turn, teachers can provide feedback to students within their groups from their perspective and experiences as STEM teachers to strengthen reasoning and improve communication. 

 

In early March 2020, during the final recruitment and preparation phase of the program, the COVID-19 pandemic started to accelerate in the united states.  As a result, all the host site campuses switched to remote learning and canceled all in-person summer programs.  The SCR2 program was faced  with a tough decision to cancel the 2020 summer program or switch to a remote only option as expeditiously as possible.  The program leadership considered the demographics of the population the program serves and came to the conclusion that canceling the program may cause additional financial hardships for a significant portion of the participants, as well as squander a valuable opportunity for them to engage in constructive activities over the summer while also providing the program with valuable information about the efficacy of moving to multi-institution remote only combined REU/RET model.  Hence, all site Co-PIs agreed to forge ahead with an expedient migration to a remote only option less than two months prior to the start of the program.

 

Program Recruitment

Once the decision was made not to cancel the SCR2 program after campuses of all host sites closed due to the pandemic, it was quite clear that fundamental changes were needed for the program to be quickly transformed for remote engagement. Since a large majority of the participants had already been accepted, it was necessary to establish the viability for them to participate in the program in a remote fashion.  It was anticipated that some participants may lack access to the necessary computing resources, cameras and a network connection with sufficient bandwidth.  Letters were sent out to all participants to inform them of the program change and to confirm their ability to continue with the program in a remote fashion.  A resource checklist was provided to ensure a proper readiness assessment could be done.  The application process was also amended with this information to ensure the final applicants selected would be aware of the new requirements. Furthermore, participants were informed that the promised stipend amounts would be unchanged for remote participation.  Surprisingly, despite the challenges posed by the pandemic, the recruitment targets for the REU program were exceed.

 

Lab Tours

In 2019, participants received a tour of the research labs at the host institution in which they would be participating.  Since that became impossible to do due to the pandemic, faculty advisors were asked to prepare an introductory video of their research activities and the associated labs.  These approximately 5-minute videos were presented to all participants during the program orientation.  It actually enabled all members of the program, including other faculty, to get a greater appreciation of the research and facilities at host sites.  This is a practice that will be adopted going forward even when the program is conducted in-person.

 

Projects & Supplies

The next challenge to tackle involved transforming existing in-person projects to be conducted remotely.  The faculty members associated with each project were tasked to review the project to assess the efficacy of transforming it to be delivered remotely.  In part, this decision was based on whether laboratory supplies could be expeditiously obtained online and at reasonable costs, whether alternative more accessible measurement processes could be implemented, and whether design and testing could be done in simulation only.  Projects that could not easily be transformed into this mode of delivery were quickly eliminated or merged into other projects.  Participants associated with each host site were presented with the modified projects to rank their preference, which were then used to assign project groups.  For the projects that required supplies, these supplies were purchased and mailed to their homes.  



 

Mentoring & Community Building

In the SCR2 program, graduate students are primarily responsible for providing daily guidance and mentoring.  In 2019, this was done in person with participants working alongside graduate students within their respective labs.  However, with the shift to a remote-only program, the process of mentorship had to be re-examined.  It was very important to establish daily contact, as would be the case for the in-person program, in order to ensure participants still feel engaged in the research.  Hence, mentors had to establish a daily meeting schedule that was recorded, along with the video conference link, and posted for all program participants.  This allowed other group participants, as well as the program coordinators and  faculty advisors to join in research meetings periodically. During these meetings, mentors established daily research goals, reviewed progress made over the last day, discuss technical challenges and solutions, as well as ideas from other group participants.  Previous work was reviewed and guidance was provided through screen sharing. During this meeting it was required that everyone turn on their cameras to provide a stronger sense of community and engagement.  

 

Breaking Down Inter-Institution Boundaries

The pandemic offered a unique opportunity to break down inter-institutional boundaries that naturally exist in multi-institutional programs. From the participant project rankings (which were across all host sites), it was clear that some had several interests that spanned institutions.  In order for those participants to get the most out of their experience, the ability to “Shadow” one or more projects was introduced.  Each participant was assigned a primary project and group, but was also allowed to shadow other projects (and groups) from other host sites. The effect of this was to expand the experiences that participants had in the program in ways that didn’t create undue stress.  This process worked well enough that it will be adopted for the hybrid program model going forward.

 

Insights from 2020 Summer Program

The SageFox Consulting Group is responsible for all SCR2 program assessments.  The insights documented in this section are derived from their assessment report. The assessment instruments used include: A pre-survey for students and teachers; A post-survey for students and teachers; a follow-up survey for students and teachers; and a post-program survey for mentors. 

 

REU Highlights

Surveyed undergraduates overwhelmingly belonged to groups underrepresented in STEM, and those participating in this program showed broad improvements in research skills, academic/career knowledge, and confidence and a consistently high desire to persist in pursuing a graduate degree.  The program had differentiated effects for men and women, and for the 2019 vs. 2020 cohorts.

  • More students participated in 2020 than 2019, however fewer women participated. 

  • The vast majority of REU participants had no prior research experience. 

  • Students joined the REU to gain research experience, develop topical knowledge, gain skills, career development and awareness and to develop a network.  This particular REU was appealing as a way to meet these goals and most students were particularly interested in the topic. 

  • The women entered the REU with less confidence yet greater perceived knowledge than the men.  

  • The 2020 cohort was similar to the 2019 cohort in terms of confidence, though the 2019 cohort had greater knowledge and sense of belongingness than the 2020 cohort at the project start. 

  • Women entered with a stronger sense of belonging than men, a reversal from 2019.

  • Students felt well prepared to work remotely including adequate technology, time and space.

  • The REU leadership team instituted daily meetings with mentors, the frequency  for which 100% of participants report was “just right.”  Likewise, the weekly program-wide lunch sessions were successful at creating a sense of community.

  • Women report greater gains in confidence than men, who also had positive gains.

  • The 2020 cohort had greater gains in knowledge concerning presenting research and ethics in research, yet lower gains in knowledge related to career options and graduate school awareness and preparing research proposals as compared to 2019.

  • Participants report increasing their sense of belonging as scientists, but not feeling like members of a scientific community.   2020 participants were much more likely to report feeling like they belonged in the field of science after the summer experience than 2019 participants did.

  • As a result of the program, women are much more likely to pursue a career in research and advanced degrees in science than men, who also had positive increases, particularly at the institution at which they had the REU experience.

  • Despite all the challenges, the participants report a higher level of satisfaction with the program than the 2019 cohort

 

At the end of the program, students reported a moderate increase in confidence in their research abilities particularly the ability to statistically analyze data. Overall, the confidence of students in 2020  in engaging in research activities were similar to the 2019 cohort. Knowledge about the research process and future graduate school/career options modestly increased, particularly for presenting research, what graduate school is like and ethics in scientific research.  The most gains in knowledge were about “defending an argument when asked questions” and “explaining my project to people outside my field.”  

2019 participants had greater growth in:

  • Research proposal write ups

  • Awareness of what graduate school is like

  • Career options in research

While 2020 participants had greater growth in: 

  • Presenting their research

  • Ethics in scientific research

  • Project management

There was a positive change in participants' sense of belonging in scientific fields.  Interestingly, there was no change in their sense of belonging to a community of scientists as a group, though women had modest changes.  In general, the sense of belongingness was lower than in 2019, particularly a feeling of belonging to the community of scientists.  It is possible that the remote nature of the program prevented the growth of identity.  Somewhat surprisingly, 2020 participants were much more likely to report feeling like they belonged in the field of science after the summer experience than 2019 participants did.

 

Looking towards the future, participants generally agreed that the program had a positive impact on their ambitions, with some modest differences between men and women.  Students credit the program with increasing their interest in the technologies involved in smart or connected cities, and increased their knowledge of the subject.  Compared to 2019, students report that participation in the program increased their interest in attending graduate school at the institution where they had the REU experience.   Women report a more positive effect of the program than men. At the conclusion of the program, participants were more likely to report interest in pursuing a PhD or an MD/PhD than prior to the program.

 

The students were successful despite the transition to a remote REU experience.  The challenges related to remote participation were largely related to communication, particularly when working in a team.  The management of expectations and relationships when not face-to-face required strong and frequent communications.  Benefits of the remote work included greater flexibility in time management including spending additional time on challenging assignments or being able to take on additional tasks when time allowed.

 

Advice for students considering an on-line REU also center around communication, specifically, feel comfortable asking questions and keep team members up-to-date on your progress.  Students also noted that time management is important, with several students noting how important it is to get “ahead” of schedule to accommodate the unanticipated, for example errors or more time needed for analysis.  The participants also spoke about the professionalism required of the program including producing quality documentation, managing time and expectations, and setting personal goals and deadlines.

 

RET Highlights

Surveyed teachers participating in the program report increased confidence and knowledge in research topics and plan to incorporate STEM into their teaching.  Gains were slightly more modest than for the 2019 cohort.

  • Surveyed teachers saw increases in many areas of confidence, such as writing research papers, understanding literature, and the ethical issues associated with research.

  • The 2019 and 2020 cohorts of teachers were extremely similar in their teaching self-efficacy after the program

 

Participants saw increases in many areas of confidence measured, with the largest effects occurring in teacher confidence in areas such as writing research papers for publication, understanding research literature,  and understanding the ethical issues associated with research.  There was a slight decrease in their confidence to statistically analyze data.  All of the teachers expect to bring lessons learned from their research into the classroom, though fewer report that they will be able to translate what they learned into the classroom than in the 2019 cohort.  As a result of the program, teachers report being much more able to improve the understanding of students who are failing and motivating students who show low interest in their work and value learning.  

 

Teachers report an increase in knowledge and interest in the technologies involved in smart and connected cities.  These increases were more modest than for the 2019 cohort of teachers. Teachers report that this increased knowledge will have an impact on what and how they teach, particularly as they can offer students more in depth and relevant examples.

 

Teachers report that they were well equipped to participate in the remote RET including technology and physical space and time.  Any challenges were individual-specific although there was a general sense of disappointment of not having the hands-on experience that would be available in a lab. Benefits include the flexibility of time.  None of the teachers participated in shadow projects, perhaps due to time constraints external to the RET program. Most teachers felt the daily group meetings were just right and that the weekly lunch sessions were helpful in creating a sense of community.  

 

Conclusion

The assessments conducted after the summer program showed that: 1) despite the move to a remote offering, the student cohort was larger than in 2019 and continues to be a diverse group in terms of demographics and majors; 2) Students and teachers were well equipped to participate in a remote research experience; 3) participants were satisfied with the program at the end of the summer; 4) students reported gains in confidence, knowledge and belongingness in STEM between the pre- and post- surveys; 5)  teachers also report gains in confidence and knowledge and report that the experience will influence how they teach and motivate their students; 6) students, teachers and mentors were all satisfied with the mentor relationship;  7) students report that they increased their desire to obtain an advanced degree; 8)   teachers report that they are better prepared to support and motivate their students and will translate what they’ve learned into the classroom.

 

References

  1. Beninson, L. A., Koski, J., Villa, E., Faram, R. & O’Connor, S. E. Evaluation of the research experiences for undergraduates (REU) sites program. Council on Undergraduate Research Quarterly 32, (2011). 

  2. Eagan, M. K. et al. Making a Difference in Science Education The Impact of Undergraduate Research Programs. Am Educ Res J 50, 683–713 (2013). 

  3. Lopatto, D. in Creating effective undergraduate research programs in science: the transformation from student to scientist (eds. Taraban, R. & Blanton, R. L.) 112–132 (Teachers College Press, 2008). 

  4. Lopatto, D. Undergraduate Research Experiences Support Science Career Decisions and Active Learning. CBE Life Sci Educ 6, 297–306 (2007). 

  5. S. H. Russell et al., “Benefits of undergraduate research experiences,” Science, vol. 316, no. 5824, pp. 548-549, Apr., 2007. 

  6. M. C. Linn et al., “Undergraduate research experiences: Impacts and opportunities,” Science, vol. 347, no. 6222, pp. 1-6, Feb., 2015. 

  7. M. K. Eagan Jr., et al., “Making a difference in science education: the impact of undergraduate research programs,” Am Educ Res J., vol. 50, no. 4, pp. 683–713. Aug., 2013. 

  8. W. Zhan and A. Lam, “Benefits of research experience for undergraduate students,” in Proceedings of American Society of Engineering Education Annual Conference, 2011. 

  9. D. Lopatto, “Undergraduate research as a high-impact student experience,” Peer Review, vol. 12, no.2, pp. 27-30, Mar., 2010. 

  10. Dubner, J. et al. Evaluating science research experience for teachers programs and their effects on student interest and academic performance: A preliminary report of an ongoing collaborative study by eight programs. in MRS Proceedings 684, GG3–6 (Cambridge Univ Press, 2001). 

  11. Dempsey, B., Hibbett, D. & Binder, M. Bridging the Gap between Classrooms and Research Laboratories. Science Teacher 74, 33–37 (2007).