Evidence

revised ePDP screenshot
Lynnette’s ePDP one semester after its creation. She has updated her resume and the section of campus and community activities and added links to her art portfolio site and her LinkedIn site.

The IUPUI Connect to Learning project centers on the electronic Personal Development Plan (ePDP), and our formal evaluation has focused on its introduction in the First-Year Seminar. Nearly all entering undergraduate students are required to take an FYS, so this one- to three-credit course lays the foundation for the undergraduate experience at IUPUI. (Learn more in our Reflection in the First Year.) The FYS course goals revolve around orienting students to collegiate learning, acquainting them with what they can expect from their IUPUI undergraduate experience, supporting them as they begin to form identities as learners and to take responsibility for their learning, helping them formulate realistic career goals and academic plans, and introducing them to college-level communication and critical thinking abilities. The Personal Development Plan serves as a primary organizing environment for these learning experiences and outcomes.

Begun in paper form in Fall 2008 sections of the First-Year Seminar, the PDP continues to be used in about half of the approximately 110 FYS sections taught each fall semester, providing a continuing control group with which to compare student and faculty opinions and student outcomes for the ePDP. The electronic PDP was introduced in pilot form in Fall 2010 in 16 FYS sections; the number jumped to nearly 50 percent in Fall 2011 and has been held approximately constant since then pending the university’s decision about adopting a new ePortfolio platform.

University College regularly conducts extensive assessment of student learning and completion or retention, using end-of-course student questionnaires and rigorous tracking of semester GPA, first-year GPA, and retention to second year. Beginning in Fall 2010, additional survey questions were added to discover perceived benefits and challenges of the ePDP. Faculty adopting the ePDP have also been surveyed each Fall, with various forms of follow-up interviews and group discussions to clarify lessons learned and opportunities for improvement.

In 2012-13 University College conducted a qualitative research project on the ePDP with funding from the IUPUI Program Review and Assessment Committee. University College staff developed a rubric for holistic assessment of ePDPs based on PDP learning outcomes for the First-Year Seminar, then used that rubric to evaluate fifty ePDPs, with two faculty reviewers for each portfolio. Scores are preliminary, and we are still coding student statements around each outcomes. Overall, students fell between the lowest (“identify”) and middle (“describe”) rubric cells; we need to decide whether this is acceptable for freshmen or if we want to work more closely with faculty on strategies to improve outcomes. Students show strongest evidence of learning in ePDP sections related to educational/career planning and weakest evidence of learning in ability to self-evaluate and identify behavioral changes that will increase their success. (See citation below for further information.)

Highlights below from the Fall 2011 quantitative assessment of student outcomes illustrate some of the data gathered. The full report is attached, and cross-year comparison and implications are discussed, in Overall Findings and Implications.

  • In fall 2011, 722 students responded to an end-of-semester questionnaire designed to assess their perceptions of the ePDP process and outcomes.
  • The five most highly rated aspects of the ePDP were related to goal setting, academic planning, career decision-making, and self-understanding (Mean on 5-point Likert-Type scale ranging from 1=Strongly Disagree to 5=Strongly Agree):
    • I have goals that are measurable, achievable, and realistic. M=4.17
    • I have chosen a major or career that supports my interests and personal values. M=4.16
    • I know what obstacles I have to overcome to succeed in college. M=4.10
    • I have chosen a major or career that matches my strengths, skills, and competencies. M=4.07
    • I clearly understand my academic strengths, skills, and competencies. M=4.06
  • The lowest rated items were related to the technology ease of use and navigation as well as sustainability or planned use of ePDP in the future. (Mean on 5-point Likert-Type scale as above.)
    • I found the technological processes associated with the PDP easy to understand. M=3.25
    • I found the technological processes associated with the PDP easy to navigate and use. M=3.21
    • I will utilize my PDP throughout my college career to help me make educational and career choices. M=3.06
    • I will collaborate with my academic advisor in updating my PDP in future semester. M=2.99
    • I will continue to update and consult my PDP in future semesters. M=2.98
  • Students who reported that they completed All Parts of the ePDP reported statistically significantly better outcomes compared to students who completed Almost All Parts of the ePDP (or were in sections that only required certain sections).
  • There were no significant differences between the 2010 and 2011 responses. The only significant and perhaps practical difference was that students in 2010 reported that they completed significantly more parts of the PDP compared to the 2011 students. This may be due to the large number of sections participating in 2011 and the fact that faculty varied greatly in terms of requiring all aspects of the ePDP compared to those in 2010.
  • Students who were enrolled in First-Year Seminar sections that required ePDP completion (some or all parts) had significantly higher Fall Cumulative GPAs (M=2.82) compared to students in sections that did not require any aspects of the ePDP (M=2.73), once academic preparation, sex, and income levels were accounted for. There were no differences with regard to First Year Cumulative GPA (fall and spring combined) or Fall-Spring Retention Rates between ePDP and No ePDP students.

In addition to use in the First-Year Seminar, the Life-Health Sciences Internship Program (LHSI) completed a successful first full year with the ePDP in 2012-13 following a limited pilot in Spring 2012. Most of the interns (sophomores,┬ájuniors, and a few seniors) had not previously created an ePDP, so the program used two “Student Ambassadors” (student mentors with previous LHSI experience) to model creation of the ePDP, troubleshoot technology questions, and engage the interns in weekly face-to-face or online discussions about their learning in these year-long internships. The ePDP clearly helped students in this program learn to recognize and reflect on their learning in new ways.

Lynnette's ePDP in Fall 2013. She has updated and expanded all sections and added home-page links to her Twitter and blog accounts, with internal links to a School of Business blog to which she also contributes. The blogs in particular demonstrate her growth in reflective capacity.
Lynnette’s ePDP in Fall 2013. She has updated and expanded all sections and added home-page links to her Twitter and blog accounts, with internal links to a School of Business blog to which she also contributes. The blogs in particular demonstrate her growth in reflective capacity.

For further information, see Buyarski, C., & Landis, C. “Using an ePortfolio to Assess the Outcomes of a First-Year Seminar: Student Narrative and Authentic Assessment,” International Journal of ePortfolios. (In press, anticipated publication April 2014)

Leave a Reply

Your email address will not be published. Required fields are marked *