Alumni Pathways: Education Outcomes Improvements

Alumni Pathways: Education Outcomes Improvements

 

Created Date

Aug 25, 2023

Target PI

PI-6: see https://docs.google.com/document/d/1K11u825jxt2gmtA2JmgxhRAbiHasYkYekfNnDi6T_JA/edit?usp=sharing

Target Release

Q4-23

Jira Epic

Document Status

Review

Epic Owner

@Gavin Esser

Stakeholder

@Lendl Meyer (Deactivated) @Dave Wallace (Deactivated) @Kaleb Trotter @Hunter Burk @Everett Bloch

Engineering Team(s) Involved

Primary: Analyst

Secondary: Micro Documents Possibly: CDOT

PART 1

Customer/User Job-to-be-Done or Problem

The JTBD and problems for this epic are the same as the previous epic: https://economicmodeling.atlassian.net/l/cp/fton6Y9h

This Epic is to take the MVP report created there and enhance it with additional features and functionality to meet more of the customer needs discussed.

From Previous Epic:

All Alumni Pathways personas need to understand what education learners completed before, at, and after their institution's offerings in order to support a variety of use cases. See details by persona in the next section.

Our new partnership with National Student Clearinghouse (NSC) provides this data and so we want to surface it in meaningful ways to our customers within Alumni Pathways.

 

Value to Customers & Users

As an institutional researcher and/or academic leader, I want to understand what education learners completed before, at, and after their institution's offerings, so that I can:

  • Answer questions from Academic Leadership (including preparing visuals and reports)

  • Provide information and insights that support reviewing and updating educational offerings

  • Evaluate strengths and weaknesses—in terms of Educational outcomes—across my institution’s portfolio of educational offerings (overall and also drilling down to the school/college, department, and program levels)

  • Provide information and insights that support grant funding applications

  • Increase awareness of, and optimize relationships with, feeder and subsequent education opportunities (overall and also drilling down to the school/college, department, and program levels)

  • Articulate/defend the value of educational offerings in preparing learners for subsequent education

As an enrollment marketer and/or program marketer, I want to understand what education learners completed before, at, and after their institution's offerings, so that I can:

  • Inform enrollment marketing targets and priorities (optimizing based on past performance)

  • Enhance marketing materials, communications, and events with information about subsequent education opportunities (overall and also drilling down to the school/college, department, and program levels)

  • Prepare talking points for my institution’s leadership

  • Articulate/defend the value of educational offerings in preparing learners for subsequent education as part of public relations and reputation-building efforts

As an administrator and/or leader in the President’s Office, I want to understand what education learners completed before, at, and after their institution's offerings, so that I can:

  • Answer questions from institution leadership (including preparing visuals and reports)

  • Evaluate strengths and weaknesses—in terms of Educational outcomes—across my institution’s portfolio of educational offerings (overall and also drilling down to the school/college, department, and program levels)

    • Specifically to inform investment decisions across my institution’s portfolio of offerings, including the possibility of sunsetting offerings

  • Provide information and insights that support grant funding applications

  • Provide information and insights that inform and support special projects and initiatives

  • Inform strategic partnership targets and priorities (optimizing based on past strengths & weaknesses)

  • Articulate/defend the value of educational offerings in preparing learners for subsequent education as part of stakeholder relations (legislative funding, government relations, community relations, etc.)

As an administrator and/or leader in Advancement, Alumni Relations, and/or Foundation, I want to understand what education learners completed before, at, and after their institution's offerings, so that I can:

  • Inform strategic donor targets and priorities (optimizing based on education journeys and outcomes)

  • Articulate/defend the value of educational offerings in preparing learners for subsequent education as part of donor relations, alumni relations, and goodwill development

  • Answer questions from donors, partners, and other stakeholders (including preparing visuals and reports)

  • Increase awareness of, and optimize relationships with, feeder and subsequent education opportunities (overall and also drilling down to the school/college, department, and program levels) in fundraising and alumni relations activities

  • Enhance materials, communications, and events with information about subsequent education opportunities (overall and also drilling down to the school/college, department, and program levels)

 

Value to Lightcast

We believe that the substantial value we are adding to customers (through serving their jobs to be done above) will be readily captured in sales acceleration, deeper market penetration, and higher retention.

We also believe that this data is difficult for institutions to find and/or compile and thus provides a competitive advantage.

Target User Role/Client/Client Category

Institutional Research, Academic, Enrollment Marketing, President’s Office, and Advancement/Alumni Relations/Foundation teams across the majority of Education segments

Note:

  • because of the dependence on NSC data, this feature will not be available to the small percentage of institutions (< 3%) and education partnership accounts that do not report data through NSC

 

Delivery Mechanism

We expect to primarily deliver this as additional tables, visualizations, filters, and exports as a part of both the existing Education Outcomes report and the Report Builder in the Alumni Pathways vertical of the Analyst Platform.

 

Success Criteria & Metrics

Ultimate Definition of Done

  • The Education Outcomes report is fully serving all of our intended use cases and user needs including:

    • Visibility into enrollments, not just completions.

    • Visibility on all future/previous education for alumni rather than just 1 step back or forwards.

    • Ability to help users understand when alumni are most likely to want to pursue additional education.

    • Helping users see award levels achieved, time spent in school, time spent between degrees, and similar useful breakdowns.

    • A report structure that better helps our users complete their work, or answer key questions rather than simply displaying all relevant data.

 

Definition of Done by PI (e.g. proposed distribution of work)

PI-6: see https://docs.google.com/document/d/1K11u825jxt2gmtA2JmgxhRAbiHasYkYekfNnDi6T_JA/edit?usp=sharing

 

North Star Metric

Alumni Pathway’s current North Star Metric is Filtered Reports per Month by Account (as a proxy for answering institutional users' questions and supporting their work). We believe this feature will move that metric via:

  • Providing additional filters for reports and giving more incentive to run reports is different ways for comparisons.

  • Adding value that will additively help lead to higher retention (e.g. more opportunities to see/use this and other reports)

Our target improvements for this metric are:

  • >20% increase in Education Outcomes reports run by Account within the first two months after these improvements are made.

  • >5% more Filtered Reports per Month by Account within the first 3 months after the first feature is released

  • The higher level of Filtered Reports per Month by Account is sustained (e.g. does not drop back to previous levels over time)

  • NOTE: we are expecting a relatively modest change in filtered reports for this change alone. I am expecting it to make a longer term impact in providing more intrinsic value that will keep customers retained. The 5% value is a guesstimate, but this feature will likely not lead to the same high impact of other epics.

 

Aspects that are out of scope (of this phase)

Anything beyond the improvements scoped for this epic. There may be additional improvements in future epics.

 

PART 2

Solution Description

We believe that this data will enable us to calculate and/or aggregate data into answers to the following questions for customers:

  • Education Timeframes: How long do learners spend pursuing various degrees?

    • What timeline gaps are there between the pursuit of degrees?

    • When might be the optimal time to reach out to a learner about pursuing another degree?

  • Transfers (complete a degree elsewhere):

    • Top institutions transferring into ours?

    • Top subsequent institutions our learners transfer to?

    • Common/typical pathways?

  • Employment outcomes: How do learners’ education pathways correlate to employment outcomes?

    • Best / worst pathways (for outcomes)

Our answers to each of these questions should include the ability to drill down by:

  • institution characteristics (e.g. school/college, department/unit, major/program/CIP, class level, etc.)

  • learner characteristics (e.g. enrollment start/end dates, enrollment status, graduation status, employment outcomes [where matched/available])

Early UX (wireframes or mockups)

 

Non-Functional Attributes & Usage Projections

Performance characteristics

  • Load times should be at least comparable to other Analyst packets & reports

  • Usage (# of users, frequency of usage) is expected to be comparable to other Alumni Pathways reports, and much lower than the Education Analyst vertical at first – rising to comparable levels to Education Analyst over several years

 

Privacy/security implications

  • We need to consider and carefully align to our FERPA and Clearinghouse data compliance requirements and corresponding privacy/security implications.

 

Localization requirements

  • Alumni Pathways is USA-only. If National Student Clearinghouse data is used in calculating the metrics to be stored, development for those calculations must be restricted to the USA only (due to security requirements in our partnership agreement with the Clearinghouse).

 

Mobile & accessibility requirements

  • We need to meet or exceed the minimum requirements for the Analyst Platform generally and Alumni Pathways vertical specifically.

 

Dependencies

PI-5 work on https://economicmodeling.atlassian.net/wiki/spaces/DPM/pages/2614788148 must be completed before this epic can be finished.

 

Legal and Ethical Considerations

Just answer yes or no.

Have you thought through these considerations (e.g. data privacy) and raised any potential concerns with the Legal team?

High-Level Rollout Strategies

This represents a modest new feature for Alumni Pathways, and thus we would want to:

  • Prepare Sales, Marketing, and Customer Success in advance so that they are prepared to demo, promote, and support it, and also ready to relay feedback to the product for subsequent improvements

  • Subsequently roll out to all customers

Risks

  • If our partnership with NSC were to end, we would lose access to this information and data source for our customers which could pose a future risk.

 

Open Questions

What are you still looking to resolve?

 


Complete with Engineering Teams

 

Effort Size Estimate

Estimated Costs

Direct Financial Costs

Are there direct costs that this feature entails? Dataset acquisition, server purchasing, software licenses, etc.?

 

Team Effort

Each team involved should give a general t-shirt size estimate of their work involved. As the epic proceeds, they can add a link to the Jira epic/issue associated with their portion of this work.

Team

Effort Estimate (T-shirt sizes)

Jira Link

Team

Effort Estimate (T-shirt sizes)

Jira Link