Facilitating Course Level R&D in an Educational Bureaucracy

My institution developed an online tool for proposing new courses and/or revising existing courses.  It is called the Course Approval and Revision Process (CARP).  The software presents 8 screens on which the instructor has to enter information that goes into a course database.  The sections are:

1. Basic Course Info.
2. Related Courses
3. Instruction Schedule
4. Grading & Restrictions
5. Learning Goals
6. Measures of Student Learning
7. Assessment Plan
8. Library Resources (Optional)

Who or what is this operation for?  It is built around the needs of a committee (the Educational Policy Subcommittee of the Faculty Executive Committee) that reviews courses before they are set before the faculty for approval.  A few other entities have also found themselves "represented" by the software.  These include the learning assessment operation run by the Office of Institutional Research and the General Education committee.

Fundamental Design Flaw

This system is designed to serve the bureaucracy.  For it to function, instructors have to be coerced to transform what they do in thinking about a course into the form that CARP can digest.
The right design would be to produce a tool that instructors would find useful and from which CARP could transparently extract what it needed.

How It Works in Practice

Instructors need to specify things like course title and description, what sorts of instruction it will include, how often it will be offered, whether there are pre-requisites, and how it will be graded.  These constitute items 1 through 4 above.  Then, items 5, 6, and 7 are about assessment.  The program picks up learning goals from the institution and from the department's records in the assessment database.  That's actually pretty useful as it means each person who proposes a new course gets to see how lame the existing learning goals are.  Unfortunately, since the goal is to get the course approved, the result is usually just that the instructor picks a few.

Then, for each goal checked, the instructor has to specify "measurable criteria."  Fortunately, the software does not yet have any AI components that can reject work in progress; one can move on by entering "TBA" as long as one repeats it enough times to hit the minimum text length requirement.

Then you haver to enter an "assessment plan."  This means describing assignments that will be used to measure student learning and then a plan for assessing student learning.  The difference between these is pretty vague, but they do have examples.  Here is what you learn about assignments:

The purpose of assessment is to improve learning. The focus of the learning is the established goals. The assessment plan is the way in which the course will be measured to determine the level of learning in the course relevant to the goals and criteria that are established for the course in order to improve the learning in the course. What evidence is collected (e.g., papers, exams, presentations, etc.)? How is the evidence evaluated against the goals for the course (e.g. rubrics, performance evaluations, etc.)? How will the evaluations be recorded (e.g., scores tabulated, written summaries, etc.)? On what basis will conclusions be drawn about the learning taking place? What will be done with the findings?

And here is an example of the assessment plan.  Or, no, we will not quote that here because it is so lame we can just describe it:  final papers will be rated on a four point scale for each learning goal.  The aggregate results will be discussed at a department meeting and plans will be made to improve.

That last step is called in the assessment business "closing the loop."

But what if this process were actually designed, start to finish, with actual teachers in mind?  What would it look like?  That's where we will go next.

Course Level R&D

Where do new (innovative, exciting) courses come from?  How do we encourage the continuous improvement of existing courses?  What do we even mean by that?

The improvement of an existing course might mean:

• adjusting the topics covered by the course so as to be more up to date, more useful to students, better coordinated with other courses (reducing duplication or improving sequencing)
• making it more attractive to students to increase enrollment
• making it more fun for those who take it
• making it more effective for those who take it
• making it easier to teach

Some of the ways that these things could be achieved are sometimes mistaken as being course improvements in and of themselves--using the latest teaching techniques; using more technology--but we are only interested in this as possible means to the above ends.

The usual source of the kind of changes we have in mind are: imitation, inspiration, and mandate.  An instructor adopts what sounds like a good practice from a colleague, suddenly gets an idea for a cool thing to do in the class, or she is told by a chair or dean to start doing X.

As Bill McKay once intoned, "There's got to be a better way!"

And as teachers by vocation we should be really interested in exploring these.  

An App Proposal

What if there were an application environment that would allow an instructor to organize her materials, present the material to students, track their performance, perform experiments, engage in continuous course revision, collaborate with others teaching the same course at her institution, a related course at another institution, and those teaching this course's pre-requisites and courses for which this course is a pre-requisite, AND have a current version of the course always on file with the central administration.

Features

1. Standard course management system (CMS)
2. The "Lecture Studio" - a platform for lecture/presentation preparation
1. Standard lectures and slide shows but also video/audio, clickers, etc.
2. Record a screen capture
3. Record a video
4. Record an audio
5. Slide library
6. Graphics library
3. Exercise/problem bank
1. Templates for problem production (allows show/don't show answer)
1. Graphic production tools and data driven problems
2. Shared problems
3. Item testing facility
4. Community with others teaching same or similar course
5. Syllabus management system with revision and forking capability (GitHub for courses)
6. Course module management: break courses into free-standing modules with clear pre-requisites and "antigens" that help determine where they will fit.
7. Integrated web bookmark organizing (social bookmarking by others teaching same course)
8. Integrated custom textbook capability
9. Integrated with library databases (contract with content providers to make journal articles available on course by course basis)
10. Integrate with your Zotero bibliography (and your associated PDF library) making it easy to include correct bibliographic information in course materials AND to assemble readers.
11. Narrative evaluation diary
12. Maybe throw in a good calendar function too.  One that had the capacity to track hours spent on teaching.
13. Social media window?  You-tube channel channel?

First stage of development to be a clustering of existing sources in an orderly fashion.  Start to develop organizational tools that would help someone deploying them keep things coordinated, organized, and minimize transaction costs and duplication of effort.

And in this corner, Minerva, goddess of wisdom...

Another online education venture in the news yesterday as "The Minerva Project" announced it had recruited Steve Kosslyn (most recently director of Center for Advanced Study in the Behavioral Sciences and before that Dean of Social Sciences at Harvard) as first dean (http://bit.ly/ZQ1JYT). 

The "Minerva Project (dot com)" : http://www.minervaproject.com/ has been around for about a year now. Their advisory board includes Larry Summers, former president of Harvard, and Bob Kerrey, ex- of the New School. Unlike Coursera and EdX, their model seems to be a little more of a direct competition with the "college experience."

They have a focus on quick progress to advanced courses that starts with a fixed introductory curriculum ("During their freshman year, all students will follow the same core curriculum focused on subjects including multimodal communications and complex-systems analysis.") and movement among several physical campuses around the world. Some sources have price point about $20k and initial recruitment focus on brightest minds in the developing world. Iniital campus (2015) to be in San Francisco.

It would behoove us (faculty) to stay on top of developments like this. The alternative is that we are going to hear about it from boards of trustees and college presidents who have been wowed at conferences and in the higher-ed-admin-media (a phenomenon we might call UVAitis). Consider this from Kerrey:

"What he's got is one of the best private-education ideas I've ever heard," says Kerrey. "I think he's going to be successful, and as a consequence, university presidents across the country are going to be able to say to their boards and their faculties, 'We have to change.' He's going to have a very, very positive impact on all of higher education in America." Leigh Buchanan, "A True Elite Education at Half the Price," Inc. Magazine, 30 October 2012 (http://bit.ly/ZPXqgj) 

Sources 

• (http://bit.ly/ZQ1JYT) "Minerva Project Wins Leading Stanford Scholar," EdSurge Blog Tony Wan, EdSurge 12 March 2013 
• (http://bit.ly/Z1NHPx) "Minerva Project: Positioning and EdTech Questions" Joshua Kim, Inside Higher Ed, April 4, 2012 
• (http://bit.ly/ZPYdOc) "A New "Elite University" Gets $25 Million in Seed Funding," Inside Higher Ed, Audrey Watters April 3, 2012 
• (http://bit.ly/ZPYBfK) "An Idea Too Sensible to Try, Until Now," Kevin Carey, Chronicle of Higher Education, November 11, 2012

King & Sen on (Trouble) Future of Universities

Gary King (Harvard) and Maya Sen (Rochester) apply solid political science analysis to the challenges facing contemporary higher education. They consider market threats, political threats, and the potential for wise responses by universities and colleges.

Other papers from the symposium:

• Editor's Introduction 
• "Chin Up; Chest Out; Don't Panic" by Michael Laver 
• “Let's Not Railroad American Higher Education!” by Henry E. Brady 
•  “Higher Education in the Twenty-First Century: Innovation, Adaptation, Preservation” by Nannerl O. Keohane 
• Comments on “The Troubled Future of College and Universities” by Virginia Sapiro 
• "Paying the Piper: Higher Education Financing and Academic Freedom” by John Mark Hansen

Click here to acess PDF if it does not appear in window below.

King and Sen on Using Social Science to Improve Teaching

Click here to acess PDF if it does not appear in window below.

The Liberal Arts and Time's Arrow

"Liberal Arts Education" as a concept is unfortunately dominated by its own legacy.

When most of us think about the liberal arts our thoughts tend to look backwards.  Some of us fondly recall our own liberal arts educations and the value we perceive it to have had for us.  Or we think about what we've been teaching for years and years.  Or we hearken back to the invention of the modern liberal arts in the late nineteenth century or to the classical liberal arts of the middle ages.

If you listen carefully, you can almost hear us thinking, "If it was good enough then, it's good enough now..."

But it's easy to miss something important.  To understand what a liberal arts education is we should not simply look at the lists in the course catalogs of bygone eras.  Instead we should look functionally at how those lists fit into their time.  Generically, what the liberal arts are is a collection of intellectual disciplines that are appropriate to the training of generalists in their time, of subjects, the mastery of which provides a foundation, a launching platform, for the leaders of an age.

I think that, often, both those who feel an imperative to discard the liberal arts and those who feel the imperative to preserve them come at the question with the wrong idea.

A higher education system that well serves the society that supports it will have a diverse array of parts. Some parts need to be tuned to producing experts at delivering current practice in the professions. Some parts need to be highly specialized, training people to be experts at producing the things of today and solving immediate problems to create the things of tomorrow. And some parts need to prepare people to solve the problems we don't yet know that we have. And we need to train people who can move back and forth among the various experts, who can consolidate their work into emergent solutions for emerging problems. And we need people who have broad capacities to examine and understand the very system in which all the above perform. And some parts of it train people broadly prior to their becoming one of those specialists so that the narrowness of their training does not become a liability.

The mistake that the discarders make is to see the importance of the practically trained and the expertly trained as telling us that we do not need the more generally trained.

The mistake of the defenders is to think that yesterday will always tell us how to train those generalists of tomorrow.

Our challenge as educators is to look forward to figure out what the liberal arts for the 21st century should look like. It's not an easy task, to be sure. But the first right step to take is to be sure we are facing in the correct direction.

Some More Thoughts abou Majors and the Future

The Canon of College Majors Persists Amid Calls for Change

The more academe changes, the more the list of majors stays the same, even though some experts say a radical reorganization of disciplines is needed.




This premium article will be available to nonsubscribers at the link above for five days and at http://chronicle.com/article/Amid-Calls-for-Change-College/48206/?key=. The article is always available to Chronicle subscribers at http://chronicle.com/article/Amid-Calls-for-Change-College/48206/

(c) The Chronicle of Higher Education
1255 23rd Street NW
Washington, DC 20037

On Getting Scooped : TEOTUAWKI*

They closed the comments on Mark Taylor's OpEd "End the University as We Know It" before I could get one in, so I'm posting it here. I often tell my students it's a good thing when you find out that your favorite idea has already been written down by some famous philosopher since that means you were on the right track. I am telling myself today that seeing something you have been thinking about show up on the NYT oped page is similar.

Like many of the other comments, mine amounts to a mixed review. Taylor, to my mind, conflates several issues relevant to the state of today's universities and thereby reduces the punch of the piece. I'm with him on the university as an enterprise in which it is far too easy to continue business as usual. But he may be overgeneralizing from his experience in a religion department in some of his other criticisms.

The point with which I resonate most strongly is that our course/major offerings are too department/discipline-bound and insufficiently dynamic and responsive to the needs of the world around us. In this blog and elsewhere I've argued for uncoupling undergraduate majors and departments as a solution. His suggestion that departments be abolished and programs reformulated around topics and then "sunsetted" every seven years is provocative and worth thinking about. I think, though, that it's not quite the right approach. Why seven years? How worthwhile to rebuild the administrative apparatus on a regular basis?

I think something more dynamic and dialectical is called for: Maybe keep departments, but abolish majors (or maybe leave the majors but create set of collaborative and cross-disciplinary majors on the fly). We faculty should have to reformulate "majors." We might, for example, cook up a program in sustainability or in institutional disruption or innovation. We'd have to really think about what portfolio of the courses we currently offer could be mixed with courses we ought to be offering to put together an educational curriculum that would have both currency and staying power. We'd force ourselves to get beyond the usual departmentally-self-serving horse-trading (I'll require one of your courses if you require one of mine). We'd stop thinking of requirements in terms of the education we wish we could have had 20 years ago and start thinking about what insights are likely to be the building blocks of how people are thinking and solving problems 20 years from now.

Most importantly, we would have to "sell" these programs to undergraduates on the basis of evidence and argument about why a particular curriculum constitutes good preparation for the world to come. And we'd develop the majors with knowing that would be our ongoing task. We would have to be not only inventors of new majors, but innovators who learned from what we were doing to make the operation a going concern in a dynamic structure that was not protected from change by decades of disciplinary convention. In short, we'd build curricular R&D into the very fabric of the university, bringing to higher education a function it's really never had.

*A variation on one of my favorite acronyms "TEOTWAWKI" = the end of the world as we know it.