Fortnightly Mailing

Taking the red pill: Sebastian Thrun's candid reflection on the AI course

After a horribly embarrassing introduction, this 25 minute talk yesterday by Sebastian Thrun gives Thrun's own candid and personal reflection on last Autumn's AI course, which had 160,000 sign-ups (nearly 100,000 of whom were on the advanced track), 46,000 submitters of the first homework, 23,000 submitters of the mid-term exam, and 20,000 who completed the final exam.

Highlights of the talk:

• the large drop-out rate from the lectures on the same course at Stanford, with students preferring to use the free video-based version;
• a volunteer army of ~2000 translators;
• individual feedback from students in terrible places in the world or under big social pressure who completed the course;
• Thrun's own epiphany concerning the wrongness of "weeder" classes;
• Thrun's decision not to teach by lecture at Stanford again and instead to concentrate his efforts on a private venture-capital funded initiative called Udacity, whose online courses will be free.

Udacity aims to enrol 500,000 students on its first two courses: CS101- Building a search engine; CS373 - Programming a robotic car.

My own and others' reports from the AI course.

 

-

 

 

Interviews about "remote learning" with Dan Cardinali, Paul Mitiguy, and Peter Norvig

Here are links to three November 2011 "canteen" interviews (audio and, in two cases helpful text transcripts) with Peter Norvig (Director of Research at Google), Dan Cardinali (President of Communities in Schools) and Paul Mitiguy (Senior Consulting Professor of Mechanical Engineering at Stanford University). The interviews form part of the resources of an undergraduate project by Deniz Kahramaner, Jon Rodriguez and Ben Kallman about "the challenges and ethical implications of implementing large-scale distance education platforms". 

The interview with Peter Norvig draws on his contemporaneous experience with the Stanford/KnowLabs Introduction to Artificial Intelligence course [November 2011 article in ALT News Online; my weekly reports from the course] and contains several perceptive and diverse insights, ranging over:

• course design (and the advantages of combining the deadline driven "discipline" of face-to-face learning with the "anytime anyplace" nature of asynchronous learning);
• the advantages (from a learner's point of view) of video, and challenges (from a course design point of view) of using it;
• the need for design/production environments for teachers and for experimentation/exploration environments for learners;
• the value, from a learning point of view, of ambiguity in presentations and open-endedness in questions;
• recruitment (and a possible business model for providers of free remote education);
• the possible motivations for teachers to contribute to the production of free courses;
• the need for collegiality and collaboration between the start-ups that are getting involved in online education.

Unexpected - how an open course's conventional exam can prompt subsequent learning

Graph by automagic using code by pedrosorio.
Other posts tagged ai-course.

The AI course finished this week. Here is a link to a discussion thread involving some very mathematically able people discussing, after the exam results were in, a "disputed" mark concerning an exam question about a statistical technique called Laplace smoothing.

I'm not making a point about the substance of the discussion (a lot of which is over my head), or about the underlying investigation (which is even more so).

But the interesting thing is the way that this open course, with its underlying conventional, assessed structure, has resulted in a great deal of very high level collaborative research, investigation, and discussion, which has gone far beyond its (first year US undergraduate) boundaries.

Even if the subject matter of the post is outside your ken, it is worth taking in the kinds of reflections that contributors are making, for example:

"This is really the sort of thing and the sort of thinking and exploring that I believe professors really want to see result from efforts to teach stuff in class (whether in K-12 or undergraduate or graduate school). This one little question has generated more serious inquiry than anything else I have seen so far from our class...... But THANK YOU for all of the serious and thoughtful responses to this question which it has been a pleasure to read."

 

Sitting in a bar with a really smart friend. The ai-class: Notes from a Lab Rat

Guest Contribution by Rob Rambusch

Picture by Ressaure (CC licensed)

Me: Test Subject

Rob Rambusch is a Project Manager for software development and implementation based in New York City. This was his first exposure to any class on Artificial Intelligence. He can be contacted at robrambusch [AT] gmail.com, or through Google+.

It: Experiment

"A bold experiment in distributed education, "Introduction to Artificial Intelligence" will be offered free and online to students worldwide from October 10th to December 18th 2011. The course will include feedback on progress and a statement of accomplishment. Taught by Sebastian Thrun and Peter Norvig, the curriculum draws from that used in Stanford's introductory Artificial Intelligence course. The instructors will offer similar materials, assignments, and exams."

Them: Experimenters

"Peter Norvig is Director of Research at Google Inc.  He is also a Fellow of the American Association for Artificial Intelligence and the Association for Computing Machinery. Norvig is co-author of the popular textbook Artificial Intelligence: A Modern Approach. Prior to joining Google he was the head of the Computation Sciences Division at NASA Ames Research Center."

"Sebastian Thrun is a Research Professor of Computer Science at Stanford University, a Google Fellow, a member of the National Academy of Engineering and the German Academy of Sciences. Thrun is best known for his research in robotics and machine learning."

 One Month Before: Preparation

I started working on the prerequisites in the month before the course began. I prepared for the class by learning Probability and Linear Algebra from Salman Khan at the Khan Academy

Final report from the Norvig/Thrun/Stanford/Know Labs Artificial Intelligence course

Word-cloud by Sea Otter

(Other posts tagged ai-course. Post originally published on 20/12/2011, with paragraph one updated to incorporate completion numbers kindly provided by David Stavens of Know Labs, and a new concluding sentence to the final paragraph.)

Along with just over 20,000 others (some 3,000 fewer than had taken the midterm exam) I completed the final examination for the free online Introduction to Artificial Intelligence course taught by Sebastian Thrun and Peter Norvig. Here is my final participant's  report from the course.

1. The final section of the course concerned Natural Language Processing. I've had an interest in machine translation for some years [e.g.]: and it was this interest that initially made me aware of Peter Norvig's work.) So for me this meant that the best part of the course came last, and if you want to gain an underlying appreciation of the science of natural language processing, it will take you a couple of hours to work through the courses 42 short videos about NLP, starting here. It is probably worth doing despite a certain amount of dependency on earlier sections of the AI course.

2. The course has been mercifully free from programming assignments: being capable of completion using pen and paper, a calculator (and on a couple of occasions a slide rule unused since 1973).  To conclude the NLP unit there were two optional programming problems, both of which could be tackled without programming. I did the second of the problems (recovering a message from a shredded version) using scissors and adhesive tape:

Food for thought. Video recording of Q&A session between Sal Khan, Peter Norvig and Sebastian Thrun

There is plenty of food for thought and a great deal to identify with - from a very diverse set of angles - in yesterday's 45 minute "Google+ Hangout" discussion between Sal Khan, Peter Norvig and Sebastian Thrun. Examples include:

• the authenticity and hence superiority of unscripted, tentative explanations, with "low" production values;
• how lectures empower lecturers not students;
• data analysis as an "underpinning process" for real personalisation;
• why their is such a mismatch between the value that a university thinks its courses provide what students think they provide;
• the challenge of combining individual study with group interaction.

 

Eighth report from the Norvig/Thrun/Stanford/Know Labs Artificial Intelligence course

(Other posts tagged ai-course.)

Here is my eighth participant's report from the Stanford Introduction to Artificial Intelligence course.

Last week's report focused mainly on the midterm exam, and on a conversation with Sebastian Thrun. This report conments on the midterm exam and reflects on the way the course has run this week.

First, some comments on the midterm exam

1. According to information just received from Sebastian Thrun, 23k students passed the midterm, with 85% currently falling into the B+ range.

2. There was a surprising amount of post-exam discussion between students about the substance of the exam. There was some whingeing, but not much given the very large numbers on the course.  The interesting discussion fell into three categories:

The shape of the next phase of online learning

Stephen Downes points to a 9 November 2011 Educause piece by Jarret Cummings and writes:

"I think the shape of the next phase of online learning is being defined: reliance on open educational resources, supported by some sort of automated, analytics based and competency based testing mechanism."

Yes.

Seventh report from the Norvig/Thrun/Stanford/Know Labs Artificial Intelligence course

(Other posts tagged ai-course.)

Substantial  changes made to A3 and small changes made to A5 and A7 below, 21 November 2011, and to introductory paragraph, 23 November 2011. [For visitors from the Aiqus discussion about student numbers, note that more light is shed on the "YouTube video counts question" - a distracting side-issue - in paragraph seven of report number two, 18 October 2011.]

Here is my seventh participant's report from the Stanford Introduction to Artificial Intelligence course. It is in three parts. A is the report proper. B picks up on a pre-arranged call from Sebastian Thrun. C lists further free CS courses that will be available from Stanford University in January 2012.

A - Report

1. A lighter week from the point of view of studying, because the last week saw less material presented, to allow students time to prepare for the midterm "examination", aka the midterm.

Using AI in formative and summative assessment

[Decided to remove image of scratch pad shortly after publishing this.]

In September 2007 I had a personally revelatory moment concerning how AI (as I now know it to be) might be used to provide formative feedback for learners. It stemmed from being involved in running the 2007 ALT conference at which Dylan Wiliam and Peter Norvig each gave keynote speeches. With Richard Noss (who now directs the ESRC/EPSRC funded TEL programme) we set up a Google Doc "scratch pad" to gather a shortlist of issues that could be worth examining. Not a lot (well, nothing!) came from it, as is often the way.

Doing the Stanford AI course (see also my piece in ALT News Online) has sensitised me to current work on the same thing.

Here are some examples. If you know of others, please post them as comments and I will collate them.

• How Khan Academy is using machine learning to assess student mastery, November 2011 piece by David Hu.
• The elements of e-assessment, 2011 article by Sophie White of the examining body OCR (Oxford, Cambridge and RSA Examinations).
• Edexcel chief puts hi-tech agenda to the test, 2010 TES article about developments in Pearson's examining body Edexcel.
• Massive scale data mining for education, November 2010 piece by Greg Linden in Communications of the ACM (via PN).
• Computer analysis of test-takers' answers to standarised tests is revealing cheats, May 2011 article in the Economist mentioning the "relentless logic of combinatorial statistics". [Thanks to DM for the link.]