Thursday, February 25, 2010

CRA Career Mentoring Workshop notes

I attended the CRA Career Mentoring Workshop this week in Washington, DC. This workshop is aimed at sharing with late-stage PhD students, postdocs, and early-stage assistant professors some advice regarding how to start and succeed in an academic career. The workshop covers topics such as how to select students, how to teach and mentor students, how to get funding, how to establish a research program, etc.

Disclosure: Having been thinking about my academic career for some time, much of what was said was review for me. Below is a summary of either new things I learnt, things I already knew that were said in new light, and things that seemed especially important to me. These are by no means complete notes and will most likely be less useful to others than to myself.

Teaching:

  • Make sure you teach the same course several times. The time savings are astronomical the second and third time around. Ask the chair for this.
  • When teaching a class, set firm rules. For example, be explicit about what your policy is on extensions and stick to that policy. Put all these policies in the syllabus. It's better to have an explicit policy that you later realize was not ideal than to leave ambiguity in the policy.
  • Idea about regrades: don't allow students to discuss regrades for 24 hours after they first see the graded item. The first reaction is anger. In 24 hours, fewer students will feel like they're entitled to argue, and those who do ask for regrades are more likely to have legitimate reasons.
  • Make old exams available. They are out there anyway.
  • Avoid doing things that penalize the organized, honest, and prepared. For example, don't create extensions. Don't take late work. Don't ask students not to look at old exams, etc.
  • Exams should start with easy questions and get harder. Everyone should get at least something right.
  • Return graded assignments and tests promptly. If it takes you two weeks to grade something, chances are you procrastinated for 13 days and then got it all done on the 14th.
  • Asking students to assess how each person in a group contributed to a joint project (by % of work) is surprising reliable and precise.
  • The ratio of preparation time to teaching time, when you start, can easily be 12:1.
  • If possible, use a textbook with slides, tests, homeworks, etc. They do exist. These can reduce the preparation time to 3:1 or 2:1.
  • First time teaching a course, keep the syllabus plan for what you will cover a little vague because you don't know if you'll hit all the topics.

Mentoring:
  • Send students to conferences.
  • Make students come back from conferences knowing 1 (then 3) new people. For example, tell them to talk to speakers. Have them return and tell you about these people. Maybe have them follow up with those people.

Getting Tenure:
  • During the first semester as an assistant professor, meet with the chair, talk about the process, ask for written evaluations (at some point).
  • When going up for tenure, your CV must have acceptance rates for conferences.
  • Keep the tenure CV updated through out the career. Every trip you take. Every conference acceptance rate. Every invited talk.
  • Do a pretenure tour. In the final year before tenure review, go around to strategically selected universities and give talks on your work (to potential letter writers).
  • "Good" incoming students don't look like you do now; they look like you did when you entered the PhD program.

Stress Management:
  • Paper deadlines and beginnings of terms are really stressful. You must take advantage of the times that are slower.
  • Pay attention to how long things take and write them down so that you can accurately plan in the future.
  • Spend money on problems to save yourself time. For example, pay someone to clean your house.
  • Exercise, get enough sleep, eat well.

Funding:
  • Call NSF program managers.
  • Call before submitting to get feedback on how an idea will fly. They may tell you "if you include ..."
  • Call after panel meeting (look up online when the panel meets) and ask how you did. Whereas you could get rejected, if you call, you might get seed funding.
  • Write to program managers asking to be a reviewer for a specific program. Send your credentials. This is the best way to learn how to write proposals.
  • Email program director one month before solicitation deadline to get on a panel.
  • A grad student costs roughly $200 a day. That means buying some equipment / books / etc. is worth it! If it takes a student 3 days to figure something out online that s/he could have learnt from a $100 book in a day, just buy it.
  • Bring in however much money you need to run your program, and not a penny more. Otherwise, you're spending too much time on the wrong things.
  • Be able to answer the George Heilmeier's Catechism questions when talking to program managers, other professors, etc.:
    • What are you trying to do? Articulate your objectives using absolutely no jargon.
    • How is it done today, and what are the limits of current practice?
    • What's new in your approach and why do you think it will be successful?
    • Who cares?
    • If you're successful, what difference will it make?
    • What are the risks and the payoffs?
    • How much will it cost?
    • How long will it take?
    • What are the midterm and final "exams" to check for success?

Advice on Getting Advice:
  • If you know the answer to a question but just need some encouragement, keep asking for advice until you get the answer you want.
Also, Dan Grossman had a good first date last week. Wish him luck!

Monday, February 22, 2010

Survivor Bias on Career Advice

I'm blogging from the NSF CRA Career Mentoring Workshop. This workshop is aimed at [very] late term graduate students and postdocs and is supposed to mentor us on how to create an academic career: how to get an academic job, how to start a successful research program, how to mentor students, how to get funding, etc.

The topic that I choose to pick on today is survivor bias in advice on how to be successful. Laura Haas, IBM, just finished giving us a plethora of advice (that she and Dave Patterson, UC Berkeley, compiled). It was good advice. It was logical advice. Having said that, I'm skeptical. I am not just skeptical for one of the two obvious reasons: (1) retrospective advice of successful people often is aimed at the perfect world and may be far more romanticized than would be useful to a young researchers, and (2) most such advice, in my opinion, is extrapolated from a single data point of the success of a particular advice giver (e.g., I wrote a shorter-than-usual research statement and I got 10 job offers, so you should write a short one!). I am skeptical because of the survivor bias, coupled with a special kind of selective memory, that I feel affects such advice.

Such advice typically includes a list of don'ts and a shorter list of dos. For example, don't follow the least publishable units model, don't work on 20-years-out problems, do follow the scientific method to the letter. To justify this advice, the advice givers then typically apply, retrospectively, this advice to their own experience. The interesting phenomenon is that, in all the examples I've encountered, this past experience contains as many dos as don'ts. The dos, especially the noble dos, such as "work with others," and "focus on quality, not quantity in publishing," are regarded as the keys to the person's success. The don'ts, however, especially the "ignoble" don'ts, such as "generate lots of publications from a single idea," get classified as mistakes and dismissed as having put the person where he/she is today.

The real problem is that we rarely hear from unsuccessful academics, who may be able to discuss whether they actually executed more don'ts than dos, as the advice we get today would imply. In fact, it's quite possible that striving to do only the dos destroyed their academic careers. It may also be possible that having the proper ratio of dos to don'ts is what led to the success of the advice givers, and, without the don't, they would not be where they are today. There is no good way to evaluate the advice, except with some sense of morality. It seems noble to work with others, it seems noble to publish quality papers, it seems noble spend your time doing deep, meaningful reviews. It seems less so to try to publish small improvements on your previous work. So our minds have little issue accepting the premise that doing these noble things is good and what will help us succeed in our careers. But that statement is far from "executing as many dos and as few don'ts as one can will lead to success in academia."

Even if the advice givers had done lots of dos and few don't, and we could at least argue correlation, it wouldn't mean causation. (On that note, Margaret Martonosi just said "I was at the original CRA Career Mentoring Workshop, and here I am, so clearly, it works!") But we cannot. There is no evidence that what they call mistakes aren't precisely the actions that brought them success.