Question: Should I get a job or go to graduate school?
Answer: If you've liked the last year or so of school, if you like working on problems that no one's solved before, if you like teaching or mentoring, you should consider graduate school. If you feel you need another year or two of school before heading into a competitive job market, then you probably want to pursue an MS. If you're a self-starting hands-on type, you may do just as well on the job. If you like working on really hard problems, well before the productization phase, you should consider getting a Ph.D. With a Ph.D. you can work in a corporate or government research lab, become a college professor, or become a university professor. The hours are long and the uncharted territory can be frustrating, but you won't really notice if you enjoy it. The pay is pretty good, if you ignore the fact that you're not working a 40 hour work week. :)
It's always good to look before you leap. You may want to talk to some graduate students (e.g., your TA's) and faculty about their experiences, careers, etc.
Question: Am I good enough to go to graduate school? Where can I get in?
Answer: Yes. Universities compete fiercly with each other to attract U.S. students. You may be surprised where you can get in. Yes, a top-ranked place like MIT, Berkeley, or Stanford might require a 3.9 GPA and GRE's pushing 700 Verbal, 800 Quantitative. Everyone applies to these places, it seems. But just a little farther down the list, you leave the stratosphere. Each place has its special characteristic that may better match your qualifications and interests; it's not just a linear scale. Also, things vary considerably year to year (maybe a factor of two in number of applications or admits). One factor is the economy--a bad economy leads more students to delay getting jobs and a glut in graduate school applications. And keep in mind that humans, typically faculty, read the applications, hence the criteria for admissions vary widely. The wise thing to do is pick a few dream schools, a few you feel you deserve to get in to, and a few safety schools. Safety in numbers! The applications are very similar, and professors will reuse most of the letter they write for you. Ten schools is barely more work than one. You could apply to twenty if you really want to play it safe.
Question: Do I take out loans to pay for grad school? Those private schools are expensive!
Answer: If you're going for a Ph.D., the department should support you through fellowships, research assistantships, and teaching assistantships (as long as you're progressing well). Good news, eh? If you're going for a Masters, you may have to get a (part-time) job or take out loans. Some departments will have assistantships for Masters students, and may even depend on them heavily for teaching assistants. But since Ph.D. students stay much longer, the tendency is to invest in them more.
Question: What can I do now to prepare? I'd have to ace 20 courses to raise my GPA now.
Answer: GPA is important, but just one element of the admissions equation. Here is what you can do:
GPA. Tricked you! If your major GPA is (much) higher than your overall GPA, make sure it gets called out (it's a spot on many forms). Also, if your upper-division or upper-division major GPA is higher, call that out.
GRE's. You can study for these. Buy the books, practice the exams, even take the courses. You can raise your score 200 points per exam. By the way, researchers communicate through writing, so don't just focus on the quantitative. Study for the verbal and writing tests, too. Not all schools require the CS subject test. This test can be hard to do well on; some of the subject matter may not match the courses you've taken, and there are fewer resources for studying and practicing. Many students opt not to take the test. The computer-based GRE presents special challenges worth preparing for. You cannot jump ahead, you cannot go back and change answers. It's harder to work problems on paper. So practice the computer-basedelement, too.
Research and projects. Anything you do that might improve your preparation as a researcher will give you an edge on both admissions and a smooth start in graduate school. Approach a professor whose classes you especially liked and volunteer to work on their project or ask if they will advise you on an independent study or honors thesis. It's not critical that their research area match your interests (as long as you can stay motivated). However, you could ask a favorite professor to give you an introduction to one you don't know. If you're aiming for a top school, it will help to have done research in your area of interest, and even have published papers. If you take this goal seriously, tell your professor you want to publish. Once you've done it, make sure you mention it in your statement, and provide URL's to the papers. You could enclose the papers themselves, but that's so 20th century.
Letters of recommendation. If you survive the numbers gauntlet, the admissions committee will give your letters of recommendation a close read. These help build a coherent, personal picture of the person behind the numbers. Letter writers like to comment on ability, preparation, passion, work habits, character, social skills, writing or presentation skills, and special accomplishments (See Research and projects). What you can do now is cultivate relationships with professors, preferably through mutually beneficial activities. Working with the professor as a researcher, programmer, or teaching assistant is common. Doing well in the courses they teach is of course important.
Statement of purpose. Every application requires a statement of purpose. This completes the picture: it shows you can write, gives you a voice, and explains your motivation for wanting to go to graduate school, at that particular university. You'll want to briefly discuss your motivation for continuing your education, your projects, your interests, and what you might like to do at the university to which you are applying. Avoid writing about your whole life history. The letter must be kept short and the admissions committee is reading a lot of letters.
E-mail appropriate faculty at the university. It is so common for foreign students to write to faculty in the U.S. that it is not possible to answer each one individually. On the other hand, it is rare for U.S. students to contact faculty. It's not a bad idea to write a couple of faculty at a university that you're particularly interested in. If there's a particular project you're interested in, you might ask how active that project still is. If you're interested in something unusual, you might ask if the faculty member were interested in that topic or issue. If you were thinking of visiting the university prior to April 15, you might ask if they could help arrange for you to speak with a couple of graduate students and the faculty member. (You should be invited out for a visit, once accepted, but the early visit is in part for them to get to know you.) Don't ask if they have research funding for you, and don't ask for their help in admission. In short, don't annoy them.
Question: There are so many schools, how do I pick where to apply? Do I just pick the best 5 or 10 that I think I can get into?
Answer: Probably not. You should consider not only the department's reputation (the university's reputation is not very relevant), but also: the reputation of the faculty in your areas of interest, as well as the the research and personality match of those faculty and the department as a whole. In short, you want to apply to places where you think you could be happy day to day, as well as have prospects after you're done. Faculty in your department will know the relative reputations of departments and research groups. They can help you identify your areas of research interest as well.