Kelm on computer architecture, global engineering management at Intel
"The other aspect of it is, even with a cache-coherent system, you can have different interleavings of memory as viewed by third parties." John Kelm on computer architecture, global engineering management
David: The date today is June 17, 2012. We're here in San Francisco, California, with our guest John Kelm. How are you doing John?
John Kelm: Doing pretty well.
David: Thank you very much.
John: I just had a delightful dinner and enjoyable company. Now we're going to enjoy a delightful time of interviewing.
David: I'm glad you're so excited. First things first, you're from Connecticut. Grew up in Stamford, I believe?
David: Norwalk. OK.
John: Which is adjacent, well, it's near Stamford.
David: It's near there. OK. What was your childhood like? You're at Intel now, you do semiconductors. Tell us what got you into engineering in the first place?
John: I don't know man. I come from kind of a blue-collar background. My father delivered oil. My mother worked at a dentist's office. I never met an engineer until I was late in high school, very late in high school. I got interested in computers when I was younger, maybe, I don't know, eighth grade, so probably 12, 13 years old. I happened to meet some people that were really into the hacker counterculture.
Ravi: In high school?
John: In high school, yeah. There's a local chapter of the 2600.
David: I remember that. People that did a lot of phone phreaking?
John: A lot of phone phreaking, but also they were just dorky people that wanted to hang out that liked computer stuff.
David: That's great. This was in the mid '90s?
John: Yeah, probably mid '90s, late '90s. Looking back on it, we didn't know anything. Between then and now, I've had a decade of training in the field. It just seems so juvenile now, in some ways. But it was a lot of fun. I got exposed to a lot of people that I wouldn't have otherwise met.
When I was a kid, I was building stuff, Legos. I like to take things apart. Things like, "Oh, here's an old VCR. I'll just rip it apart. Don't know how to put it back together, but..." That comes with age.
David: When you were in high school, I know you were pretty involved with the yearbook. Did you do any kind of extracurriculars, anything like that?
John: I was president of the Honor Society. I was on the tennis team.
Originally, it kind of started as...There's this perception that to get into a good college you need to be doing an extracurricular. You need to be doing a sport. You need to have good grades. You need to be taking these certain classes. I don't know. In the end, I did a sport. I did an activity. I showed some leadership capability, I suppose. I ended up going to a state university.
David: So pretty average childhood. Nothing really outstanding.
John: I wouldn't say it's anything outstanding. What got me was when I went into college, I just focused in a way that I think a lot of my colleagues, a lot of other students I saw didn't.
David: I see. You actually got into undergrad at the University of Connecticut. Honestly, you're one of the hardest working people I know. What was the culture like there? You said it was a big state school. Where is it located? It's in Connecticut, right?
John: Yeah, but it's in the middle of nowhere.
Ravi Sathyam: Is it in New Haven?
John: No, that's where Yale is. It's in Storrs, which is probably about half an hour away from Hartford, very much in cow country. You drive along a desolate state road to the school. Then there's nothing really around the school, and the city that surrounds the school is rather small.
Ravi: For some reason, I can't imagine cow country in the Northeast.
John: I think the coastal areas of the Northeast and the areas along rivers are very developed. Just look at Google maps' satellite imagery, and you'll just see very high population density around the coast and up the rivers, and then sparse farmland in between.
It's interesting. I don't think I would've realized what my college experience was really like until I had gone to Illinois, which is also a big state school, but much more focused on engineering. A much larger state school, a much higher ranked state school, a much larger state school. I think I gained something from both experiences.
David: That's interesting. UConn is the big state school in Connecticut, and it's fairly large. I think they have about 30,000 people there?
John: Maybe across all the campuses, but I think the Storrs campus has got to be about 20,000, maybe less.
David: OK. What's the culture like there? We've interviewed people that have gone to a lot of places, Berkley, obviously, University of Illinois, up in Canada as well.
David: Yeah, Waterloo was just the other day. What do people do? You went for computer science, if I remember correctly.
John: Computer science and engineering.
David: Yeah, so computer science and engineering. Do people get jobs when they graduate, or do a lot of people go to graduate school? What are the paths?
John: I can't really say, because I was not close friends with a lot of people in my department. I knew a lot of people through my job in school, but for the most part, people I knew got jobs afterwards. Very few of them went to graduate school. If they did, they were doing professional masters programs.
David: Not a lot of people going into research programs, you're saying?
John: No, not that I know of. There may have been, but not that I knew of.
Ravi: Was there a big financial presence there? I'm sure, since you said a lot of people go to jobs, they go into the finance industry, do IT at a bank?
John: In computer science, I got the feeling like some number of people went into the finance industry—the insurance industry—because Hartford is a big insurance center. Also, defense; there's a couple electric boat, submarine manufacturing companies in Connecticut: Pratt & Whitney, United Technologies, and General are based there.
David: They recruited from Illinois, as well, but I never knew what they did.
John: They're a couple different companies. Otis Elevator is one of the companies. Sikorsky, the helicopter company, is one of the companies. Pratt & Whitney makes engines, jets, that kind of thing, is one of the companies.
Ravi: Hamilton Sundstrand. United Technology's essentially, from what I understand, is a conglomerate of these individual companies that do various engineering things.
John: Some of it's defense related. Some of it's commercial.
David: What do people aspire to? It's interesting, again, when you look across the United States, people are very shaped by their college experiences. Sometimes they're told to go to startups. Other times, go get jobs at big companies, go to graduate school, maybe get into government. What were the messages that were sent there?
John: None of those. I really felt like it was...I think it was a good place if you were really engaged and you were pushing yourself, but I think it was a lot easier to slip by than it was at Illinois. I think Illinois, in a lot of ways for the undergrads, was very much a pressure cooker whereas at Connecticut I felt like there were hard classes and a lot of people who started didn't graduate, but I don't think there was really a driving force.
I think there was a much larger range of capability of student at Connecticut than there was at Illinois, but there's a large spread at Illinois, too. I think most people didn't really know what they wanted to do. I don't know. I think that's part of going to a state school that's not particularly well known for engineering. I'm not trying to downplay the quality of the program. I'm just saying that if you're Illinois, people know Illinois is an engineering school. It's just like Berkeley's known as a really big engineering school. If you go there and you're an engineer, well, you're in for a pretty rigorous education.
But at UConn, I think a lot of people just headed up there because, "Oh, you know, it seemed like something good to do. I'm good at math. I like hacking on computers." But from there, I never got a feeling like there was a strong culture of, "This is what I do after I graduate."
David: For the curriculum there, was it pretty project oriented or was there a lot of mathematics and theory. What was it like?
John: Very theory oriented. I would go many semesters without really writing much code. It's just very theory oriented. A lot of people would pooh-pooh this and say this is a bad thing, but I think it really prepared me for my post-graduate education.
David: What kind work experience did you have? Did you ever co-op or intern?
John: No. I don't think that we had really good connections for co-oping and interning. I applied at a couple places but nothing ever came of it when I was an undergraduate.
David: What did you do during your summers? Did you work?
John: I worked.
David: Oh, OK. One thing I want to bring up is you've done a lot of jobs in your life, among them you were pretty involved with transpo, transportation at UConn. We've had a lot of talks over the years about how that's prepared you for things later, like working your way up through the ranks and learning how to manage people. Can you comment on that a little bit?
John: I was in an interesting situation when I started college. I was pretty much on my own to pay for it. You can take loans out, but I never felt like that was a good option for me, so I applied for jobs working for res life and IT infrastructure for the university. They all denied me. [laughs]
John: I ended up working for the bus company because it seemed to have a very high value to effort ratio. They paid quite well. The one thing is that you had to get a commercial drivers license to do it. And so, they put you through a bunch of training which actually...This is a model that I've come to appreciate, having gone through graduate school and working in large companies, is that you have to vet your employees before you actually hire them.
And so the way it worked is you had to scrounge for hours. You made less money. You trained and eventually you passed this test so that you could get a license and you could drive busses on campus.
David: Before you were driving, you were working as a mechanic, I think?
John: No, no, no, no, no.
Basically you would train to become a bus driver and then I became a bus driver. That actually caused a lot of people to wash out. I don't know what the ratio is, but a large portion of people never really made it to become bus drivers.
Anyway, after being there about a year, I became a manager; I was assistant manager of preventative maintenance. I was in charge of maintenance for the busses, dealing with the mechanics on campus.
David: You were young, too. You were 20 at the time or 21?
David: You were an assistant manager at 19, you were managing people in their 30s...
John: 20s, yeah.
David: 20s, OK.
John: It was a good job. I liked working with the mechanics. They're a little bit more blue collar than a lot of the other people I worked with. I felt a little more affinity for them and I respected their position. There was often a lot of friction because you've got a bunch of college students that want to go out, and go boozing on the weekends, they're not that responsible. Though, I think we hired a lot of reasonably responsible people.
Then you had state employees that are, "I've got an education so the state takes good care of me. I don't get my hands dirty." And then there are the motor pool guys that are like, "We turn wrenches and make sure that things work." I respected those guys a lot. I think that's why, it was one of the things that made my job easier, is that I showed people that had skills respect and I continued to take that forward. People that have got skills, you want to treat them with respect.
David: You help them to get into positions where they can contribute, right?
John: Yeah, exactly. And just isolate them from things that they don't want to deal with.
David: You went through all this, eventually went to graduate school. I guess the first question is why?
John: Why go to graduate school? I felt like, when I was coming closer to graduation at UConn, that I didn't have good job prospects. I wasn't really sure what I wanted to do with a graduate education. I wasn't really thinking.
David: What year did you graduate with your undergrad again?
John: What year?
John: 2005, which actually, timing is kind of interesting. Speaking of transpo, I was hired for transpo September 11th, 2001.
John: I was sitting in the office filling out paperwork and the hiring manager's like, "Planes just flew into the World Trade Center."
I was like, "No, that didn't happen."
He's like, "Yes, it did."
I thought it was a total joke. The whole campus was cleared out. "What's going on?" I had been filling out paperwork this entire time.
Everybody's like, "Go back to the dorms. Don't go outside. We don't know what's going to happen next."
I feel like that actually set the tone for the next couple years of the world. Entering college at a time when something so traumatic for the country was happening is kind of interesting.
A lot of guys that I went to school with in high school all went into the military. "Oh, you know, it's a good job. We have these wars. Nothing really happens. You sit on a base for couple months. No big deal." They ended up going to places like Afghanistan and Iraq. One of my girlfriends in college, her brother went to Afghanistan. Lots of good friends of mine that I went to school with ended up in Iraq.
I remember them coming back afterward. In particular, I remember one incident coming back from the summer of 2001 or 2002 I suppose it would be, after my first year of college, and just talking to some of the guys I went to high school with telling me about the things they saw in Afghanistan. It was unreal. Coming from a college campus, it's like people talk about the Vietnam war, so disconnected from what's going on. This guy's talking about, "You've got to watch out for little kids with grenades." Crazy stuff. It blows your mind.
These are guys that..."College isn't my thing. I'm going to go the military route." And then it blew up in everybody's face. It set the tone for the next couple years.
I graduated in 2005. Again, it's kind of like, "What's going on in the world? We're still embroiled in these wars, but the economy's cooking." Everything's really exciting.
David: Shock and Awe was in 2003. That was about two years into the U.S. being in Iraq.
John: The thing is, it was kind of like an afterthought by the time I graduated. People just don't think about that anymore. It's like, "Oh, we were just there."
Ravi: 2003 was when the U.S. invaded Iraq.
David: One other comment you made, too, is that at different universities you get more heavily recruited by a lot of places. At Crittercism, where I work now, we have three interns. They're all in their 20s, and Kleiner Perkins and lot of the other venture capital firms came in and pulled these kids out of college from MIT and Cornell. Again, it's just access to opportunities can be very different depending on where you go. But it's also that a lot of companies don't have the budgets to visit every school in the U.S.
John: It's totally a value proposition, right? It creates a niche for some of these companies to come into a place like UConn or some school where they have a good alumni association or relationship and they can hire whoever they want.
I think, wherever you go, there's going to be a bell curve. There's going to be some distribution. I won't say that it's necessarily bell shaped, but you have some sort of distribution of skills. What you really want to do is maximize how many quality applicants you get.
If you go to a place like Illinois, if you go to a place like MIT, even though there is still a distribution of skills, even the median student is really good. You go to a place like Illinois, the median student's OK. You go to some small state school somewhere that doesn't have a well-regarded program, the best students are still really good.
David: Yeah, but at the same time, there's going to be less competition. I guess it just depends on how you want to run your recruiting.
I guess, coming back to your PhD, what was your motivation? You said that there wasn't a lot of recruiting going on. You didn't particularly care for the opportunities that you were offered so it seems like you thought it was the best choice at the time.
John: Yeah. I applied to...Let me see, where did I apply? I don't even remember where I got in anymore. I think I got into more than one school. I think I got into Georgia Tech and I got into Illinois.
Ravi: When you were applying for your PhD, what specific area were you interested in?
John: The funny thing is, I feel like, for all the things that my university education provided for me, knowing what I was talking about they did not provide. I had no idea how to talk about what I was interested in, what I should be interested in.
I look at undergrads that we had working with us at Illinois and they had a much better idea of what graduate schools expected, what faculty expected. They had much better connections with the faculty, much better connections with senior graduate students.
If somebody was willing to keep coming back and talking to us...People would come and we'd talk to these undergraduates when we were PhD students and tell them how the world worked. If they needed help writing recommendation letters, writing statements of purpose, dealing with their applications, we gave it to them. I didn't feel like I had that when I was an undergraduate.
David: When you got to Illinois, one of the important choices that a lot of graduate students initially face, is that they have to pick an academic advisor. I know you eventually ended up with, obviously, a full committee. I think Steve Lumetta was your graduate advisor. What was the process there? Was that quick? What happened?
John: I met Steve [Lumetta] in March of 2005 at a recruitment weekend.
David: Right, that was before you started.
John: Yeah. So that kind of sealed the deal for me as I went out there. As I was saying before, I got accepted to two schools. I got rejected from one school, Princeton.
John: I'm not Ivy caliber.
John: I went to recruitment weekend. This is actually kind of a funny story because I went from Connecticut in March and it was freezing, I mean, it was close to zero Fahrenheit.
David: Just for the benefit of the listeners, this is a PhD admit tour, right?
David: So you're going around and all the schools that you've been admitted to, invite you to come.
John: The top quality students are invited to come. A lot of times they'll take the mediocre students and not invite them and they'll sort of fill up the Master's ranks. But the people they really are actively trying to recruit they invite out.
Ravi: So at this point, were you already accepted?
Ravi: OK. Before we get to the PhD tour really quick, could you describe to us the admission process, what all you have to submit to get accepted into a PhD program?
John: Oh boy, I don't even remember anymore.
Ravi: For the Bachelor's it's pretty standardized. You have your SAT or ACT scores. You have a high school GPA, and an essay.
John: Yeah, so this is actually another interesting story. You have to take your GRE's for most graduate schools. It's like the graduate equivalent of an SAT standardized test. You take it on a computer.
I went back to Norwalk, where I'm from, to take it and stay with my mother that night and the next morning I go to take it and it's like the last day I can take it before the admissions deadlines for the schools.
I show up at the place at eight in the morning and their computers are all failing and they're having all kinds of problems getting everything running. I'm like, man, I'm not going to be able to take this. He's like, "Oh, we can reschedule for December." I'm like, "December is two months too late, that's not going to work." I'm like, super stressed out. This went on for about an hour and a half or two hours. They finally got it working and are like you can take the test now. I'm like "Great, after all this stress, right?" I go and take the thing right. I kind of know what I would get on the exam because I'd taken a bunch of practice tests, like I really worked pretty hard at it. I spent a bunch of time that previous summer just going through the examples.
David: Very prepared.
John: Yeah, I felt very prepared, but I think you can only prepare yourself so much for these things. So I take it and the nice thing about, or maybe the bad thing about taking one of these computerized tests is they tell you your score at the end. So I'm going through and I'm taking the test and of course I was all stressed out because they almost told me I wasn't going to be able to take it.
And I get a really good score, and I almost like, kiss the monitor and walk out of there. [laughter]
John: I think that had a lot to do with my getting into Illinois. There are certain winnowing functions, just certain things, cut offs, and if you don't hit that cut off you don't get a good opportunity, so getting a good score really helped a lot.
David: OK, letters of recommendation obviously, too.
David: So you were on the PhD admit tour. It sounds like there were a couple of faculty members that had already looked at your application by the time you had gotten there, and you were being recruited directly by them.
David: What kind of research were you looking to do when you first started? Did you have a specific idea?
John: No idea. I kind of liked architecture but I didn't even know what it really meant.
David: Computer architecture again, chips.
John: No, not even chips, just computer system design. I wasn't a coder. I wasn't really a semiconductor guy. I was not a circuits guy. I was, "How do we put these big pieces together to make something work?"
David: I see. So one of your first papers, if I remember correctly, was on optics.
John: Yeah, when I was an undergraduate.
David: What was that one about?
John: Again, in high school you sort of need these check boxes to get into a good college, there's these check boxes you need to get into a good graduate school.
And they're not for no reason. If anything, the fact that you say, "OK, you're not going to get into graduate school if you don't do these things" is a good forcing function for a lot of students that are otherwise not well focused to go and do some things that are actually good for them. For me, it was like OK, I need to do some graduate research. And this is an interesting sort of agreement between faculty and undergraduates that "OK, you're going to come work for me. And I'm not going to spend any time on you, but you can work with my graduate students and maybe find a project for you, but you might just end up doing complete work on scripts or just..."
David: Right. Pretty low level work.
John: Pretty low level work, but you get exposed to the process. I think often in life, understanding the process is more important than the actual details. You don't need to be a genius in optics to get papers done in optics. I don't know that much about optics, and certainly not anymore, but I spent a bunch of time with the grad students and I spent a bunch of time with a faculty member, Bing Wang who was a new faculty at the time so he was invested in finding students. I think for new faculty they end up recruiting a lot of undergraduates because they are willing to work for them and don't require a lot of funding and there's low risk there.
Ravi: So even though you are doing low level work it essentially showcases an interest in research, which I think, as you said, is a forcing function.
I've seen this with my applications, too, when I was applying for Master's. Some universities that I got rejected at, I essentially figured out that just because of lack of research on my part or lack of publishing papers, not getting my name out there on research papers. That cost me.
David: It shows initiative, too.
John: It shows initiative. I think I happened to be in a good situation where my needs and the faculty's needs were well aligned. He needs to publish papers, he doesn't care what they are, and I'm an undergraduate, and I'm willing to work on stuff, and that meant I got papers. Whereas if you work for a more senior guy the graduate students are the ones getting the papers. Maybe you might get third authorship or something, which is still really great if you're...
Ravi: An undergrad.
John: An undergrad, sure. And I got to travel. I got to go to two conferences when I was an undergraduate, which was an interesting experience. Though I have to say I had no idea what I was doing when I was there.
David: Sure. Talking about the incentives in academia, it's interesting. You said the graduate student wants to get papers published, the professor also wants to get papers published, so what are people going after? In other words, what sets you apart?
John: What sets me apart personally?
David: As a professor, as somebody that's taking an academic career.
John: Apart from what? Why do you...
David: Apart from their peers, how do you excel?
John: At being a good faculty? I think some of it is chance. I think junior faculty are an interesting position. I think being someone who...
David: By junior you mean someone who doesn't have tenure yet?
John: Right, being an assistant professor. It's kind of interesting for me because I'm of the age range of an assistant professor, people that graduate with the end of...
David: Late twenties, sure.
John: No, I don't even mean that, I mean I'm a couple years out from graduating, and I had that option, that I could have gone the assistant faculty route instead of going to work for Intel.
I think it's an interesting opportunity in the sense that you have at least five or six years, depending on the tenure sequence for you. You can do whatever you want, you get a small startup fund, but as soon as you start you need to start getting papers published. My experience as a graduate student is that even good ideas take a year or two to get published just because there's a certain amount of randomness to the process.
David: So for a faculty member they'll start, you're saying, they'll get a little bit of money like when they first join the university, they take an appointment.
David: They have to produce some results, right? So again, what are these results, what are they going for?
John: So what's interesting is that new faculty, the advice I've gotten in my own experience is that you should never do something conservative.
When you first start you should be swinging for the home run because you have this interesting opportunity.
Also, doing something completely different than what everyone else is working on is often...a less well tread area, so it's easier to publish. The publications may not be as good or may not end up being interesting in the end and might not get as much visibility. But you also have that small probability that you do something that's just groundbreaking and that's kind of fundamental for the whole entire field.
The problem you run into is many of these things go through this arc right, where the initial publications are complete garbage because they had no idea what they were talking about, but they had this kernel of an idea that kind of the middle age publications figure out, "Oh that's the right thing to do, and we're going to do really rigorous work around that", and then eventually it just tails off and it's all incremental.
For new faculty I think it's all about trying to sow the seeds of a new area and do something really interesting and different. The problem there is you run the risk of either doing something that ends up taking off during your tenure process, or it just ends up being this thing that lives off in oblivion and ends up not really going anywhere, like, "Who's actually looking at this"? This is completely synthetic.
David: That's why you talked about chance, because it sounds like chance does play a fairly large part in this.
John: It does. I think it actually feeds back into the university process. That is, the most senior faculty are the ones that don't want to take risks. They just want to keep the funding coming in, run a big lab, work on things that only big teams can work on.
They're often the ones running administrative positions. They're dealing with a lot of bureaucracy.
David: Provosts, department heads and people like that.
John: Some of that but also just being a really senior faculty member. It seems like they spend a lot of their time on advising committees. They spend a lot of time being an adviser, that kind of thing, to outside interests.
Ravi: From my experience as a graduate student just observing the academic process in general, it seems like obviously the kind of conferences you publish to also makes a difference. There's obviously ratings of conference, not ratings, but some conferences are deemed...
Ravi: Well these aren't official ranks, but some conference are deemed prestigious while others aren't. It seems like if you are an established professor, if your name is on a paper, you may not be the first person on the paper, but if your name is on the paper, you get accepted in this conference, versus if you're an unknown assistant professor, you might get screwed out of this prestigious conference. How do you fight that?
John: Well, most of these conferences are double-blind reviews. You have no idea who's reviewing you and they have no idea who you are, which has its ups and its downs; that's maybe a different conversation, but basically you can't evaluate somebody's work based on who they are, which means there's a lot of people who produce work that's subpar and they can write it in a way that you're willing to believe it so they end up getting in because they know how to work the system.
On the flip side, people can say oh, I kind of know who this is and this is really bad work so I'm just going to say this is bad. I think, as new faculty, one of the interesting things a faculty member who didn't get tenure once told me is one of his biggest mistakes was not pushing his junior students to publish at less prestigious conferences because he felt like the process of publishing anywhere and presenting anywhere was just lost on them
So they were always swinging for the home run, when what they just needed was to get on base. It also builds a lot of confidence. It gets your ideas out there. It allows you to move on.
If you have a paper, don't give up on it. If it gets rejected from one conference, you try and resubmit it.
David: Some kind of idea, right?
John: Yeah. But if you end up getting accepted somewhere, then you move on to something else.
David: Sure. Even though publishing is a major focus, we've talked a bit earlier about the role of teaching in graduate education.
David: And often how the professors spend a lot of time trying to really mentor and build their students. If they aren't getting good research output, it's often not productive for them. So can you comment on that a bit?
John: Like how some students aren't necessarily productive?
David: Not some students, but some professors. Because the complicated thing about a research university is there are a lot of different incentives that are at work and they don't all align with each other.
You have undergraduates that are theoretically looking for a good education and some kind of a degree. You have professors who need publications in order to build their careers. Then you have graduate students and they're probably trying to get some PhD or maybe start their careers themselves. It's interesting to me how these all interact.
Can you talk a bit about that, about how sometimes teaching gets lost in the mix, I think.
John: The problem is that tenure committees now...I don't know anything about what tenure committees actually do, but what I've heard secondhand is that the research you do defines whether you get tenure or not. With rare exception does teaching matter in the tenure decision.
I think there's a sliding scale. I think at a tier one research university, research is really all that matters. I think at a more teaching focused school, not necessarily a bad school, there's just a lot of teaching-focused schools that just don't produce PhDs. They're masters only or only undergraduate.
Ravi: Like Rose-Hulman.
John: Rose-Hulman's a good example of this, where teaching matters a lot more. I think it's interesting. My advisor, is very focused on teaching. It actually made him a better advisor in a lot of ways.
David: It's funny you mentioned that, John, because I was just looking back as Steve Lumetta's lecture notes on signals the other day for my job, actually at Crittercism, we were doing something with signal handling in mobile devices. I still look at that stuff today.
John: The guy's just a really great teacher and focuses a lot on teaching. For him, research and education—I don't want to speak for him—but I think the two are inseparable in the sense that research is a process of educating the student. I learned a lot through my research but I also learned a lot through teaching.
I think it depends on what the university's charter is. In some ways, universities are a lot like businesses. It's about, "OK, we can get lots of undergraduates in, paying $30,000 a year to come here. And then we're going to hire faculty that are going to get a million dollars in grant money coming in so they can hire a bunch of graduate students." It's all about, how much money can we get coming into the university?
But I think the charter of the university is also to produce really good students. To produce really good students it's not just about how much funding you can pull in. I think probably get enough funding so that your students don't have to worry about...stipends are important, but also giving them the opportunity to teach, I think, is really important. It forces them to interact with undergraduates.
If you're not from the university, like I didn't do my undergraduate at Illinois, working with undergraduates my first year gave me a much stronger connection to the university and I got to see how the undergraduate curriculum worked. But it also forced you to understand fundamentals. I think there's a lot of classes you could teach that.
David: Disciplinary fundamentals. Computer science fundamentals or electrical engineering fundamentals.
John: It's frightening, standing in front of a bunch of students in a first year class and you know that at least two thirds of them know more about the material than you do.
David: It's just been a while, right?
John: It's not even that it's been a while. The classes at Illinois are really hard! [laughter]
John: It's also that, for them, this is the hardest class that they've ever taken in their lives! For me it was like, I was taking three graduate classes, trying to figure out research, and trying to understand what I'm doing. For these guys, it's like, "Oh, this is the hardest class I have so I'm going to work on it a lot."
Ravi: That's precisely the reason why I chose never to be a TA, because of this.
David: John, you and I both have been TAs, so I've had this experience.
John: We TAed together. That's how we met each other.
David: Yeah. In fact...Yeah, that's true.
Ravi: What course was it that you guys TAed together?
David: It was the Introductory Computer Systems Engineering course.
Can you talk about some of the technical stuff that you worked on in graduate school? Obviously, you started with optics. That was undergrad. You came in. You were working with Steve. I think that was in the IMPACT group or which group?
John: No, that's Wen-Mei's group. It's actually interesting.
David: Shane Ryoo, one of our former interviewers, is in that group.
John: He was.
One thing that's kind of funny is I worked in optics, and Steve had students working in optical communications networks. I did not want to continue doing optics when I went into graduate school.
Ravi: Was there any pretty good reason why? Because you did optics undergrad. Was that something you did not like?
John: That was opportunistic. It was that there was a professor that I was taking, it was Introductory Circuits or something. Some basic class that he was teaching and he just said, "I want undergraduates to work with me."
I was like, "Oh, I'll try something different." I think one thing that's defined a lot of things I've done in my life is taking advantage of opportunities when they're there. I never say, "Oh, that's not me." If it's close enough and it's a good opportunity, I'll give it a shot.
Doing optics as my undergraduate research is one of those things. It taught me a lot of important things, but it also gave me a lot of opportunities that I wouldn't have otherwise had if I didn't go that route.
David: Computer architecture, though, is really the big focus, right?
John: Yeah. When I moved into graduate school I wanted to work on architecture. I think I fell more into it during my first year of graduate school.
I didn't quite know where I was going. Steve was working on a project at the time related to...I don't quite understand how it all fit together, but there was a large research consortium that was funding cross-discipline research between circuits, people, architects, application people, compiler people called GSRC. That was funding me for the first couple semesters I was there. Actually, the first two semesters I had a teaching assistantship and after that I was on GSRC money.
David: I see. What were some of the projects you did? I remember Rigel was big.
John: Rigel was actually my second phase of graduate school. My first phase of graduate school I worked a lot with FPGAs, reconfigurable logic.
David: Field programmable gate arrays. They're used for prototyping electronics a lot.
John: It is. There's also a belief for a long time that they could be used for computation. You have a problem that can be mapped efficiently to FPGAs where you have loads of bit level parallelism. You can do something faster and more efficiently.
And you see this. FPGAs go into base stations for cell phones where they're just doing bit manipulation for data streams. It's not worth making a chip for it.
David: A bit of background. You have a semiconductor. It can do various things. The FPGA is a custom semiconductor. From what I understand, they have a bit higher unit cost than fully fabricated semiconductors. Can you talk a bit about that?
John: A FPGA is basically a sea of programmable gates made up of little clusters of SRAM look-up table, some logic, and some connections to a communication fabric basically.
Basically you have CAD tools that let you express circuits that can either be mapped down to an actual silicon chip or you could map them onto one of these field programmable gate arrays, an FPGA. It would be much larger per area but you can reprogram that thing whereas when you want to build a chip, you've sent it off to fab and you come back with a chip weeks or months later and it costs you millions of dollars.
David: And it can't be changed, right?
John: Right, whereas an FPGA can be changed. Often you'll see FPGAs as glue logic for other chips. You build some big ASIC and you'll hook it up to an FPGA. The ASIC does 99% of the stuff and all the malleable things you do in the FPGA.
There was a belief for while in the research community that people would really want to use FPGAs for computation.
David: Special purpose computation?
John: Special purpose computation. It would be like, "I want to do MPEG decoding. I'm going to set up the FPGA to be a really good MPEG decoder."
David: Video decoding, right?
John: Video decoding, encoding media.
I think part of the reason why people focused on this is because these are very bit level, parallel things. Data parallel things that map all the FPGAs. But I think the community moved away from that because not that many problems mapped well to that platform.
David: Was the idea that you would have a single FPGA inside of your computer and then you could reprogram it as necessary?
John: Potentially. One of the platforms we worked on was built by this company called Xilinx that builds FPGAs. It has an embedded processor in it, so it was a PowerPC processor and it ran a full Linux stack. We worked on getting that Linux stack working on the PowerPC. You could create encoders or decoders or any sort of special-purpose logic logic in the FPGA fabric and connect it up to a fully capable computer system.
My master's thesis was actually based on building an interface that lets you plug in reconfigurable accelerators for different applications. One of the examples was a JPEG encoder. You would have all the blocks that represented data parallel computation in the FPGA and the rest of it would be on the CPU. You could reconfigure on the fly to have these pieces in the FPGA logic and you'd see a speed up.
David: So speed up was the big advantage you're saying?
John: Energy efficiency was part of it, too.
David: It's interesting to talk a bit about the tradeoffs in semiconductor design. I know a few of them power is really important. Speed is really important. What are some of the other ones?
John: Power, speed, area, complexity, cost, which is often related to area and time-to-market. But, energy and power can be two different things. I think it's important for people to realize.
David: Can you elaborate a bit on that?
John: You know, like TDP, total dissipation power is one of these things that people think about. That's the max power that something can run at. If it runs at that, it's going to overheat or it's going to burn somebody. There're sort of different metrics for TDP.
David: And, that can get pretty high, right? So, you think about a light bulb with 100 watts.
David: You have a little semiconductor. It's what? One square centimeter...
John: Yeah, it's like a postage stamp.
David: ...throwing off 100 watts.
John: I think we've moved away from 100 watts these days. [laughter]
John: I don't know what you can buy. But, I think the high-end server parts are maybe close to 100 watts. But, a laptop part is going to be 20 watts, 25 watts.
David: Total dissipation. So, that's the highest it's ever uses.
John: Yeah. And, if you want to be fanless, you're talking, I don't know, one to three watts. I don't really know exactly what it is. But, that's like TDP.
The other question is, energy dissipation. So, that's power. Energy is more a question of, "OK. I need to do something." And, maybe there's some real-time guarantee like, "I need to decode a movie. I need to decode one frame every thirtieth of a second. It doesn't matter if I power up my machine and run it for a microsecond, or I power it up and I run it for one-thirtieth of a second", as long as I do it before the user notices that there's a problem. And, the way that mobile devices want to work is they want to do it the most energy efficient way possible because they have a limited energy supply which is the battery. So often, it's better to do something that's slower, but more efficient, as long as it meets the constraints of the system.
David: Right. And, I think that's a big departure from how things were done even a few years ago.
John: Yeah. I think moving to mobile devices, in particular phones, has moved people more towards the focus on energy efficiency. Power did matter in desktops and the datacenter because there's only so much wasted heat you can dissipate with air cooling. So, if you want to go beyond about 150 watts, you're talking about liquid cooling. And, that's just the non-starter for most people.
David: For mobile devices certainly.
John: Well, even for your desktop computer, you don't want to have some sort of glycol-chilled kind of thing going on.
David: That's ridiculous.
John: Yeah. Well, if you want to go above 150, 200 watts, that's where you're looking.
David: I see. So, can you talk about some of the other stuff you worked on in the graduate school? I'm just prompting you here because I think it's actually quite interesting work. You did a lot of stuff with cache coherence. So, can you kind of talk about that problem a little bit?
John: Actually, it's kind of a side story. So, after I finished my master's, I kind of had a dead semester. I finish in the winter of 2006, so after a year and a half. And, I had kind of hit a dead semester where I was like, "Oh, I don't know what I'm going to do my PhD on. I don't think this is going to go much further. I don't really want to keep working on it."
David: The FPGA stuff.
John: Yeah. I tried a bunch of different things. And then, I got an internship working at AMD for the Summer of 2007, which was really good for me.
David: Those were in Sunnyvale.
John: Yeah. It was in Sunnyvale, which is funny because it's not very far from where I live today. I live like, three blocks away from where I lived when I was here in 2007—small world. But, it exposed me to a bunch of different things. What I worked on at AMD had nothing to do with what I work on now, nor does it have anything to do with what I worked on previously in graduate school.
I came back to graduate school in Fall of 2007, and was like, "Well, I need a project to work on." And Sanjay Patel, at the time, was like, "Oh, you know, I have this idea for this Rigel thing." So, he'd been coming off of working on Ageia...
John: Ageia's PhysX processor. But now he's back at the university and wants to do something else. And said, "Oh, you know, this Rigel thing is something I think is really great." I'm like, "Oh, it sounds kind of interesting." So, I started working on it, building up a bunch of infrastructure.
Ravi: So, what ever happened to Ageia?
John: They got purchased by NVIDIA.
Ravi: I see. OK. NVIDIA now actually makes these PhysX processing units?
John: I don't think they make any chips. I think they mostly bought them for the libraries that they build. I think the office is still there in St. Louis where Ageia was based. So, they still have all the guys that worked on their other Ageia stuff.
So, it was interesting because it's something that happens to graduate students that it's hard to do research, but it's easy to do infrastructure because it's just kind of a well defined task. It's engineering. It's like, "I need to build something." And, it's well defined. I know what it is. Even when I finished it, I can kind of tweak this thing. I can build some sort of Flash-based interface to my really neat technical gizmo that I built in some other crazy language. [laughter]
But, it's true. The problem is that it's deceptively easy. And, you don't end up working on...
David: Important problems.
John: ...important problems, where infrastructure is really just a means to an end.
David: Right. So, I think anytime you're doing work, it's important to identify what the goals are.
John: Exactly. And for research...
David: Infrastructure is not the goal.
John: Infrastructure is not the goal. Very few graduate students get their PhD on infrastructure. But, it's fun to do. It's easy. It's well defined. But like, research ideas are just undefined.
John: If they were well-defined they wouldn't be research.
David: And, I think one of the things you did infrastructure-wise was you were working on this architecture called Rigel. And, it was totally new. So, taking all the GNU binutils.
John: Yeah. I mean, we...
David: So, you built a compiler that targeted this architecture that didn't even exist in silicon.
John: We targeted LLVM to our architecture.
Ravi: So, what was the purpose of Rigel from a basic point of view?
John: The idea was that you would build a system that had just, massive amounts of parallelism, lots and lots of processors. And, they'd be quite different from what people where building at the time which was using GPUs for doing massive parallelism.
We were going to have a different memory model, we were going to have a different execution model. So, GPUs tend to be more SIMD where you basically have a lot of threads that kind of execute in unison, whereas we wanted to allow threads to just execute in any arbitrary order. And, the idea was it would be much easier for programmers to target.
I don't know, it was an interesting vehicle for doing a lot of research. I don't know if the idea in the end was a good idea, but I think it's kind of irrelevant for a PhD student. If they just let us ask a lot of interesting questions in which we built up the infrastructure, we could actually do that.
David: Right. This is a very ambitious project. I think modern CPUs typically have anywhere from, what two to four, maybe eight cores tops. And, you guys are trying to build a hypothetical CPU with like, a 3000 core...
John: 3000 cores, yeah. Basically pushing the limits of silicon technology.
David: Yeah, I was impressed by the level of infrastructure support for this. Again, we talked about why infrastructure is not always great. But, you built a complete simulator for this. You had a chip that was hypothetical, like it didn't actually exist. But, you built an entire software system that allowed you to run code on it as if it was a real computer, right?
David: Can you talk about that a little bit? That's a big achievement.
John: Well, it wasn't just the work of me. I mean, they're other graduate students that joined the project after a while. I was sort of the only student working on it from maybe six months, nine months. And then, one or two other students joined.
David: Yeah. I mean, it was a huge software project.
John: It's boot strapping.
John: It's boot strapping. Basically, nothing works until something works. And then, you can sort of move from there and iterate from there.
I think the first steps that I took was building a single core simulator for the ISA [instruction set architecture], building a binutils so that you could compile binaries written in Rigel Assembly to run on.
David: So, that's like objdump...
John: Yeah, no compilers. No compiler at this point.
John: And then, writing a couple of simple apps to test the thing. And then, the next phase was, "We need a compiler because we don't want to write all of our code in assembly"
John: So, I retargeted the LLVM which is another Illinois project, to target Rigel, then, worked on a parallel version of our simulators. We took steps. So, it was like going from one core, to eight cores, to basically 1000 cores. And, we picked up a couple of tools along the way, more benchmarks.
And then, one of the interesting research projects I wanted to work on was memory models and coherence. The problem with these projects is that you can call yourself the architect of the project, but you're not going to really get a thesis from that. Rigel was not a thesis. Rigel, maybe it was a startup opportunity, maybe it was a vehicle for research. But, it certainly wasn't a thesis in itself.
David: Right. At the end of the day, it's the problems that you discover that matter, right?
John: Right. Right. And so, for me, I got to a certain point where I thought, "Well, this maybe could become something."
There have been examples in the past of graduate students working on some chip project and said, "OK, I want to do a startup from this" And, they end up maybe not finishing their thesis, or they do and it's not a very academic thesis. And, it's good for them.
David: Yeah. Just talk about that a bit more. You know, when you're trying to ship a product like out in the commercial world, the emphasis is really on shipping, right, on making the thing and making sure it works.
David: So, what is that with academia? It's not necessarily about building the whole system.
David: Like, what's interesting?
John: I think it's not about building the whole system. It's about pushing yourself into the corner of the design space where you contribute. I think any company in the valley could build a way better design than we ever could.
I think there is the possibility that nobody was actually trying to build the thing that we were trying to build. So maybe if we built it, even if it wasn't so good, it would have been compelling enough that we would have gained some interest from somebody.
But I think academia is about defining problems in a particular way, and then executing on them. Part of the problem is that you need to build up all this infrastructure to get to that point. We were looking at a research area that wasn't well-treaded, and so we had to build up a lot of our own infrastructure.
Actually, it's been released on Github, since then. Danny Johnson, Matt Johnson went and released this thing, so anybody can use the infrastructure that we built; but I decided at this point, I don't see where this is going as a project, other than for me to use it as a vehicle for me graduating. So, I focused on memory model and cache coherence for thousand core processors and got some school papers out of it.
David: So, cache coherence. Can you talk about what that is, because I think it's a very tough problem, but it's very interesting.
John: Right. The problem is, people writing code, when they first approach writing code, think about it in a sequential instruction stream. A happens and B happens and C happens, and maybe I access memory, and the access of memory happens in a single sequential order.
As you add multiple course of the situation, you have many processors, many streams of execution. And if you're using a shared memory model of execution, they all share the same memory. And so, if processor A accesses memory location X and processor B accesses memory location X, how do you define the interaction? What values they see? That's kind of coherenence; how do you see a single sequence of updates to this value, in memory?
David: So, there's one value, but there's two things that can be reading and writing to it at the same time, right?
John: Exactly. The problem is, to speed up processors you give them these local memories, where they're basically caching things; that means that they can access it very quickly, but if somebody from one of these other processors wants to access it, it's a bit slower, so you need some protocol to make sure they don't get some stale view of memory.
David: Right, because they each have their own copy of memory.
John: Exactly. So you need a protocol for keeping that memory consistent—cache coherence.
The other aspect of it is, even with a cache-coherent system, you can have different interleavings of memory as viewed by third parties. If you have three processors in a system, A and B are writing things in one order and reading them in one order, how does processor C see that?
A lot of what my PhD work was on, was how to reason about that in a system that didn't have hardware managing that protocol, but had software managing that protocol, and also, if you had heterogeneous systems where software managed some part of it, and hardware managed another part of it, how did these things interact with each other?
John: That's quite complicated.
David: Eh! [laughter]
David: What else did you do in graduate school? You said cache coherence was the big thing. A little bit of optics. What else?
John: Beer brewing.
David: Beer brewing, of course. That's actually worth talking about for a minute because it's a social thing. You have a bit of people over. It's important. Get people out and meet them, and maybe create a little bit of a separate environment, right?
John: Yes, I think in graduate school networking is really important. Just meeting people and experiencing different things.
David: I think something you said too is that it's kind of, "total war", graduate school. You sit in the evening and on the weekends, and you think about these problems. They don't ever leave your brain.
John: No, they don't ever leave your brain!
David: Maybe you're sitting there on a Saturday night, and you're drinking beer with your buddy. You're going to have some realization. You're going to discuss it, right?
John: Oh, yeah. I think one of the great things about my undergraduate career, and also my graduate career, is that I associated with a lot of people that were not engineers. I got a lot of different perspectives and views. I kept from being a part of a monoculture, "This is the way the world is."
I've noticed being out here, the Bay Area, San Francisco, it can be a lot easier to be stuck in this echo chamber where it's like everybody thinks the same way. At least they may say that they conflict, but it's on these kind of minute things. Whereas, if you're hanging around with a bunch of people that study art history, or physics, or maybe they're not even in college, maybe college doesn't interest them, you just get a much broader perspective on things.
Ravi: I'm not sure about that. Here we are, three people from the Bay Area. We work in companies in the Bay Area, and I think we work in very different sectors of computer engineering or computer science.
John: Sure. I don't think we necessarily agree, but again, I think it's just the difference. It's just how big the gradient is, I suppose.
David: Yeah. There's a lot of stuff that I'd hoped to get into. I don't think we'll have time, but one thing I did want to bring up is it's a very large company. There's a lot of different time zones, cultures, and geographies, and I think a big part of your job is to go to these different places, whether it's in Barcelona or Israel, or any of these other places, and just be an advocate for best practices and to reach consensus on things. Can you talk a little bit about that?
David: It seems very difficult.
John: It is, it's hard. I work with a lot of guys in Spain. I have history, worked all over Europe. Fortunately, I haven't had to do both Asia and Europe at the same time, which would be pretty difficult because they keep completely different schedules.
Ravi: You'd have to be up at all times of the night.
John: Basically, but it's very interesting, because the way people work in America and the United States in general is a little bit different than the one people work in Europe, which is a little different than how people work in, say, Israel. I've worked with some guys in Israel.
I think it's taught me how big the world is and how small the worldview of a lot of people, and how even my view was previously.
David: Yeah. You travel a lot. You probably leave the country once a month.
John: I don't think it's that high, but I've been out of the country maybe three, four, five times a year, every other month at the most. I've come to really appreciate face time with people you work with. I think it's really easy to think there's an "us versus them" thing if you don't actually go and meet with them, talk with them, see how they live, see what they do, and be there.
David: It's harder to hate somebody that you actually know. [laughs]
John: Yeah. There's a lot of guys I work with that I'm really good friends with in Spain. And until I went out to have beers with them, it broke that barrier. Now they come here, I go there, we chat, we talk about things, we argue.
That's the thing. You can't be in a company building important things that require a lot of time and effort if you're not able to argue efficiently.
Arguing is necessary. It's how you cut out the bad ideas, focus on the good ideas, and push the right agenda forward. I think if you just argue in circles, or if you've failed to argue, you're in a bad spot. What you need to be able to do is argue with each other, say, "This, we're going to do," make a decision, and move on.
And respect each other at the end of it. It can't be about, "Dave thinks this and Ravi thinks this."
Ravi: Coming back to world culture at a company like Intel, could you comment on the culture of innovation at Intel versus somewhere like at a small company? Like what Shane, the other Ph.D. in computer engineering, he worked at ZeroSoft, and you worked at Intel immediately after graduating with a Ph.D. Could you comment on the culture of innovation at Intel?
John: I think Intel is very innovation-focused. I think they're...
David: Right. And I think the pace of change across every industry that touches technology is just much, much faster than the rest of the economy. You talked about Intel's tick-tock schedule. They're pushing out new parts every 12 to 18 months, right?
John: Yeah. The big difference there, I think, is starter culture, social media, building user-facing apps that are web-based take weeks. Things can change in days.
David: It's just software.
John: Yeah. It's just software. It's software that you can deploy. It's not to say that it doesn't take a lot of effort to get to that point, but you can deploy new versions of it very quickly, whereas the process of getting from concept to physical chip that sits inside of some Apple laptop or Dell laptop is years, right?
David: Even thinking about how you manage a supply chain and how things physically move around is something that, in the software world, where I work, we don't really do.
John: Yeah. I think that's even beyond Intel. You look at Apple, and it's like, "Well, we want to make this phone that gets these displays from Korea, gets these batteries from Singapore, it gets assembled in China and then it gets shipped to the US." Testing happens here. There's just so many different things you need to think about that'll need to come together at the same time. It's really crazy.
David: Sure. I want to wrap up pretty soon. I know we're pretty close to time. Do you have any advice for younger people? Anything that surprised you or anything that's...?
Ravi: People intending to join a Ph.D. program now, or you're an undergrad, or people intending to join a larger company like Intel, or...?
John: Yeah. I think work hard, focus...
David: Be prepared, right?
John: Be prepared.
David: Scout motto.
John: Yes, nothing's easy. Be prepared for failure. Nothing's worth doing if you're not going to fail at it a couple of times. Just be responsible. Be responsible with life, money, love. I don't know, I think just don't do stupid things.
Surround yourself with good people. That's a big thing. Work really hard to surround yourself with good people, because that's where good ideas come from. That's where you get good perspective. You get good advice.
People are going to be honest with you. People are going to be hard with you. Life isn't about finding the easy path. The easy path, it'll get you so far. But I think if you want to be really good, you need to be around really good people.
David: All right, sounds good. Thanks for coming out tonight. Pleasure's ours, John.
Ravi: Thanks a lot, John. Appreciate it.