Piazza considered harmful

2018/12/21

Background: what is Piazza?

Piazza is a web-based threaded message board widely used in college courses, introduced in 2012. Faculty and teaching assistants post various announcements on Piazza, and students post questions, which are answered by the faculty and other classmates. It is widely adopted in engineering and computer science courses. For example, in UC San Diego’s Computer Science an Engineering department, almost all courses, both undergraduate and graduate, rely on Piazza for online course discussion.

In fact, Piazza is so extensively adopted within the CS department that it’s name has become synonymous with the entire concept of online discussion. Much like Kleenex became a general term for tissue, and Xerox became a general term for photocopier, Piazza effectively means “web-based message board” for our students and faculty.

One reason for Piazza’s popularity is prior to its introduction in the early 2010s, the alternatives were just terrible. UC San Diego, like most colleges and universities, offers a “learning management system (LMS)” such as WebCT or Blackboard. These tools are considered terrible by faculty and students alike, and their threaded message boards are difficult to use and search, and so faculty and students had not used them. When Piazza was introduced, it offered an easy to use alternative that was a much better experience, both for students and faculty. Setting up a Piazza site took only a few minutes, and required no maintenance or upkeep. It was, simply, a fantastic alternative to LMS Q&A discussion sites.

Another reason faculty like Piazza is that when a student asks a question, other students can answer that question. If the faculty or TAs like that student answer, they can “endorse” the answer, indicating to the rest of the class that the answer is correct and saving the teaching staff time, since the students in the class act as de facto tutors.

Piazza is used A LOT

Piazza's claimed usage

Usage claims from Piazza’s website as of Dec 2018

Piazza claims that the average student spends three hours per night on their website. This is a huge amount of time. To put that in perspective, Mediakix reports that people spend the following amount of time on the major social media sites:

It is difficult to overstate just how intensely Piazza is used. For large, multi-hundred student courses, it is common to have thousands of message threads, posts, and comments per course per term. As an example, the last four offerings of my upper-division undergraduate networking course (CSE 124) have had 4,464, 4,317, 2,647, and 5,468 posts and replies per 10-week term. That works out to between 264 and 546 posts per week.

What is Piazza’s business model?

Piazza's Career portal

The main Piazza Careers website

Piazza has chosen a business model in which is offers its message board to students and faculty for “free”, and makes its money by selling student data to 3rd party companies. In particular, the data Piazza sells is not anonymous or aggregate data, but personally identifiable data linked to individual students.

In many ways, Piazza is much like Facebook. Users interact with and are engaged with one “side” of the website (the social media page in the case of Facebook, and the threaded message board in the case of Piazza). Both companies offer these user-facing parts of their service without requiring direct payment, and make their money by the other side of their service (to advertisers in the case of Facebook, and to companies in the case of Piazza Careers).

Piazza's partners

The employer search engine interface to the underlying Piazza data

Companies can join the Piazza Careers portal by paying Piazza. Once they join, they’re given a search engine interface to student data, including a list of every class a student has taken, and when, and how active they were on the message board, as well as the extent to which the student’s posts were “endorsed” by the faculty.

What’s good about Piazza Careers?

Piazza's partners

A sample of companies subscribing to Piazza Careers

When students sign up for Piazza, they get to choose whether they also join the Piazza Careers portal. If they do, they are then potentially connected to employers who want to interview or hire them. Employers can notify them of career fairs or internship recruiting happening at their college or university, and depending on which classes the student has taken, they can get highly targeted recruitment contacts from companies. From the corporate side, companies can send messages to students who took certain combinations of classes (e.g. a machine learning course and an operating systems course). They can also find out which students were active on the site and received a large number of faculty endorsements, indicating (indirectly) that the student is providing good answers to questions.

These types of searches are not hypothetical, they are quite clearly spelled out in Piazza’s marketing materials. Here are a set of searches that are featured in the materials used to sell their service to 3rd party companies:

With Piazza, every student you hire has a high probability of having a Piazza account. And we have rich information on our students – which classes they’ve taken, which classes they’ve TA’d, and which classes they’ve received professor endorsements in. We’re able to provide you with a list of academic traits for any of your past interns or recent grads.

Professor endorsement searching means that whenever you as an instructor mark a student’s answer as correct, you’re adding metadata to their account that becomes searchable in the Careers portal search engine.

“Show me the top students in CS 2112 at Cornell.”

Here “top student” means students who are the most active on the message board. Later in the marketing materials, a similar search is specified as:

Run a search like: “Show me opted-in PhD students who have taken Computer Security classes at Cornell and received professor endorsements.” Send them a message: “Hey, our Founder and CTO Maryanna will be on campus to discuss the problems we’re solving in the security space. I saw you have strong domain expertise — I would love to extend a personal invitation for you to come by and say hi to her.”

Professor endorsement and posting frequency are highlighted throughout the Piazza materials. Another hypothetical search is provided as:

Combine all of the above to craft powerful searches like: “Show me mobile developers who have received professor endorsements in iOS programming classes, have published apps to the App Store, are student leaders on-campus, and previously interned at one of my talent competitors.”

What’s so bad about Piazza Careers?

The same aspects of the Piazza Careers portal that might benefit some students by definition might be used to hurt other students. Piazza’s marketing materials tout the fact that companies can find the “top” students and those that have large numbers of faculty endorsements. It isn’t hard to imagine how this might backfire, though. Imagine that two students from the same university apply for the same job at a company. The recruiter searches for those students in the Piazza Careers database, and sees that one student has tons of faculty endorsements, and the other doesn’t have any. They might choose to interview the first student, and pass on the second.

It is important to realize that the student who got passed over for not posting in Piazza a lot (and thus not getting many faculty endorsements) would have no way of knowing why their application was passed over. They would just know that they didn’t get a call back from their application.

At this point, you might be thinking that it would be silly for a company to base a recruiting decision on how active a student was on Piazza, or whether their posts received faculty endorsements. I would agree with you! But it is important to realize that Piazza thinks it matters, and has based its entire business model on this hypothesis. The private VC firms that fund Piazza believe that this business model will be successful and have staked millions of dollars of their fund on this premise. Every company that pays to be members of the Careers portal also believes that this data is useful to them in making recruitment decisions. It is possible that all of these parties is wrong (Silicon Valley isn’t infallible after all), but it would be wrong to dismiss concerns about student data privacy and the potential impact that using this resource has on our students.

Student data vs. student metadata

If you read through the terms of service (at least as of December 2018), they claim that they do not include the content of the message board posts in the student data they sell to third parties. However, if you look at the marketing materials they use to attract paying corporate customers, they claim that companies can search for “top students” (who are defined as students who post the most messages), as well as a list of which students have large numbers of faculty endorsements. It is true that a count of how many faculty endorsements one has isn’t technically based on the content of the posts. However, no faculty or students I’ve spoken with have any idea that posting in Piazza or marking down a student answer as the “correct” one could eventually be packaged up and sold to companies considering whether to interview or hire that student. The phrase “opted in” is used extensively in Piazza’s online materials, though again, what students are opting into isn’t necessarily clear from the outset.

Does using Piazza violate FERPA?

It depends.

If you use Piazza in the most typical way, then using Piazza probably violates FERPA laws. This is because personally identifiable information about students, the classes they take, when they took those courses, and indirect information about how well they did in those courses (the faculty endorsement value) are stored on 3rd party servers and sold to 3rd parties. If you consult Piazza’s page on FERPA (https://piazza.com/legal/ferpa), they make it seem like their system is FERPA compliant, even though it is generally not. FERPA has to do with data storage and transfer policies, and a company cannot simply assert that their service is “FERPA compliant.” The details really matter here, which is why several colleges and universities have raised concerns about the service.

Technically students could all sign in using pseudonyms and register for the service with throw-away email addresses. If used in this way, then Piazza would likely be FERPA compliant. From a pedagogical point of view, though, not having any idea who is posting questions or who are answering those questions would reduce the value of a class-wide discussion board. Further, it is not possible for the TAs or instructor to remain anonymous, since their posts would immediately “out” them, for example posting that one’s office hours are moved due to a schedule conflict. Having students post questions in “anonymous mode” doesn’t help the situation, since the underlying data still keeps track of the student identification, even if it is hidden from their classmates.

Can’t students just opt out?

Yes they can, but the vast majority do not. In the marketing material that Piazza uses to advertise their Careers portal to 3rd party companies, they proudly point out that all but a few percent of students are members of the Career portal. This is not to say that such a large percent of students are fully aware and consent to the way their data is repackaged and sold to 3rd parties. For years I believed that the Piazza Careers portal was just a way for companies to post job advertisements and career fairs. I had no idea that the product being sold by Piazza was the personally identifiable student data. I suspect that many students are not aware of this fact as well, or have fully thought through the implications of having this data on a global marketplace.

It is true that all students that are part of the Careers portal have “opted in” to the service. It is not necessarily true, however, that they have truly made an informed consent of how their data is managed and sold to third parties. As faculty members, we have a responsibility to ensure that our students privacy is protected and that the services that we adopt model ethical behavior and ethical business models.

So what could we use instead?

Piazza is a significant improvement compared to LMS message boards. But when you compare Piazza to other more modern message boards, I personally do not think that it is particularly that good. My students and TAs routinely complain about Piazza’s inadequate search feature, for example.

UC San Diego has adopted Google’s web-based infrastructure (GMail, Google Docs, etc). Included in that suite of services is Google Groups. Groups offers a mailing list, but now offers a “Q&A” message board. This message board is very similar to Piazza, in that students can post questions, they can “upvote” questions to indicate that they also have that question, and students can answer each others’ posts and add comments. Questions can be marked as duplicate, and as students type in new questions, Google’s search engine automatically searches to find potentially similar posts (like they do with Google Search). Questions and discussions can be organized into categories (projects, lectures, homeworks, etc), and also tags can be used to cluster topics. Messages can also be moderated, so that the teaching staff can take a quick look before it “goes live” to avoid public posts of questions that include answers to homeworks or program/cost listings.

So on a feature-to-feature basis, Google Q&A seems like a reasonable alternative to Piazza. But more importantly, at least at UC San Diego, our contract with Google means that the content is truly FERPA compliant, and Google does not resell or use the message board data at all.

Conclusions

Piazza is a service that meets a need, and has thus far kept faculty and students alike pretty happy. But their underlying business model is based on monetizing the personal information of our students. I am sure that the employees and creators of Piazza are fine people with good intentions, but regardless of their intentions, a business based on mining data will eventually have to… mine and resell the data to stay in business. Once our student’s data is collected, we lose our ability to control that data in the future. My concern isn’t isolated. Dipayan Ghosh and Jim Steyer just published an article in the New York Times about this topic a few weeks ago: Kids Shouldn’t Have to Sacrifice Privacy for Education.