Ongoing Investigations

Category: Digital Scholarship

  • Ethics Week 8: What Do You Do with Academic Software?

    So this week (and the past couple weeks, actually) I’ve been doing some reading on the legal policies of a company called Gradescope. While this isn’t a full write-up, more an extension of some annotation I’ve been doing lately, I thought it would make a good blog post.

    So, Gradescope. What is it? Who is it? What does it mean for us?

    What is Gradescope?

    Gradescope is a web-based subscription software that allows educators to organize and delegate the collection and grading of assignments, in order to allow them to better focus on the needs of their students.

    Or, simply put: it’s like Moodle (turn in your assignments online) but it also grades stuff if you ask it to and tell it how.

    They’re not giving it to humans to grade, either — they’re grading algorithmically, based on a rubric.

    It’s aimed primarily at STEM, econ and business classes, which makes sense, since quantitative classes would be easier to grade consistently by machine. But their website does boast that it handles papers, even if it’s a little unclear on what aspects of the papers it handles (collection, maybe grammar) vs what aspects are handed to the educator (style and content, hopefully).

    Who is Gradescope?

    Well, as of October 2018, Turnitin owns Gradescope. Turnitin is another web-based software company, this one oriented towards catching plagiarism and determining authorship of papers. They partner with large databases, including Crossref, Elsevier, Springer, Wiley-Blackwell, IEEE and Taylor & Francis, in order to check what percentage of a paper matches with other papers out there. In fact, they claim their content database includes “70 billion current and archived webpages”.

    But their plagiarism checker is a little concerning, missing 44% of plagiarized sources in a 2015 experiment at UT-Austin. Penn State University, in their online guide to Turnitin, outlined various ways to get false positives, none of which inspire confidence in Turnitin’s algorithms.

    • Any rough draft stored in Turnitin’s repository could come up as a false positive when the final draft is submitted.
    • Any copies of the paper submitted to Turnitin to check for plagiarism before the paper is officially turned in for class will be stored in the repository – which can generate a false positive.
    • Turnitin doesn’t check whether a student’s original work is being plagiarized, just whether their work matches with another source in the repository.
    • A paper that expands on/includes material from a previous paper on the same topic can also generate a false positive.

    It’s remarkably hard to find any information on Gradescope’s own effectiveness or practices, incidentally. Their terms of service and privacy policy mention third-party partners and third-party recruiters, but decline to elaborate on either. One can sign in with Google, making Google a third-party partner with access to their data, but I can’t find any clear indication of what recruiters might be present on Gradescope’s site.

    Additionally, though their privacy policy claims to comply with GDPR and Privacy Shield (an opt-in U.S.-E.U. and U.S.-Switzerland privacy rubric that allows members certain privileges), their terms of service were last updated in November 2014 — which means they predate GDPR (adopted April 2016), Privacy Shield (July 2016) and the CCPA (fall 2019), a California-based privacy law which they are also subject to. Their privacy policy was last updated in June of 2018.

    What does it mean for Carleton?

    At present, not much. I haven’t heard of any courses that use either Gradescope or Turnitin, which are subscription-based models that would need to be funded by whatever department chose to use them (or by the college at large).

    While Gradescope’s privacy policy and terms of service are mostly standard legal jargon, there are a couple of things I tend to sideeye – like, what happens if your account is deleted due to a false positive?

    The first of my major issues is that they’re not publicly forthcoming with their revenue streams. Of course they charge for subscriptions from the schools they work with, and their client base is obviously private, but as I said above, I can’t find out what third-party recruiters have access to their site. Nor can I find out how these recruiters get access — do they pay for it? Does Gradescope seek them out, or do they contact Gradescope? What sort of data do they get to look at, and how can users manage their privacy?

    The second is their grading methods. Since they’re owned by Turnitin, it’s very likely they use Turnitin’s plagiarism checker — which, as we know, is faulty, despite its wide access to databases.

    Which brings me to my third point: Gradescope keeps student work, and makes it accessible to the third-party partners they use to do their work (like, for example, Turnitin). Turnitin keeps student work and adds it to their databases so they can check for plagiarism more expansively. This is a feature schools pay for.

    Or, put another way: schools pay for Gradescope and Turnitin to make money from other schools off of their students’ work.

    Gradescope has policies in place that comply with GDPR – meaning, any user has the right to be forgotten. You can write to Gradescope, ask them what information they have on you, and tell them to delete it, and they will do it (with a few caveats related to anonymized and group data).

    But, at the same time, their terms of service are six years out of date. Their privacy policy doesn’t mention the California Consumer Privacy act, and I haven’t been able to find any official statements from Gradescope on the topic. And, possibly most worryingly, it is very unclear whether their partners have the same policies in place. And even if they do, would requesting that Gradescope delete your information would extend to any party with whom they’ve shared that information?

    Sources

    “Turnitin Acquires Gradescope”, Zeligs Hand, Amanda, Turnitin, 3 October 2018. https://www.turnitin.com/press/turnitin-acquires-gradescope

    “Database Content”, Turnitin. https://www.turnitin.com/about/content

    “How Much Do You Know About Turnitin’s Content?”, Lee, Christine, Turnitin, 30 May 2019. https://www.turnitin.com/blog/how-much-do-you-know-about-turnitins-content

    “Optimizing the Originality Report”, Penn State University. https://turnitin.psu.edu/bestpractices/usingreports/

    “UT-Austin test results for Turnitin”, Schorn, Susan, University of Texas — Austin. https://utexas.instructure.com/courses/633028/pages/ut-austin-test-results-for-turnitin

    “Privacy Policy”, Gradescope, 25 June 2018. https://www.gradescope.com/privacy

    “Terms of Use”, Gradescope, 3 November 2014. https://www.gradescope.com/tos

  • Ethics Week 7: The New Travel Ban

    So. The travel ban.

    It’s not strictly digital humanities, but it’s definitely ethics and it’s definitely relevant to Carleton. So that’s my blog post for today: what’s happening, what does it mean, and how is it going to impact us?

    What’s happening?

    Basically, the Trump administration is expanding their limitations on visas for immigration into the United States. This policy was announced at the end of January, and it’s taking effect on February 21st, 2020. Spot check: that’s tomorrow.

    Up until now, there have been restrictions on immigration from Iran, Libya, North Korea, Venezuela, Yemen, Somalia and Syria. Citizens from those countries can’t obtain visas at all. With the new policy, there will also be limitations on Myanmar/Burma, Sudan, Nigeria, Tanzania, Kyrgyzstan, and Eritrea. Citizens from these countries aren’t barred from visiting the States entirely, but they mostly won’t be able to get permanent residency.

    What does it mean?

    The Trump administration is calling this a national security measure, though the effectiveness of the policy is in doubt. Critics are calling it out for racist policies — specifically, the appearance of four African nations on the list of newly-restricted countries after many remarks by President Trump denigrating African countries has caused a backlash.

    Legally, there are a variety of visas the United States offers. They are divided into two main categories: Nonimmigrant Visas, and Immigrant Visas. This new update to the travel ban focuses primarily on immigrant visas, which include sponsorship by a spouse or family member, sponsorship by an employer, diversity visas, and returning residents.

    The updated ban bars citizens from the affected countries (Myanmar/Burma, Sudan, Nigeria, Tanzania, Kyrgyzstan and Eritrea) from the diversity visa lottery, and only citizens from Tanzania and Sudan will be able to gain visas that would let them immigrate to the US. There are other specific restrictions of varying sorts on different countries related to which types of visas can be obtained (there are 34 varieties of nonimmigrant visa that I found, and 30 varieties which might apply to citizens of the affected countries).

    What does this mean for Carleton?

    This is going to hit Carleton specifically with regards to international students and faculty. Those who already have visas aren’t going to have them revoked, and student visas are among the nonimmigrant visas which aren’t going to be restricted, but students might think twice about applying to United States schools if they’re unlikely to be able to gain long-term residency in the country.

    Additionally, faculty visas seem to be a bit of a gray area. Employer-sponsored visas fall into the immigrant visa category, which the travel ban is restricting, but there is a non-immigrant visa called “Professor, scholar, teacher (exchange visitor)”, which is a J-visa. J-visas are specifically for individuals who are part of exchange visitor programs, which is unlikely to help with long-term employees of the college.

    The takeaway is… I’m not sure, honestly. The political science department hosts an off-campus studies program travelling to Thailand and Myanmar. International students may still be able to get visas, but it will likely be difficult and may lower the desire to attend Carleton. And employees of the college currently on an employer-sponsored visa shouldn’t have theirs revoked, but it’s unclear whether they’ll be able to renew it once it expires.

    Sources:

  • Ethics Week 6: The Road So Far

    It’s midterms, and you know what that means. Time to review!

    … my work so far this term, that is. I’ve already taken my exams.

    I did a little inventory of my past blog posts, as well as some behind-the-scenes stuff, so today I’m going to be talking about what I said I’d do versus what I actually did, and whether there’s room to pick back up.

    What I Planned

    It’s actually pretty easy to see what I meant to do this term, because I wrote a blog post about it. It sums up a couple of the themes I was hoping to touch on and a couple of projects I was hoping to do.

    The executive summary of it all is thus. I had four major projects:

    1. look into different ways of organizing work, especially digital vs. analog systems
    2. study open source software communities for the Digital Scholarship Interns’ winter project
    3. investigate data security laws and how they affect college software
    4. develop my Ruby coding skills

    Did I do these things? Sort of!

    What I Did

    It’s a little hard to look back through my work the past couple weeks, because some of my blog posts are password-locked and I don’t write about everything I work on (it’s important to have focused, informative posts).

    That said, I have followed through on these plans in different ways throughout the term so far.

    First: Organizing Systems

    This is very solidly a behind-the-scenes thing. I’ve experimented a little with Google Keep, comparing it to my past experience with to-do lists in the Notes app and Wunderlist, plus bullet journals. No conclusive results yet, but this investigation is ongoing. Expect a recommendation report sometime in the future!

    Second: the Digital Scholarship Interns project

    My part of the project has been reshaped a couple times to better fit with what we’re doing. Our project is a revamp of the DSI blog (you may have noticed it’s a little snazzier than it used to be). But, since we’re focusing on blogging at Carleton, open source software didn’t really fit.

    Instead, I’m going to be doing an investigation of Carleton social media culture, comparing the Carleton Livejournal community (still up online, though not very active these days) to Facebook and how people at Carleton use it. This way, we can get a good idea of the common questions Carleton users ask, and what they look for in a platform.

    Third: Investigate Data Security Laws

    This has been a way bigger part of my term than previously anticipated. When I wrote my initial blog post, setting out my goals for the term, I thought digital systems of organization and the intern project would be my two major priorities, with some legalese on the side and a little Ruby to fill in the cracks.

    In reality, my term has focused a lot on rights management and data security. Week three had me looking into some new data privacy laws in the United States (this post was password locked, for internal use only); week four saw me thinking about what it means to safely store your work with different digital entities; and week five was all about reflecting on technology in the Iowa caucuses.

    And it’s going to keep going from there! The more digging around I do, the more stuff I find. Next week looks like it’ll see me looking through the Terms of Service and Privacy Policy of some scholastic software companies, to think about what control we have over our work at Carleton. Can’t wait!

    Of course, since I’ve been spending a lot more time than expected on this, that means…

    Fourth: Ruby Coding

    My Ruby plan for the term has shifted a bit, from coding up projects to investigating its uses. Ruby’s been used in some interesting archival projects, and one of my goals is to understand its affordances in that area.

    That said, because it has less practical ramifications in the here-and-now than understanding how we organize our work, investigating Carleton’s social media footprint, or unpacking the new laws around data privacy, this project’s been temporarily shelved.

    And frankly, I’m okay with that. One of the important lessons this term has been solid prioritizing.

    Where To Next?

    Honestly, keeping up what I’ve been doing. There are next steps for everything: writing up the different organization systems, putting together my content for the intern project, and decoding the mysteries of Terms of Service.

    I also have some new projects coming down the pipeline! They’re a secret for now, but might make an appearance in future blog posts.

  • Ethics Week 5: Let’s Talk About Iowa

    My post this week was going to be about how people use libraries and the information they contain.

    Then Iowa happened.

    So this week, we’re summarizing the technological debacle that was Monday’s Iowa caucus. What happened? Did anyone expect it? How did this come about?

    A quick overview

    In brief: in the Iowa caucus, people go to their precinct and vote for the candidate they want. The precinct then reports the totals in, so they can be counted up and people can figure out who got the most votes. In the past, the totals have been reported over the phone lines, but this year, the Iowa Democratic Party decided to commission a phone app from the tech company Shadow, Inc. to report the results. The Nevada Democratic Party also helped pay for the app’s development, hoping to use it for their upcoming caucus on February 22nd (this plan has since been scrapped).

    Some important things to note, first and foremost: this project was commissioned by the Iowa Democratic Party, not the Democratic National Committee; the app was for reporting caucus results, not for voting electronically; and, while there were security concerns, the primary issues were with the app’s performance.

    The app, whose name I can’t find (various sources call it ‘the Shadow app’, ‘the app’, or ‘the app from Shadow, Inc.’), was still in development this past weekend — meaning, days before the primary. In the tech world, this might not be too unusual: it’s pretty standard for games and apps to release patches in the early days after their release date. But for voting technology, that’s not an acceptable practice.

    So what’s the problem?

    Security was the obvious concern with the app, and has been since at least mid-January, when NPR put out a piece on the app’s development. It’s hard to get a general impression of the app’s security features, because the chairman of the Iowa State Party, Troy Price, declined to give any details on tests it might have undergone, companies that might have checked it, or issues they might have found. His goal was to avoid leaking information that might entice hackers, but many cybersecurity experts say that “security through obscurity” isn’t actually an effective countermeasure against hacking.

    That said, while the app’s poor performance definitely raised security concerns (it was meant to be used on the personal devices of the volunteers running the caucus), the main issue was just that: poor performance. The app was apparently hard to find, difficult to download, and glitchy.

    While the Iowa Democratic Party claimed it would have backup procedures in place, they weren’t prepared for the volume of failures. When caucus organizers gave up on the app and decided to report results the old-fashioned way, over the phone lines, the sheer volume of calls blocked most people from getting through, which contributed to the delay in results.

    So what now?

    Well, Nevada’s not going to be using the app. Iowa’s results have been checked against a paper trail, which was pretty foreseeable given the scrutiny this incident has caused.

    There are some upsides: Iowa was considering implementing smartphone voting for this caucus, but nixed that idea a while ago. The question of election security is also front and central, and hopefully people will be more cautious of tech’s place in the democratic process going forward.

    Also, just in case you were worried about what this incident means for the company behind the app, Shadow, Inc. released a tweet stating that they “regret the delay… and the uncertainty it caused” and that “[they] will apply the lessons learned in the future”.

    And as always, XKCD has a comic for this.

    Sources

    “Iowa’s caucus app was a disaster waiting to happen”, Zach Whittaker, TechCrunch. https://techcrunch.com/2020/02/04/iowa-caucus-app-disaster/

    “Despite Election Security Fears, Iowa Caucuses Will Use New Smartphone App”, Kate Payne and Miles Parks, NPR. https://www.npr.org/2020/01/14/795906732/despite-election-security-fears-iowa-caucuses-will-use-new-smartphone-app

    “The Shoestring App Developer Behind the Iowa Caucus Debacle”, Emily Glazer, Deepa Seetharaman, and Alexa Corse, The Wall Street Journal. https://www.wsj.com/articles/the-shoestring-app-developer-behind-the-iowa-caucus-debacle-11580904037

    “What We Know About The App That Delayed Iowa’s Caucus Results”, Avie Schneider, NPR. https://www.npr.org/2020/02/04/802583844/what-we-know-about-the-app-that-delayed-iowas-caucus-results

    Shadow, Inc.’s Twitter Statement: https://twitter.com/ShadowIncHQ/status/1224773796307050497

    “Voting Software”, Randall Munroe, XKCD. https://xkcd.com/2030/

  • Ethics Week 4: It’s 2020. Where is Your Work?

    This week, I’ve been reading up on archival and maintenance systems, with a special eye to digital scholarship. How do we store and maintain things? How are systems designed to facilitate their own updates, and to keep their contents both safe and accessible?

    What I’m talking about has a couple of different aspects attached to it, since there are a lot of different factors in data storage. Here are some quick questions to ask when it comes to archiving your work:

    • Who is the entity organizing this storage?
      Different entities store different data for different reasons and in different ways. Consider: Google Books, the Gould Library, JSTOR, the Internet Archives 
    • What types of information are being stored?
      Consider file formats, storage limits, and what types of information the storing body might prioritize (and why). 
    • When was this system created, and how is it being maintained?
      Digital systems require maintenance, just like physical ones. Old websites are hard to navigate; outdated software can cause security issues. 
    • Where is your information stored, and where is it accessible?
      Is your paper being paywalled? Can the people who need to find it, find it? And conversely, is the storing entity capable of secure storage? 
    • Why is this entity storing your information?
      Have they expressed a commitment to the long-term accessibility of knowledge? Do they have a profit motive (and, if they lose profits, will that commitment waver)? 
    • How is this entity demonstrating their commitment to preserving information?
      Are they transparent with their motives, their methods, and their means? Are they likely to waver in difficult times?

    No system is perfect, but looking to an entity’s motives and methods can be particularly illuminating. Also worth investigating is how comparable companies that have shut down dealt with their records, whether they kept them after closing down, and whether those items are still accessible by their creators, or by the general public.

    Vine, for example, hosted short videos which could be made through its phone app, but went defunct after struggling to find a way to consistently monetize its content. The site still has videos available to view and to download, but people can no longer post new videos, and I couldn’t find a source on whether creators can remove their old videos or control who can see them.

    Moving forward, I’ll continue looking into how digital systems are maintained and what they might store; this will also tie into a side project I’m working on, where I will be researching the history of the internet.

  • Ethics Week 3: What’s the CCPA?

    This week I’ve been following up on looking into the CCPA, or California Consumer Privacy Act. Quick intro: the CCPA is a law that came into effect in California on January 1st, 2020, which addresses data security with regards to companies that collect and sell user data. There is a cheatsheet provided by the California government which summarizes the law, available at the end of this post.

    Essentially, what it boils down to is pretty similar to the GDPR, the European Union’s data privacy regulation which took effect in May of 2018. California consumers have the right to know what data is being collected, the right to have that data removed from records, the right to know what sorts of companies are buying their information, the right to tell people not to sell their data, and the right to not be retaliated against for exercising any other rights granted by the CCPA.

    The laws also only apply to certain businesses (specifically, ones that meet certain thresholds for revenue or user traffic), and they only apply to the sale of data. That’s caused some concern, since companies like Google and Facebook technically don’t sell their users’ data: they sell access to their users, based on the data they have on file.

    The laws also only apply to for-profit businesses, which means that non-profits which operate in California and might otherwise meet the CCPA’s criteria are not subject to those regulations; however, if they do business with for-profit institutions that involves user data, they may still have to comply with the CCPA.

    While I looked into the CCPA’s effect on for-profit institutions that work with higher education, such as ProQuest and Elseviere, I couldn’t find an official statement from those companies. I did, however, find a cybersecurity law report published by Elseviere. This report mostly touches on those facts already stated earlier, but also makes a point of emphasizing that the CCPA has defined ‘personal information’ in a much broader way than the GDPR, or any other current privacy regulation.

    Going forward, it’s not clear what the ramifications of the CCPA will be. Some people claim it goes too far; others are worried that restricting only the sale of personal information doesn’t go far enough, and won’t actually protect people’s privacy. California tends to be a trendsetter where state legislature is concerned, however, so I’ll be looking into other potential privacy bills and seeing where we’ll be going from here.

    References:

  • Ethics Week 2: Laying the Groundwork

    This being the second in a series of weekly blog posts, intended to edify the public on the various goings-on within the ethical strata of the Digital Scholarship community of Carleton, particularly with regards to…

    I kid, I kid. Here’s a quick catch-up on what I’ve been doing this week!

    The process of my work has been primarily logistical, getting up and running for the rest of term. I’ve been exploring various topics, deciding which ones are worth pursuing, and solidifying them into project ideas and questions to answer. I’ve determined four projects/areas of research, two major and two supporting, which I’ve outlined below and which feed into each other in various ways.

    After some exploration, my common question for this term will be regarding the tradeoffs between organizational affordances in the cloud, and data security. This will consist of testing various organizational systems, seeing what conveniences they offer and what drawbacks, and researching how this data is stored and what it might be used for. I will also be keeping an eye on project logistics and multi-person information/storage systems, because my major project will center around open source software.

    The Digital Scholarship Interns’ winter 2019 project will consist of revamping our website and, in the process, exploring the history of collaboration, accessibility and performance in web design. Part of my job will be examining open source communities and how they work with each other, as well as how that workflow affects the final product of their projects and designs.

    In addition, I will be following up on my earlier data security question by looking at some common sites used by colleges for logistical purposes, investigating their practices, and researching the laws being passed about data security which affect those sites. Some of the writeups on these topics may not be made public, but rest assured, I’m looking!

    And finally, I will also be engaging in a side project of mine, which is to develop my coding skills, particularly in the programming language Ruby. I’m hoping to learn more about Ruby on Rails, and possibly use it in order to build a web-based presentation of my research for the DSI project.

    Happy Week 2!

  • Fall 2019: A Term of [REDACTED]

    The common theme to my work as the Ethics Intern this term has been anonymization, particularly of data. The two projects taking the bulk of my attention both deal with the requirements of ethical data collection; one in a practical manner, one in a theoretical. First is my work with the Public Memory of Myanmar project, on behalf of Professor Tun Myint; second is my research into the Institutional Review Board and its requirements for Carleton.

    The Public Memory of Myanmar project is an ongoing effort to document daily life in Burma/Myanmar, in various towns and cities around the country. The Carleton Political Science department sponsors an annual trip, organized by Tun Myint, where participants photograph and interview locals. Burma/Myanmar is a politically turbulent area, and the Public Memory exists in order to preserve ways of life that are in jeopardy and to put narratives of life into those people whose experiences may not be depicted in the official political narratives put out by the state. However, this political turbulence also means that local people participating in this project may be in danger if they can be identified. Much of my work this term has therefore consisted of going through each item in the collection (a photograph/video, accompanied by descriptive metadata) and combing through the information associated with each object in order to find any specific names or locations that could be used to identify individuals.

    This focus on the particulars of individual data also comes up in my other major project, which has been to do research with regards to the Institutional Review Board at Carleton, the standards it must meet, and the standards it must uphold. The IRB is an overseeing body at Carleton that approves project proposals for work which involves research using human subjects. These proposals must explain what data they wish to collect, how they will collect it, and what their plans are for safe storage. These concerns come up most frequently with regards to the psychology and economics departments. However, it is definitely possible for other fields to propose projects that fall under the purview of the IRB. The regulations the IRB is subject to are relevant to the library and to digital scholarship because the library is responsible for the storage of data after it is collected by student researchers (it must be kept for a minimum of three years after the study is concluded, according to federal law). Carleton is also subject to GDPR regulations where data collection overlaps with GDPR jurisdiction (data collected in regards to people currently located in the EU falls under this jurisdiction). 

    Ethical data collection under the IRB involves looking not only at which pieces of information are collected, but how these pieces might work together to identify an individual even if personal IDs are not collected. For example, while collecting college ID numbers or Social Security numbers would make participants easy to identify and therefore would not be allowable under the IRB, collecting, for example, age, gender, hometown and current dorm creates the potential to identify specific individuals by combining this information. Finding, for example, a 23-year-old woman from Albany living in a specific building is a distinct possibility based on collected data which the researcher might not even mean to use.

    You can see why it’s been difficult for me to come up with a visual to convey the work I’ve done this term, since so much of my work centered on stripping away identity. In the end, what I settled for was somewhat tongue-in-cheek. Please enjoy Sanitized Data, by Pilot Irwin (2019).

  • Pilot Irwin 2020 Introduction

    Hi, my name is Pilot Irwin, and I’m a senior CAMS major from New York City. In addition to my work in Cinema and Media Studies, I also have a strong interest in Computer Science and Economics, and I’m fascinated by the ways STEM disciplines and the humanities can not only intersect but inspire each other. As the Ethics intern, I will be responsible for examining how we interact with digital technology and how the construction of digital systems affects people’s lives, both intentionally and accidentally (2019-2020).

css.php