This dataset includes data from the Scratch Online Community created by the Lifelong Kindergarten Group at the MIT Media Lab. The dataset includes the first five years of data from Scratch (roughly 2007-2012). The data includes public data collected by the MySQL database that runs the Scratch online community website. Because the data is from a large website that was under both active use and active development, it contains a number of inconsistencies and errors. We have tried to document known issues.
These datasets and this documentation was prepared by Benjamin Mako Hill at the University of Washington and Andrés Monroy-Hernández at Microsoft Research. If you find inconsistencies and errors in these data, please contact them at makohill@uw.edu and andresmh@andresmh.com to adjust the data or the documentation.
All datasets are provided in RData format suitable for loading into GNU R with the load() function. In the RData files, all underscores ("_") are converted to periods (".") as per R variable naming conventions
Although the documentation you are reading is publicly, all tables with user data are access restricted. Individuals should request access to these data by following the instructions, providing details about yourself and your research, and agreeing to the Scratch Research Data Sharing Agreement (SRDSA). All these steps are detailed in the Scratch Research Data access request form. Every individual with access to the data (including collaborators on teams and students in a lab working with these data) must fill out this form and recieve approval before they access the data.
The users of the Scratch online community are young people, many under 18 years of age, from countries across the world. In designing the release of these data, we have worked to insure that the data collected here were all contributed to the public Scratch website and that the dataset contains no private data. The Scratch website rules prohibit users from sharing identifiable information in any form and even discourages usernames that might hint at "real" names or identities. Even so, and even though these data are public and were contributed by users who knew they would be public, there may still be risks involved in research using these data. As a result, the Scratch Team at MIT is requiring that all researchers seeking access to these data provide their name, affiliation, and a description of their project so that it can be reviewed by the Scratch Team first. Moreover, they are requiring that researchers using these data accept a data usage agreement.
The protocol for the publication of these data was approved by the Committee on the Use of Humans as Experimental Subjects at MIT (COUHES). Researchers using these data are responsible for ensuring that their own use and research using these data is conducted with any necessary approval from their own ethics or institutional review boards. We have made available the full text of the proposal to release these data submitted to COUHES and COUHES' letter of approval.
Core datasets are the datasets that describe the major objects and relationships captured by the Scratch Online Community website.
In addition to the RData files, these datasets are provided in bzip2 compressed comma-separated values format (CSV) with strings in double-quotes(" ").
These tables are much larger tables that include user submitted text useful for text and NLP based analyses.
In addition to the RData files, these datasets are provided in bzip2 compressed JSON format. Because the files are very large, the JSON files are not formatted as a single JSON object. Instead, they are formatted as a "stream" of JSON dictionaries where each row in the dataset is formatted a separate JSON dictionary. Each dictionary or row is separated by newlines. As a result, newlines and carriage returns within text fields are escaped ("\n" and "\r"). This format is very similar to the format used for other social media including the Twitter streaming API or GitHub event data.
These tables contain quantitative summaries of the content of each project file. Because a small number of project files were not parseable, likely because of data corruption or because they were created with user-modified versions of the Scratch programming environment, data is misisng for these projects.
In addition to the RData files, these datasets are provided in bzip2 compressed comma-separated values format (CSV) with strings in double-quotes(" ").
example_scripts it includes example Python programs to parse and import the CSV and JSON files and to detect and mark incorrectly encoded data.