Getting online: Experiment hosting sites
Once your study is up and running on your own computer, we need to get it up online. You can upload directly to server space, if you have access to some, or using an experiment hosting site.
Server Space
One option would be to simply upload your experiment to some server space you have access to. If you are at a university you will likely have access to some at your university. If not, you can get server space at the the University of Edinburgh. You will likely need to activate your server space, and upload your HTML/JS/CSS files using an STFP (e.g. FileZilla) into your wwwroot (e.g. public_html folder). In order to save data with this method, check out the page on saving data on the left.
Cognition.run
cognition.run is completely free and incredibly easy to use - just create a new task and upload your JavaScript to the source code section! It doesn’t accept HTML, so your CSS will need to be in a separate file. This seems too good to be true - the site is run by one person and I’m sure it can’t stay free forever, but it’s fantastic for the moment.
Pavlovia
The cheapest of the paid experiment hosting sites is Pavlovia, run by the University of Nottingham staff that created PsychoPy off of a grant from the Welcome trust, meaning it’ll only cost you 20p per participant! They are a great organisation to be supporting and don’t seem to be a commercial enterprise, and an institutional licence is only £1500(hint hint). Here’s the information on how to integrate jsPsych with Pavlovia: pavlovia.org/js-psych - this is a bit involved for me to try out here but if anyone tries it out and runs across any issues and solutions do let me know!
Gorilla
All of this can be done through the online platform Gorilla. As the university aren’t subscribed to Gorilla, it will cost a fair bit of money to run participants. Note PhD students will qualify for the researcher package (‘tokens’ just means ‘participants’), but this will still push the budget for most PhD students, especially if you are already paying participants. The researcher package would be £150 for sign up, which includes 200 free participants, and amongst other options you can get 500 more participants for £350, so £500 for your first 700 participants.
Anwyl-Irvine, A. L., Massonnié, J., Flitton, A., Kirkham, N., & Evershed, J. K. (2020). Gorilla in our midst: An online behavioral experiment builder. Behavior Research Methods, 52(1), 388-407. https://doi.org/10.3758/s13428-019-01237-x
If you do choose this option, note that Gorilla currently uses the previous version of JavaScript (ES5) and so if your code has relied on changes made since the current version was released (ES6), you might need to make some changes to your code to get it to work. I contacted them about this, and it should be fixed by the end of the year, apparently.
This is the tutorial to get jsPsych working in Gorilla. Don’t forget to add your HTML <script> section to the ‘head’ tab in the format suggested in the example and upload your plugins etc to the ‘resources’ section.
Qualtrics
It’s also possible to just place your study into a Qualtrics survey using their JavaScript question editor. This tutorial shows you how to do that: https://kywch.github.io/jsPsych-in-Qualtrics/. This approach is useful for longitudianl studies where you need Qualtrics to manage participants and store their sensitive data securely.
The problem with this approach, however, is that your data won’t save to Qualtrics. Instead, we’ll have to tell the JavaScript to send the data to a folder stored in your personal server space offered by the university. Before you follow the tutorial, note that the sections on saving your data won’t work if you’re at Sussex - I detail below how to get them to work. Regarding Qualtrics and JavaScript:
1. saving your data with json and not csv is recommended
2. template literals will not work properly in Qualtrics
3. The JavaScript version used by Qualtrics is in fact relying on the JavaScript version run by your browser, and so it’s best to not use code from the latest version of JavaScript (ES6) if you can avoid it, to ensure maximum usability for users with older/not updated browsers.
PsiTurk
https://psiturk.org/Pushkin
https://languagelearninglab.gitbook.io/pushkin/Hartshorne, J. K., de Leeuw, J. R., Goodman, N. D., Jennings, M., & O’Donnell, T. J. (2019). A thousand studies for the price of one: Accelerating psychological science with Pushkin. Behavior Research Methods, 51(4), 1782-1803. https://doi.org/10.3758/s13428-018-1155-z
PsyToolkit
https://www.psytoolkit.org/Stoet, G. (2017). PsyToolkit: A novel web-based method for running online questionnaires and reaction-time experiments. Teaching of Psychology, 44(1), 24-31. https://journals.sagepub.com/doi/10.1177/0098628316677643
Finding Five
https://www.findingfive.com/Labvanced
https://www.labvanced.com/Finger, H., Goeke, C., Diekamp, D., Standvoß, K., & König, P. (2017). LabVanced: a unified JavaScript framework for online studies. In International Conference on Computational Social Science (Cologne). [OPEN ACCESS] https://www.labvanced.com/publication.html
Testable
https://www.testable.org/Rezlescu, C., Danaila, I., Miron, A., & Amariei, C. (2020). More time for science: Using Testable to create and share behavioral experiments faster, recruit better participants, and engage students in hands-on research. Progress in Brain Research, 253, 243-262. https://doi.org/10.1016/bs.pbr.2020.06.005
Concerto
https://concertoplatform.com/aboutiSpring
https://www.ispringsolutions.com/SoSci
https://www.soscisurvey.de/Millisecond
https://www.millisecond.com/Experimaker
https://experimaker.com/ExpFactory
https://expfactory.github.io/resources
You might also want to check out these resources on experiment hosting sites where I got some of the above info from:
https://kennysmithed.github.io/oels2020/oels_wk11.html
https://www.spatialhearing.org/remotetesting/Resources/Web-basedPlatforms