.. _site:

Website
======================

User registration
-------------------

A SyntaxGym account is required to create new test suites or add new language models.
No account is needed to browse existing suites, models, or results on the site.

To sign up for an account, please click `here <http://syntaxgym.org/user/create>`_.

Exports
--------------------

SyntaxGym allows users to download test suites, raw surprisals, and prediction results
in a variety of human-readable formats.

Exporting test suites
**********************

.. note::
    
    We recommend that you familiarize yourself with our 
    `test suite format <https://cpllab.github.io/syntaxgym-core/architecture.html#the-structure-of-test-suites>`_ before reading further.

Users can download any test suite on SyntaxGym in ``.json`` or ``.csv`` format.
The ``.json`` format is documented in full `here <https://cpllab.github.io/syntaxgym-core/suite_json.html>`_.

When you download a test suite as a ``.csv`` file, the columns are 
``["item_number", "condition_name", "region_number", "region_name", "content"]``.
These refer to the basic building blocks of our test suite format, 
as described `here <https://cpllab.github.io/syntaxgym-core/architecture.html#the-structure-of-test-suites>`_. 
Please note that the ``.csv`` file represents the test suite data in **long format**.

Here is an example test suite item in ``.csv`` format:

.. code-block:: csv
    :linenos:

    item_number,condition_name,region_number,region_name,content
    1,number_match,1,subject,The boy
    1,number_match,2,verb,swims
    1,number_match,3,post_verb,today.
    1,number_mismatch,1,subject,The boy
    1,number_mismatch,2,verb,swim
    1,number_mismatch,3,post_verb,today.

.. note:: 

    The ``.csv`` format only represents the **items** of a test suite,
    and does not represent the **predictions**. To make use of the prediction
    formulae, we recommend that you download the test suite in ``.json`` format.

Exporting raw surprisals
*************************

Users can download the raw surprisal results across all models for any test suite
by visiting the test suite page and clicking on the ``Download all results``
dropdown menu at the bottom of the page.

The raw results are formatted as a ``.csv`` file with the following columns:

suite
    Name of the test suite. (Note: this is the short name, not the long name
    displayed on the website for readability.)

model_name
    Name of the language model.

item_number, condition_name, region_number, region_name, content
    See the `test suite documentation <https://cpllab.github.io/syntaxgym-core/architecture.html#the-structure-of-test-suites>`_.

surprisal
    The aggregate surprisal value at the relevant region, in bits.

oovs
    Any out-of-vocabulary items in the relevant region.

Exporting prediction results
*****************************

Users can download the prediction results across all models for any test suite
by visiting the test suite page and clicking on the ``Download all results``
dropdown menu at the bottom of the page.