.. _tools:

Open-source tools
======================

In addition to the clearinghouse `website <http://syntaxgym.org>`_,
the SyntaxGym ecosystem provides two open-source tools:
``syntaxgym`` and ``lm-zoo``.

``syntaxgym`` is a Python package which provides easy, standardized, reproducible 
access to targeted syntactic evaluations of language models. It replicates the 
core behavior of the SyntaxGym website. ``syntaxgym`` is documented in full
`here <https://cpllab.github.io/syntaxgym-core/index.html>`_.

The Language Model Zoo (LM Zoo) is an open-source repository of 
state-of-the-art language models, designed to support black-box access to model 
predictions and representations. It provides the command line tool ``lm-zoo``, 
a standard interface for interacting with language models. ``lm-zoo`` is
documented in full `here <https://cpllab.github.io/lm-zoo/index.html>`_. 

.. note::

    To gain a better understanding of how these three tools work together
    (`syntaxgym.org <http://syntaxgym.org>`_, ``syntaxgym``, and ``lm-zoo``),
    please take a look at our `ACL 2020 System Demonstration paper <https://www.aclweb.org/anthology/2020.acl-demos.10/>`_.