In addition to the clearinghouse website,
the SyntaxGym ecosystem provides two open-source tools:
syntaxgym is a Python package which provides easy, standardized, reproducible
access to targeted syntactic evaluations of language models. It replicates the
core behavior of the SyntaxGym website.
syntaxgym is documented in full
The Language Model Zoo (LM Zoo) is an open-source repository of
state-of-the-art language models, designed to support black-box access to model
predictions and representations. It provides the command line tool
a standard interface for interacting with language models.
documented in full here.