ScienceWorld

ScienceWorld is a text-based virtual environment centered around accomplishing tasks from the standardized elementary science curriculum. This code accompanies the paper ScienceWorld: Is your Textual Agent Smarter than a 5th grader?.

Examples

Example play-through transcripts are available at: https://sciworld.apps.allenai.org/explore

Quickstart

Before running: You will have to have Java 1.8+ installed on your system (shipped with most linux distributions).

Install with pip:

conda create --name scienceworld python=3.8
conda activate scienceworld
pip install .

Run an example random agent, on task 13 (classification: place a non-living thing in a box), for 5 episodes:

python examples/random_agent.py --task-num=13 --num-episodes=5 --simplifications-preset easy

Run a user console where you can interact with the environment, on task 3 (change of state: melting):

python examples/human.py --task-num=3 --num-episodes=5

Web Server Demo

A web server demo is also available, that allows running a ScienceWorld user console that can be interacted with in a web browser.

To run the web server demo:

conda create --name scienceworld python=3.8
conda activate scienceworld
pip install .[webserver]

Run the web server:

python examples/scienceworld-web-server-example.py

Point your web browser to: localhost:8080

ScienceWorld Design

ScienceWorld is written in Scala (2.12.9), and compiles using sbt into a JAR file that is run with java. For convenience, a python API is provided, which interfaces using the py4j package.

Ports: ScienceWorld is nominally run as a server, which interfaces to the Python API with py4j through a port. The default port is 25335. The actual port used will be 25335 + the thread number provided when a ScienceWorld class is instantiated.

Threads: ScienceWorld is designed to run many threads simultaneously, if desired. To do so, initialize one ScienceWorldEnv object per thread, and provide a unique threadNum for each thread. Don't forget to close down servers that you instantiate using the env.shutdown() command. If you are spawning many threads (10+) at the same time, you may wish to add a short delay (5-10 seconds) after initialization to wait for all the servers to initialize.

Tasks

The subtasks and their associated subtask indices are listed below. Subtask indices are automatically assigned by sorting task names:

Citation

Bibtex

TODO