pyleus 0.3.0

Organize your files

This is an example of the directory tree of a simple topology:

my_first_topology/
|-- my_first_topology/
|   |-- __init__.py
|   |-- dummy_bolt.py
|   |-- dummy_spout.py
|-- pyleus_topology.yaml
|-- requirements.txt

Define the topology layout

A simple pyleus_topology.yaml should look like the following:

name: my_first_topology

topology:

    - spout:
        name: my-first-spout
        module: my_first_topology.dummy_spout

    - bolt:
        name: my-first-bolt
        module: my_first_topology.dummy_bolt
        groupings:
            - shuffle_grouping: my-first-spout

This defines a topology where a single bolt subscribes to the output stream of a single spout. As simple as it is.

Write your first spout

This is the code implementing dummy_spout.py:

from pyleus.storm import Spout


class DummySpout(Spout):

    OUTPUT_FIELDS = ['sentence', 'name']

    def next_tuple(self):
        self.emit(("This is a sentence.", "spout",))


if __name__ == '__main__':
    DummySpout().run()

Write your first bolt

Let’s now look at dummy_bolt.py:

from pyleus.storm import SimpleBolt


class DummyBolt(SimpleBolt):

    OUTPUT_FIELDS = ['sentence']

    def process_tuple(self, tup):
        sentence, name = tup.values
        new_sentence = "{0} says, \"{1}\"".format(name, sentence)
        self.emit((new_sentence,), anchors=[tup])


if __name__ == '__main__':
    DummyBolt().run()

Run your topology

Run the topology on your local machine for debugging:

pyleus build my_first_topology/pyleus_topology.yaml
pyleus local --debug my_first_topology.jar

When you are done, hit C-C.

Configuration File

You can set default values for many configuration options by placing a .pyleus.conf file in your home directory:

[storm]
nimbus_host: 10.11.12.13
jvm_opts: -Djava.io.tmpdir=/home/myuser/tmp

[build]
pypi_index_url: http://pypi.ninjacorp.com/simple/