Skip to main content

Reusable django app for collecting and visualizing network topology

Project description

https://travis-ci.org/netjson/django-netjsongraph.svg https://coveralls.io/repos/netjson/django-netjsongraph/badge.svg Requirements Status https://badge.fury.io/py/django-netjsongraph.svg

Reusable django app for collecting and visualizing network topology.

https://raw.githubusercontent.com/netjson/django-netjsongraph/master/docs/images/visualizer.png https://raw.githubusercontent.com/netjson/django-netjsongraph/master/docs/images/admin.png

Current features

  • network topology collector supporting different formats:
    • NetJSON NetworkGraph

    • OLSR (jsoninfo/txtinfo)

    • batman-adv (jsondoc/txtinfo)

    • BMX6 (q6m)

    • CNML 1.0

    • additional formats can be added by specifying custom parsers

  • network topology visualizer based on netjsongraph.js

  • simple HTTP API that exposes data in NetJSON NetworkGraph format

  • admin interface that allows to easily manage, audit and debug topologies and their relative data (nodes, links)

  • receive topology from multiple nodes

Project goals

  • make it easy to visualize network topology data for the formats supported by netdiff

  • expose topology data via RESTful resources in NetJSON NetworkGraph format

  • make it easy to integrate in larger django projects to improve reusability

  • make it easy to extend its models by providing abstract models (needs improvement in this point)

  • provide ways to customize or replace the visualizer (needs improvement in this point)

  • keep the core very simple

  • provide ways to extend the default behaviour

  • encourage new features to be published as extensions

Install stable version from pypi

Install from pypi:

pip install django-netjsongraph

Install development version

Install tarball:

pip install https://github.com/netjson/django-netjsongraph/tarball/master

Alternatively you can install via pip using git:

pip install -e git+git://github.com/netjson/django-netjsongraph#egg=django-netjsongraph

If you want to contribute, install your cloned fork:

git clone git@github.com:<your_fork>/django-netjsongraph.git
cd django-netjsongraph
python setup.py develop

Setup (integrate in an existing django project)

Add rest_framework and django_netjsongraph to INSTALLED_APPS:

INSTALLED_APPS = [
    # other apps
    'rest_framework',
    'django_netjsongraph'
    # ...
]

Include urls in your urlconf (you can change the prefixes according to your needs):

from django.conf.urls import include, url

from django_netjsongraph.api import urls as netjsongraph_api
from django_netjsongraph.visualizer import urls as netjsongraph_visualizer

urlpatterns = [
    # your URLs ...
    url(r'^api/', include(netjsongraph_api)),
    url(r'', include(netjsongraph_visualizer)),
]

Create database tables:

./manage.py migrate

Management Commands

update_topology

After topology URLs (URLs exposing the files that the topology of the network) have been added in the admin, the update_topology management command can be used to collect data and start playing with the network graph:

./manage.py update_topology

The management command accepts a --label argument that will be used to search in topology labels, eg:

./manage.py update_topology --label mytopology

Logging

The update_topology management command will automatically try to log errors.

For a good default LOGGING configuration refer to the test settings.

Strategies

There are mainly two ways of collecting topology information:

  • FETCH strategy

  • RECEIVE strategy

Each Topology instance has a strategy field which can be set to the desired setting.

FETCH strategy

Topology data will be fetched from a URL.

When some links are not detected anymore they will be flagged as “down” straightaway.

RECEIVE strategy

Topology data is sent directly from one or more nodes of the network.

The collector waits to receive data in the payload of a POST HTTP request; when such a request is received, a key parameter it’s first checked against the Topology key.

If the request is authorized the collector proceeds to update the topology.

If the data is sent from one node only, it’s highly advised to set the expiration_time of the Topology instance to 0 (seconds), this way the system works just like in the FETCH strategy, with the only difference that the data is sent by one node instead of fetched by the collector.

If the data is sent from multiple nodes, you SHOULD set the expiration_time of the Topology instance to a value slightly higher than the interval used by nodes to send the topology, this way links will be flagged as “down” only if they haven’t been detected for a while. This mechanism allows to visualize the topology even if the network has been split in several parts, the disadvantage is that it will take a bit more time to detect links that go offline.

Settings

NETJSONGRAPH_PARSERS

type:

list

default:

[]

Additional custom netdiff parsers.

NETJSONGRAPH_SIGNALS

type:

str

default:

None

String representing python module to import on initialization.

Useful for loading django signals or to define custom behaviour.

NETJSONGRAPH_TIMEOUT

type:

int

default:

8

Timeout when fetching topology URLs.

NETJSONGRAPH_VISUALIZER_CSS

type:

str

default:

netjsongraph/style.css

Path of the visualizer css file. Allows customization of css according to user’s preferences.

Overriding visualizer templates

The following steps can be followed for overriding the visualizer’s default templates.

  • Create a directory in the project and put it in TEMPLATES['DIRS'], which can be found in settings.py file.

  • Create a sub directory named netjsongraph and add all the templates which shall override the default netjsongraph/* templates.

  • The name of the template file should be the same as the one it shall override.

Extending django-netjsongraph

django-netjsongraph provides a set of models, admin classes and generic views which can be imported, extended and reused by third party apps.

To extend django-netjsongraph, you MUST NOT add it to settings.INSTALLED_APPS, but you must create your own app (which goes into settings.INSTALLED_APPS), import the base classes from django-netjsongraph and add your customizations.

Extending models

This example provides an example of how to extend the base models of django-netjsongraph.

# models.py of your custom ``network`` app
from django.db import models

from django_netjsongraph.base.link import AbstractLink
from django_netjsongraph.base.node import AbstractNode
from django_netjsongraph.base.topology import AbstractTopology


class OrganizationMixin(models.Model):
    organization = models.ForeignKey('organization.Organization')

    class Meta:
        abstract = True


class Topology(OrganizationMixin, AbstractTopology):
    def clean(self):
        # your own logic here
        pass

    class Meta:
        abstract = False


class Node(AbstractNode):
    class Meta(AbstractNode.Meta):
        abstract = False


class Link(AbstractLink):
    class Meta(AbstractLink.Meta):
        abstract = False

Extending the admin

Following the above example, you can avoid duplicating the admin code by importing the base admin classes and registering your models with.

# admin.py of your app
from django.contrib import admin
from django_netjsongraph.base.admin import (AbstractLinkAdmin,
                                            AbstractNodeAdmin,
                                            AbstractTopologyAdmin)
# these are you custom models
from .models import Link, Node, Topology


class TopologyAdmin(AbstractTopologyAdmin):
    model = Topology


class NodeAdmin(AbstractNodeAdmin):
    model = Node


class LinkAdmin(AbstractLinkAdmin):
    model = Link


admin.site.register(Link, LinkAdmin)
admin.site.register(Node, NodeAdmin)
admin.site.register(Topology, TopologyAdmin)

Extending API views

If your use case doesn’t vary much from the base, you may also want to try to reuse the API views:

# your app.api.views
from ..models import Topology
from django_netjsongraph.api.generics import (BaseNetworkCollectionView,
                                              BaseNetworkGraphView,
                                              BaseReceiveTopologyView)


class NetworkCollectionView(BaseNetworkCollectionView):
    queryset = Topology.objects.filter(published=True)


class NetworkGraphView(BaseNetworkGraphView):
    queryset = Topology.objects.filter(published=True)


class ReceiveTopologyView(BaseReceiveTopologyView):
    model = Topology

network_collection = NetworkCollectionView.as_view() network_graph = NetworkGraphView.as_view() receive_topology = ReceiveTopologyView.as_view()

API URLs

If you are not making drastic changes to the api views, you can avoid duplicating the URL logic by using the get_api_urls function. Put this in your api urls.py:

# your app.api.urls
from django_netjsongraph.utils import get_api_urls
from . import views

urlpatterns = get_api_urls(views)

Extending AppConfig

You may want to reuse the AppConfig class of django-netjsongraph too:

from django_netjsongraph.apps import DjangoNetjsongraphConfig

class MyOwnConfig(DjangoNetjsongraphConfig):
    name = 'yourapp'
    label = 'yourapp'

Installing for development

Install sqlite:

sudo apt-get install sqlite3 libsqlite3-dev

Install your forked repo:

git clone git://github.com/<your_fork>/django-netjsongraph
cd django-netjsongraph/
python setup.py develop

Install test requirements:

pip install -r requirements-test.txt

Create database:

cd tests/
./manage.py migrate
./manage.py createsuperuser

Launch development server:

./manage.py runserver

You can access the visualizer at http://127.0.0.1:8000/ and the admin interface at http://127.0.0.1:8000/admin/.

Run tests with:

./runtests.py

Contributing

First off, thanks for taking the time to read these guidelines.

Trying to follow these guidelines is important in order to minimize waste and avoid misunderstandings.

  1. Ensure your changes meet the Project Goals

  2. If you found a bug please send a failing test with a patch

  3. If you want to add a new feature, announce your intentions in the issue tracker

  4. Fork this repo and install it by following the instructions in Installing for development

  5. Follow PEP8, Style Guide for Python Code

  6. Write code

  7. Write tests for your code

  8. Ensure all tests pass

  9. Ensure test coverage is not under 90%

  10. Document your changes

  11. Send pull request

Changelog

See CHANGES.

License

See LICENSE.

This projects bundles third-party javascript libraries in its source code:

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

django-netjsongraph-0.3.tar.gz (93.4 kB view details)

Uploaded Source

Built Distribution

django_netjsongraph-0.3-py2.py3-none-any.whl (108.7 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file django-netjsongraph-0.3.tar.gz.

File metadata

File hashes

Hashes for django-netjsongraph-0.3.tar.gz
Algorithm Hash digest
SHA256 642a9bb7c6cf5e799b50bc70b5b0815de9141f181a69ffcda94f89aa24ea6210
MD5 1d241601450cc13b414e6a54b4a96e0b
BLAKE2b-256 7acc10cff80aedad605505af4f17fcc067063fc713fc48fe65aad96787d477e9

See more details on using hashes here.

Provenance

File details

Details for the file django_netjsongraph-0.3-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for django_netjsongraph-0.3-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 22528ea535b639f8875b1de3a1b973730f2de8840abf25ac74eb480dada34e7d
MD5 34d76ee77cb8f624070f8324c70acd5b
BLAKE2b-256 e996fe348a22429101044796b927924216cb97f3f4b0e6a61ac0cecb1a1827e3

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page