Jupyter metakernel for apache spark and scala
Project description
# spylon-kernel
[![Build Status](https://travis-ci.org/maxpoint/spylon-kernel.svg?branch=master)](https://travis-ci.org/maxpoint/spylon-kernel)
[![codecov](https://codecov.io/gh/maxpoint/spylon-kernel/branch/master/graph/badge.svg)](https://codecov.io/gh/maxpoint/spylon-kernel)
A Scala [Jupyter kernel](http://jupyter.readthedocs.io/en/latest/projects/kernels.html) that uses [metakernel](https://github.com/Calysto/metakernel) in combination with [py4j](https://www.py4j.org/).
## Prerequisites
* Apache Spark 2.1.1 compiled for Scala 2.11
* Jupyter Notebook
* Python 3.5+
## Install
You can install the spylon-kernel package using `pip` or `conda`.
```bash
pip install spylon-kernel
# or
conda install -c conda-forge spylon-kernel
```
## Using it as a Scala Kernel
You can use spylon-kernel as Scala kernel for Jupyter Notebook. Do this when you want
to work with Spark in Scala with a bit of Python code mixed in.
Create a kernel spec for Jupyter notebook by running the following command:
```bash
python -m spylon_kernel install
```
Launch `jupyter notebook` and you should see a `spylon-kernel` as an option
in the *New* dropdown menu.
See [the basic example notebook](./examples/basic_example.ipynb) for information
about how to intiialize a Spark session and use it both in Scala and Python.
## Using it as an IPython Magic
You can also use spylon-kernel as a magic in an IPython notebook. Do this when
you want to mix a little bit of Scala into your primarily Python notebook.
```python
from spylon_kernel import register_ipython_magics
register_ipython_magics()
```
```scala
%%scala
val x = 8
x
```
## Using it as a Library
Finally, you can use spylon-kernel as a Python library. Do this when you
want to evaluate a string of Scala code in a Python script or shell.
```python
from spylon_kernel import get_scala_interpreter
interp = get_scala_interpreter()
# Evaluate the result of a scala code block.
interp.interpret("""
val x = 8
x
""")
interp.last_result()
```
# Release Process
Push a tag and submit a source dist to PyPI.
```
git commit -m 'REL: 0.2.1' --allow-empty
git tag -a 0.2.1 # and enter the same message as the commit
git push origin master # or send a PR
# if everything builds / tests cleanly, release to pypi
make release
```
Then update https://github.com/conda-forge/spylon-kernel-feedstock.
[![Build Status](https://travis-ci.org/maxpoint/spylon-kernel.svg?branch=master)](https://travis-ci.org/maxpoint/spylon-kernel)
[![codecov](https://codecov.io/gh/maxpoint/spylon-kernel/branch/master/graph/badge.svg)](https://codecov.io/gh/maxpoint/spylon-kernel)
A Scala [Jupyter kernel](http://jupyter.readthedocs.io/en/latest/projects/kernels.html) that uses [metakernel](https://github.com/Calysto/metakernel) in combination with [py4j](https://www.py4j.org/).
## Prerequisites
* Apache Spark 2.1.1 compiled for Scala 2.11
* Jupyter Notebook
* Python 3.5+
## Install
You can install the spylon-kernel package using `pip` or `conda`.
```bash
pip install spylon-kernel
# or
conda install -c conda-forge spylon-kernel
```
## Using it as a Scala Kernel
You can use spylon-kernel as Scala kernel for Jupyter Notebook. Do this when you want
to work with Spark in Scala with a bit of Python code mixed in.
Create a kernel spec for Jupyter notebook by running the following command:
```bash
python -m spylon_kernel install
```
Launch `jupyter notebook` and you should see a `spylon-kernel` as an option
in the *New* dropdown menu.
See [the basic example notebook](./examples/basic_example.ipynb) for information
about how to intiialize a Spark session and use it both in Scala and Python.
## Using it as an IPython Magic
You can also use spylon-kernel as a magic in an IPython notebook. Do this when
you want to mix a little bit of Scala into your primarily Python notebook.
```python
from spylon_kernel import register_ipython_magics
register_ipython_magics()
```
```scala
%%scala
val x = 8
x
```
## Using it as a Library
Finally, you can use spylon-kernel as a Python library. Do this when you
want to evaluate a string of Scala code in a Python script or shell.
```python
from spylon_kernel import get_scala_interpreter
interp = get_scala_interpreter()
# Evaluate the result of a scala code block.
interp.interpret("""
val x = 8
x
""")
interp.last_result()
```
# Release Process
Push a tag and submit a source dist to PyPI.
```
git commit -m 'REL: 0.2.1' --allow-empty
git tag -a 0.2.1 # and enter the same message as the commit
git push origin master # or send a PR
# if everything builds / tests cleanly, release to pypi
make release
```
Then update https://github.com/conda-forge/spylon-kernel-feedstock.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
spylon-kernel-0.3.0.tar.gz
(33.4 kB
view details)
File details
Details for the file spylon-kernel-0.3.0.tar.gz
.
File metadata
- Download URL: spylon-kernel-0.3.0.tar.gz
- Upload date:
- Size: 33.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 370a0a2f340d9191810004e4a0465db38de16f614d9bdf94b210f6f563832c81 |
|
MD5 | 0dc255822908e2ca97eb85df3ad22287 |
|
BLAKE2b-256 | a0d065e3dbb271051faee9ef43dddbccec29bcbfb59a0c09875957f25fc70473 |