Struct2Tensor is a package for parsing and manipulating structured data for TensorFlow

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for google_opensource from gravatar.com google_opensource Avatar for tfx_oss from gravatar.com tfx_oss

Unverified details

These details have not been verified by PyPI
Meta
  • License: Apache Software License (Apache 2.0)
  • Author: Google LLC
  • Tags tensorflow, protobuf, machine, learning
Classifiers

Project description

Struct2Tensor

Python PyPI

Introduction

struct2tensor is a library for parsing structured data inside of tensorflow. In particular, it makes it easy to manipulate structured data, e.g., slicing, flattening, copying substructures, and so on, as part of a TensorFlow model graph. The notebook in 'examples/prensor_playground.ipynb' provides a few examples of struct2tensor in action and an introduction to the main concepts. You can run the notebook in your browser through Google's colab environment, or download the file to run it in your own Jupyter environment.

There are two main use cases of this repo:

  1. To create a PIP package. The PIP package contains plug-ins (OpKernels) to an existing tensorflow installation.
  2. To staticlly link with tensorflow-serving.

As these processes are independent, one can follow either set of directions below.

Use a pre-built Linux PIP package.

From a virtual environment, run:

pip install struct2tensor

Nightly Packages

Struct2Tensor also hosts nightly packages at https://pypi-nightly.tensorflow.org on Google Cloud. To install the latest nightly package, please use the following command:

pip install -i https://pypi-nightly.tensorflow.org/simple struct2tensor

This will install the nightly packages for the major dependencies of Fairness Indicators such as TensorFlow Metadata (TFMD).

Creating a PIP package.

The struct2tensor PIP package is useful for creating models. It works with tensorflow 2.x.

In order to unify the process, we recommend compiling struct2tensor inside a docker container.

Downloading the Code

Go to your home directory.

Download the source code.

git clone https://github.com/google/struct2tensor.git
cd ~/struct2tensor

Use docker-compose

Install docker-compose.

Use it to build a pip wheel for Python 3.6 with tensorflow version 2:

docker-compose build manylinux2010
docker-compose run -e PYTHON_VERSION=36 -e TF_VERSION=RELEASED_TF_2 manylinux2010

Or build a pip wheel for Python 3.7 with tensorflow version 2 (note that if you run one of these docker-compose commands after the other, the second will erase the result from the first):

docker-compose build manylinux2010
docker-compose run -e PYTHON_VERSION=37 -e TF_VERSION=RELEASED_TF_2 manylinux2010

This will create a manylinux package in the ~/struct2tensor/dist directory.

Creating a static library

In order to construct a static library for tensorflow-serving, we run:

bazel build -c opt struct2tensor:prensor_kernels_and_ops

This can also be linked into another library.

TensorFlow Serving docker image

struct2tensor needs a couple of custom TensorFlow ops to function. If you train a model with struct2tensor and wants to serve it with TensorFlow Serving, the TensorFlow Serving binary needs to link with those custom ops. We have a pre-built docker image that contains such a binary. The Dockerfile is available at tools/tf_serving_docker/Dockerfile. The image is available at gcr.io/tfx-oss-public/s2t_tf_serving.

Please see the Dockerfile for details. But in brief, the image exposes port 8500 as the gRPC endpoint and port 8501 as the REST endpoint. You can set two environment variables MODEL_BASE_PATH and MODEL_NAME to point it to your model (either mount it to the container, or put your model on GCS). It will look for a saved model at ${MODEL_BASE_PATH}/${MODEL_NAME}/${VERSION_NUMBER}, where VERSION_NUMBER is an integer.

Compatibility

struct2tensor tensorflow
0.33.0 2.5.0
0.32.0 2.5.0
0.31.0 2.5.0
0.30.0 2.4.0
0.29.0 2.4.0
0.28.0 2.4.0
0.27.0 2.4.0
0.26.0 2.3.0
0.25.0 2.3.0
0.24.0 2.3.0
0.23.0 2.3.0
0.22.0 2.2.0
0.21.1 2.1.0
0.21.0 2.1.0
0.0.1.dev* 1.15

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for google_opensource from gravatar.com google_opensource Avatar for tfx_oss from gravatar.com tfx_oss

Unverified details

These details have not been verified by PyPI
Meta
  • License: Apache Software License (Apache 2.0)
  • Author: Google LLC
  • Tags tensorflow, protobuf, machine, learning
Classifiers

Release history Release notifications | RSS feed

0.47.0

0.46.0

0.45.0

0.44.0

0.43.0

0.42.0

0.41.0

0.40.0

0.39.0

0.38.0

0.37.0

0.36.0

0.35.0

0.34.0

This version

0.33.0

0.32.0

0.31.0

0.30.0

0.29.0

0.28.0

0.27.0

0.26.0

0.25.0

0.24.0

0.23.0

0.22.0

0.21.1

0.21.0

0.0.1.dev6 pre-release

0.0.1.dev5 pre-release

0.0.1.dev4 pre-release

0.0.1.dev3 pre-release

0.0.1.dev2 pre-release

0.0.1.dev1 pre-release

0.0.1.dev0 pre-release

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

struct2tensor-0.33.0-cp38-cp38-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64

struct2tensor-0.33.0-cp38-cp38-macosx_10_9_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.8 macOS 10.9+ x86-64

struct2tensor-0.33.0-cp37-cp37m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64

struct2tensor-0.33.0-cp37-cp37m-macosx_10_9_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.7m macOS 10.9+ x86-64

struct2tensor-0.33.0-cp36-cp36m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.12+ x86-64

struct2tensor-0.33.0-cp36-cp36m-macosx_10_9_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.6m macOS 10.9+ x86-64

File details

Details for the file struct2tensor-0.33.0-cp38-cp38-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.

File metadata

File hashes

Hashes for struct2tensor-0.33.0-cp38-cp38-manylinux2010_x86_64.manylinux_2_12_x86_64.whl
Algorithm Hash digest
SHA256 59e4cbbda99c93247f80e03ac5f1506c9e4e95541ce6ed774a0d38c49da14e88
MD5 d6867fd3a31d462477ef98ce3cef4cd2
BLAKE2b-256 202b1c0dd5cdd07c0ac91bcd38f60bea2a3a7f8d9f3f054d99b030ebe8ec3851

See more details on using hashes here.

File details

Details for the file struct2tensor-0.33.0-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: struct2tensor-0.33.0-cp38-cp38-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.8, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.8.2

File hashes

Hashes for struct2tensor-0.33.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 5f6c3c3166a66d28b52df3b7fd59d026f6fad7a3ada7e0f4b73d0a987a805d84
MD5 d083531896a3f5b1cae627b76b1c78c0
BLAKE2b-256 b8c8c16e64d5462563a92546710d387d587e60236e11df314143e1dfed84d3e3

See more details on using hashes here.

File details

Details for the file struct2tensor-0.33.0-cp37-cp37m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.

File metadata

File hashes

Hashes for struct2tensor-0.33.0-cp37-cp37m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl
Algorithm Hash digest
SHA256 b3e0a01ad3140fc04b083f0c9a2fcdf883f84d66a4f8205aff1a425c1c736073
MD5 769a5a2f50359b87d5e6b4eff1c0bf9f
BLAKE2b-256 6e2c32d6b044722fdaf6e21ac3ae6b48854a87a7dd3f5a02c569a69799e1fcbe

See more details on using hashes here.

File details

Details for the file struct2tensor-0.33.0-cp37-cp37m-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: struct2tensor-0.33.0-cp37-cp37m-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.7m, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.7.3

File hashes

Hashes for struct2tensor-0.33.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 b686d70bf771486f51878080131c0044d1e764288bd464ea6704dd41a6c0b01a
MD5 f15dc16ed2682c839f72f3ec3139f4de
BLAKE2b-256 065146ca3df9baaac8413f25cf76cdf6e5fc50f90cd97100b3718175d73797cb

See more details on using hashes here.

File details

Details for the file struct2tensor-0.33.0-cp36-cp36m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.

File metadata

File hashes

Hashes for struct2tensor-0.33.0-cp36-cp36m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl
Algorithm Hash digest
SHA256 d048a73ead78636a724f6ffccf04d445373a29859757da38e34618d20ce6b87e
MD5 4da8f46982bbfbc8fcc6291a494f5874
BLAKE2b-256 34ff4062ea3cede45164bf4a96ef678aeb2a7b4882484cc2b29d04183e7b2aba

See more details on using hashes here.

File details

Details for the file struct2tensor-0.33.0-cp36-cp36m-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: struct2tensor-0.33.0-cp36-cp36m-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.6m, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.6.8

File hashes

Hashes for struct2tensor-0.33.0-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 44b24b97ed791023554998a15ad21f8b84adf55ff4c6a0b56400dbfb00496bce
MD5 b5501e782e49a2577084837dafcda201
BLAKE2b-256 e5f6fd08a0fe440ef165fbaadb8f69e0a9849f31540e965693b10105f4c4dd66

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page