Struct2Tensor is a package for parsing and manipulating structured data for TensorFlow

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for google_opensource from gravatar.com google_opensource Avatar for tfx_oss from gravatar.com tfx_oss

Unverified details

These details have not been verified by PyPI
Meta
  • License: Apache Software License (Apache 2.0)
  • Author: Google LLC
  • Tags tensorflow, protobuf, machine, learning
Classifiers

Project description

Struct2Tensor

Python PyPI

Introduction

struct2tensor is a library for parsing structured data inside of tensorflow. In particular, it makes it easy to manipulate structured data, e.g., slicing, flattening, copying substructures, and so on, as part of a TensorFlow model graph. The notebook in 'examples/prensor_playground.ipynb' provides a few examples of struct2tensor in action and an introduction to the main concepts. You can run the notebook in your browser through Google's colab environment, or download the file to run it in your own Jupyter environment.

There are two main use cases of this repo:

  1. To create a PIP package. The PIP package contains plug-ins (OpKernels) to an existing tensorflow installation.
  2. To staticlly link with tensorflow-serving.

As these processes are independent, one can follow either set of directions below.

Use a pre-built Linux PIP package.

From a virtual environment, run:

pip install struct2tensor

Nightly Packages

Struct2Tensor also hosts nightly packages at https://pypi-nightly.tensorflow.org on Google Cloud. To install the latest nightly package, please use the following command:

pip install -i https://pypi-nightly.tensorflow.org/simple struct2tensor

This will install the nightly packages for the major dependencies of Fairness Indicators such as TensorFlow Metadata (TFMD).

Creating a PIP package.

The struct2tensor PIP package is useful for creating models. It works with tensorflow 2.x.

In order to unify the process, we recommend compiling struct2tensor inside a docker container.

Downloading the Code

Go to your home directory.

Download the source code.

git clone https://github.com/google/struct2tensor.git
cd ~/struct2tensor

Use docker-compose

Install docker-compose.

Use it to build a pip wheel for Python 3.6 with tensorflow version 2:

docker-compose build manylinux2010
docker-compose run -e PYTHON_VERSION=36 -e TF_VERSION=RELEASED_TF_2 manylinux2010

Or build a pip wheel for Python 3.7 with tensorflow version 2 (note that if you run one of these docker-compose commands after the other, the second will erase the result from the first):

docker-compose build manylinux2010
docker-compose run -e PYTHON_VERSION=37 -e TF_VERSION=RELEASED_TF_2 manylinux2010

This will create a manylinux package in the ~/struct2tensor/dist directory.

Creating a static library

In order to construct a static library for tensorflow-serving, we run:

bazel build -c opt struct2tensor:prensor_kernels_and_ops

This can also be linked into another library.

TensorFlow Serving docker image

struct2tensor needs a couple of custom TensorFlow ops to function. If you train a model with struct2tensor and wants to serve it with TensorFlow Serving, the TensorFlow Serving binary needs to link with those custom ops. We have a pre-built docker image that contains such a binary. The Dockerfile is available at tools/tf_serving_docker/Dockerfile. The image is available at gcr.io/tfx-oss-public/s2t_tf_serving.

Please see the Dockerfile for details. But in brief, the image exposes port 8500 as the gRPC endpoint and port 8501 as the REST endpoint. You can set two environment variables MODEL_BASE_PATH and MODEL_NAME to point it to your model (either mount it to the container, or put your model on GCS). It will look for a saved model at ${MODEL_BASE_PATH}/${MODEL_NAME}/${VERSION_NUMBER}, where VERSION_NUMBER is an integer.

Compatibility

struct2tensor tensorflow
0.32.0 2.5.0
0.31.0 2.5.0
0.30.0 2.4.0
0.29.0 2.4.0
0.28.0 2.4.0
0.27.0 2.4.0
0.26.0 2.3.0
0.25.0 2.3.0
0.24.0 2.3.0
0.23.0 2.3.0
0.22.0 2.2.0
0.21.1 2.1.0
0.21.0 2.1.0
0.0.1.dev* 1.15

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for google_opensource from gravatar.com google_opensource Avatar for tfx_oss from gravatar.com tfx_oss

Unverified details

These details have not been verified by PyPI
Meta
  • License: Apache Software License (Apache 2.0)
  • Author: Google LLC
  • Tags tensorflow, protobuf, machine, learning
Classifiers

Release history Release notifications | RSS feed

0.47.0

0.46.0

0.45.0

0.44.0

0.43.0

0.42.0

0.41.0

0.40.0

0.39.0

0.38.0

0.37.0

0.36.0

0.35.0

0.34.0

0.33.0

This version

0.32.0

0.31.0

0.30.0

0.29.0

0.28.0

0.27.0

0.26.0

0.25.0

0.24.0

0.23.0

0.22.0

0.21.1

0.21.0

0.0.1.dev6 pre-release

0.0.1.dev5 pre-release

0.0.1.dev4 pre-release

0.0.1.dev3 pre-release

0.0.1.dev2 pre-release

0.0.1.dev1 pre-release

0.0.1.dev0 pre-release

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

struct2tensor-0.32.0-cp38-cp38-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64

struct2tensor-0.32.0-cp38-cp38-macosx_10_9_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.8 macOS 10.9+ x86-64

struct2tensor-0.32.0-cp37-cp37m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64

struct2tensor-0.32.0-cp37-cp37m-macosx_10_9_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.7m macOS 10.9+ x86-64

struct2tensor-0.32.0-cp36-cp36m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.12+ x86-64

struct2tensor-0.32.0-cp36-cp36m-macosx_10_9_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.6m macOS 10.9+ x86-64

File details

Details for the file struct2tensor-0.32.0-cp38-cp38-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.

File metadata

File hashes

Hashes for struct2tensor-0.32.0-cp38-cp38-manylinux2010_x86_64.manylinux_2_12_x86_64.whl
Algorithm Hash digest
SHA256 46bf1f98a833d47a5ebaf3fad8e0205bf4e1ecf178fdb2f40080670ce2888db7
MD5 1d778a7c1c75d30ff6bc111f7a2cbe58
BLAKE2b-256 3ddf4dcf6914aa4c7bed1f1d4626c33a7c6ee09a36fdee58a82031ede8354792

See more details on using hashes here.

File details

Details for the file struct2tensor-0.32.0-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: struct2tensor-0.32.0-cp38-cp38-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.8, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.8.2

File hashes

Hashes for struct2tensor-0.32.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 e0b71bb9d25f95845c89040896d716d25bc0b7fb33c6e038d0b028b95f8b1a9a
MD5 d50bab9f97e0172c0cdf98b317dcac6e
BLAKE2b-256 76a2e1a3687801c07c877e9e71fc015004b148931fc40978ac3486667130f9ec

See more details on using hashes here.

File details

Details for the file struct2tensor-0.32.0-cp37-cp37m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.

File metadata

File hashes

Hashes for struct2tensor-0.32.0-cp37-cp37m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl
Algorithm Hash digest
SHA256 cfca8bdcfb21d600350a740e7b2fc96df18dfcfaca44fd5c58baa0d69335963b
MD5 04c1f671f483e361eef1c5393e714ead
BLAKE2b-256 75bd1abb11d44b0a0261e3f2ce622166ac93295b08fb32deab4cd7d9fdf9f90a

See more details on using hashes here.

File details

Details for the file struct2tensor-0.32.0-cp37-cp37m-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: struct2tensor-0.32.0-cp37-cp37m-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.7m, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.7.3

File hashes

Hashes for struct2tensor-0.32.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 5a115927f7ffc2b602a6d4133521ae03254a9686c737187ba91e6dfe98a48e56
MD5 f0225e6b3d21bd710ddf1c260558ab1f
BLAKE2b-256 28daab6a259ff632e4757b7b2eb8f91ed9289d967738e15c95187979a75fa7ee

See more details on using hashes here.

File details

Details for the file struct2tensor-0.32.0-cp36-cp36m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.

File metadata

File hashes

Hashes for struct2tensor-0.32.0-cp36-cp36m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl
Algorithm Hash digest
SHA256 b9a7e138cae6e0b5cbabfd1dbf0479c83835a16b4882608ba6a0f4bdb0fc36a4
MD5 d4d08aec6f64883b3b90ecf927c2487b
BLAKE2b-256 13e19dd4b0e42409cc4529ed578031e4dcd6d1c76766b7b023abffc368b6dcd1

See more details on using hashes here.

File details

Details for the file struct2tensor-0.32.0-cp36-cp36m-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: struct2tensor-0.32.0-cp36-cp36m-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.6m, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.6.8

File hashes

Hashes for struct2tensor-0.32.0-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 d3d3f2cfa9ff9d29ad83192c55cc042d20f35992910deab994fbce5369358b98
MD5 8d7fe1bffd82e18a777d85b1ab66155b
BLAKE2b-256 0e99691a412d97b7b0815c6d4494f3842e7aa8b7acb98e8ffe42eee3faef6d4a

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page