Struct2Tensor is a package for parsing and manipulating structured data for TensorFlow

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for google_opensource from gravatar.com google_opensource Avatar for tfx_oss from gravatar.com tfx_oss

Unverified details

These details have not been verified by PyPI
Meta
  • License: Apache Software License (Apache 2.0)
  • Author: Google LLC
  • Tags tensorflow, protobuf, machine, learning
Classifiers

Project description

Struct2Tensor

Python PyPI

Introduction

struct2tensor is a library for parsing structured data inside of tensorflow. In particular, it makes it easy to manipulate structured data, e.g., slicing, flattening, copying substructures, and so on, as part of a TensorFlow model graph. The notebook in 'examples/prensor_playground.ipynb' provides a few examples of struct2tensor in action and an introduction to the main concepts. You can run the notebook in your browser through Google's colab environment, or download the file to run it in your own Jupyter environment.

There are two main use cases of this repo:

  1. To create a PIP package. The PIP package contains plug-ins (OpKernels) to an existing tensorflow installation.
  2. To staticlly link with tensorflow-serving.

As these processes are independent, one can follow either set of directions below.

Use a pre-built Linux PIP package.

From a virtual environment, run:

pip install struct2tensor

Nightly Packages

Struct2Tensor also hosts nightly packages at https://pypi-nightly.tensorflow.org on Google Cloud. To install the latest nightly package, please use the following command:

pip install -i https://pypi-nightly.tensorflow.org/simple struct2tensor

This will install the nightly packages for the major dependencies of Fairness Indicators such as TensorFlow Metadata (TFMD).

Creating a PIP package.

The struct2tensor PIP package is useful for creating models. It works with tensorflow 2.x.

In order to unify the process, we recommend compiling struct2tensor inside a docker container.

Downloading the Code

Go to your home directory.

Download the source code.

git clone https://github.com/google/struct2tensor.git
cd ~/struct2tensor

Use docker-compose

Install docker-compose.

Use it to build a pip wheel for Python 3.6 with tensorflow version 2:

docker-compose build manylinux2010
docker-compose run -e PYTHON_VERSION=36 -e TF_VERSION=RELEASED_TF_2 manylinux2010

Or build a pip wheel for Python 3.7 with tensorflow version 2 (note that if you run one of these docker-compose commands after the other, the second will erase the result from the first):

docker-compose build manylinux2010
docker-compose run -e PYTHON_VERSION=37 -e TF_VERSION=RELEASED_TF_2 manylinux2010

This will create a manylinux package in the ~/struct2tensor/dist directory.

Creating a static library

In order to construct a static library for tensorflow-serving, we run:

bazel build -c opt struct2tensor:prensor_kernels_and_ops

This can also be linked into another library.

TensorFlow Serving docker image

struct2tensor needs a couple of custom TensorFlow ops to function. If you train a model with struct2tensor and wants to serve it with TensorFlow Serving, the TensorFlow Serving binary needs to link with those custom ops. We have a pre-built docker image that contains such a binary. The Dockerfile is available at tools/tf_serving_docker/Dockerfile. The image is available at gcr.io/tfx-oss-public/s2t_tf_serving.

Please see the Dockerfile for details. But in brief, the image exposes port 8500 as the gRPC endpoint and port 8501 as the REST endpoint. You can set two environment variables MODEL_BASE_PATH and MODEL_NAME to point it to your model (either mount it to the container, or put your model on GCS). It will look for a saved model at ${MODEL_BASE_PATH}/${MODEL_NAME}/${VERSION_NUMBER}, where VERSION_NUMBER is an integer.

Compatibility

struct2tensor tensorflow
0.34.0 2.6.0
0.33.0 2.5.0
0.32.0 2.5.0
0.31.0 2.5.0
0.30.0 2.4.0
0.29.0 2.4.0
0.28.0 2.4.0
0.27.0 2.4.0
0.26.0 2.3.0
0.25.0 2.3.0
0.24.0 2.3.0
0.23.0 2.3.0
0.22.0 2.2.0
0.21.1 2.1.0
0.21.0 2.1.0
0.0.1.dev* 1.15

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for google_opensource from gravatar.com google_opensource Avatar for tfx_oss from gravatar.com tfx_oss

Unverified details

These details have not been verified by PyPI
Meta
  • License: Apache Software License (Apache 2.0)
  • Author: Google LLC
  • Tags tensorflow, protobuf, machine, learning
Classifiers

Release history Release notifications | RSS feed

0.47.0

0.46.0

0.45.0

0.44.0

0.43.0

0.42.0

0.41.0

0.40.0

0.39.0

0.38.0

0.37.0

0.36.0

0.35.0

This version

0.34.0

0.33.0

0.32.0

0.31.0

0.30.0

0.29.0

0.28.0

0.27.0

0.26.0

0.25.0

0.24.0

0.23.0

0.22.0

0.21.1

0.21.0

0.0.1.dev6 pre-release

0.0.1.dev5 pre-release

0.0.1.dev4 pre-release

0.0.1.dev3 pre-release

0.0.1.dev2 pre-release

0.0.1.dev1 pre-release

0.0.1.dev0 pre-release

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

struct2tensor-0.34.0-cp38-cp38-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64

struct2tensor-0.34.0-cp38-cp38-macosx_10_9_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.8 macOS 10.9+ x86-64

struct2tensor-0.34.0-cp37-cp37m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64

struct2tensor-0.34.0-cp37-cp37m-macosx_10_9_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.7m macOS 10.9+ x86-64

struct2tensor-0.34.0-cp36-cp36m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (2.2 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.12+ x86-64

struct2tensor-0.34.0-cp36-cp36m-macosx_10_9_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.6m macOS 10.9+ x86-64

File details

Details for the file struct2tensor-0.34.0-cp38-cp38-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.

File metadata

File hashes

Hashes for struct2tensor-0.34.0-cp38-cp38-manylinux2010_x86_64.manylinux_2_12_x86_64.whl
Algorithm Hash digest
SHA256 9d1c217753b3350c4a8b4f327b8b5117c8f2e80f35e9431a2f8e7aa102812e62
MD5 c27b85e047f74baf4ead0cc4ee3c7554
BLAKE2b-256 1b16b08bf3704360708346611ab28f4e0920831e5391b02907d7dc448bba768b

See more details on using hashes here.

File details

Details for the file struct2tensor-0.34.0-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: struct2tensor-0.34.0-cp38-cp38-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.8, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.4 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.8.2

File hashes

Hashes for struct2tensor-0.34.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 9af3bc46278fffa6fbff98fdd2da0a90c3bb2b0a59df6086b6f86f14f9ba1288
MD5 51529d1077522bf4f441df76e1c7d9f7
BLAKE2b-256 475bfdebc03dc26e9631850c7e73dbb703a566d01435e819c7fdce7ef7bc2b60

See more details on using hashes here.

File details

Details for the file struct2tensor-0.34.0-cp37-cp37m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.

File metadata

File hashes

Hashes for struct2tensor-0.34.0-cp37-cp37m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl
Algorithm Hash digest
SHA256 6816f80ca821d8785f8fb2622438e72b39dba329bae189b7b40ed011a44a9037
MD5 e8c4e9dd32a2cf5085c1af2a413a9a5e
BLAKE2b-256 86df5068ebf985480d3c8c76ef6d879e4ca4bf94a3b2dda73c4cf64e6cc5a024

See more details on using hashes here.

File details

Details for the file struct2tensor-0.34.0-cp37-cp37m-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: struct2tensor-0.34.0-cp37-cp37m-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.7m, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.4 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.7.3

File hashes

Hashes for struct2tensor-0.34.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 bc6cc1ea533181a7af78968778b9173c0297ae46520c1fbada3ab876df24a511
MD5 b82b6366506a285fcbbc87c2c0ff4c6c
BLAKE2b-256 d57be0e56cd96479d6024f3fefd4538c063a4a600c1259d03f1faffb58b1bf74

See more details on using hashes here.

File details

Details for the file struct2tensor-0.34.0-cp36-cp36m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.

File metadata

File hashes

Hashes for struct2tensor-0.34.0-cp36-cp36m-manylinux2010_x86_64.manylinux_2_12_x86_64.whl
Algorithm Hash digest
SHA256 eb1cea2eb32d5e9b76f106aff238fca88c1f835c909680bfb43dc0979ea7be70
MD5 6761086b2c76ff27333a928b58b0430d
BLAKE2b-256 2e438e1ad80e6331f55f6825b18340cd3f2e4f680f99e6bd3fe2a4041b62ac84

See more details on using hashes here.

File details

Details for the file struct2tensor-0.34.0-cp36-cp36m-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: struct2tensor-0.34.0-cp36-cp36m-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.6m, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.6.4 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.6.8

File hashes

Hashes for struct2tensor-0.34.0-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 72f1c817f5fcc3b08e641542cd6be9336588c53451554531321c468a0d0b884c
MD5 313389930df3959816d35dbce9ce8fb7
BLAKE2b-256 7f1de8519a6114404460068824bdd03613b729ef7f28f7b4440e995c6b03a0c3

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page