d2a · PyPI

It translates django models into sqlalchemy orm/expression objects.

These details have not been verified by PyPI

Project links

Homepage

Project description

https://github.com/walkframe/d2a/workflows/build/badge.svg?branch=master

https://img.shields.io/badge/License-MIT-blue.svg

Requirements

Python: 3.5 or later.
- Tested with 3.6, 3.9
Django: 2.x, 3.x
- Tested with 2.2.9, 3.0.1, 3.1.1
SQLAlchemy: 1.1 or later.
- Tested with 1.1.0, 1.4.1

Installation

$ pip install d2a

Add d2a to settings.INSTALLED_APPS.

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    # top or here
    'd2a',

    # example apps
    'books',
    'sales',
]

Usage

Code generation

$ python manage.py sqla_codegen

Config example

Output example

Auto loading [deprecated]

AUTOLOAD will be enabled when D2A_CONFIG["AUTOLOAD"] is specified.

Config example

models_sqla (default) will be made as a module in all apps.

>>> from books import models_sqla
>>> models_sqla.  # tab completion
models_sqla.Author(            models_sqla.BookCategory(      models_sqla.CategoryRelation(  models_sqla.transfer(
models_sqla.Book(              models_sqla.Category(          models_sqla.models

>>> models_sqla.Book
<class 'd2a.book'>

>>> models_sqla.Book.  # tab completion
models_sqla.Book.author       models_sqla.Book.content      models_sqla.Book.metadata     models_sqla.Book.tags
models_sqla.Book.author_id    models_sqla.Book.description  models_sqla.Book.mro(         models_sqla.Book.title
models_sqla.Book.category     models_sqla.Book.id           models_sqla.Book.price
# SQL Expression schema
>>> models_sqla.Book.__table__
Table(
  'book', MetaData(bind=None),
  Column('id', UUID(), table=<book>, primary_key=True, nullable=False, default=ColumnDefault(<function uuid4 at 0x7f3cebe7e598>)),
  Column('price', JSON(astext_type=Text()), table=<book>, nullable=False),
  Column('title', VARCHAR(length=255), table=<book>, nullable=False),
  Column('description', TEXT(), table=<book>),
  Column('author_id', INTEGER(), ForeignKey('author.id'), table=<book>),
  Column('content', BYTEA(), table=<book>, nullable=False),
  Column('tags', ARRAY(VARCHAR()), table=<book>, nullable=False),
  schema=None
)

Also it can extract model declared implicitly depending on m2m field. (in this case, BookCategory)

Note

You can set configrations to settings.py.

Example:

# This variable can be omitted.
D2A_CONFIG = {
    'AUTOLOAD': { # optional
        # module name: It can be used different module name from `models_sqla`.
        'module': 'modelsa',  # optional, default: 'models_sqla'
        # transfer function's args after 'exports' arg.
        'option': {  # optional
            'db_type': 'postgresql',  # default: 'default'
            'back_type': 'backref',  # default: 'backref'
            'as_table': True,  # default: False
            'name_formatter': str.upper,  # default: get_camelcase
        }
    },
    # converting rules for customized fields
    'ALIASES': {  # optional
        # Evaluates ExtendedImageField as ImageField
        ExtendedImageField: models.ImageField,
    },
    'USE_GEOALCHEMY2': True,  # default: False
}

Per models module

If you want to create a module manually, create a models_sqla.py in the apps.

Write like the following to it:

from d2a import transfer
from . import models
transfer(models, globals())

models_sqla.py exists, auto module creation will be omitted.

And if you create every models_sqla.py manually, it is unnecessary to set d2a to settings.INSTALLED_APPS.

Example:

project_postgresql/books/models_sqla.py
You can omit specifying db_type, then it automatically detects a database type from settings.DATABASES['default'].
- Now postgresql, mysql and oracle are allowed, the other types will be converted to the following types as default type: sqlalchemy/types.py

Per model

If you just want to convert one model, you should use declare function.

>>> from d2a import declare
>>> from sales.models import Sales
>>> sales = declare(Sales)
>>> sales
<class 'd2a.sales'>

>>> sales.__table__
Table(
  'sales', MetaData(bind=None),
  Column('id', BIGINT(), table=<sales>, primary_key=True, nullable=False),
  Column('book_id', UUID(), ForeignKey('book.id'), table=<sales>, nullable=False),
  Column('sold', TIMESTAMP(), table=<sales>, nullable=False),
  Column('reservation', INTERVAL(), table=<sales>),
  Column('source', INET(), table=<sales>),
  schema=None
)

>>> sales.
sales.book         sales.id           sales.mro(         sales.sold
sales.book_id      sales.metadata     sales.reservation  sales.source

Custom fields

If you are using customized field (not built-in), you can register the field as the other field using alias or alias_dict method.

from django.db.models import ImageField

class ExtendedImageField(ImageField):
    """something customizing"""

from d2a import alias
alias(ExtendedImageField, ImageField)

# or
alias_dict({
    ExtendedImageField: ImageField,
})

When the translation rule is not found, it will warn you and continue. (2.6.x later)

You can change the behavior by specifying the following values to D2A_CONFIG['MISSING'].

None:

Ignores the warning.

Field:

Uses the specifying field instead of the unknown field.

e.g. Using CharField

from django.db.models import CharField

D2A_CONFIG = {
    'MISSING': CharField,
}

Note

Before 2.1.x d2a maps django.contrib.postgres.fields.JSONField to JSON by mistake. It should have mapped it to JSONB.

Since 2.2.0 the mapping is fixed.

If you want to use JSON type as before, then you are able to map some 3rd-party jsonfield to JSON as follows:

from jsonfield import JSONField  # e.g. https://github.com/dmkoch/django-jsonfield
import d2a

d2a.alias(JSONField, d2a.JSONType)

Or add to settings.D2A_CONFIG['ALIASES'].

Querying shortcut

Expression

There are two functions.

query_expression:: To retrieve SELECT results, and returns a list containing record.
execute_expression:: To execute INSERT, DELETE, UPDATE statements, and returns num of records having been affected.

>>> from sqlalchemy import (
...     select,
...     insert,
... )

>>> from d2a import query_expression, execute_expression

# if you try on `project_mysql` demo, you should write ``from books.modelsa import Author``
>>> from books.models_sqla import Author

>>> AuthorTable = Author.__table__

>>> records = [
...     {'name': 'a', 'age': 10},
...     {'name': 'b', 'age': 30},
...     {'name': 'c', 'age': 20},
... ]

>>> # insert
>>> stmt = insert(AuthorTable).values(records)
>>> execute_expression(stmt)
3

>>> # select
>>> stmt = select([
...     AuthorTable.c.id,
...     AuthorTable.c.name,
...     AuthorTable.c.age,
... ]).select_from(AuthorTable).order_by(AuthorTable.c.age)

>>> query_expression(stmt)
[
  OrderedDict([('id', 12), ('name', 'a'), ('age', 10)]),
  OrderedDict([('id', 14), ('name', 'c'), ('age', 20)]),
  OrderedDict([('id', 13), ('name', 'b'), ('age', 30)])
]

>>> # record as tuple
>>> query_expression(stmt, as_col_dict=False)
[(12, 'a', 10), (14, 'c', 20), (13, 'b', 30)]

>>> query_expression(stmt, as_col_dict=False, debug={'printer': print, 'show_explain': True, 'sql_format': True})
====================================================================================================
SELECT author.id,
       author.name,
       author.age
FROM author
ORDER BY author.age
====================================================================================================
Sort  (cost=16.39..16.74 rows=140 width=522) (actual time=0.027..0.028 rows=18 loops=1)
  Sort Key: age
  Sort Method: quicksort  Memory: 25kB
  ->  Seq Scan on author  (cost=0.00..11.40 rows=140 width=522) (actual time=0.007..0.009 rows=18 loops=1)
Planning time: 0.072 ms
Execution time: 0.047 ms
[(12, 'a', 10), (14, 'c', 20), (13, 'b', 30)]

Note

I added argument of query_expression() to see debugging information.

Specify options as dict type like the following:

query_expression(stmt, debug={  # all options can be skipped.
    'show_sql': True, # if showing the sql query or not.
    'show_explain': False, # if showing explain for the sql query or not.
    'sql_format': False, # if formatting the sql query or not.
    'sql_reindent': True, # if setting indent the sql query or not.
    'sql_keyword_case': 'upper', # A rule converting reserved words.
    'explain_prefix': depends on the database type. unless you specify it, an appropriate prefix will be automatically used.
    'printer': logger.debug, # printing method, if you use python3, then try `print` function.
    'delimiter': '=' * 100, # characters dividing debug informations.
    'database': 'default' # django database
})

Default is {} (An empty dict means disabling debug.)

ORM

There is a function named make_session for ORM mode.

>>> from d2a import make_session
>>> from books.models_sqla import Author

>>> with make_session() as session:
...     # it commits and flushes automatically when the scope exits.
...     a = Author()
...     a.name = 'righ'
...     a.age = 30
...     session.add(a)
...
>>> with make_session() as session:
...     # when the session was rolled back or causes some exception in the context,
...     # it won't register records in the session.
...     a = Author()
...     a.name = 'teruhiko'
...     a.age = 85
...     session.add(a)
...     session.rollback()
...
>>> with make_session() as session:
...     session.query(Author.name, Author.age).all()
...
[('righ', 30)]

It receives the following arguments:

engine:: engine object or database-type (string) (default: None). When it is omitted, it guesses database type and gets an engine automatically.
autoflush:: It is the same as sessionmaker (default: True)
autocommit:: It is the same as sessionmaker (default: False)
expire_on_commit:: It is the same as sessionmaker (default: True)
info:: It is the same as sessionmaker (default: None)

All arguments can be omitted.

Demo

start up environment

$ git clone git@github.com:walkframe/d2a.git
$ cd d2a
$ docker-compose up

preparation

$ docker exec -it d2a_app /bin/bash
# python -m venv venv # only first time
# source venv/bin/activate
(venv) # cd project_postgresql/ # (or mysql)
(venv) project_postgresql # ./manage.py migrate

execute

(venv) project_postgresql # ./manage.py shell

>>> from books import models_sqla
>>> book = models_sqla.Book()
>>> author = models_sqla.Author()
>>> book.author = author
>>> author.books
[<d2a.book object at 0x7f3cec539358>]
# And do something you want to do ;)

GeoDjango

GeoDjango-GeoAlchemy2 translation demo

History

3.0.x:

Code generation debut.
AUTOLOAD got deprecated.
- When you do not specify "AUTOLOAD" in your``settings.D2A_CONFIG``, AUTOLOAD get disabled.

2.8.x:

2021-03-19
Dropped Python 3.5.x.
Supported Python 3.9.x.
Supported SQLAlchemy 1.4.x.

2.7.x:

Dropped sleep option.

2.6.x:

2020-06-06
Add MISSING option.

2.5.x:

2020-05-26
Dropped support for django1.11
Changed key format. (prefix:_ to around:__)
Set up continuous deployment to PyPI.

2.4.x:

2020-05-26
Add postgres fields
- CIText fields
  - CICharField
  - CIEmailField
  - CITextField
- Range fields
  - IntegerRangeField
  - BigIntegerRangeField
  - DecimalRangeField
  - FloatRangeField
  - DateTimeRangeField
  - DateRangeField
- HStoreField

2.3.x:

2020-05-25
GeoAlchemy2 support.
- It translates GeoDjango fields into GeoAlchemy2 fields.

2.2.x:

2020-01-03:

Release

2020-02-13:

dealt with failing CI
Supported the following fields:
- PositiveBigIntegerField
- SmallAutoField
Dropped support for the following versions:
- Python: < 3.5.0.
- SQLAlchemy: < 1.1.0.
d2a.make_engine can receive all create_engine arguments now.
Remapped django JSONField to JSONB (it was JSON before)
Migrated to GitHub Actions from CircleCI.

2.1.x:

Changed:
- Warning: Changed arg name as_dict to as_col_dict
Added:

as_row_list:

If result set being list type or not.

default is True.

dict_method:

A method making row to dict. You got to be able to change the method to dict().

default is collections.OrderedDict.

debug:

If showing debug information or not. specify options dict.

2.0.x:

Added a shortcut function for executing in ORM mode.
Added two shortcut functions for executing in EXPRESSION mode.

1.1.x:

(2019-02-17)
Added a function to load all models automatically.

1.0.2:

(2018-07-10)
Improved a little.

1.0.1:

(2018-07-06)
Fixed a bug, that it will be provided None even though it’s not specified default argument.

1.0.0:

(2018-07-05)
Fixed bugs.
Added unit tests.

0.0.6:

Fixed a bug that abstract models become the targets.
Deleted install_requires.

0.0.5:

added alias method.

0.0.4:

fixed bugs.

0.0.3:

it got easy to declare custom field.
transfer method can define secondary table.

0.0.2:

it supported m2m field.
it limited django version less than 1.9.

0.0.1:

first release (2017-12-27)

Project details

These details have not been verified by PyPI

Project links

Homepage

Release history Release notifications | RSS feed

3.0.1

Apr 18, 2021

3.0.0

Apr 17, 2021

3.0.0b12 pre-release

Apr 17, 2021

3.0.0b11 pre-release

Apr 17, 2021

3.0.0b10 pre-release

Apr 17, 2021

3.0.0b9 pre-release

Apr 17, 2021

3.0.0b8 pre-release

Apr 17, 2021

This version

3.0.0b7 pre-release

Apr 17, 2021

3.0.0b6 pre-release

Apr 16, 2021

3.0.0b5 pre-release

Apr 16, 2021

3.0.0b4 pre-release

Apr 16, 2021

3.0.0b3 pre-release

Apr 16, 2021

3.0.0b2 pre-release

Apr 16, 2021

3.0.0b1 pre-release

Apr 16, 2021

2.8.0

Mar 19, 2021

2.7.0

Sep 19, 2020

2.6.2

Jun 23, 2020

2.6.1

Jun 21, 2020

2.6.0

Jun 6, 2020

2.5.0

May 26, 2020

2.4.0

May 26, 2020

2.3.0

May 25, 2020

2.2.2

Feb 13, 2020

2.2.1

Jan 3, 2020

2.2.0

Jan 3, 2020

2.1.2

Jun 15, 2019

2.1.1

May 5, 2019

2.1.0

Apr 24, 2019

2.0.0

Mar 20, 2019

1.1.4

Feb 28, 2019

1.1.3

Feb 21, 2019

1.1.2

Feb 21, 2019

1.1.1

Feb 20, 2019

1.1.0

Feb 17, 2019

1.0.2

Jul 10, 2018

1.0.1

Jul 6, 2018

1.0.0

Jul 5, 2018

0.0.6

Feb 9, 2018

0.0.5

Feb 6, 2018

0.0.4

Jan 9, 2018

0.0.3

Jan 4, 2018

0.0.2

Dec 29, 2017

0.0.1

Dec 27, 2017

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

d2a-3.0.0b7.tar.gz (25.4 kB view details)

Uploaded Apr 17, 2021 Source

Built Distribution

d2a-3.0.0b7-py3-none-any.whl (21.7 kB view details)

Uploaded Apr 17, 2021 Python 3

File details

Details for the file d2a-3.0.0b7.tar.gz.

File metadata

Download URL: d2a-3.0.0b7.tar.gz
Upload date: Apr 17, 2021
Size: 25.4 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/3.4.1 importlib_metadata/3.10.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.4

File hashes

Hashes for d2a-3.0.0b7.tar.gz
Algorithm	Hash digest
SHA256	`3c72af4b8eff111d9a0d06d4333ad231753716ce185b82cf36b062f1e19673a6`
MD5	`d8c44d89195989266f7a5cf4b199d77b`
BLAKE2b-256	`d74eda85b3bc3463f6106e834785ab254e3e17be2e18581be5328d68f067bd4d`

See more details on using hashes here.

File details

Details for the file d2a-3.0.0b7-py3-none-any.whl.

File metadata

Download URL: d2a-3.0.0b7-py3-none-any.whl
Upload date: Apr 17, 2021
Size: 21.7 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/3.4.1 importlib_metadata/3.10.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.4

File hashes

Hashes for d2a-3.0.0b7-py3-none-any.whl
Algorithm	Hash digest
SHA256	`2a3aaa2fb758f22e7ed4ffa714a2f807d5c8670ddafddea9385a61ead6718636`
MD5	`33b9ac17f366d4839fdc9e3d82b03373`
BLAKE2b-256	`7d808799443b89df5b550b9600f55d2637423273836a8298664be8e84d7f9348`

See more details on using hashes here.

d2a 3.0.0b7

Navigation

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Project description

Requirements

Installation

Usage

Code generation

Auto loading [deprecated]

Per models module

Per model

Custom fields

Querying shortcut

Expression

ORM

Demo

start up environment

preparation

execute

GeoDjango

Links

History

Project details

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Release history Release notifications | RSS feed

Download files

Source Distribution

Built Distribution

File details

File metadata

File hashes

File details

File metadata

File hashes