freezer-api 6.0.0.0rc1

Install freezer-api

# git clone https://git.openstack.org/openstack/freezer-api.git
# cd freezer-api
# pip install ./

edit config file

# sudo mkdir -p /etc/freezer
# sudo cp etc/freezer/freezer-api.conf.sample /etc/freezer/freezer-api.conf
# sudo cp etc/freezer/freezer-paste.ini /etc/freezer/freezer-paste.ini
# sudo cp etc/freezer/policy.json /etc/freezer/policy.json
# sudo vi /etc/freezer/freezer-api.conf
# sudo vi /etc/freezer/freezer-paste.ini

setup/configure the db

The currently supported db is Elasticsearch.

In case you are using a dedicated instance of the server, you’ll need to install Elasticsearch and start it. Depending on the OS flavor it might be different.

Below you can find an example for Ubuntu Xenial.

First of all JRE and Elasticsearch should be installed:

# apt install openjdk-8-jre
# wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/deb/elasticsearch/2.3.1/elasticsearch-2.3.1.deb
# dpkg -i elasticsearch-2.3.1.deb

and started:

# systemctl start elasticsearch

Elasticsearch needs to know what type of data each document’s field contains. This information is contained in the mapping, or schema definition. Elasticsearch will use dynamic mapping to try to guess the field type from the basic datatypes available in JSON, but some field’s properties have to be explicitly declared to tune the indexing engine. To do that, use the freezer-manage command:

# freezer-manage db sync

You should have updated your configuration files before doing this step. freezer-manage has the following options:

• To create the db mappings use the following command:

  # freezer-manage db sync

• To update the db mappings using the following command. Update means that you might have some mappings and you want to update it with a more recent ones

  # freezer-manage db update

• To remove the db mappings using the following command

  # freezer-manage db remove

• To print the db mappings using the following command

  # freezer-manage db show

• To update your settings (number of replicas) all what you need to do is to change its value in the configuration file and then run the following command

  # freezer-manage db update-settings

If you provided an invalid number of replicas that will cause problems later on, so it’s highly recommended to make sure that you are using the correct number of replicas. For more info click here Elasticsearch_Replicas_instructions

• To get information about optional additional parameters:

  # freezer-manage -h

• If you want to add any additional parameter like –yes or –erase, they should be before the db option. Check the following examples:

Wrong Example:

# freezer-manage db sync -y -e

Correct Example:

# freezer-manage -y -e db sync

run simple instance

# freezer-api

examples running using uwsgi

# uwsgi --http :9090 --need-app --master --module freezer_api.cmd.wsgi:application

# uwsgi --https :9090,foobar.crt,foobar.key --need-app --master --module freezer_api.cmd.wsgi:application

example running freezer-api with apache2

# sudo vi /etc/apache2/sites-enabled/freezer-api.conf

<VirtualHost ...>
    WSGIDaemonProcess freezer-api processes=2 threads=2 user=freezer display-name=%{GROUP}
    WSGIProcessGroup freezer-api
    WSGIApplicationGroup freezer-api
    WSGIScriptAlias / /opt/stack/freezer_api/cmd/wsgi.py

    ErrorLog /var/log/freezer-api/freezer-api.log
    CustomLog /var/log/freezer-api/freezer-api_access.log combined
    LogLevel info

    <Directory /opt/stack/freezer_api>
      Options Indexes FollowSymLinks MultiViews
      Require all granted
      AllowOverride None
      Order allow,deny
      allow from all
      LimitRequestBody 102400
    </Directory>
</VirtualHost>

Edit local.conf

To configure the Freezer API with DevStack, you will need to enable the freezer-api plugin by adding one line to the [[local|localrc]] section of your local.conf file:

enable_plugin freezer-api <GITURL> [GITREF]

where

<GITURL> is the URL of a freezer-api repository
[GITREF] is an optional git ref (branch/ref/tag).  The default is master.

For example

enable_plugin freezer-api https://git.openstack.org/openstack/freezer-api.git master

Plugin Options

The plugin makes use of apache2 by default. To use the uwsgi server set the following environment variable

export FREEZER_API_SERVER_TYPE=uwsgi

The default port is 9090. To configure the api to listen on a different port set the variable FREEZER_API_PORT. For example to make use of port 19090 use

export FREEZER_API_PORT=19090

For more information, see openstack_devstack_plugins_install

General

GET /       List API version
GET /v1     JSON Home document, see http://tools.ietf.org/html/draft-nottingham-json-home-03

Backup metadata

GET    /v1/backups(?limit,offset)  Lists backups
POST   /v1/backups                 Creates backup entry

GET    /v1/backups/{backup_id}     Get backup details
DELETE /v1/backups/{backup_id}     Deletes the specified backup

Freezer clients management

GET    /v1/clients(?limit,offset)       Lists registered clients
POST   /v1/clients                      Creates client entry

GET    /v1/clients/{freezerc_id}     Get client details
PATCH  /v1/clients/{freezerc_id}     Updates the specified client information
DELETE /v1/clients/{freezerc_id}     Deletes the specified client information

Freezer jobs management

GET    /v1/jobs(?limit,offset)     Lists registered jobs
POST   /v1/jobs                    Creates job entry

GET    /v1/jobs/{jobs_id}          Get job details
POST   /v1/jobs/{jobs_id}          creates or replaces a job entry using the specified job_id
DELETE /v1/jobs/{jobs_id}          Deletes the specified job information
PATCH  /v1/jobs/{jobs_id}          Updates part of the document

Freezer actions management

GET    /v1/actions(?limit,offset)  Lists registered action
POST   /v1/actions                 Creates action entry

GET    /v1/actions/{actions_id}    Get action details
POST   /v1/actions/{actions_id}    creates or replaces a action entry using the specified action_id
DELETE /v1/actions/{actions_id}    Deletes the specified action information
PATCH  /v1/actions/{actions_id}    Updates part of the action document

Freezer sessions management

GET    /v1/sessions(?limit,offset)  Lists registered session
POST   /v1/sessions                 Creates session entry

GET    /v1/sessions/{sessions_id}    Get session details
POST   /v1/sessions/{sessions_id}    creates or replaces a session entry using the specified session_id
DELETE /v1/sessions/{sessions_id}    Deletes the specified session information
PATCH  /v1/sessions/{sessions_id}    Updates part of the session document

POST   /v1/sessions/{sessions_id}/action           requests actions (e.g. start/end) upon a specific session

PUT    /v1/sessions/{sessions_id}/jobs/{job_id}    adds the job to the session
DELETE /v1/sessions/{sessions_id}/jobs/{job_id}    removes the job from the session

Scheduling Time Information

Three types of scheduling can be identified

• date - used for single run jobs

• interval - periodic jobs, providing an interval value

• cron-like jobs

Each type has specific parameters which can be given.

date scheduling

"schedule_date":      : datetime isoformat

interval scheduling

"schedule_interval"   : "continuous", "N weeks" / "N days" / "N hours" / "N minutes" / "N seconds"

"schedule_start_date" : datetime isoformat
"schedule_end_date"   : datetime isoformat

cron-like scheduling

"schedule_year"       : 4 digit year
"schedule_month"      : 1-12
"schedule_day"        : 1-31
"schedule_week"       : 1-53
"schedule_day_of_week": 0-6 or string mon,tue,wed,thu,fri,sat,sun
"schedule_hour"       : 0-23
"schedule_minute"     : 0-59
"schedule_second"     : 0-59

"schedule_start_date" : datetime isoformat
"schedule_end_date"   : datetime isoformat

Job examples

example backup freezer_action

"freezer_action": {
  "action" : "backup"
  "mode" : "fs"
  "src_file" : "/home/tylerdurden/project_mayhem"
  "backup_name" : "project_mayhem_backup"
  "container" : "my_backup_container"
  "max_backup_level" : int
  "always_backup_level": int
  "restart_always_backup": int
  "no_incremental" : bool
  "encrypt_pass_file" : private_key_file
  "log_file" : "/var/log/freezer.log"
  "hostname" : false
  "max_cpu_priority" : false
}

example restore freezer_action

"freezer_action": {
  "action": "restore"
  "restore-abs-path": "/home/tylerdurden/project_mayhem"
  "container" : "my_backup_container"
  "backup_name": "project_mayhem_backup"
  "restore-from-host": "another_host"
  "max_cpu_priority": true
}

example scheduled backup job. job will be executed once at the provided datetime

"job": {
    "job_actions":
        [
            {
                "freezer_action":
                    {
                        "action" : "backup",
                        "mode" : "fs",
                        "src_file" : "/home/tylerdurden/project_mayhem",
                        "backup_name" : "project_mayhem_backup",
                        "container" : "my_backup_container",
                    }
                "exit_status": "fail|success"
                "max_retries": int,
                "max_retries_interval": secs,
                "mandatory": bool
            },
            {
                action
                ...
            },
            {
                action
                ...
            }
        ],
    "job_schedule":
        {
            "time_created": 1234,
            "time_started": 1234,
            "time_ended":   0,
            "status":  "stop | scheduled | running",
            "schedule_date": "2015-06-02T16:20:00",
        }
    "job_id": "blabla",
    "client_id": "blabla",
    "user_id": "blabla",
    "description": "scheduled one shot",
}


"job": {
    "job_actions":
        [ ... ],
    "job_schedule":
        {
            "time_created": 1234,
            "time_started": 1234,
            "time_ended":   0,

            "status":  "stop",
            "event": "start"
            "schedule_interval" : "1 days"
            "schedule_start_date" : "2015-06-02T16:20:00"
        },
    "job_id": "4822e482fcbb439189a1ad616ac0a72f",
    "client_id": "26b4ea367ac64702868653912e9428cc_freezer.mydomain.myid",
    "user_id": "35a322dfb2b14f40bc53a29a14309021",
    "description": "daily backup",
}

multiple scheduling choices allowed

"job": {
    "job_actions":
        [ ... ],
    "job_schedule":
        {
            "time_created": 1234,
            "time_started": 1234,
            "time_ended":   0,
            "status":  "scheduled"
            "schedule_month" : "1-6, 9-12"
            "schedule_day" : "mon, wed, fri"
            "schedule_hour": "03"
            "schedule_minute": "25"
        }
    "job_id": "blabla",
    "client_id": "blabla",
    "user_id": "blabla",
    "description": "daily backup",
}

Finished job with result

"job": {
    "job_actions": [ ... ],
    "job_schedule":
        {
            "time_created": 1234,
            "time_started": 1234,
            "time_ended":   4321,
            "status":  "stop",
            "event": "",
            "result": "success",
            "schedule_time": "2015-06-02T16:20:00"
        },
    "job_id": "blabla",
    "client_id": "blabla",
    "user_id": "blabla",
    "description": "one shot job",
}

Actions default values

It is possible to define properties that span across multiple actions This allow not to rewrite values that might be the same in multiple actions. If properties are specifically set in one action, then the specified value is the one used.

Example

"job": {
    "action_defaults": {
        "log_file": "/tmp/freezer_tmp_log",
        "container": "my_backup_container"
    },
    "job_actions": [{
        "freezer_action": {
            "action": "backup",
            "mode": "fs",
            "src_file": "/home/user1/file",
            "backup_name": "user1_backup"
        }
    }, {
        "freezer_action": {
            "action": "backup",
            "mode": "fs",
            "src_file": "/home/user2/file",
            "backup_name": "user2_backup"
        }
    }, {
        "freezer_action": {
            "action": "backup",
            "mode": "fs",
            "src_file": "/home/user3/file",
            "backup_name": "user2_backup",
            "log_file": "/home/user3/specific_log_file"
        }
    }],
    "description": "scheduled one shot"
}

Is Equivalent to

"job": {
    "job_actions": [{
        "freezer_action": {
            "action": "backup",
            "mode": "fs",
            "src_file": "/home/user1/file",
            "backup_name": "user1_backup",
            "log_file": "/tmp/freezer_tmp_log",
            "container": "my_backup_container"
        }
    }, {
        "freezer_action": {
            "action": "backup",
            "mode": "fs",
            "src_file": "/home/user2/file",
            "backup_name": "user2_backup",
            "log_file": "/tmp/freezer_tmp_log",
            "container": "my_backup_container"
        }
    }, {
        "freezer_action": {
            "action": "backup",
            "mode": "fs",
            "src_file": "/home/user3/file",
            "backup_name": "user2_backup",
            "log_file": "/home/user3/specific_log_file",
            "container": "my_backup_container"
        }
    }],
    "description": "scheduled one shot"
}

Session Data Structure

session =
{
  "session_id": string,
  "session_tag": int,
  "description": string,
  "hold_off": int (seconds),
  "schedule": { scheduling information, same as jobs },
  "jobs": { 'job_id_1': {
              "client_id": string,
              "status": string,
              "result": string
              "time_started": int  (timestamp),
              "time_ended":   int  (timestamp),
            },
            'job_id_2': {
              "client_id": string,
              "status": string,
              "result": string
              "time_started": int  (timestamp),
              "time_ended":   int  (timestamp),
            }
          }
  "time_start": int timestamp,
  "time_end": int timestamp,
  "time_started": int  (timestamp),
  "time_ended":   int  (timestamp),
  "status": string "completed" "running",
  "result": string "success" "fail",
  "user_id": string
}

Session actions

When the freezer scheduler running on a node wants to start a session, it sends a POST request to the following endpoint:

POST   /v1/sessions/{sessions_id}/action

The body of the request bears the action and parameters

Session START action

{
    "start": {
        "job_id": "JOB_ID_HERE",
        "current_tag": 22
    }
}

Example of a successful response

{
    'result': 'success',
    'session_tag': 23
}

Session STOP action

{
    "end": {
        "job_id": "JOB_ID_HERE",
        "current_tag": 23,
        "result": "success|fail"
    }
}

Session-Job association

PUT    /v1/sessions/{sessions_id}/jobs/{job_id}    adds the job to the session
DELETE /v1/sessions/{sessions_id}/jobs/{job_id}    removes the job from the session