co2mpas 1.3.1rc0

All-In-One Installation under Windows

• Download all-in-one archive from http://files.co2mpas.io/. Ensure that you download the correct 32/64 architecture for your PC (the 64bit archive CANNOT run on 32bit PCs).

• Use the original “7z” extraxtor, since “plain-zip” produces out-of-memory errors when expanding long directories.

• Run INSTALL.bat script contained in the root of the unzipped folder. It will install links for commons CO₂MPAS tasks under your Windows Start-Menu.

• Visit the guidelines for its usage: allinone (also contained within the archive).

Python Installation

If you already have a suitable python-3 installation with all scientific packages updated to their latest versions, you may skip this 1st stage.

Install co2mpas package

1. Install CO₂MPAS executable internally into your python-environment with the following console-commands (there is no prob if the 1st uninstall command fails):

   > pip uninstall co2mpas
   > pip install co2mpas
   Collecting co2mpas
   Downloading http://pypi.co2mpas.io/packages/co2mpas-...
   ...
   Installing collected packages: co2mpas
   Successfully installed co2mpas-1.3.1.rc0

2. Check that when you run co2mpas, the version executed is indeed the one installed above (check both version-identifiers and paths):

   > co2mpas -vV
   co2mpas_version: 1.3.1.rc0
   co2mpas_rel_date: 2016-07-18 17:48:05
   co2mpas_path: d:\co2mpas_ALLINONE-64bit-v1.3.1.rc0\Apps\WinPython\python-3.4.3\lib\site-packages\co2mpas
   python_path: D:\co2mpas_ALLINONE-64bit-v1.3.1.rc0\WinPython\python-3.4.3
   python_version: 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600 XXX]
   PATH: D:\co2mpas_ALLINONE-64bit-v1.3.1.rc0\WinPython...

> pip uninstall co2mpas
Uninstalling co2mpas-<installed-version>
...
Proceed (y/n)?

Autocompletion

In order to press [Tab] and get completions, do the following in your environment (ALLINONE is pre-configured with them):

• For the Clink environment, on cmd.exe, add the following lua script inside clink’s profile folder: clink/profile/co2mpas_autocompletion.lua

  --[[ clink-autocompletion for CO2MPAS
  --]]
  local handle = io.popen('co2mpas-autocompletions')
  words_str = handle:read("*a")
  handle:close()
  
  function words_generator(prefix, first, last)
      local cmd = 'co2mpas'
      local prefix_len = #prefix
  
      --print('P:'..prefix..', F:'..first..', L:'..last..', l:'..rl_state.line_buffer)
      if prefix_len == 0 or rl_state.line_buffer:sub(1, cmd:len()) ~= cmd then
          return false
      end
  
      for w in string.gmatch(words_str, "%S+") do
          -- Add matching app-words.
          --
          if w:sub(1, prefix_len) == prefix then
              clink.add_match(w)
          end
  
          -- Add matching files & dirs.
          --
          full_path = true
          nf = clink.match_files(prefix..'*', full_path)
          if nf > 0 then
              clink.matches_are_files()
          end
      end
      return clink.match_count() > 0
  end
  
  sort_id = 100
  clink.register_match_generator(words_generator)

• For the bash shell just add this command in your ~/.bashrc (or type it every time you open a new console):

  complete -fdev -W "`co2mpas-autocompletions`" co2mpas

Demo files

The simulator contains input-files for demo-vehicles that are a nice starting point to try out:

To run them, do the following:

1. Choose a folder where you will store the input and output files:

   ## Skip this if ``tutorial`` folder already exists.
   $ mkdir tutorial
   $ cd tutorial
   
   ## Skip also this if folders exist.
   $ mkdir input output

2. Create the demo vehicles inside the input-folder with the demo sub-command:

   $ co2mpas demo input
   INFO:co2mpas.__main__:Creating INPUT-DEMO file 't\co2mpas_demo-0.xlsx'...
   INFO:co2mpas.__main__:Creating INPUT-DEMO file 't\co2mpas_demo-1.xlsx'...
   INFO:co2mpas.__main__:Creating INPUT-DEMO file 't\co2mpas_demo-10.xlsx'...
   INFO:co2mpas.__main__:Creating INPUT-DEMO file 't\co2mpas_demo-2.xlsx'...
   INFO:co2mpas.__main__:Creating INPUT-DEMO file 't\co2mpas_demo-3.xlsx'...
   INFO:co2mpas.__main__:Creating INPUT-DEMO file 't\co2mpas_demo-4.xlsx'...
   INFO:co2mpas.__main__:Creating INPUT-DEMO file 't\co2mpas_demo-5.xlsx'...
   INFO:co2mpas.__main__:Creating INPUT-DEMO file 't\co2mpas_demo-6.xlsx'...
   INFO:co2mpas.__main__:Creating INPUT-DEMO file 't\co2mpas_demo-7.xlsx'...
   INFO:co2mpas.__main__:Creating INPUT-DEMO file 't\co2mpas_demo-8.xlsx'...
   INFO:co2mpas.__main__:Creating INPUT-DEMO file 't\co2mpas_demo-9.xlsx'...
   INFO:co2mpas.__main__:You may run DEMOS with:
       co2mpas batch input

3. Run the simulator on all demo-files (note, it might take considerable time):

   $ co2mpas batch input -O output
   Processing ['input'] --> 'output'...
   Processing: co2mpas_demo-0
   ...
   ...
   Done! [499.579 sec]

4. Inspect the results (explained in the next section):

   $ start output/*summary.xlsx       ## More summaries might exist in the folder from previous runs.
   $ start output                     ## View the folder with all files generated.

Output files

The output-files produced on each run are the following:

• One file per vehicle, named as <timestamp>-<inp-fname>.xls: This file contains all the inputs and calculation results for each vehicle contained in the batch-run: scalar-parameters and time series for target, calibration and prediction phases, for all cycles. In addition, the file contains all the specific submodel-functions that generated the results, a comparison summary, and information on the python libraries installed on the system (for investigating reproducibility issues).

• A Summary-file named as <timestamp>-summary.xls: Major CO₂ emissions values, optimized CO₂ parameters values and success/fail flags of CO₂MPAS submodels for all vehicles in the batch-run.

Entering new vehicles

You may modify the samples vehicles and run again the model. But to be sure that your vehicle does not contain by accident any of the sample-data, use the template sub-command to make an empty input excel-file:

1. Decide the input/output folders. Assuming we are still in the tutorial folder and we wish to re-use the input/output folders from the example above, we may clear all their contents with this:

   $ rm -r ./input/* ./output/*      ## Replace `rm` with `del` in *Windows* (`cmd.exe`)

2. Create an empty vehicle template-file (eg. vehicle_1.xlsx) inside the input-folder with the template sub-command:

   $ co2mpas template input/vehicle_1.xlsx  ## Note that here we specify the filename, not the folder!
   Creating TEMPLATE INPUT file 'input/vehicle_1.xlsx'...

3. Open the template excel-file to fill-in your vehicle data (and save it afterwards):

   $ start input/vehicle_1.xlsx      ## Opens the excel-file. Use `start` in *cmd.exe*.

   The generated file contains help descriptions to help you populate it with vehicle data. For items where an array of values is required (i.e. gear-box ratios) you may reference different parts of the spreadsheet following the syntax of the “xlref” mini-language.

   You may repeat these last 2 steps if you want to add more vehicles in the batch-run.

4. Run the simulator. Specify the single excel-file as input:

   $ co2mpas batch ./input/vehicle_1.xlsx -O output
   Processing './input/vehicle_1.xlsx' --> 'output'...
   Processing: vehicle_1
   ...
   Done! [12.938986 sec]

5. Assuming you do receive any error, you may now inspect the results:

   $ start output/*summary.xlsx      ## More summaries might open from previous runs.
   $ start output                    ## View all files generated (see below).

6. In the case of errors, or if the results are not satisfactory, repeat the above procedure from step 3 to modify the vehicle and re-run the model. See also ref: debug, below.

Simulation plan

It is possible to launch co2mpas once, and have it run the model multiple times, with variations on the input-data, all contained in a single (or more) input file(s).

The data for base model are contained in the regular sheets, and any variations are provided in additional sheets which names starting with the plan. prefix. These sheets must contain a table where each row is a single simulation, while the columns names are the parameters that the user want to vary. The columns of these tables must contain the following special names:

• id: Identifies the variation id.

• base: this is a file path of a CO2MPAS excel input, this model will be used as new base vehicle.

• defaults: this is a a list of file paths. The calibrated models of these files are used as default models of the base model. This behavior is needed to simulate, for example, a manual car (base) as A/T, because the A/T strategy and the torque converter are not in the base model.

Using custom output xl-files as templates

You may have defined customized xl-files for summarizing time-series and scalar parameters. To have CO₂MPAS fill those “output-template” files with its results, execute it with the --out-template option.

To create/modify one output-template yourself, do the following:

1. Open a typical CO₂MPAS output-file for some vehicle.

2. Add one or more sheets and specify/referring CO₂MPAS result-data using named-ranges.

3. (Optional) Delete the old sheets and save your file.

4. Use that file together with the --out-template argument.

Launch CO₂MPAS from Jupyter(aka IPython)

You may enter the data for a single vehicle and run its simulation, plot its results and experiment in your browser using IPython.

The usage pattern is similar to “demos” but requires to have ipython installed:

1. Ensure ipython with notebook “extra” is installed:

   $ pip install ipython[notebook]
   Installing collected packages: ipython[notebook]
   ...
   Successfully installed ipython-x.x.x notebook-x.x.x

2. Then create the demo ipython-notebook(s) into some folder (i.e. assuming the same setup from above, tutorial/input):

   $ pwd                     ## Check our current folder (``cd`` alone for Windows).
   .../tutorial
   
   $ co2mpas ipynb ./input

3. Start-up the server and open a browser page to run the vehicle-simulation:

   $ ipython notebook ./input

4. A new window should open to your default browser (AVOID IEXPLORER) listing the simVehicle.ipynb notebook (and all the demo xls-files). Click on the *.ippynb file to “load” the notebook in a new tab.

   The results are of a simulation run already pre-generated for this notebook but you may run it yourself again, by clicking the menu:

   "menu" --> `Cell` --> `Run All`

   And watch it as it re-calculates cell by cell.

5. You may edit the python code on the cells by selecting them and clicking Enter (the frame should become green), and then re-run them, with Ctrl + Enter.

   Navigate your self around by taking the tutorial at:

   "menu" --> `Help` --> `User Interface Tour`

   And study the example code and diagrams.

6. When you have finished, return to the console and issue twice Ctrl + C to shutdown the ipython-server.

Synchronizing time-series

The model might fail in case your time-series signals are time-shifted and/or with different sampling rates. Even if the run succeeds, the results will not be accurate enough.

As an aid tool, you may use the datasync command-line tool to “synchronize” your data-tables. This command reads one or more tables from excel-files and synchronizes their columns. The syntax of this utility command is given by typing datasync --help in the command line (listing below just the main fields):

Shift and resample excel-tables; see http://co2mpas.io/usage.html#Synchronizing-time-series.

Usage:
  datasync          [(-v | --verbose) | --logconf <conf-file>] [--force | -f]
                    [--interp <method>] [--no-clone] [--prefix-cols]
                    [-O <output>] <x-label> <y-label> <ref-table>
                    [<sync-table> ...]
  datasync          [--verbose | -v]  (--version | -V)
  datasync          [--interp-methods | -l]
  datasync          --help
  datasync template [-f] [--cycle <cycle>] [<excel-file-path> ...]

Options:
  <x-label>              Column-name of the common x-axis (e.g. 'times') to be
                         re-sampled if needed.
  <y-label>              Column-name of y-axis cross-correlated between all
                         <sync-table> and <ref-table>.
  <ref-table>            The reference table, in *xl-ref* notation (usually
                         given as `file#sheet!`); synced columns will be
                         appended into this table.
                         The captured table must contain <x_label> & <y_label>
                         as column labels.
                         If hash(`#`) symbol missing, assumed as file-path and
                         the table is read from its 1st sheet .
  <sync-table>           Sheets to be synced in relation to <ref-table>, also in
                         *xl-ref* notation.
                         All tables must contain <x_label> & <y_label> as column
                         labels.
                         Each xlref may omit file or sheet-name parts; in that
                         case, those from the previous xlref(s) are reused.
                         If hash(`#`) symbol missing, assumed as sheet-name.
                         If none given, all non-empty sheets of <ref-table> are
                         synced against the 1st one.
  -O <output>            Output folder or file path to write the results:
                         - Non-existent path: taken as the new file-path; fails
                           if intermediate folders do not exist, unless --force.
                         - Existent file: file-path to overwrite if --force,
                           fails otherwise.
                         - Existent folder: writes a new file
                           `<ref-file>.sync<.ext>` in that folder; --force
                           required if that file exists.
                         [default: .].
  -f, --force            Overwrite excel-file(s) and create any missing
                         intermediate folders.
  --prefix-cols          Prefix all synced column names with their source
                         sheet-names. By default, only clashing column-names are
                         prefixed.
  --no-clone             Do not clone excel-sheets contained in <ref-table>
                         workbook into output.
  --interp <method>      Interpolation method used in the resampling
                         [default: linear]: 'linear', 'nearest', 'zero',
                         'slinear', 'quadratic', 'cubic', 'barycentric',
                         'polynomial', 'spline' is passed to
                         scipy.interpolate.interp1d. Both 'polynomial' and
                         'spline' require that you also specify an order (int),
                         e.g. df.interpolate(--interp=polynomial4).
                         'krogh', 'piecewise_polynomial', 'pchip' and 'akima'
                         are all wrappers around the scipy interpolation methods
                         of similar names. 'integral'
  -l, --interp-methods   List of all interpolation methods that can be used in
                         the resampling.
  --cycle <cycle>        If set (e.g., --cycle=nedc.manual), the <ref-table> is
                         populated with the theoretical velocity profile.
                         Options: 'nedc.manual', 'nedc.automatic',
                         'wltp.class1', 'wltp.class2', 'wltp.class3a', and
                         'wltp.class3b'.
  <excel-file-path>      Output file.

All input tables must share 2 common columns: <x-label> and <y-label>, as if those tables describe 2D cartesian data, with a common X-axis and multiple data-series on the Y-Axis.

The 1st table given (<ref-table>) is considered to contain the “reference” X/Y values; the data-columns to shift-and-resample are contained in one or more tables (<sync-table>) specified subsequently in the command line, that are possibly read from different excel work-books.

• Shifting is based on the cross-correlation of <y-label> columns;

• resampling is based on the values of <x-label> columns among the different tables.

All tables are read from excel-sheets using the xl-ref syntax, which is best explained with some examples.

Debugging and investigating results

• Make sure that you have installed graphviz, and when running the simulation, append also the --plot-workflow option.

  $ co2mpas batch --plot-workflow bad-file.xlsx

  A browser tab will open at the end with the nodes processed.

• Use the modelgraph sub-command to plot the offending model (or just out of curiosity). For instance:

  $ co2mpas modelgraph co2mpas.model.physical.wheels.wheels

• Inspect the functions mentioned in the workflow and models and search them in CO2MPAS documentation ensuring you are visiting the documents for the actual version you are using.

Execution Model

The execution of CO2MPAS model for a single vehicle is a stepwise procedure of 3 stages: precondition, calibration, and prediction. These are invoked repeatedly, and subsequently combined, for the various cycles, as shown in the “active” flow-diagram of the execution, below:

1. Precondition: identifies the initial state of the vehicle by running a preconditioning WLTP cycle, before running the WLTP-H and WLTP-L cycles. The inputs are defined by the input.precondition.wltp_p node, while the outputs are stored in output.precondition.wltp_p.

2. Calibration: the scope of the stage is to identify, calibrate and select (see next sections) the best physical models from the WLTP-H and WLTP-L inputs (input.calibration.wltp_x). If some of the inputs needed to calibrate the physical models are not provided (e.g. initial_state_of_charge), the model will select the missing ones from precondition-stage’s outputs (output.precondition.wltp_p). Note that all data provided in input.calibration.wltp_x overwrite those in output.precondition.wltp_p.

3. Prediction: executed for the NEDC and as well as for the WLTP-H and WLTP-L cycles. All predictions use the calibrated_models. The inputs to predict the cycles are defined by the user in input.prediction.xxx nodes. If some or all inputs for the prediction of WLTP-H and WLTP-L cycles are not provided, the model will select from `output.calibration.wltp_x nodes a minimum set required to predict CO₂ emissions.

Excel input: data naming conventions

This section describes the data naming convention used in the CO₂MPAS template (.xlsx file). In it, the names used as sheet-names, parameter-names and column-names are “sensitive”, in the sense that they construct a data-values tree which is then fed into into the simulation model as input. These names are split in “parts”, as explained below with examples:

• sheet-names parts:

              base.input.precondition.WLTP-H.ts
              └┬─┘ └─┬─┘ └────┬─────┘ └─┬──┘ └┬┘
  scope────────┘     │        │         │     │
  usage──────────────┘        │         │     │
  stage───────────────────────┘         │     │
  cycle─────────────────────────────────┘     │
  sheet_type──────────────────────────────────┘

  First 4 parts above are optional, but at least one of them must be present on a sheet-name; those parts are then used as defaults for all parameter-names contained in that sheet. type is optional and specify the type of sheet.

• parameter-names/columns-names parts:

                 plan.target.prediction.initial_state_of_charge.WLTP-H
                 └┬─┘ └─┬─┘ └────┬────┘ └──────────┬──────────┘ └──┬─┘
  scope(optional)─┘     │        │                 │               │
  usage(optional)───────┘        │                 │               │
  stage(optional)────────────────┘                 │               │
  parameter────────────────────────────────────────┘               │
  cycle(optional)──────────────────────────────────────────────────┘

  OR with the last 2 parts reversed:

                plan.target.prediction.WLTP-H.initial_state_of_charge
                                       └──┬─┘ └──────────┬──────────┘
  cycle(optional)─────────────────────────┘              │
  parameter──────────────────────────────────────────────┘

Calibrated Physical Models

There are potentially eight models calibrated from input scalar-values and time-series (see reference):

1. AT_model,

2. electric_model,

3. clutch_torque_converter_model,

4. co2_params,

5. engine_cold_start_speed_model,

6. engine_coolant_temperature_model,

7. engine_speed_model, and

8. start_stop_model.

Each model is calibrated separately over WLTP_H and WLTP_L. A model can contain one or several functions predicting different quantities. For example, the electric_model contains the following functions/data:

• alternator_current_model,

• alternator_status_model,

• electric_load,

• max_battery_charging_current,

• start_demand.

These functions/data are calibrated/estimated based on the provided input (in the particular case: alternator current, battery current, and initial SOC) over both cycles, assuming that data for both WLTP_H and WLTP_L are provided.

Model selection

To select which is the best calibration (from WLTP_H or WLTP_L or ALL) to be used in the prediction phase, the results of each stage are compared against the provided input data (used in the calibration). The calibrated models are THEN used to recalculate (predict) the inputs of the WLTP_H and WLTP_L cycles. A score (weighted average of all computed metrics) is attributed to each calibration of each model as a result of this comparison.

In addition to the above, a success flag is defined according to upper or lower limits of scores which have been defined empirically by the JRC. If a model fails these limits, priority is then given to a model that succeeds, even if it has achieved a worse score.

The following table describes the scores, targets, and metrics for each model: