Semi-ATE's STDF library

Standard Test Data Format

This library is NOT intended to be the fastest in the world!

Often people are searching for 'the fastest' STDF parser. If this is what you are after, keep on looking and by all means, hit the wall later on, and at that point you might consider to return! 🤣

Ok, a fast parser is first of all writen in probably C/C++, and it has to dispence of a lot of the checking/correcting in order to become realy fast, and probably throwing away information not deemed interesting enough (and later turns out to be vital). However in real life STDF files are far from perfect, meaning that fast parsers will FAIL to do their intended job! You might tweak them for one or another ATE in your environment, but it will not be a can-do-everything parser!

In any case, when you start parsing STDF's at the moment you want to interact with the data, you are, as they say, too little too late ... you must still be living in the last century (not to say last millennium 🤪)

A good parser is written in a higher level language (like Python) and it does an awefull lot of checking (and if needed correcting) and doesn't throw any information away, so as to return reliably with full, meaningfull and correct data! This of course makes it slower. One can optimize that a bit by using Cython or maybe numba but that is besides the point.

The point is that STDF data should be converted to a useable format like pandas (numpy alone will not do as plenty of data is not numerical) WHILE the data is being generated, preferrably not post-factum and definitely not pre-usage!

Think of it like this: STDF is a very good format from the point of view of the ATE, because if a test program is crashing, we lost virtually no data! Also, in STDF everything conserning an ATE has his defined place! (as opposed to CSV or similar ... naaah, you can not call it a 'format' can you?) Anyway, STDF is an un-usable format from the point of view of data analysis! Therefore we need to convert the data to a format that is usable. (and if now you are thinking 'SQL', then I can confirm that you are a die-hard masochist that still lives in the last millennium because you are clearly not up to speed when it comes to data science! 🧐)

Anyway, I did put pandas forward, because Semi-ATE is Python (>=3.7) based, but to be fair one could also go the SAS- or the R way but those make less sense in the Semi-ATE concept.

In any case, Semi-ATE is outputting STDF data, so whatever (legacy) system(s) you have, Semi-ATE will play along nicely!

The Semi-ATE-Metis project builds on Semi-ATE-STDF/numpy/scipy/pandas/GStreamer/HDF5/matplotlib to deliver data analysis tailored to the semiconductor test industry ... in open source!

Eat that Mentor/Galaxy! For years you took money-for-nothing, and in the end you still screwed your customers (cfr. PAT). My-silver-lining: now we will do some screwing! See how that feels! 😋

It is also NOT just a parser!

In Semi-ATE we also need to write STDF files!

Infact here are the specifications of the Semi-ATE-STDF library:

• Endianness: Little & Big
• Formats: STDF & ATDF
• Versions & Extensions:
  • V3: support depricated
  • V4:
    • standard
    • V4-2007
    • Memory:2010.1 (planned but not implemented yet)
• Modes: read & write
• compressions: (in all modes!)
  • gzip
  • lzma → turns out to be the best compressor for STDF files. 🤫
  • bz2
• encodings:
  • ASCII
  • UTF-8 (added to support things like 'ηA', 'μV', '°C', '-∞', ... but also to make STDF compatible with python3 itself 😎)
• floating point extensions:
  • IEEE 754-1985 (aka: NaN, nan, Infinity, Inf, inf, ...)
• (support for python2 is depricated)
  • Python 3.7
  • Python 3.8 ---add-badges-here--- (code coverage, build)
  • Python 3.9
• Packaging:

Installation

Stand alone

conda

$ conda install Semi-ATE-STDF

pip

$ pip install Semi-ATE-STDF

As part of the Semi-ATE suit

conda (preferred)

$ conda install Semi-ATE

pip (discouraged as Semi-ATE holds a plugin for Spyder)

$ pip install Semi-ATE

Usage examples

This STDF library is a part of the Semi-ATE suit, and it shares the namespace.

print an STDF in a human readable form on the standard output

from Semi_ATE import STDF

for REC in STDF.records_from_file("blahbla.stdf"):
    print(REC)

work with a STDF file storred in compressed form (lzma)

from Semi_ATE import STDF

for REC in STDF.records_from_file("blahbla.stdf.xz"):
    print(REC)

convert an STDF file into an ATDF file

from Semi_ATE import STDF

basename = "blahblah"

with open(f"{basename}.atdf", "w") as atdf:
   for REC in STDF.records_from_file(f"{basename}.stdf"):
       atdf.write(REC.to_atdf())

work with a STDF record class :

import time
from Semi_ATE import STDF

def test_records_from_file():
        
#   Make 2 records and put them into one STDF file

    record = STDF.FAR()
#   Example of getting binary data:   
    data = record.__repr__()
    
    record = STDF.WIR()
#   Example of set_value functon:   
    record.set_value('HEAD_NUM', 1)
    record.set_value('SITE_GRP', 1)
    record.set_value('START_T', int(time.time()))
    record.set_value('WAFER_ID', "WFR_ID_123456789")

#   Example of collecting all records:   
    data += record.__repr__()

#   Example of saving file :   
    f = open("test.stdf", mode="wb")
    file_name = f.name
    f.write(data)
    f.close()
    
    f = open(file_name)

    print("\nDump content of the STDF file in text format")
#   Example of printng binary data from the STDF file in text format:   
    for REC in STDF.records_from_file(file_name):
        print(REC)

    print("\nShow usage of get_fields function")
#   Example of getting information about available fields:   
    for REC in STDF.records_from_file(file_name):
#   Print name of the record
        print(f" RECORD {REC.id}")
        print(REC.get_fields())

    print("\nShow usage of get_value function")
#   Example of getting fields values:   
    for REC in STDF.records_from_file(file_name):
#   Print name of the record
        print(f" RECORD {REC.id}")
        fields = REC.get_fields()
        for field in fields:
            value = REC.get_value(field)
            print(f" Field {field} = {value}")

    print("\nShow usage of to_dict function")
#   Example of usage to_dict function:   
    for REC in STDF.records_from_file(file_name):
        stdf_dict = REC.to_dict()
        if REC.id=="WIR":
            print(f"Get HEAD_NUM field value from dictinary  : {stdf_dict['HEAD_NUM']}")
            print(f"Get SITE_GRP field value from dictinary  : {stdf_dict['SITE_GRP']}")
            print(f"Get START_T  field value from dictinary  : {stdf_dict['START_T']}")
            print(f"Get WAFER_ID field value from dictinary  : {stdf_dict['WAFER_ID']}")

    print("\nShow usage of reset function")
#   Example of reseting data in a single record:   
    for REC in STDF.records_from_file(file_name):
        if REC.id=="WIR":
            REC.reset()
            print(REC)

    
    f.close()

Note

You could use this library to make your own "converters", however this is the goal of the Semi-ATE-Metis project, so by unsing Semi-ATE-Metis (which depends on Semi-ATE-STDF) you don't need to handle the 'conversion' anymore and you can directly make your hands dirty with the 'tool' you want to have !!! :thumbsup: