Model Card Toolkit

The Model Card Toolkit (MCT) streamlines and automates generation of
Model Cards [1], machine learning documents
that provide context and transparency into a model’s development and performance.
Integrating the MCT into your ML pipeline enables you to share model metadata and
metrics with researchers, developers, reporters, and more.

Some use cases of model cards include:

• Facilitating the exchange of information between model builders and product developers.
• Informing users of ML models to make better-informed decisions about how to use them (or how not to use them).
• Providing model information required for effective public oversight and accountability.

Installation

The Model Card Toolkit is hosted on PyPI,
and requires Python 3.7 or later.

Installing the basic, framework agnostic package:

pip install model-card-toolkit

If you are generating model cards for TensorFlow models, install the optional
TensorFlow dependencies to use Model Card Toolkit’s TensorFlow utilities:

pip install model-card-toolkit[tensorflow]

You may need to append the --use-deprecated=legacy-resolver flag when running
versions of pip starting with 20.3.

See the installation guide
for more installation options.

Getting Started

import model_card_toolkit as mct

# Initialize the Model Card Toolkit with a path to store generate assets
model_card_output_path = ...
toolkit = mct.ModelCardToolkit(model_card_output_path)

# Initialize the ModelCard, which can be freely populated
model_card = toolkit.scaffold_assets()
model_card.model_details.name = 'My Model'

# Write the model card data to a proto file
toolkit.update_model_card(model_card)

# Return the model card document as an HTML page
html = toolkit.export_format()

Model Card Generation on TFX

If you are using TensorFlow Extended (TFX), you can
incorporate model card generation into your TFX pipeline via the ModelCardGenerator
component.

The ModelCardGenerator component has moved to the
TFX Addons library and is no longer
packaged in Model Card Toolkit from version 2.0.0. Before you can use the
component, you will need to install the tfx-addons package:

pip install tfx-addons[model_card_generator]

See the ModelCardGenerator guide
and run the case study notebook
to learn more about the component.

Schema

Model cards are stored in proto as an intermediate format. You can see the model
card JSON schema in the schema directory.

References

[1] https://arxiv.org/abs/1810.03993