Types

tfx.v1.types

TFX types module.

MODULE	DESCRIPTION
standard_artifacts	Public API for standard_artifacts.

CLASS	DESCRIPTION
BaseBeamComponent	Base class for a TFX Beam pipeline component.
BaseChannel	An abstraction for component (BaseNode) artifact inputs.
BaseComponent	Base class for a TFX pipeline component.
BaseFunctionalComponent	Base class for functional components.
BaseFunctionalComponentFactory	Serves to declare the return type below.
BaseNode	Base class for a node in TFX pipeline.

Classes

BaseBeamComponent

BaseBeamComponent(spec: ComponentSpec, custom_executor_spec: Optional[ExecutorSpec] = None)

Bases: BaseComponent

Base class for a TFX Beam pipeline component.

An instance of a subclass of BaseBaseComponent represents the parameters for a single execution of that TFX Beam pipeline component.

Beam based components should subclass BaseBeamComponent instead of BaseComponent in order to inherit Beam related SDKs. All subclasses of BaseBeamComponent should override the required class level attributes specified in BaseComponent.

Initialize a component.

PARAMETER	DESCRIPTION
spec	types.ComponentSpec object for this component instance. TYPE: ComponentSpec
custom_executor_spec	Optional custom executor spec overriding the default executor specified in the component attribute. TYPE: Optional[ExecutorSpec] DEFAULT: None

METHOD	DESCRIPTION
add_downstream_node	Experimental: Add another component that must run after this one.
add_downstream_nodes	Experimental: Add another component that must run after this one.
add_upstream_node	Experimental: Add another component that must run before this one.
add_upstream_nodes	Experimental: Add components that must run before this one.
from_json_dict	Convert from dictionary data to an object.
to_json_dict	Convert from an object to a JSON serializable dictionary.
with_beam_pipeline_args	Add per component Beam pipeline args.
with_platform_config	Attaches a proto-form platform config to a component.

ATTRIBUTE	DESCRIPTION
id	Node id, unique across all TFX nodes in a pipeline. TYPE: str
outputs	Component's output channel dict. TYPE: Dict[str, OutputChannel]

Source code in tfx/dsl/components/base/base_component.py

def __init__(
    self,
    spec: types.ComponentSpec,
    custom_executor_spec: Optional[executor_spec.ExecutorSpec] = None):
  """Initialize a component.

  Args:
    spec: types.ComponentSpec object for this component instance.
    custom_executor_spec: Optional custom executor spec overriding the default
      executor specified in the component attribute.
  """
  if custom_executor_spec:
    if not isinstance(custom_executor_spec, executor_spec.ExecutorSpec):
      raise TypeError(
          ('Custom executor spec override %s for %s should be an instance of '
           'ExecutorSpec') % (custom_executor_spec, self.__class__))

  executor_spec_obj = custom_executor_spec or self.__class__.EXECUTOR_SPEC
  # TODO(b/171742415): Remove this try-catch block once we migrate Beam
  # DAG runner to IR-based stack. The deep copy will only fail for function
  # based components due to pickle workaround we created in ExecutorClassSpec.
  try:
    executor_spec_obj = executor_spec_obj.copy()
  except Exception as e:  # pylint:disable = bare-except
    # This will only happen for function based components, which is fine.
    raise ValueError(
        f'The executor spec {executor_spec_obj!r} of {self.__class__} class '
        'is not copyable.'
    ) from e

  driver_class = self.__class__.DRIVER_CLASS
  # Set self.spec before super.__init__() where node registration happens.
  # This enable node input checking on node context registration.
  self._validate_spec(spec)
  spec.migrate_output_channels(self)
  self.spec = spec
  super().__init__(
      executor_spec=executor_spec_obj,
      driver_class=driver_class,
  )
  self._validate_component_class()
  self.platform_config = None
  self._pip_dependencies = []

Attributes

id property writable

id: str

Node id, unique across all TFX nodes in a pipeline.

If id is set by the user, return it directly. Otherwise, return .

RETURNS	DESCRIPTION
str	node id.

outputs property

outputs: Dict[str, OutputChannel]

Component's output channel dict.

Functions

add_downstream_node

add_downstream_node(downstream_node)

Experimental: Add another component that must run after this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

It is symmetric with add_upstream_node.

PARAMETER	DESCRIPTION
downstream_node	a component that must run after this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_downstream_node(self, downstream_node):
  """Experimental: Add another component that must run after this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.

  It is symmetric with `add_upstream_node`.

  Args:
    downstream_node: a component that must run after this node.
  """
  self._downstream_nodes.add(downstream_node)
  if self not in downstream_node.upstream_nodes:
    downstream_node.add_upstream_node(self)

add_downstream_nodes

add_downstream_nodes(downstream_nodes)

Experimental: Add another component that must run after this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

It is symmetric with add_upstream_nodes.

PARAMETER	DESCRIPTION
downstream_nodes	a list of components that must run after this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_downstream_nodes(self, downstream_nodes):
  """Experimental: Add another component that must run after this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.

  It is symmetric with `add_upstream_nodes`.

  Args:
    downstream_nodes: a list of components that must run after this node.
  """
  self._downstream_nodes.update(downstream_nodes)
  for downstream_node in downstream_nodes:
    if self not in downstream_node.upstream_nodes:
      downstream_node.add_upstream_node(self)

add_upstream_node

add_upstream_node(upstream_node)

Experimental: Add another component that must run before this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

It is symmetric with add_downstream_node.

PARAMETER	DESCRIPTION
upstream_node	a component that must run before this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_upstream_node(self, upstream_node):
  """Experimental: Add another component that must run before this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.

  It is symmetric with `add_downstream_node`.

  Args:
    upstream_node: a component that must run before this node.
  """
  self._upstream_nodes.add(upstream_node)
  if self not in upstream_node.downstream_nodes:
    upstream_node.add_downstream_node(self)

add_upstream_nodes

add_upstream_nodes(upstream_nodes)

Experimental: Add components that must run before this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

PARAMETER	DESCRIPTION
upstream_nodes	a list of components that must run before this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_upstream_nodes(self, upstream_nodes):
  """Experimental: Add components that must run before this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.


  Args:
    upstream_nodes: a list of components that must run before this node.
  """
  self._upstream_nodes.update(upstream_nodes)
  for upstream_node in upstream_nodes:
    if self not in upstream_node.downstream_nodes:
      upstream_node.add_downstream_node(self)

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/utils/json_utils.py

@classmethod
@doc_controls.do_not_doc_in_subclasses
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  """Convert from dictionary data to an object."""
  instance = cls.__new__(cls)
  instance.__dict__ = dict_data
  return instance

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def to_json_dict(self) -> Dict[str, Any]:
  """Convert from an object to a JSON serializable dictionary."""
  return dict((k, v)
              for k, v in self.__dict__.items()
              if k not in ['_upstream_nodes', '_downstream_nodes'])

with_beam_pipeline_args

with_beam_pipeline_args(beam_pipeline_args: Iterable[Union[str, Placeholder]]) -> BaseBeamComponent

Add per component Beam pipeline args.

PARAMETER	DESCRIPTION
beam_pipeline_args	List of Beam pipeline args to be added to the Beam executor spec. TYPE: Iterable[Union[str, Placeholder]]

RETURNS	DESCRIPTION
BaseBeamComponent	the same component itself.

Source code in tfx/dsl/components/base/base_beam_component.py

def with_beam_pipeline_args(
    self, beam_pipeline_args: Iterable[Union[str, placeholder.Placeholder]]
) -> 'BaseBeamComponent':
  """Add per component Beam pipeline args.

  Args:
    beam_pipeline_args: List of Beam pipeline args to be added to the Beam
      executor spec.

  Returns:
    the same component itself.
  """
  cast(executor_spec.BeamExecutorSpec,
       self.executor_spec).add_beam_pipeline_args(beam_pipeline_args)
  return self

with_platform_config

with_platform_config(config: Message) -> Self

Attaches a proto-form platform config to a component.

The config will be a per-node platform-specific config.

PARAMETER	DESCRIPTION
config	platform config to attach to the component. TYPE: Message

RETURNS	DESCRIPTION
Self	the same component itself.

Source code in tfx/dsl/components/base/base_component.py

@doc_controls.do_not_doc_in_subclasses
def with_platform_config(
    self, config: message.Message
) -> typing_extensions.Self:
  """Attaches a proto-form platform config to a component.

  The config will be a per-node platform-specific config.

  Args:
    config: platform config to attach to the component.

  Returns:
    the same component itself.
  """
  self.platform_config = config
  return self

BaseChannel

BaseChannel(type: Type[_AT], is_optional: Optional[bool] = None)

Bases: ABC, Generic[_AT]

An abstraction for component (BaseNode) artifact inputs.

BaseChannel is often interchangeably used with the term 'channel' (not capital Channel which points to the legacy class name).

Component takes artifact inputs distinguished by each "input key". For example:

trainer = Trainer(
    examples=example_gen.outputs['examples'],
) # ^^^^^^^^
  # input key
           # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
           # channel

Here "examples" is the input key of the Examples artifact type. example_gen.outputs["examples"] is a channel. Typically a single channel refers to a list of Artifact of a homogeneous type. Since channel is a declarative abstraction it is not strictly bound to the actual artifact, but is more of an input selector.

The most commonly used channel type is an OutputChannel (in the form of component.outputs["key"], which selects the artifact produced by the component in the same pipeline run (in synchronous execution mode; more information on OutputChannel docstring), and is typically a single artifact.

ATTRIBUTE	DESCRIPTION
type	The artifact type class that the Channel takes. TYPE: Type[_AT]
is_optional	If this channel is optional (e.g. may trigger components at run time if there are no artifacts in the channel). None if not explicetely set. TYPE: Optional[bool]

METHOD	DESCRIPTION
as_optional	Creates an optional version of self.
get_data_dependent_node_ids	Get data dependent nodes of this channel.

ATTRIBUTE	DESCRIPTION
is_optional	If this is an "optional" channel. Changes Pipeline runtime behavior. TYPE: Optional[bool]
type_name	Name of the artifact type class that Channel takes.

Source code in tfx/types/channel.py

def __init__(self, type: Type[_AT], is_optional: Optional[bool] = None):  # pylint: disable=redefined-builtin
  if not _is_artifact_type(type):
    raise ValueError(
        'Argument "type" of BaseChannel constructor must be a subclass of '
        f'tfx.Artifact (got {type}).')
  self._artifact_type = type
  self._input_trigger = None
  self._original_channel = None
  self._is_optional = is_optional

Attributes

is_optional property

is_optional: Optional[bool]

If this is an "optional" channel. Changes Pipeline runtime behavior.

type_name property

type_name

Name of the artifact type class that Channel takes.

Functions

as_optional

as_optional() -> Self

Creates an optional version of self.

By default component input channels are considered required, meaning if the channel does not contain at least 1 artifact, the component will be skipped. Making channel optional disables this requirement and allows componenst to be executed with no artifacts from this channel.

RETURNS	DESCRIPTION
Self	A copy of self which is optional.

Source code in tfx/types/channel.py

def as_optional(self) -> typing_extensions.Self:
  """Creates an optional version of self.

  By default component input channels are considered required, meaning
  if the channel does not contain at least 1 artifact, the component
  will be skipped. Making channel optional disables this requirement and
  allows componenst to be executed with no artifacts from this channel.

  Returns:
    A copy of self which is optional.
  """
  new_channel = copy.copy(self)
  new_channel._is_optional = True  # pylint: disable=protected-access
  return new_channel

get_data_dependent_node_ids abstractmethod

get_data_dependent_node_ids() -> Set[str]

Get data dependent nodes of this channel.

Currently only the OutputChannel directly imposes the data dependency, but other channels can also indirectly have a data dependency if they depend on the OutputChannel. Use this abstract method to define transitive data dependency.

RETURNS	DESCRIPTION
Set[str]	A set of data-dependent node IDs.

Source code in tfx/types/channel.py

@abc.abstractmethod
def get_data_dependent_node_ids(self) -> Set[str]:
  """Get data dependent nodes of this channel.

  Currently only the `OutputChannel` directly imposes the data dependency,
  but other channels can also indirectly have a data dependency if they depend
  on the OutputChannel. Use this abstract method to define transitive data
  dependency.

  Returns:
    A set of data-dependent node IDs.
  """

BaseComponent

BaseComponent(spec: ComponentSpec, custom_executor_spec: Optional[ExecutorSpec] = None)

Bases: BaseNode, ABC

Base class for a TFX pipeline component.

An instance of a subclass of BaseComponent represents the parameters for a single execution of that TFX pipeline component.

All subclasses of BaseComponent must override the SPEC_CLASS field with the ComponentSpec subclass that defines the interface of this component.

ATTRIBUTE	DESCRIPTION
SPEC_CLASS	a subclass of types.ComponentSpec used by this component (required). This is a class level value.
EXECUTOR_SPEC	an instance of executor_spec.ExecutorSpec which describes how to execute this component (required). This is a class level value.
DRIVER_CLASS	a subclass of base_driver.BaseDriver as a custom driver for this component (optional, defaults to base_driver.BaseDriver). This is a class level value.
PRE_EXECUTABLE_SPEC	an optional PythonClassExecutableSpec of pre-execution hook.
spec	an instance of SPEC_CLASS. See types.ComponentSpec for more details.
platform_config	a protobuf message representing platform config for a component instance.

Initialize a component.

PARAMETER	DESCRIPTION
spec	types.ComponentSpec object for this component instance. TYPE: ComponentSpec
custom_executor_spec	Optional custom executor spec overriding the default executor specified in the component attribute. TYPE: Optional[ExecutorSpec] DEFAULT: None

METHOD	DESCRIPTION
add_downstream_node	Experimental: Add another component that must run after this one.
add_downstream_nodes	Experimental: Add another component that must run after this one.
add_upstream_node	Experimental: Add another component that must run before this one.
add_upstream_nodes	Experimental: Add components that must run before this one.
from_json_dict	Convert from dictionary data to an object.
to_json_dict	Convert from an object to a JSON serializable dictionary.
with_platform_config	Attaches a proto-form platform config to a component.

ATTRIBUTE	DESCRIPTION
id	Node id, unique across all TFX nodes in a pipeline. TYPE: str
outputs	Component's output channel dict. TYPE: Dict[str, OutputChannel]

Source code in tfx/dsl/components/base/base_component.py

def __init__(
    self,
    spec: types.ComponentSpec,
    custom_executor_spec: Optional[executor_spec.ExecutorSpec] = None):
  """Initialize a component.

  Args:
    spec: types.ComponentSpec object for this component instance.
    custom_executor_spec: Optional custom executor spec overriding the default
      executor specified in the component attribute.
  """
  if custom_executor_spec:
    if not isinstance(custom_executor_spec, executor_spec.ExecutorSpec):
      raise TypeError(
          ('Custom executor spec override %s for %s should be an instance of '
           'ExecutorSpec') % (custom_executor_spec, self.__class__))

  executor_spec_obj = custom_executor_spec or self.__class__.EXECUTOR_SPEC
  # TODO(b/171742415): Remove this try-catch block once we migrate Beam
  # DAG runner to IR-based stack. The deep copy will only fail for function
  # based components due to pickle workaround we created in ExecutorClassSpec.
  try:
    executor_spec_obj = executor_spec_obj.copy()
  except Exception as e:  # pylint:disable = bare-except
    # This will only happen for function based components, which is fine.
    raise ValueError(
        f'The executor spec {executor_spec_obj!r} of {self.__class__} class '
        'is not copyable.'
    ) from e

  driver_class = self.__class__.DRIVER_CLASS
  # Set self.spec before super.__init__() where node registration happens.
  # This enable node input checking on node context registration.
  self._validate_spec(spec)
  spec.migrate_output_channels(self)
  self.spec = spec
  super().__init__(
      executor_spec=executor_spec_obj,
      driver_class=driver_class,
  )
  self._validate_component_class()
  self.platform_config = None
  self._pip_dependencies = []

Attributes

id property writable

id: str

Node id, unique across all TFX nodes in a pipeline.

If id is set by the user, return it directly. Otherwise, return .

RETURNS	DESCRIPTION
str	node id.

outputs property

outputs: Dict[str, OutputChannel]

Component's output channel dict.

Functions

add_downstream_node

add_downstream_node(downstream_node)

Experimental: Add another component that must run after this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

It is symmetric with add_upstream_node.

PARAMETER	DESCRIPTION
downstream_node	a component that must run after this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_downstream_node(self, downstream_node):
  """Experimental: Add another component that must run after this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.

  It is symmetric with `add_upstream_node`.

  Args:
    downstream_node: a component that must run after this node.
  """
  self._downstream_nodes.add(downstream_node)
  if self not in downstream_node.upstream_nodes:
    downstream_node.add_upstream_node(self)

add_downstream_nodes

add_downstream_nodes(downstream_nodes)

Experimental: Add another component that must run after this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

It is symmetric with add_upstream_nodes.

PARAMETER	DESCRIPTION
downstream_nodes	a list of components that must run after this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_downstream_nodes(self, downstream_nodes):
  """Experimental: Add another component that must run after this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.

  It is symmetric with `add_upstream_nodes`.

  Args:
    downstream_nodes: a list of components that must run after this node.
  """
  self._downstream_nodes.update(downstream_nodes)
  for downstream_node in downstream_nodes:
    if self not in downstream_node.upstream_nodes:
      downstream_node.add_upstream_node(self)

add_upstream_node

add_upstream_node(upstream_node)

Experimental: Add another component that must run before this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

It is symmetric with add_downstream_node.

PARAMETER	DESCRIPTION
upstream_node	a component that must run before this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_upstream_node(self, upstream_node):
  """Experimental: Add another component that must run before this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.

  It is symmetric with `add_downstream_node`.

  Args:
    upstream_node: a component that must run before this node.
  """
  self._upstream_nodes.add(upstream_node)
  if self not in upstream_node.downstream_nodes:
    upstream_node.add_downstream_node(self)

add_upstream_nodes

add_upstream_nodes(upstream_nodes)

Experimental: Add components that must run before this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

PARAMETER	DESCRIPTION
upstream_nodes	a list of components that must run before this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_upstream_nodes(self, upstream_nodes):
  """Experimental: Add components that must run before this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.


  Args:
    upstream_nodes: a list of components that must run before this node.
  """
  self._upstream_nodes.update(upstream_nodes)
  for upstream_node in upstream_nodes:
    if self not in upstream_node.downstream_nodes:
      upstream_node.add_downstream_node(self)

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/utils/json_utils.py

@classmethod
@doc_controls.do_not_doc_in_subclasses
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  """Convert from dictionary data to an object."""
  instance = cls.__new__(cls)
  instance.__dict__ = dict_data
  return instance

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def to_json_dict(self) -> Dict[str, Any]:
  """Convert from an object to a JSON serializable dictionary."""
  return dict((k, v)
              for k, v in self.__dict__.items()
              if k not in ['_upstream_nodes', '_downstream_nodes'])

with_platform_config

with_platform_config(config: Message) -> Self

Attaches a proto-form platform config to a component.

The config will be a per-node platform-specific config.

PARAMETER	DESCRIPTION
config	platform config to attach to the component. TYPE: Message

RETURNS	DESCRIPTION
Self	the same component itself.

Source code in tfx/dsl/components/base/base_component.py

@doc_controls.do_not_doc_in_subclasses
def with_platform_config(
    self, config: message.Message
) -> typing_extensions.Self:
  """Attaches a proto-form platform config to a component.

  The config will be a per-node platform-specific config.

  Args:
    config: platform config to attach to the component.

  Returns:
    the same component itself.
  """
  self.platform_config = config
  return self

BaseFunctionalComponent

BaseFunctionalComponent(spec: ComponentSpec, custom_executor_spec: Optional[ExecutorSpec] = None)

Bases: BaseComponent

Base class for functional components.

Initialize a component.

PARAMETER	DESCRIPTION
spec	types.ComponentSpec object for this component instance. TYPE: ComponentSpec
custom_executor_spec	Optional custom executor spec overriding the default executor specified in the component attribute. TYPE: Optional[ExecutorSpec] DEFAULT: None

METHOD	DESCRIPTION
add_downstream_node	Experimental: Add another component that must run after this one.
add_downstream_nodes	Experimental: Add another component that must run after this one.
add_upstream_node	Experimental: Add another component that must run before this one.
add_upstream_nodes	Experimental: Add components that must run before this one.
from_json_dict	Convert from dictionary data to an object.
to_json_dict	Convert from an object to a JSON serializable dictionary.
with_platform_config	Attaches a proto-form platform config to a component.

ATTRIBUTE	DESCRIPTION
id	Node id, unique across all TFX nodes in a pipeline. TYPE: str
outputs	Component's output channel dict. TYPE: Dict[str, OutputChannel]

Source code in tfx/dsl/components/base/base_component.py

def __init__(
    self,
    spec: types.ComponentSpec,
    custom_executor_spec: Optional[executor_spec.ExecutorSpec] = None):
  """Initialize a component.

  Args:
    spec: types.ComponentSpec object for this component instance.
    custom_executor_spec: Optional custom executor spec overriding the default
      executor specified in the component attribute.
  """
  if custom_executor_spec:
    if not isinstance(custom_executor_spec, executor_spec.ExecutorSpec):
      raise TypeError(
          ('Custom executor spec override %s for %s should be an instance of '
           'ExecutorSpec') % (custom_executor_spec, self.__class__))

  executor_spec_obj = custom_executor_spec or self.__class__.EXECUTOR_SPEC
  # TODO(b/171742415): Remove this try-catch block once we migrate Beam
  # DAG runner to IR-based stack. The deep copy will only fail for function
  # based components due to pickle workaround we created in ExecutorClassSpec.
  try:
    executor_spec_obj = executor_spec_obj.copy()
  except Exception as e:  # pylint:disable = bare-except
    # This will only happen for function based components, which is fine.
    raise ValueError(
        f'The executor spec {executor_spec_obj!r} of {self.__class__} class '
        'is not copyable.'
    ) from e

  driver_class = self.__class__.DRIVER_CLASS
  # Set self.spec before super.__init__() where node registration happens.
  # This enable node input checking on node context registration.
  self._validate_spec(spec)
  spec.migrate_output_channels(self)
  self.spec = spec
  super().__init__(
      executor_spec=executor_spec_obj,
      driver_class=driver_class,
  )
  self._validate_component_class()
  self.platform_config = None
  self._pip_dependencies = []

Attributes

id property writable

id: str

Node id, unique across all TFX nodes in a pipeline.

If id is set by the user, return it directly. Otherwise, return .

RETURNS	DESCRIPTION
str	node id.

outputs property

outputs: Dict[str, OutputChannel]

Component's output channel dict.

Functions

add_downstream_node

add_downstream_node(downstream_node)

Experimental: Add another component that must run after this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

It is symmetric with add_upstream_node.

PARAMETER	DESCRIPTION
downstream_node	a component that must run after this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_downstream_node(self, downstream_node):
  """Experimental: Add another component that must run after this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.

  It is symmetric with `add_upstream_node`.

  Args:
    downstream_node: a component that must run after this node.
  """
  self._downstream_nodes.add(downstream_node)
  if self not in downstream_node.upstream_nodes:
    downstream_node.add_upstream_node(self)

add_downstream_nodes

add_downstream_nodes(downstream_nodes)

Experimental: Add another component that must run after this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

It is symmetric with add_upstream_nodes.

PARAMETER	DESCRIPTION
downstream_nodes	a list of components that must run after this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_downstream_nodes(self, downstream_nodes):
  """Experimental: Add another component that must run after this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.

  It is symmetric with `add_upstream_nodes`.

  Args:
    downstream_nodes: a list of components that must run after this node.
  """
  self._downstream_nodes.update(downstream_nodes)
  for downstream_node in downstream_nodes:
    if self not in downstream_node.upstream_nodes:
      downstream_node.add_upstream_node(self)

add_upstream_node

add_upstream_node(upstream_node)

Experimental: Add another component that must run before this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

It is symmetric with add_downstream_node.

PARAMETER	DESCRIPTION
upstream_node	a component that must run before this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_upstream_node(self, upstream_node):
  """Experimental: Add another component that must run before this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.

  It is symmetric with `add_downstream_node`.

  Args:
    upstream_node: a component that must run before this node.
  """
  self._upstream_nodes.add(upstream_node)
  if self not in upstream_node.downstream_nodes:
    upstream_node.add_downstream_node(self)

add_upstream_nodes

add_upstream_nodes(upstream_nodes)

Experimental: Add components that must run before this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

PARAMETER	DESCRIPTION
upstream_nodes	a list of components that must run before this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_upstream_nodes(self, upstream_nodes):
  """Experimental: Add components that must run before this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.


  Args:
    upstream_nodes: a list of components that must run before this node.
  """
  self._upstream_nodes.update(upstream_nodes)
  for upstream_node in upstream_nodes:
    if self not in upstream_node.downstream_nodes:
      upstream_node.add_downstream_node(self)

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/utils/json_utils.py

@classmethod
@doc_controls.do_not_doc_in_subclasses
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  """Convert from dictionary data to an object."""
  instance = cls.__new__(cls)
  instance.__dict__ = dict_data
  return instance

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def to_json_dict(self) -> Dict[str, Any]:
  """Convert from an object to a JSON serializable dictionary."""
  return dict((k, v)
              for k, v in self.__dict__.items()
              if k not in ['_upstream_nodes', '_downstream_nodes'])

with_platform_config

with_platform_config(config: Message) -> Self

Attaches a proto-form platform config to a component.

The config will be a per-node platform-specific config.

PARAMETER	DESCRIPTION
config	platform config to attach to the component. TYPE: Message

RETURNS	DESCRIPTION
Self	the same component itself.

Source code in tfx/dsl/components/base/base_component.py

@doc_controls.do_not_doc_in_subclasses
def with_platform_config(
    self, config: message.Message
) -> typing_extensions.Self:
  """Attaches a proto-form platform config to a component.

  The config will be a per-node platform-specific config.

  Args:
    config: platform config to attach to the component.

  Returns:
    the same component itself.
  """
  self.platform_config = config
  return self

BaseFunctionalComponentFactory

Bases: Protocol

Serves to declare the return type below.

METHOD	DESCRIPTION
__call__	This corresponds to BaseFunctionalComponent.init.
test_call	This corresponds to the static BaseFunctionalComponent.test_call().

Functions

__call__

__call__(*args: Any, **kwargs: Any) -> BaseFunctionalComponent

This corresponds to BaseFunctionalComponent.init.

Source code in tfx/dsl/component/experimental/decorators.py

def __call__(self, *args: Any, **kwargs: Any) -> BaseFunctionalComponent:
  """This corresponds to BaseFunctionalComponent.__init__."""
  ...

test_call

test_call(*args: Any, **kwargs: Any) -> Any

This corresponds to the static BaseFunctionalComponent.test_call().

Source code in tfx/dsl/component/experimental/decorators.py

def test_call(self, *args: Any, **kwargs: Any) -> Any:
  """This corresponds to the static BaseFunctionalComponent.test_call()."""
  ...

BaseNode

BaseNode(executor_spec: Optional[ExecutorSpec] = None, driver_class: Optional[Type[BaseDriver]] = None)

Bases: Jsonable, ABC

Base class for a node in TFX pipeline.

Initialize a node.

PARAMETER	DESCRIPTION
executor_spec	Optional instance of executor_spec.ExecutorSpec which describes how to execute this node (optional, defaults to an empty executor indicates no-op. TYPE: Optional[ExecutorSpec] DEFAULT: None
driver_class	Optional subclass of base_driver.BaseDriver as a custom driver for this node (optional, defaults to base_driver.BaseDriver). Nodes usually use the default driver class, but may override it. TYPE: Optional[Type[BaseDriver]] DEFAULT: None

METHOD	DESCRIPTION
add_downstream_node	Experimental: Add another component that must run after this one.
add_downstream_nodes	Experimental: Add another component that must run after this one.
add_upstream_node	Experimental: Add another component that must run before this one.
add_upstream_nodes	Experimental: Add components that must run before this one.
from_json_dict	Convert from dictionary data to an object.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
id	Node id, unique across all TFX nodes in a pipeline. TYPE: str

Source code in tfx/dsl/components/base/base_node.py

def __init__(
    self,
    executor_spec: Optional[executor_spec_module.ExecutorSpec] = None,
    driver_class: Optional[Type[base_driver.BaseDriver]] = None,
):
  """Initialize a node.

  Args:
    executor_spec: Optional instance of executor_spec.ExecutorSpec which
      describes how to execute this node (optional, defaults to an empty
      executor indicates no-op.
    driver_class: Optional subclass of base_driver.BaseDriver as a custom
      driver for this node (optional, defaults to base_driver.BaseDriver).
      Nodes usually use the default driver class, but may override it.
  """
  if executor_spec is None:
    executor_spec = executor_spec_module.ExecutorClassSpec(
        base_executor.EmptyExecutor)
  if driver_class is None:
    driver_class = base_driver.BaseDriver
  self.executor_spec = executor_spec
  self.driver_class = driver_class
  self._upstream_nodes = set()
  self._downstream_nodes = set()
  self._id = None
  self._node_execution_options: Optional[utils.NodeExecutionOptions] = None
  dsl_context_registry.get().put_node(self)

Attributes

id property writable

id: str

Node id, unique across all TFX nodes in a pipeline.

If id is set by the user, return it directly. Otherwise, return .

RETURNS	DESCRIPTION
str	node id.

Functions

add_downstream_node

add_downstream_node(downstream_node)

Experimental: Add another component that must run after this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

It is symmetric with add_upstream_node.

PARAMETER	DESCRIPTION
downstream_node	a component that must run after this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_downstream_node(self, downstream_node):
  """Experimental: Add another component that must run after this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.

  It is symmetric with `add_upstream_node`.

  Args:
    downstream_node: a component that must run after this node.
  """
  self._downstream_nodes.add(downstream_node)
  if self not in downstream_node.upstream_nodes:
    downstream_node.add_upstream_node(self)

add_downstream_nodes

add_downstream_nodes(downstream_nodes)

Experimental: Add another component that must run after this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

It is symmetric with add_upstream_nodes.

PARAMETER	DESCRIPTION
downstream_nodes	a list of components that must run after this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_downstream_nodes(self, downstream_nodes):
  """Experimental: Add another component that must run after this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.

  It is symmetric with `add_upstream_nodes`.

  Args:
    downstream_nodes: a list of components that must run after this node.
  """
  self._downstream_nodes.update(downstream_nodes)
  for downstream_node in downstream_nodes:
    if self not in downstream_node.upstream_nodes:
      downstream_node.add_upstream_node(self)

add_upstream_node

add_upstream_node(upstream_node)

Experimental: Add another component that must run before this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

It is symmetric with add_downstream_node.

PARAMETER	DESCRIPTION
upstream_node	a component that must run before this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_upstream_node(self, upstream_node):
  """Experimental: Add another component that must run before this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.

  It is symmetric with `add_downstream_node`.

  Args:
    upstream_node: a component that must run before this node.
  """
  self._upstream_nodes.add(upstream_node)
  if self not in upstream_node.downstream_nodes:
    upstream_node.add_downstream_node(self)

add_upstream_nodes

add_upstream_nodes(upstream_nodes)

Experimental: Add components that must run before this one.

This method enables task-based dependencies by enforcing execution order for synchronous pipelines on supported platforms. Currently, the supported platforms are Airflow, Beam, and Kubeflow Pipelines.

Note that this API call should be considered experimental, and may not work with asynchronous pipelines, sub-pipelines and pipelines with conditional nodes. We also recommend relying on data for capturing dependencies where possible to ensure data lineage is fully captured within MLMD.

PARAMETER	DESCRIPTION
upstream_nodes	a list of components that must run before this node.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def add_upstream_nodes(self, upstream_nodes):
  """Experimental: Add components that must run before this one.

  This method enables task-based dependencies by enforcing execution order for
  synchronous pipelines on supported platforms. Currently, the supported
  platforms are Airflow, Beam, and Kubeflow Pipelines.

  Note that this API call should be considered experimental, and may not work
  with asynchronous pipelines, sub-pipelines and pipelines with conditional
  nodes. We also recommend relying on data for capturing dependencies where
  possible to ensure data lineage is fully captured within MLMD.


  Args:
    upstream_nodes: a list of components that must run before this node.
  """
  self._upstream_nodes.update(upstream_nodes)
  for upstream_node in upstream_nodes:
    if self not in upstream_node.downstream_nodes:
      upstream_node.add_downstream_node(self)

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/utils/json_utils.py

@classmethod
@doc_controls.do_not_doc_in_subclasses
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  """Convert from dictionary data to an object."""
  instance = cls.__new__(cls)
  instance.__dict__ = dict_data
  return instance

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/dsl/components/base/base_node.py

@doc_controls.do_not_doc_in_subclasses
def to_json_dict(self) -> Dict[str, Any]:
  """Convert from an object to a JSON serializable dictionary."""
  return dict((k, v)
              for k, v in self.__dict__.items()
              if k not in ['_upstream_nodes', '_downstream_nodes'])

Modules

standard_artifacts

Public API for standard_artifacts.

CLASS	DESCRIPTION
Boolean	Artifacts representing a boolean.
Bytes	Artifacts representing raw bytes.
ExampleAnomalies	TFX first-party component artifact definition.
ExampleStatistics	TFX first-party component artifact definition.
Examples	Artifact that contains the training data.
Float	Float-typed artifact.
HyperParameters	TFX first-party component artifact definition.
InferenceResult	TFX first-party component artifact definition.
InfraBlessing	TFX first-party component artifact definition.
Integer	Integer-typed artifact.
JsonValue	Artifacts representing a Jsonable value.
Model	Artifact that contains the actual persisted model.
ModelBlessing	Artifact that contains the evaluation of a trained model.
ModelEvaluation	TFX first-party component artifact definition.
ModelRun	TFX first-party component artifact definition.
PushedModel	TFX first-party component artifact definition.
Schema	Artifact that contains the schema of the data.
String	String-typed artifact.
TransformCache	TFX first-party component artifact definition.
TransformGraph	TFX first-party component artifact definition.
TunerResults	TFX first-party component artifact definition.

Classes

Boolean

Boolean(*args, **kwargs)

Bases: ValueArtifact

Artifacts representing a boolean.

Boolean value artifacts are encoded as "1" for True and "0" for False.

Initializes ValueArtifact.

METHOD	DESCRIPTION
annotate_as	Annotate the value artifact type with a system artifact class.
copy_from	Set uri, properties and custom properties from a given Artifact.
decode	Method decoding the file content. Implemented by subclasses.
encode	Method encoding the file content. Implemented by subclasses.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str
value	Value stored in the artifact.

Source code in tfx/types/value_artifact.py

def __init__(self, *args, **kwargs):
  """Initializes ValueArtifact."""
  self._has_value = False
  self._modified = False
  self._value = None
  super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

value property writable

value

Value stored in the artifact.

Functions

annotate_as classmethod

annotate_as(type_annotation: Optional[Type[SystemArtifact]] = None)

Annotate the value artifact type with a system artifact class.

Example usage

from tfx import v1 as tfx

OutputArtifact = tfx.dsl.components.OutputArtifact
String = tfx.types.standard_artifacts.String
Model = tfx.dsl.standard_annotations.Model


@tfx.dsl.components.component
def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...

PARAMETER	DESCRIPTION
type_annotation	the standard annotations used to annotate the value artifact type. The possible values are in tfx.v1.dsl.standard_annotations. TYPE: Optional[Type[SystemArtifact]] DEFAULT: None

RETURNS	DESCRIPTION
	A subclass of the method caller class (e.g., standard_artifacts.String, standard_artifacts.Float) with TYPE_ANNOTATION attribute set to be type_annotation; returns the original class iftype_annotation is None.

Source code in tfx/types/value_artifact.py

@classmethod
def annotate_as(cls, type_annotation: Optional[Type[SystemArtifact]] = None):
  """Annotate the value artifact type with a system artifact class.

  !!! example "Example usage"

      ```python
      from tfx import v1 as tfx

      OutputArtifact = tfx.dsl.components.OutputArtifact
      String = tfx.types.standard_artifacts.String
      Model = tfx.dsl.standard_annotations.Model


      @tfx.dsl.components.component
      def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...
      ```

  Args:
    type_annotation: the standard annotations used to annotate the value
      artifact type. The possible values are in
      `tfx.v1.dsl.standard_annotations`.

  Returns:
    A subclass of the method caller class (e.g., [`standard_artifacts.String`][tfx.v1.types.standard_artifacts.String],
      [`standard_artifacts.Float`][tfx.v1.types.standard_artifacts.Float]) with TYPE_ANNOTATION attribute set to be
      `type_annotation`; returns the original class if`type_annotation` is None.
  """
  if not type_annotation:
    return cls
  if not issubclass(type_annotation, SystemArtifact):
    raise ValueError(
        'type_annotation %s is not a subclass of SystemArtifact.' %
        type_annotation)
  type_annotation_str = str(type_annotation.__name__)
  return type(
      str(cls.__name__) + '_' + type_annotation_str,
      (cls,),
      {
          'TYPE_NAME': str(cls.TYPE_NAME) + '_' + type_annotation_str,
          'TYPE_ANNOTATION': type_annotation,
          '__module__': cls.__module__,
      },
  )

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

decode

decode(serialized_value: bytes)

Method decoding the file content. Implemented by subclasses.

Source code in tfx/types/standard_artifacts.py

def decode(self, serialized_value: bytes):
    return int(serialized_value) != 0

encode

encode(value: bool)

Method encoding the file content. Implemented by subclasses.

Source code in tfx/types/standard_artifacts.py

def encode(self, value: bool):
    if not isinstance(value, bool):
        raise TypeError(
            f"Expecting bytes but got value {value} of type {type(value)}"
        )
    return b"1" if value else b"0"

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

Bytes

Bytes(*args, **kwargs)

Bases: ValueArtifact

Artifacts representing raw bytes.

Initializes ValueArtifact.

METHOD	DESCRIPTION
annotate_as	Annotate the value artifact type with a system artifact class.
copy_from	Set uri, properties and custom properties from a given Artifact.
decode	Method decoding the file content. Implemented by subclasses.
encode	Method encoding the file content. Implemented by subclasses.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str
value	Value stored in the artifact.

Source code in tfx/types/value_artifact.py

def __init__(self, *args, **kwargs):
  """Initializes ValueArtifact."""
  self._has_value = False
  self._modified = False
  self._value = None
  super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

value property writable

value

Value stored in the artifact.

Functions

annotate_as classmethod

annotate_as(type_annotation: Optional[Type[SystemArtifact]] = None)

Annotate the value artifact type with a system artifact class.

Example usage

from tfx import v1 as tfx

OutputArtifact = tfx.dsl.components.OutputArtifact
String = tfx.types.standard_artifacts.String
Model = tfx.dsl.standard_annotations.Model


@tfx.dsl.components.component
def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...

PARAMETER	DESCRIPTION
type_annotation	the standard annotations used to annotate the value artifact type. The possible values are in tfx.v1.dsl.standard_annotations. TYPE: Optional[Type[SystemArtifact]] DEFAULT: None

RETURNS	DESCRIPTION
	A subclass of the method caller class (e.g., standard_artifacts.String, standard_artifacts.Float) with TYPE_ANNOTATION attribute set to be type_annotation; returns the original class iftype_annotation is None.

Source code in tfx/types/value_artifact.py

@classmethod
def annotate_as(cls, type_annotation: Optional[Type[SystemArtifact]] = None):
  """Annotate the value artifact type with a system artifact class.

  !!! example "Example usage"

      ```python
      from tfx import v1 as tfx

      OutputArtifact = tfx.dsl.components.OutputArtifact
      String = tfx.types.standard_artifacts.String
      Model = tfx.dsl.standard_annotations.Model


      @tfx.dsl.components.component
      def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...
      ```

  Args:
    type_annotation: the standard annotations used to annotate the value
      artifact type. The possible values are in
      `tfx.v1.dsl.standard_annotations`.

  Returns:
    A subclass of the method caller class (e.g., [`standard_artifacts.String`][tfx.v1.types.standard_artifacts.String],
      [`standard_artifacts.Float`][tfx.v1.types.standard_artifacts.Float]) with TYPE_ANNOTATION attribute set to be
      `type_annotation`; returns the original class if`type_annotation` is None.
  """
  if not type_annotation:
    return cls
  if not issubclass(type_annotation, SystemArtifact):
    raise ValueError(
        'type_annotation %s is not a subclass of SystemArtifact.' %
        type_annotation)
  type_annotation_str = str(type_annotation.__name__)
  return type(
      str(cls.__name__) + '_' + type_annotation_str,
      (cls,),
      {
          'TYPE_NAME': str(cls.TYPE_NAME) + '_' + type_annotation_str,
          'TYPE_ANNOTATION': type_annotation,
          '__module__': cls.__module__,
      },
  )

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

decode

decode(serialized_value: bytes)

Method decoding the file content. Implemented by subclasses.

Source code in tfx/types/standard_artifacts.py

def decode(self, serialized_value: bytes):
    return serialized_value

encode

encode(value: bytes)

Method encoding the file content. Implemented by subclasses.

Source code in tfx/types/standard_artifacts.py

def encode(self, value: bytes):
    if not isinstance(value, bytes):
        raise TypeError(
            "Expecting bytes but got value %s of type %s"
            % (str(value), type(value))
        )
    return value

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

ExampleAnomalies

ExampleAnomalies(*args, **kwargs)

Bases: _TfxArtifact

TFX first-party component artifact definition.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

ExampleStatistics

ExampleStatistics(*args, **kwargs)

Bases: _TfxArtifact

TFX first-party component artifact definition.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

Examples

Examples(*args, **kwargs)

Bases: _TfxArtifact

Artifact that contains the training data.

Training data should be brought in to the TFX pipeline using components like ExampleGen. Data in Examples artifact is split and stored separately. The file and payload format must be specified as optional custom properties if not using default formats. Please see the ExampleGen guide to understand about span, version and splits.

• Properties:
• span: Integer to distinguish group of Examples.
• version: Integer to represent updated data.

• splits: A list of split names. For example, ["train", "test"].

• File structure:

• {uri}/

  • Split-{split_name1}/: Files for split
    • All direct children files are recognized as the data.
    • File format and payload format are determined by custom properties.
  • Split-{split_name2}/: Another split...

• Commonly used custom properties of the Examples artifact:

• file_format: a string that represents the file format. See tfx/components/util/tfxio_utils.py:make_tfxio for available values.
• payload_format: int (enum) value of the data payload format. See tfx/proto/example_gen.proto:PayloadFormat for available formats.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
path	Path to the artifact URI's split subdirectory.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

path

path(*, split: str) -> str

Path to the artifact URI's split subdirectory.

This method DOES NOT create a directory path it returns; caller must make a directory of the returned path value before writing.

PARAMETER	DESCRIPTION
split	A name of the split, e.g. "train", "validation", "test". TYPE: str

RAISES	DESCRIPTION
ValueError	if the split is not in the self.splits.

RETURNS	DESCRIPTION
str	A path to {self.uri}/Split-{split}.

Source code in tfx/types/standard_artifacts.py

def path(self, *, split: str) -> str:
    """Path to the artifact URI's split subdirectory.

    This method DOES NOT create a directory path it returns; caller must make
    a directory of the returned path value before writing.

    Args:
      split: A name of the split, e.g. `"train"`, `"validation"`, `"test"`.

    Raises:
      ValueError: if the `split` is not in the `self.splits`.

    Returns:
      A path to `{self.uri}/Split-{split}`.
    """
    if split not in self.splits:
        raise ValueError(
            f"Split {split} not found in {self.splits=}. Did you forget to update"
            " Examples.splits first?"
        )
    return standard_artifact_utils.get_split_uris([self], split)[0]

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

Float

Float(*args, **kwargs)

Bases: ValueArtifact

Float-typed artifact.

Float value artifacts are encoded using Python str() class. However, Nan and Infinity are handled separately. See string constants in the class.

Initializes ValueArtifact.

METHOD	DESCRIPTION
annotate_as	Annotate the value artifact type with a system artifact class.
copy_from	Set uri, properties and custom properties from a given Artifact.
decode	Method decoding the file content. Implemented by subclasses.
encode	Method encoding the file content. Implemented by subclasses.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str
value	Value stored in the artifact.

Source code in tfx/types/value_artifact.py

def __init__(self, *args, **kwargs):
  """Initializes ValueArtifact."""
  self._has_value = False
  self._modified = False
  self._value = None
  super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

value property writable

value

Value stored in the artifact.

Functions

annotate_as classmethod

annotate_as(type_annotation: Optional[Type[SystemArtifact]] = None)

Annotate the value artifact type with a system artifact class.

Example usage

from tfx import v1 as tfx

OutputArtifact = tfx.dsl.components.OutputArtifact
String = tfx.types.standard_artifacts.String
Model = tfx.dsl.standard_annotations.Model


@tfx.dsl.components.component
def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...

PARAMETER	DESCRIPTION
type_annotation	the standard annotations used to annotate the value artifact type. The possible values are in tfx.v1.dsl.standard_annotations. TYPE: Optional[Type[SystemArtifact]] DEFAULT: None

RETURNS	DESCRIPTION
	A subclass of the method caller class (e.g., standard_artifacts.String, standard_artifacts.Float) with TYPE_ANNOTATION attribute set to be type_annotation; returns the original class iftype_annotation is None.

Source code in tfx/types/value_artifact.py

@classmethod
def annotate_as(cls, type_annotation: Optional[Type[SystemArtifact]] = None):
  """Annotate the value artifact type with a system artifact class.

  !!! example "Example usage"

      ```python
      from tfx import v1 as tfx

      OutputArtifact = tfx.dsl.components.OutputArtifact
      String = tfx.types.standard_artifacts.String
      Model = tfx.dsl.standard_annotations.Model


      @tfx.dsl.components.component
      def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...
      ```

  Args:
    type_annotation: the standard annotations used to annotate the value
      artifact type. The possible values are in
      `tfx.v1.dsl.standard_annotations`.

  Returns:
    A subclass of the method caller class (e.g., [`standard_artifacts.String`][tfx.v1.types.standard_artifacts.String],
      [`standard_artifacts.Float`][tfx.v1.types.standard_artifacts.Float]) with TYPE_ANNOTATION attribute set to be
      `type_annotation`; returns the original class if`type_annotation` is None.
  """
  if not type_annotation:
    return cls
  if not issubclass(type_annotation, SystemArtifact):
    raise ValueError(
        'type_annotation %s is not a subclass of SystemArtifact.' %
        type_annotation)
  type_annotation_str = str(type_annotation.__name__)
  return type(
      str(cls.__name__) + '_' + type_annotation_str,
      (cls,),
      {
          'TYPE_NAME': str(cls.TYPE_NAME) + '_' + type_annotation_str,
          'TYPE_ANNOTATION': type_annotation,
          '__module__': cls.__module__,
      },
  )

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

decode

decode(serialized_value: bytes) -> float

Method decoding the file content. Implemented by subclasses.

Source code in tfx/types/standard_artifacts.py

def decode(self, serialized_value: bytes) -> float:
    result = float(serialized_value)

    # Check that the decoded value exactly matches the encoded string.
    # Note that float() can handle bytes, but Decimal() cannot.
    serialized_string = serialized_value.decode("utf-8")
    reserialized_string = str(result)
    is_exact = decimal.Decimal(serialized_string) == decimal.Decimal(
        reserialized_string
    )
    if not is_exact:
        logging.warning(
            'The number "%s" has lost precision when converted to float "%s"',
            serialized_value,
            reserialized_string,
        )

    return result

encode

encode(value: float) -> bytes

Method encoding the file content. Implemented by subclasses.

Source code in tfx/types/standard_artifacts.py

def encode(self, value: float) -> bytes:
    if not isinstance(value, float):
        raise TypeError(
            f"Expecting float but got value {value} of type {type(value)}"
        )
    if math.isinf(value) or math.isnan(value):
        logging.warning(
            '! The number "%s" may be unsupported by non-python components.', value
        )
    str_value = str(value)
    # Special encoding for infinities and NaN to increase comatibility with
    # other languages.
    # Decoding works automatically.
    if math.isinf(value):
        if value >= 0:
            str_value = Float._ENCODED_POSITIVE_INFINITY
        else:
            str_value = Float._ENCODED_NEGATIVE_INFINITY
    if math.isnan(value):
        str_value = Float._ENCODED_NAN

    return str_value.encode("utf-8")

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

HyperParameters

HyperParameters(*args, **kwargs)

Bases: _TfxArtifact

TFX first-party component artifact definition.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

InferenceResult

InferenceResult(*args, **kwargs)

Bases: _TfxArtifact

TFX first-party component artifact definition.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

InfraBlessing

InfraBlessing(*args, **kwargs)

Bases: _TfxArtifact

TFX first-party component artifact definition.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

Integer

Integer(*args, **kwargs)

Bases: ValueArtifact

Integer-typed artifact.

Integer value artifacts are encoded as a decimal string.

Initializes ValueArtifact.

METHOD	DESCRIPTION
annotate_as	Annotate the value artifact type with a system artifact class.
copy_from	Set uri, properties and custom properties from a given Artifact.
decode	Method decoding the file content. Implemented by subclasses.
encode	Method encoding the file content. Implemented by subclasses.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str
value	Value stored in the artifact.

Source code in tfx/types/value_artifact.py

def __init__(self, *args, **kwargs):
  """Initializes ValueArtifact."""
  self._has_value = False
  self._modified = False
  self._value = None
  super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

value property writable

value

Value stored in the artifact.

Functions

annotate_as classmethod

annotate_as(type_annotation: Optional[Type[SystemArtifact]] = None)

Annotate the value artifact type with a system artifact class.

Example usage

from tfx import v1 as tfx

OutputArtifact = tfx.dsl.components.OutputArtifact
String = tfx.types.standard_artifacts.String
Model = tfx.dsl.standard_annotations.Model


@tfx.dsl.components.component
def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...

PARAMETER	DESCRIPTION
type_annotation	the standard annotations used to annotate the value artifact type. The possible values are in tfx.v1.dsl.standard_annotations. TYPE: Optional[Type[SystemArtifact]] DEFAULT: None

RETURNS	DESCRIPTION
	A subclass of the method caller class (e.g., standard_artifacts.String, standard_artifacts.Float) with TYPE_ANNOTATION attribute set to be type_annotation; returns the original class iftype_annotation is None.

Source code in tfx/types/value_artifact.py

@classmethod
def annotate_as(cls, type_annotation: Optional[Type[SystemArtifact]] = None):
  """Annotate the value artifact type with a system artifact class.

  !!! example "Example usage"

      ```python
      from tfx import v1 as tfx

      OutputArtifact = tfx.dsl.components.OutputArtifact
      String = tfx.types.standard_artifacts.String
      Model = tfx.dsl.standard_annotations.Model


      @tfx.dsl.components.component
      def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...
      ```

  Args:
    type_annotation: the standard annotations used to annotate the value
      artifact type. The possible values are in
      `tfx.v1.dsl.standard_annotations`.

  Returns:
    A subclass of the method caller class (e.g., [`standard_artifacts.String`][tfx.v1.types.standard_artifacts.String],
      [`standard_artifacts.Float`][tfx.v1.types.standard_artifacts.Float]) with TYPE_ANNOTATION attribute set to be
      `type_annotation`; returns the original class if`type_annotation` is None.
  """
  if not type_annotation:
    return cls
  if not issubclass(type_annotation, SystemArtifact):
    raise ValueError(
        'type_annotation %s is not a subclass of SystemArtifact.' %
        type_annotation)
  type_annotation_str = str(type_annotation.__name__)
  return type(
      str(cls.__name__) + '_' + type_annotation_str,
      (cls,),
      {
          'TYPE_NAME': str(cls.TYPE_NAME) + '_' + type_annotation_str,
          'TYPE_ANNOTATION': type_annotation,
          '__module__': cls.__module__,
      },
  )

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

decode

decode(serialized_value: bytes) -> int

Method decoding the file content. Implemented by subclasses.

Source code in tfx/types/standard_artifacts.py

def decode(self, serialized_value: bytes) -> int:
    return int(serialized_value)

encode

encode(value: int) -> bytes

Method encoding the file content. Implemented by subclasses.

Source code in tfx/types/standard_artifacts.py

def encode(self, value: int) -> bytes:
    if not isinstance(value, int):
        raise TypeError(
            f"Expecting int but got value {value} of type {type(value)}"
        )
    return str(value).encode("utf-8")

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

JsonValue

JsonValue(*args, **kwargs)

Bases: ValueArtifact

Artifacts representing a Jsonable value.

Initializes ValueArtifact.

METHOD	DESCRIPTION
annotate_as	Annotate the value artifact type with a system artifact class.
copy_from	Set uri, properties and custom properties from a given Artifact.
decode	Method decoding the file content. Implemented by subclasses.
encode	Method encoding the file content. Implemented by subclasses.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str
value	Value stored in the artifact.

Source code in tfx/types/value_artifact.py

def __init__(self, *args, **kwargs):
  """Initializes ValueArtifact."""
  self._has_value = False
  self._modified = False
  self._value = None
  super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

value property writable

value

Value stored in the artifact.

Functions

annotate_as classmethod

annotate_as(type_annotation: Optional[Type[SystemArtifact]] = None)

Annotate the value artifact type with a system artifact class.

Example usage

from tfx import v1 as tfx

OutputArtifact = tfx.dsl.components.OutputArtifact
String = tfx.types.standard_artifacts.String
Model = tfx.dsl.standard_annotations.Model


@tfx.dsl.components.component
def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...

PARAMETER	DESCRIPTION
type_annotation	the standard annotations used to annotate the value artifact type. The possible values are in tfx.v1.dsl.standard_annotations. TYPE: Optional[Type[SystemArtifact]] DEFAULT: None

RETURNS	DESCRIPTION
	A subclass of the method caller class (e.g., standard_artifacts.String, standard_artifacts.Float) with TYPE_ANNOTATION attribute set to be type_annotation; returns the original class iftype_annotation is None.

Source code in tfx/types/value_artifact.py

@classmethod
def annotate_as(cls, type_annotation: Optional[Type[SystemArtifact]] = None):
  """Annotate the value artifact type with a system artifact class.

  !!! example "Example usage"

      ```python
      from tfx import v1 as tfx

      OutputArtifact = tfx.dsl.components.OutputArtifact
      String = tfx.types.standard_artifacts.String
      Model = tfx.dsl.standard_annotations.Model


      @tfx.dsl.components.component
      def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...
      ```

  Args:
    type_annotation: the standard annotations used to annotate the value
      artifact type. The possible values are in
      `tfx.v1.dsl.standard_annotations`.

  Returns:
    A subclass of the method caller class (e.g., [`standard_artifacts.String`][tfx.v1.types.standard_artifacts.String],
      [`standard_artifacts.Float`][tfx.v1.types.standard_artifacts.Float]) with TYPE_ANNOTATION attribute set to be
      `type_annotation`; returns the original class if`type_annotation` is None.
  """
  if not type_annotation:
    return cls
  if not issubclass(type_annotation, SystemArtifact):
    raise ValueError(
        'type_annotation %s is not a subclass of SystemArtifact.' %
        type_annotation)
  type_annotation_str = str(type_annotation.__name__)
  return type(
      str(cls.__name__) + '_' + type_annotation_str,
      (cls,),
      {
          'TYPE_NAME': str(cls.TYPE_NAME) + '_' + type_annotation_str,
          'TYPE_ANNOTATION': type_annotation,
          '__module__': cls.__module__,
      },
  )

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

decode

decode(serialized_value: str) -> JsonableType

Method decoding the file content. Implemented by subclasses.

Source code in tfx/types/standard_artifacts.py

def decode(self, serialized_value: str) -> json_utils.JsonableType:
    return json_utils.loads(serialized_value)

encode

encode(value: JsonableType) -> str

Method encoding the file content. Implemented by subclasses.

Source code in tfx/types/standard_artifacts.py

def encode(self, value: json_utils.JsonableType) -> str:
    return json_utils.dumps(value)

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

Model

Model(*args, **kwargs)

Bases: _TfxArtifact

Artifact that contains the actual persisted model.

Training components stores the trained model like a saved model in this artifact. A Model artifact contains serialization of the trained model in one or more formats, each suitable for different usage (e.g. serving, evaluation), and serving environments.

• File structure:

• {uri}/

  • Format-Serving/: Model exported for serving.
    • saved_model.pb
    • Other actual model files.
  • Format-TFMA/: Model exported for evaluation.
    • saved_model.pb
    • Other actual model files.

• Commonly used custom properties of the Model artifact:

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

ModelBlessing

ModelBlessing(*args, **kwargs)

Bases: _TfxArtifact

Artifact that contains the evaluation of a trained model.

This artifact is usually used with Conditional when determining whether to push this model on service or not.

# Run pusher if evaluator has blessed the model.
with tfx.dsl.Cond(evaluator.outputs['blessing'].future()
                  [0].custom_property('blessed') == 1):
  pusher = Pusher(...)

• File structure:
• {uri}/
  • BLESSED: if the evaluator has blessed the model.
  • NOT_BLESSED: if the evaluator has not blessed the model.

• See tfx/components/evaluator/executor.py for how to write ModelBlessing.

• Commonly used custom properties of the ModelBlessing artifact:

• blessed: int value that represents whether the evaluator has blessed its model or not.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

ModelEvaluation

ModelEvaluation(*args, **kwargs)

Bases: _TfxArtifact

TFX first-party component artifact definition.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

ModelRun

ModelRun(*args, **kwargs)

Bases: _TfxArtifact

TFX first-party component artifact definition.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

PushedModel

PushedModel(*args, **kwargs)

Bases: _TfxArtifact

TFX first-party component artifact definition.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

Schema

Schema(*args, **kwargs)

Bases: _TfxArtifact

Artifact that contains the schema of the data.

Schema artifact is used to store the schema of the data. The schema is a proto that describes the data, including the type of each feature, the range of values for each feature, and other properties. The schema is usually generated by the SchemaGen component, which uses the statistics of the data to infer the schema. The schema can be used by other components in the pipeline to validate the data and to generate models.

• File structure:
• {uri}/
  • schema.pbtxt: Text-proto format serialization of tensorflow_metadata.proto.v0.schema.Schema proto message.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

String

String(*args, **kwargs)

Bases: ValueArtifact

String-typed artifact.

String value artifacts are encoded using UTF-8.

Initializes ValueArtifact.

METHOD	DESCRIPTION
annotate_as	Annotate the value artifact type with a system artifact class.
copy_from	Set uri, properties and custom properties from a given Artifact.
decode	Method decoding the file content. Implemented by subclasses.
encode	Method encoding the file content. Implemented by subclasses.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str
value	Value stored in the artifact.

Source code in tfx/types/value_artifact.py

def __init__(self, *args, **kwargs):
  """Initializes ValueArtifact."""
  self._has_value = False
  self._modified = False
  self._value = None
  super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

value property writable

value

Value stored in the artifact.

Functions

annotate_as classmethod

annotate_as(type_annotation: Optional[Type[SystemArtifact]] = None)

Annotate the value artifact type with a system artifact class.

Example usage

from tfx import v1 as tfx

OutputArtifact = tfx.dsl.components.OutputArtifact
String = tfx.types.standard_artifacts.String
Model = tfx.dsl.standard_annotations.Model


@tfx.dsl.components.component
def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...

PARAMETER	DESCRIPTION
type_annotation	the standard annotations used to annotate the value artifact type. The possible values are in tfx.v1.dsl.standard_annotations. TYPE: Optional[Type[SystemArtifact]] DEFAULT: None

RETURNS	DESCRIPTION
	A subclass of the method caller class (e.g., standard_artifacts.String, standard_artifacts.Float) with TYPE_ANNOTATION attribute set to be type_annotation; returns the original class iftype_annotation is None.

Source code in tfx/types/value_artifact.py

@classmethod
def annotate_as(cls, type_annotation: Optional[Type[SystemArtifact]] = None):
  """Annotate the value artifact type with a system artifact class.

  !!! example "Example usage"

      ```python
      from tfx import v1 as tfx

      OutputArtifact = tfx.dsl.components.OutputArtifact
      String = tfx.types.standard_artifacts.String
      Model = tfx.dsl.standard_annotations.Model


      @tfx.dsl.components.component
      def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...
      ```

  Args:
    type_annotation: the standard annotations used to annotate the value
      artifact type. The possible values are in
      `tfx.v1.dsl.standard_annotations`.

  Returns:
    A subclass of the method caller class (e.g., [`standard_artifacts.String`][tfx.v1.types.standard_artifacts.String],
      [`standard_artifacts.Float`][tfx.v1.types.standard_artifacts.Float]) with TYPE_ANNOTATION attribute set to be
      `type_annotation`; returns the original class if`type_annotation` is None.
  """
  if not type_annotation:
    return cls
  if not issubclass(type_annotation, SystemArtifact):
    raise ValueError(
        'type_annotation %s is not a subclass of SystemArtifact.' %
        type_annotation)
  type_annotation_str = str(type_annotation.__name__)
  return type(
      str(cls.__name__) + '_' + type_annotation_str,
      (cls,),
      {
          'TYPE_NAME': str(cls.TYPE_NAME) + '_' + type_annotation_str,
          'TYPE_ANNOTATION': type_annotation,
          '__module__': cls.__module__,
      },
  )

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

decode

decode(serialized_value: bytes) -> str

Method decoding the file content. Implemented by subclasses.

Source code in tfx/types/standard_artifacts.py

def decode(self, serialized_value: bytes) -> str:
    return serialized_value.decode("utf-8")

encode

encode(value: str) -> bytes

Method encoding the file content. Implemented by subclasses.

Source code in tfx/types/standard_artifacts.py

def encode(self, value: str) -> bytes:
    if not isinstance(value, str):
        raise TypeError(
            "Expecting Text but got value %s of type %s" % (str(value), type(value))
        )
    return value.encode("utf-8")

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

TransformCache

TransformCache(*args, **kwargs)

Bases: _TfxArtifact

TFX first-party component artifact definition.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

TransformGraph

TransformGraph(*args, **kwargs)

Bases: _TfxArtifact

TFX first-party component artifact definition.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }

TunerResults

TunerResults(*args, **kwargs)

Bases: _TfxArtifact

TFX first-party component artifact definition.

Construct TFX first-party component artifact.

METHOD	DESCRIPTION
copy_from	Set uri, properties and custom properties from a given Artifact.
from_json_dict	Convert from dictionary data to an object.
get_bool_custom_property	Get a custom property of bool type.
get_custom_property	Gets a custom property with key. Return None if not found.
get_float_custom_property	Gets a custom property of float type.
get_int_custom_property	Get a custom property of int type.
get_json_value_custom_property	Get a custom property of JSON type.
get_proto_custom_property	Get a custom property of proto type.
get_string_custom_property	Get a custom property of string type.
set_bool_custom_property	Sets a custom property of bool type.
set_float_custom_property	Sets a custom property of float type.
set_int_custom_property	Set a custom property of int type.
set_json_value_custom_property	Sets a custom property of JSON type.
set_mlmd_artifact	Replace the MLMD artifact object on this artifact.
set_mlmd_artifact_type	Set entire ArtifactType in this object.
set_proto_custom_property	Sets a custom property of proto type.
set_string_custom_property	Set a custom property of string type.
to_json_dict	Convert from an object to a JSON serializable dictionary.

ATTRIBUTE	DESCRIPTION
artifact_type	Type of the underlying mlmd artifact.
external_id	external id of the underlying artifact. TYPE: str
id	Id of the underlying mlmd artifact. TYPE: int
is_external	Returns true if the artifact is external. TYPE: bool
mlmd_artifact	Underlying mlmd artifact.
name	Name of the underlying mlmd artifact. TYPE: str
pipeline_name	Name of the pipeline that produce the artifact. TYPE: str
producer_component	Producer component of the artifact. TYPE: str
state	State of the underlying mlmd artifact. TYPE: str
type	Type of the artifact.
type_id	Type id of the underlying mlmd artifact. TYPE: int
type_name	Type name of the underlying mlmd artifact.
uri	Artifact URI. TYPE: str

Source code in tfx/types/standard_artifacts.py

def __init__(self, *args, **kwargs):
    """Construct TFX first-party component artifact."""
    # TODO(b/176795331): Refactor directory structure to make it clearer that
    # TFX-specific artifacts require the full "tfx" package be installed.
    #
    # Do not allow usage of TFX-specific artifact if only the core pipeline
    # SDK package is installed.
    try:
        import setuptools # pytype: disable=import-error  # noqa: F401

        # Test import only when setuptools is available.
        try:
            # `extensions` is not included in ml_pipelines_sdk and doesn't have any
            # transitive import.
            import tfx.extensions as _  # type: ignore  # noqa: F401 # pylint: disable=g-import-not-at-top
        except ModuleNotFoundError as err:
            # The following condition detects exactly whether only the DSL package
            # is installed, and is bypassed when tests run in Bazel.
            raise RuntimeError(
                'The "tfx" and all dependent packages need to be '
                "installed to use this functionality."
            ) from err
    except ModuleNotFoundError:
        pass

    super().__init__(*args, **kwargs)

Attributes

artifact_type property

artifact_type

Type of the underlying mlmd artifact.

external_id property

external_id: str

external id of the underlying artifact.

id property writable

id: int

Id of the underlying mlmd artifact.

is_external property writable

is_external: bool

Returns true if the artifact is external.

mlmd_artifact property

mlmd_artifact

Underlying mlmd artifact.

name property writable

name: str

Name of the underlying mlmd artifact.

pipeline_name property writable

pipeline_name: str

Name of the pipeline that produce the artifact.

producer_component property writable

producer_component: str

Producer component of the artifact.

state property writable

state: str

State of the underlying mlmd artifact.

type property

type

Type of the artifact.

type_id property writable

type_id: int

Type id of the underlying mlmd artifact.

type_name property

type_name

Type name of the underlying mlmd artifact.

uri property writable

uri: str

Artifact URI.

Functions

copy_from

copy_from(other: Artifact)

Set uri, properties and custom properties from a given Artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def copy_from(self, other: 'Artifact'):
  """Set uri, properties and custom properties from a given Artifact."""
  assert self.type is other.type, (
      'Unable to set properties from an artifact of different type: {} vs {}'
      .format(self.type_name, other.type_name))
  self.uri = other.uri
  if other.artifact_type.HasField('id'):
    self.type_id = other.artifact_type.id

  self._artifact.properties.clear()
  self._artifact.properties.MergeFrom(other._artifact.properties)  # pylint: disable=protected-access
  self._artifact.custom_properties.clear()
  self._artifact.custom_properties.MergeFrom(
      other._artifact.custom_properties)  # pylint: disable=protected-access
  self._cached_modifiable_properties = copy.deepcopy(
      other._cached_modifiable_properties)  # pylint: disable=protected-access
  self._cached_modifiable_custom_properties = copy.deepcopy(
      other._cached_modifiable_custom_properties)  # pylint: disable=protected-access

from_json_dict classmethod

from_json_dict(dict_data: Dict[str, Any]) -> Any

Convert from dictionary data to an object.

Source code in tfx/types/artifact.py

@classmethod
@doc_controls.do_not_doc_inheritable
def from_json_dict(cls, dict_data: Dict[str, Any]) -> Any:
  module_name = dict_data['__artifact_class_module__']
  class_name = dict_data['__artifact_class_name__']
  artifact = metadata_store_pb2.Artifact()
  artifact_type = metadata_store_pb2.ArtifactType()
  json_format.Parse(json.dumps(dict_data['artifact']), artifact)
  json_format.Parse(json.dumps(dict_data['artifact_type']), artifact_type)

  # First, try to resolve the specific class used for the artifact; if this
  # is not possible, use a generic artifact.Artifact object.
  result = None
  try:
    artifact_cls = getattr(importlib.import_module(module_name), class_name)
    # If the artifact type is the base Artifact class, do not construct the
    # object here since that constructor requires the mlmd_artifact_type
    # argument.
    if artifact_cls != Artifact:
      result = artifact_cls()
  except (AttributeError, ImportError, ValueError):
    logging.warning((
        'Could not load artifact class %s.%s; using fallback deserialization '
        'for the relevant artifact. Please make sure that any artifact '
        'classes can be imported within your container or environment.'),
                    module_name, class_name)
  if not result:
    result = Artifact(mlmd_artifact_type=artifact_type)
  result.set_mlmd_artifact_type(artifact_type)
  result.set_mlmd_artifact(artifact)
  return result

get_bool_custom_property

get_bool_custom_property(key: str) -> bool

Get a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_bool_custom_property(self, key: str) -> bool:
  """Get a custom property of bool type."""
  if key not in self._artifact.custom_properties:
    return False
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, bool):
    return json_value
  return self._artifact.custom_properties[key].bool_value

get_custom_property

get_custom_property(key: str) -> Optional[Union[int, float, str, bool, JsonValueType]]

Gets a custom property with key. Return None if not found.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_custom_property(
    self, key: str
) -> Optional[Union[int, float, str, bool, JsonValueType]]:
  """Gets a custom property with key. Return None if not found."""
  if key not in self._artifact.custom_properties:
    return None

  json_value = self.get_json_value_custom_property(key)
  if json_value:
    return json_value

  mlmd_value = self._artifact.custom_properties[key]
  if mlmd_value.HasField('int_value'):
    return mlmd_value.int_value
  elif mlmd_value.HasField('double_value'):
    return mlmd_value.double_value
  elif mlmd_value.HasField('string_value'):
    return mlmd_value.string_value
  elif mlmd_value.HasField('bool_value'):
    return mlmd_value.bool_value
  return None

get_float_custom_property

get_float_custom_property(key: str) -> float

Gets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_float_custom_property(self, key: str) -> float:
  """Gets a custom property of float type."""
  if key not in self._artifact.custom_properties:
    return 0.0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return json_value
  return self._artifact.custom_properties[key].double_value

get_int_custom_property

get_int_custom_property(key: str) -> int

Get a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_int_custom_property(self, key: str) -> int:
  """Get a custom property of int type."""
  if key not in self._artifact.custom_properties:
    return 0
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, float):
    return int(json_value)
  return self._artifact.custom_properties[key].int_value

get_json_value_custom_property

get_json_value_custom_property(key: str) -> JsonValueType

Get a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_json_value_custom_property(self, key: str) -> JsonValueType:
  """Get a custom property of JSON type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('struct_value')):
    return None
  value = _decode_struct_value(
      self._artifact.custom_properties[key].struct_value)
  # We must cache the decoded lists or dictionaries returned here so that
  # if their recursive contents are modified, the Metadata proto message
  # can be updated to reflect this.
  if isinstance(value, (dict, list)):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_proto_custom_property

get_proto_custom_property(key: str) -> Optional[Message]

Get a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def get_proto_custom_property(self, key: str) -> Optional[message.Message]:
  """Get a custom property of proto type."""
  if key in self._cached_modifiable_custom_properties:
    return self._cached_modifiable_custom_properties[key]
  if (key not in self._artifact.custom_properties or
      not self._artifact.custom_properties[key].HasField('proto_value')):
    return None
  value = proto_utils.unpack_proto_any(
      self._artifact.custom_properties[key].proto_value)
  # We must cache the protobuf message here so that if its contents are
  # modified, the Metadata proto message can be updated to reflect this.
  if isinstance(value, message.Message):
    self._cached_modifiable_custom_properties[key] = value
  return value

get_string_custom_property

get_string_custom_property(key: str) -> str

Get a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def get_string_custom_property(self, key: str) -> str:
  """Get a custom property of string type."""
  if key not in self._artifact.custom_properties:
    return ''
  json_value = self.get_json_value_custom_property(key)
  if isinstance(json_value, str):
    return json_value
  return self._artifact.custom_properties[key].string_value

set_bool_custom_property

set_bool_custom_property(key: str, value: bool)

Sets a custom property of bool type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_bool_custom_property(self, key: str, value: bool):
  """Sets a custom property of bool type."""
  self._artifact.custom_properties[key].bool_value = value

set_float_custom_property

set_float_custom_property(key: str, value: float)

Sets a custom property of float type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_float_custom_property(self, key: str, value: float):
  """Sets a custom property of float type."""
  self._artifact.custom_properties[key].double_value = builtins.float(value)

set_int_custom_property

set_int_custom_property(key: str, value: int)

Set a custom property of int type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_int_custom_property(self, key: str, value: int):
  """Set a custom property of int type."""
  self._artifact.custom_properties[key].int_value = builtins.int(value)

set_json_value_custom_property

set_json_value_custom_property(key: str, value: JsonValueType)

Sets a custom property of JSON type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_json_value_custom_property(self, key: str, value: JsonValueType):
  """Sets a custom property of JSON type."""
  self._cached_modifiable_custom_properties[key] = value

set_mlmd_artifact

set_mlmd_artifact(artifact: Artifact)

Replace the MLMD artifact object on this artifact.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact(self, artifact: metadata_store_pb2.Artifact):
  """Replace the MLMD artifact object on this artifact."""
  if not isinstance(artifact, metadata_store_pb2.Artifact):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.Artifact, got %s '
         'instead.') % (artifact,))
  self._artifact = artifact
  self._cached_modifiable_properties = {}
  self._cached_modifiable_custom_properties = {}

set_mlmd_artifact_type

set_mlmd_artifact_type(artifact_type: ArtifactType)

Set entire ArtifactType in this object.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_mlmd_artifact_type(self,
                           artifact_type: metadata_store_pb2.ArtifactType):
  """Set entire ArtifactType in this object."""
  if not isinstance(artifact_type, metadata_store_pb2.ArtifactType):
    raise ValueError(
        ('Expected instance of metadata_store_pb2.ArtifactType, got %s '
         'instead.') % (artifact_type,))
  self._artifact_type = artifact_type
  self._artifact.type_id = artifact_type.id

set_proto_custom_property

set_proto_custom_property(key: str, value: Message)

Sets a custom property of proto type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def set_proto_custom_property(self, key: str, value: message.Message):
  """Sets a custom property of proto type."""
  self._cached_modifiable_custom_properties[key] = value

set_string_custom_property

set_string_custom_property(key: str, value: str)

Set a custom property of string type.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_in_subclasses
def set_string_custom_property(self, key: str, value: str):
  """Set a custom property of string type."""
  self._artifact.custom_properties[key].string_value = value

to_json_dict

to_json_dict() -> Dict[str, Any]

Convert from an object to a JSON serializable dictionary.

Source code in tfx/types/artifact.py

@doc_controls.do_not_doc_inheritable
def to_json_dict(self) -> Dict[str, Any]:
  return {
      'artifact':
          json.loads(
              json_format.MessageToJson(
                  message=self.mlmd_artifact,
                  preserving_proto_field_name=True)),
      'artifact_type':
          json.loads(
              json_format.MessageToJson(
                  message=self._artifact_type,
                  preserving_proto_field_name=True)),
      '__artifact_class_module__':
          self.__class__.__module__,
      '__artifact_class_name__':
          self.__class__.__name__,
  }