dml_util.adapters.base

Base adapter class.

This module provides the base adapter class for the DaggerML utilities. Adapters are used to connect DaggerML to various execution environments, such as AWS Lambda or local runners.

Classes

AdapterBase()	Base class for DaggerML adapters.
SafeCloudWatchLogHandler(*args[, boto3_client])
VerboseArgumentParser([prog, usage, ...])

class dml_util.adapters.base.AdapterBase

Bases: object

Base class for DaggerML adapters.

This class provides a CLI interface for executing DaggerML functions iteratively, passing environment variables along. It supports different adapters for remote execution, such as AWS Lambda or local runners.

ADAPTER

The name of the adapter (to be defined in subclasses).

Type:

str

ADAPTERS

Type:

dict

ADAPTER = None

ADAPTERS = {'lambda': <class 'dml_util.adapters.lambda_.LambdaAdapter'>, 'local': <class 'dml_util.adapters.local.LocalAdapter'>}

classmethod cli(args=None)

Command-line interface for the adapter.

This method reads input data from a file or stdin, sends it to a remote service specified by the URI, and writes the response to an output file or stdout. If an error occurs, it writes the error message to an error file or stderr.

Cli Parameters

uristr

URI of the function to invoke.

–input FILE, -i FILEpath, optional (default: STDIN)

Input data file or stdin or s3 location of where to read the dump from.

–output FILE, -o FILEpath, optional (default: STDOUT)

Output location for the response data (can be a file, stdout, or s3 location).

–error FILE, -e FILEpath, optional (default: STDERR)

Error output location (can be a file, stderr, or s3 location).

–n-iters N, -n Nint, optional (default: 1)

Number of iterations to run. Set to 0 to run indefinitely.

–debugflag, optional

Enables debug logging.

returns:

Exit code: 0 on success, 1 on error.

rtype:

int

classmethod funkify(uri, data)

classmethod send_to_remote(uri, config, dump)

Send data to a remote service specified by the URI.

Parameters:

• uri (str) – The URI of the remote service.

• config (EnvConfig) – Configuration for the run, including cache path, cache key, S3 bucket, etc.

• dump (str) – The opaque blob to send to the remote service.

Returns:

A tuple containing the response data and a message. If the response is truthy, we pass it on to the caller via –output flag. The message is written to the –error flag.

Return type:

tuple[str, str]

class dml_util.adapters.base.SafeCloudWatchLogHandler(*args, boto3_client=None, **kwargs)

Bases: CloudWatchLogHandler

emit(record)

Do whatever it takes to actually log the specified logging record.

This version is intended to be implemented by subclasses and so raises a NotImplementedError.

class dml_util.adapters.base.VerboseArgumentParser(prog=None, usage=None, description=None, epilog=None, parents=[], formatter_class=<class 'argparse.HelpFormatter'>, prefix_chars='-', fromfile_prefix_chars=None, argument_default=None, conflict_handler='error', add_help=True, allow_abbrev=True, exit_on_error=True)

Bases: ArgumentParser

error(message: string)

Prints a usage message incorporating the message to stderr and exits.

If you override this in a subclass, it should not return – it should either exit or raise an exception.