dml_util.core.utils

Core utilities with no external dependencies.

This module contains utility functions that are used throughout the DaggerML utilities package. These functions have minimal or no external dependencies on DaggerML itself, making them suitable for use in environments where DaggerML is not available.

Functions

tree_map

Apply a function to elements of a nested data structure that satisfy a predicate.

dict_product

Generate all combinations of dictionary values for parameter sweeps.

now

Get current time.

_run_cli

Run a command-line program and capture output.

if_read_file

Read a file if it exists.

proc_exists

Check if a process exists.

js_dump

Dump data as JSON with consistent formatting.

compute_hash

Compute hash of a file-like object.

exactly_one

Ensure exactly one of multiple parameters is provided.

Functions

batched(iterable, n, *[, strict])
compute_hash(obj[, chunk_size, hash_algorithm])
dict_product(d)	Given a dictionary of lists, yield all possible combinations of the lists.
exactly_one(**kw)
if_read_file(path)
js_dump(data, **kw)
now()
proc_exists(pid)
tree_map(predicate, fn, item)

dml_util.core.utils.batched(iterable, n, *, strict=False)

dml_util.core.utils.compute_hash(obj, chunk_size=8192, hash_algorithm='sha256')

dml_util.core.utils.dict_product(d)

Given a dictionary of lists, yield all possible combinations of the lists. Good for grid searches.

Parameters:

d (dict) – A dictionary where the keys are strings and the values are lists. The keys represent the names of the parameters, and the values are the possible values for those parameters.

Yields:

dict – A dictionary representing a single combination of parameter values. The keys are the same as the input dictionary, and the values are the corresponding values from the input lists.

Examples

>>> d = {'a': [1, 2], 'b': ['x', 'y']}
>>> for combination in dict_product(d):
...     print(combination)
{'a': 1, 'b': 'x'}
{'a': 1, 'b': 'y'}
{'a': 2, 'b': 'x'}
{'a': 2, 'b': 'y'}

dml_util.core.utils.exactly_one(**kw)

dml_util.core.utils.if_read_file(path)

dml_util.core.utils.js_dump(data, **kw)

dml_util.core.utils.now()

dml_util.core.utils.proc_exists(pid)

dml_util.core.utils.tree_map(predicate, fn, item)