API Reference¶
Mapping Functions¶
- htmap.map(func, args, map_options=None, tag=None, quiet=False)[source]¶
Map a function call over a one-dimensional iterable of arguments. The function must take exactly one positional argument and no keyword arguments.
- Parameters:
func (
Callable
) – The function to map the arguments over.args (
Iterable
[Any
]) – An iterable of arguments to pass to the mapped function.map_options (
Optional
[MapOptions
]) – An instance ofhtmap.MapOptions
.quiet (
bool
) – Do not print the map name in an interactive shell.
- Return type:
- Returns:
map – A
htmap.Map
representing the map.
- htmap.starmap(func, args=None, kwargs=None, map_options=None, tag=None, quiet=False)[source]¶
Map a function call over aligned iterables of arguments and keyword arguments. Each element of
args
andkwargs
is unpacked into the signature of the function, so their elements should be tuples and dictionaries corresponding to position and keyword arguments of the mapped function.- Parameters:
func (
Callable
) – The function to map the arguments over.args (
Optional
[Iterable
[Tuple
[Any
,...
]]]) – An iterable of tuples of positional arguments to unpack into the mapped function.kwargs (
Optional
[Iterable
[Dict
[str
,Any
]]]) – An iterable of dictionaries of keyword arguments to unpack into the mapped function.map_options (
Optional
[MapOptions
]) – An instance ofhtmap.MapOptions
.quiet (
bool
) – Do not print the map name in an interactive shell.
- Return type:
- Returns:
map – A
htmap.Map
representing the map.
- htmap.build_map(func, map_options=None, tag=None)[source]¶
Return a
MapBuilder
for the given function.- Parameters:
func (
Callable
) – The function to map over.map_options (
Optional
[MapOptions
]) – An instance ofhtmap.MapOptions
.
- Return type:
- Returns:
map_builder – A
MapBuilder
for the given function.
Map Builder¶
- class htmap.MapBuilder(func, map_options=None, tag=None)[source]¶
The
htmap.MapBuilder
provides an alternate way to create maps. Once created viahtmap.build_map()
or similar as a context manager, the map builder can be called as if it were the function you’re mapping over. When thewith
block exits, the inputs are collected and submitted as a single map.with htmap.build_map(tag="pow", func=lambda x, p: x ** p) as builder: for x in range(1, 4): builder(x, x) map = builder.map print(list(map)) # [1, 4, 27]
- __len__()[source]¶
The length of a
MapBuilder
is the number of inputs it has been sent.- Return type:
- property map: Map¶
The
Map
associated with thisMapBuilder
. Will raisehtmap.exceptions.NoMapYet
when accessed until thewith
block for thisMapBuilder
completes.
MappedFunction¶
A more convenient and flexible way to work with HTMap is to use the htmap()
decorator to build a MappedFunction
.
- htmap.mapped(map_options=None)[source]¶
A decorator that wraps a function in an
MappedFunction
, which provides an interface for mapping functions calls out to an HTCondor cluster.- Parameters:
map_options (
Optional
[MapOptions
]) – An instance ofhtmap.MapOptions
. Any map calls from theMappedFunction
produced by this decorator will inherit from this.- Return type:
- Returns:
mapped_function – A
MappedFunction
that wraps the function (or a wrapper function that does the wrapping).
- class htmap.MappedFunction(func, map_options=None)[source]¶
- Parameters:
func (
Callable
) – A function to wrap in aMappedFunction
.map_options (
Optional
[MapOptions
]) – An instance ofhtmap.MapOptions
. Any map calls from theMappedFunction
produced by this decorator will inherit from this.
- map(args, tag=None, map_options=None)[source]¶
As
htmap.map()
, but thefunc
argument is the mapped function.- Return type:
- starmap(args=None, kwargs=None, tag=None, map_options=None)[source]¶
As
htmap.starmap()
, but thefunc
argument is the mapped function.- Return type:
- build_map(tag=None, map_options=None)[source]¶
As
htmap.build_map()
, but thefunc
argument is the mapped function.- Return type:
Map¶
The Map
is your window into the status and output of your map.
Once you get a map result back from a map call,
you can use its methods to get the status of jobs,
change the properties of the map while its running,
pause, restart, or cancel the map,
and finally retrieve the output once the map is done.
The various methods that allow you to get and iterate over components will raise exceptions if something has gone wrong with your map:
htmap.exceptions.MapComponentError
if a component experienced an error while executing.htmap.exceptions.MapComponentHeld
if a component was held by HTCondor, likely because an input file did not exist or the component used too much memory or disk.
The exception message will contain information about what caused the error. See Error Handling for more details on error handling.
- class htmap.Map(*, tag, map_dir)[source]¶
Represents the results from a map call.
Warning
You should never instantiate a
Map
directly! Instead, you’ll get yourMap
by calling a top-level mapping function likehtmap.map()
, aMappedFunction
mapping method, or by usinghtmap.load()
. We are not responsible for whatever vile contraption you build if you bypass the correct methods!- __getitem__(item)[source]¶
Return the output associated with the input index. Does not block.
- Return type:
- classmethod load(tag)[source]¶
Load a
Map
by looking up itstag
.Raises
htmap.exceptions.TagNotFound
if thetag
does not exist.
- property components: Tuple[int, ...]¶
Return a tuple containing the component indices for the
htmap.Map
.
- wait(timeout=None, show_progress_bar=False, holds_ok=False, errors_ok=False)[source]¶
Wait until all output associated with this
Map
is available.If any components in the map are held or experience an execution error, this method will raise an exception (
htmap.exceptions.MapComponentHeld
orhtmap.exceptions.MapComponentError
, respectively).- Parameters:
timeout (
Union
[int
,float
,timedelta
,None
]) – How long to wait for the map to complete before raising ahtmap.exceptions.TimeoutError
. IfNone
, wait forever.show_progress_bar (
bool
) – IfTrue
, a progress bar will be displayed.holds_ok (
bool
) – IfTrue
, will not raise exceptions if components are held.errors_ok (
bool
) – IfTrue
, will not raise exceptions if components experience execution errors.
- Return type:
- get(component, timeout=None)[source]¶
Return the output associated with the input component index. If the component experienced an execution error, this will raise
htmap.exceptions.MapComponentError
. Useget_err()
,errors()
,error_reports()
to see what went wrong!
- get_err(component, timeout=None)[source]¶
Return the error associated with the input component index. If the component actually succeeded, this will raise
htmap.exceptions.ExpectedError
.- Parameters:
- Return type:
- iter(timeout=None)[source]¶
Returns an iterator over the output of the
htmap.Map
in the same order as the inputs, waiting on each individual output to become available.
- iter_with_inputs(timeout=None)[source]¶
Returns an iterator over the inputs and output of the
htmap.Map
in the same order as the inputs, waiting on each individual output to become available.
- iter_as_available(timeout=None)[source]¶
Returns an iterator over the output of the
htmap.Map
, yielding individual outputs as they become available.The iteration order is initially random, but is consistent within a single interpreter session once the map is completed.
- iter_as_available_with_inputs(timeout=None)[source]¶
Returns an iterator over the inputs and output of the
htmap.Map
, yielding individual(input, output)
pairs as they become available.The iteration order is initially random, but is consistent within a single interpreter session once the map is completed.
- property component_statuses: List[ComponentStatus]¶
Return the current
state.ComponentStatus
of each component in the map.
- components_by_status()[source]¶
Return the component indices grouped by their states.
- Return type:
Examples
This example finds the completed jobs for a submitted map, and processes those results:
from time import sleep import htmap def job(x): sleep(x) return 1 / x m = htmap.map(job, [0, 2, 4, 6, 8], tag="foo") # Wait for all jobs to finish. # Alternatively, use `futures = htmap.load("foo")` on a different process sleep(10) completed = m.components_by_status()[htmap.JobStatus.COMPLETED] for component in completed: result = m.get(future) # Whatever processing needs to be done print(result) # prints "2", "4", "6", and "8"
- property holds: Dict[int, ComponentHold]¶
A dictionary of component indices to their
Hold
(if they are held).
- hold_report()[source]¶
Return a string containing a formatted table describing any held components.
- Return type:
- property errors: Dict[int, ComponentError]¶
A dictionary of component indices to their
ExecutionError
(if that component experienced an error).
- error_reports()[source]¶
Yields the error reports for any components that experienced an error during execution.
- property memory_usage: List[int]¶
Return the latest peak memory usage of each map component, measured in MB. A component that hasn’t reported yet will show a
0
.Warning
Due to current limitations in HTCondor, memory use for very short-lived components (<5 seconds) will not be accurate.
- remove(force=False)[source]¶
This command removes a map from the Condor queue. Functionally, this command aborts a job.
This function will completely remove a map from the Condor queue regardless of job state (running, executing, waiting, etc). All data associated with a removed map is permanently deleted.
- property exists: bool¶
True
if and only if the map has not been successfully removed. Otherwise,False
.
- hold()[source]¶
This command holds a map. The components of the map will not be allowed to run until released (see
Map.release()
).HTCondor may itself hold your map components if it detects that something has gone wrong with them. Resolve the underlying problem, then use the
Map.release()
command to allow the components to run again.- Return type:
- release()[source]¶
This command releases a map, undoing holds (see
Map.hold()
). The held components of a released map will become idle again.HTCondor may itself hold your map components if it detects that something has gone wrong with them. Resolve the underlying problem, then use this command to allow the components to run again.
- Return type:
- pause()[source]¶
This command pauses a map. The running components of a paused map will keep their resource claims, but will stop actively executing. The map can be un-paused by resuming it (see the
Map.resume()
command).- Return type:
- resume()[source]¶
This command resumes a map (reverses the
Map.pause()
command). The running components of a resumed map will resume execution on their claimed resources.- Return type:
- vacate()[source]¶
This command vacates a map. The running components of a vacated map will give up their claimed resources and become idle again.
Checkpointing maps will still have access to their last checkpoint, and will resume from it as if execution was interrupted for any other reason.
- Return type:
- set_memory(memory)[source]¶
Change the amount of memory (RAM) each map component needs.
Warning
Edits do not affect components that are currently running. To “restart” components so that they see the new attribute value, consider vacating their map (see the vacate command).
- set_disk(disk)[source]¶
Change the amount of disk space each map component needs.
Warning
Edits do not affect components that are currently running. To “restart” components so that they see the new attribute value, consider vacating their map (see the vacate command).
- rerun(components=None)[source]¶
Re-run (part of) the map from scratch. The selected components must be completed or errored.
Any existing output of re-run components is removed; they are re-submitted to the HTCondor queue with their original map options (i.e., without any subsequent edits).
- retag(tag)[source]¶
Give this map a new
tag
. The oldtag
will be available for re-use immediately.Retagging a map makes it not transient. Maps that have never had an explicit tag given to them are transient and can be easily cleaned up via the clean command.
- property stdout: MapStdOut¶
A sequence containing the
stdout
for each map component. You can index into it (with a component index) to get thestdout
for that component, or iterate over the sequence to get all of thestdout
from the map.
- property stderr: MapStdErr¶
A sequence containing the
stderr
for each map component. You can index into it (with a component index) to get thestderr
for that component, or iterate over the sequence to get all of thestderr
from the map.
- property output_files: MapOutputFiles¶
A sequence containing the path to the directory containing the output files for each map component. You can index into it (with a component index) to get the path for that component, or iterate over the sequence to get all of the paths from the map.
- count(value) integer -- return number of occurrences of value ¶
- index(value[, start[, stop]]) integer -- return first index of value. ¶
Raises ValueError if the value is not present.
Supporting start and stop arguments is optional, but recommended.
- class htmap.ComponentStatus(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]¶
An enumeration of the possible statuses that a map component can be in. These are mostly identical to the HTCondor job statuses of the same name.
- UNKNOWN = 'UNKNOWN'¶
- UNMATERIALIZED = 'UNMATERIALIZED'¶
- IDLE = 'IDLE'¶
- RUNNING = 'RUNNING'¶
- REMOVED = 'REMOVED'¶
- COMPLETED = 'COMPLETED'¶
- HELD = 'HELD'¶
- SUSPENDED = 'SUSPENDED'¶
- ERRORED = 'ERRORED'¶
- class htmap.MapStdOut(map)[source]¶
An object that helps implement a map’s sequence over its
stdout
. Don’t both instantiating one yourself: use theMap.stdout
attribute instead.- get(component, timeout=None)¶
Return a string containing the stdout/stderr from a single map component.
- class htmap.MapStdErr(map)[source]¶
An object that helps implement a map’s sequence over its
stderr
. Don’t both instantiating one yourself: use theMap.stderr
attribute instead.- get(component, timeout=None)¶
Return a string containing the stdout/stderr from a single map component.
- class htmap.MapOutputFiles(map)[source]¶
An object that helps implement a map’s sequence over its output file directories. Don’t both instantiating one yourself: use the
Map.output_files
attribute instead.- get(component, timeout=None)[source]¶
Return the
pathlib.Path
to the directory containing the output files for the given component.- Parameters:
- Return type:
- Returns:
path – The path to the directory containing the output files for the given component.
Error Handling¶
Map components can generally encounter two kinds of errors:
An exception occurred inside your function on the execute node.
HTCondor was unable to run the map component for some reason.
The first kind will result in HTMap transporting a htmap.ComponentError
back to you,
which you can access via htmap.Map.get_err()
.
The htmap.ComponentError.report()
method returns a formatted error report for your perusal.
htmap.Map.error_reports()
is a shortcut that returns all of the error reports for all of the components of your map.
If you want to access the error programmatically, you can grab it using htmap.get_err()
.
The second kind of error doesn’t provide as much information.
The method htmap.Map.holds()
will give you a dictionary mapping components to their htmap.ComponentHold
, if they have one.
htmap.Map.hold_report()
will return a formatted table showing any holds in your map.
The hold’s reason
attribute will tell you a lot about what HTCondor doesn’t like about your component.
- class htmap.ComponentError(*, map, component, exception_msg, node_info, python_info, scratch_dir_contents, stack_summary)[source]¶
Represents an error experienced by a map component during remote execution.
- node_info¶
A tuple containing information about the HTCondor execute node the component ran on.
- Type:
- python_info¶
A tuple containing information about the Python installation on the execute node.
- Type:
- scratch_dir_contents¶
A list of paths in the scratch directory on the execute node.
- Type:
List[pathlib.Path]
- stack_summary¶
The Python stack frames at the time of execution, excluding HTMap’s own stack frame.
- Type:
MapOptions¶
Map options are the equivalent of HTCondor’s submit descriptors. All HTCondor submit descriptors are valid map options except those reserved by HTMap for internal use (see below).
Fixed options are the most basic option. The entire map will used the fixed option. If you pass a single string as the value of a map option, it will become a fixed option.
Variadic options are options that are given individually to each component of a map.
For example, each component of a map might need a different amount of memory.
In that case you could pass a list to request_memory
, with the same number of elements as the number of inputs to the map.
Inherited options are given to a htmap.MappedFunction
when it is created.
Any maps made using that function can inherit these options.
Options that are passed in the actual map call override inherited options (excepting fixed_input_files
, see the note).
For example, if you know that a certain function always takes a large amount of memory, you could give it a large request_memory
at the htmap.MappedFunction
level so that you don’t have to do it for every individual map.
Additionally, default map options can be set globally via settings['MAP_OPTIONS.<option_name>'] = <option_value>
.
Warning
Only certain options make sense as inherited options. For example, they shouldn’t be variadic options.
fixed_input_files
has special behavior as an inherited option: they are merged together instead of overridden.
Note
When looking at examples of raw HTCondor submit files, you may see submit descriptors that are prefixed with a +
or a MY.
.
Those options should be passed to htmap.MapOptions
via the custom_options
keyword arguments.
- class htmap.MapOptions(*, fixed_input_files=None, input_files=None, output_remaps=None, custom_options=None, **kwargs)[source]¶
- Parameters:
fixed_input_files (
Union
[PathLike
,TransferPath
,Iterable
[Union
[PathLike
,TransferPath
]],None
]) – A single file, or an iterable of files, to send to all components of the map.input_files (
Union
[Iterable
[Union
[PathLike
,TransferPath
]],Iterable
[Iterable
[Union
[PathLike
,TransferPath
]]],None
]) – An iterable of single files or iterables of files to map over. This may be useful if you want additional files to be sent to each map component, but don’t want them in your mapped function’s arguments.output_remaps (
Union
[Mapping
[str
,TransferPath
],Iterable
[Mapping
[str
,TransferPath
]],None
]) – A dictionary, or an iterable of dictionaries, specifying output transfer remaps. A remapped output file is sent to a specified destination instead of back to the submit machine. If a single dictionary is passed, it will be applied to every map component (in this case, you may want to use the$(component)
submit macro to differentiate them). Each dictionary should be a “mapping” between the names (last path component, as a string) of o utput files and their destinations, given as aTransferPath
. You must still calltransfer_output_files()
on the files for the them to be transferred at all; listing them here only sets up the remapping.custom_options (
Optional
[Dict
[str
,str
]]) – A dictionary of submit descriptors that are not built-in HTCondor descriptors. These are the descriptors that, if you were writing a submit file, would have a leading+
orMY.
. The leading characters are unnecessary here, but can be included if you’d like.kwargs (
Union
[str
,Iterable
[str
]]) – Additional keyword arguments are interpreted as HTCondor submit descriptors. Values that are single strings are used for all components of the map. Providing an iterable for the value will map that option. Certain keywords are reserved for internal use (see the RESERVED_KEYS class attribute).
Notes
Warning
The representation of the values in
fixed_input_files
,input_files
,custom_options
andkwargs
should exactly match the characters in the submit file after the=
.For example, let’s say your job requires this submit file:
# file: job.submit foo = "bar" aaa = xyz bbb = false ccc = 1
The
MapOptions
that express the same submit options would be:>>> options = {"foo": '"bar"', "aaa": "xyz", "bbb": "false", "ccc": "1"} >>> print(options["foo"]) # exactly matches the value in the submit file ... "bar" >>> options["foo"] = "\"bar\"" # alternative value >>> MapOptions(**options)
Submit file values with quotes require escaped quotes in the Python string.
- RESERVED_KEYS = {'+IsHTMapJob', '+component', 'IsHTMapJob', 'MY.IsHTMapJob', 'MY.component', 'arguments', 'component', 'executable', 'jobbatchname', 'log', 'should_transfer_files', 'stderr', 'stdout', 'submit_event_notes', 'transfer_executable', 'transfer_input_files', 'transfer_output_files', 'transfer_output_remaps', 'universe', 'when_to_transfer_output'}¶
- classmethod merge(*others)[source]¶
Merge any number of
MapOptions
together, like acollections.ChainMap
. Options closer to the left take priority. :rtype:MapOptions
Note
fixed_input_files
is a special case, and is merged up the chain instead of being overwritten.requirements
are also combined, in a way where all requirements must be satisfied.
File Transfer¶
- class htmap.TransferPath(path, protocol=None, location=None)[source]¶
A
TransferPath
describes the location of a file or directory. If theprotocol
andlocation
are bothNone
, it describes a location on the local filesystem. If either are given, it describes a remote location.When used as an argument to a mapped function, a
TransferPath
tells HTMap to arrange for the specified files/directories to be transferred to the execute machine from some location, which may be the local filesystem on the submit machine or some remote location like an HTTP address or an S3 server.Transfer paths are recognized in mapped function inputs as long as they are either:
Arguments or keyword arguments of the mapped function.
Stored inside a primitive container (tuple, list, set, dictionary value) that is an argument or keyword argument of the mapped function. Nested containers are inspected recursively.
When the mapped function runs execute-side, it will receive (instead of this object) a normal
pathlib.Path
object pointing to the execute-side path of the file/directory.TransferPath
is also used to specify the locations for output files to be sent, if they are not to be returned to the submit machine. For example, output files could be sent to an S3 server. See theoutput_remaps
argument ofMapOptions
for more details on “remapped” output file transfer.Where appropriate,
TransferPath
has the same interface as apathlib.Path
. See the examples for some ways to leverage this API to efficiently construct transfer paths.Attention
You may need to pass additional submit descriptors to your map to actually be able to use input/output transfers for certain protocols. For example, to transfer to and from an S3 server, you also need to pass
aws_access_key_id_file
andaws_secret_access_key_file
. See the condor_submit documentation for more details.Examples
Transfer a file stored in your home directory using HTCondor file transfer:
transfer_path = htmap.TransferPath.cwd() / 'file.txt'
Transfer a local file at an absolute path using HTCondor file transfer:
transfer_path = htmap.TransferPath("/foo/bar/baz.txt")
Get a file from an HTTP server, located at
http://htmap.readthedocs.io/en/latest/_static/htmap-logo.svg
:transfer_path = htmap.TransferPath( path = "en/latest/_static/htmap-logo.svg", protocol = "http", location = "htmap.readthedocs.io", )
or
base_path = htmap.TransferPath( path = "/", protocol = "http", location = "htmap.readthedocs.io", ) transfer_path = base_path / 'en' / 'latest' / '_static' / 'htmap-logo.svg'
- Parameters:
path (
Union
[TransferPath
,PathLike
]) – The path to the file or directory to transfer.protocol (
Optional
[str
]) – The protocol to perform for the transfer with. If set toNone
(the default), use HTCondor local file transfer.location (
Optional
[str
]) – The location to find a remote file when using a protocol transfer. This could be the address of a server, for example.
- htmap.transfer_output_files(*paths)[source]¶
Informs HTMap about the existence of output files.
Attention
This function is a no-op when executing locally, so you if you’re testing your function it won’t do anything.
Attention
The files will be moved by this function, so they will not be available in their original locations.
Checkpointing¶
- htmap.checkpoint(*paths)[source]¶
Informs HTMap about the existence of checkpoint files. This function should be called every time the checkpoint files are changed, even if they have the same names as before.
Attention
This function is a no-op when executing locally (i.e., not execute-side), so you if you’re testing your function locally it won’t do anything.
Attention
The files will be copied by this function, so try not to make the checkpoint files too large.
Management¶
These functions help you manage your maps.
- htmap.status(maps=None, include_state=True, include_meta=True)[source]¶
Return a formatted table containing information on the given maps.
- Parameters:
maps (
Optional
[Iterable
[Map
]]) – The maps to display information on. IfNone
, displays information on all existing maps.include_state (
bool
) – IfTrue
, include information on the state of the map’s components.include_meta (
bool
) – IfTrue
, include information about the map’s memory usage, disk usage, and runtime.
- Return type:
- Returns:
table – A text table containing information on the given maps.
- htmap.get_tags(pattern=None)[source]¶
Return a tuple containing the
tag
for all existing maps, with optional filtering based on a glob-style pattern.
- htmap.load_maps(pattern=None)[source]¶
Return a
tuple
containing theMap
for all existing maps, with optional filtering based on a glob-style pattern.
- htmap.clean(*, all=False)[source]¶
Clean up transient maps by removing them.
Maps that have never had a tag explicitly set are assigned randomized tags and marked as “transient”. This command removes maps marked transient (and can also remove all maps, not just transient ones, if the –all option is passed).
Programmatic Status Messages¶
These functions are useful for generating machine-readable status information.
- htmap.status_json(maps=None, include_state=True, include_meta=True, compact=False)[source]¶
Return a JSON-formatted string containing information on the given maps.
Disk and memory usage are reported in bytes. Runtimes are reported in seconds.
- Parameters:
maps (
Optional
[Iterable
[Map
]]) – The maps to display information on. IfNone
, displays information on all existing maps.include_state (
bool
) – IfTrue
, include information on the state of the map’s components.include_meta (
bool
) – IfTrue
, include information about the map’s memory usage, disk usage, and runtime.compact (
bool
) – IfTrue
, the JSON will be formatted in the most compact possible representation.
- Return type:
- Returns:
json – A JSON-formatted dictionary containing information on the given maps.
- htmap.status_csv(maps=None, include_state=True, include_meta=True)[source]¶
Return a CSV-formatted string containing information on the given maps.
Disk and memory usage are reported in bytes. Runtimes are reported in seconds.
- Parameters:
maps (
Optional
[Iterable
[Map
]]) – The maps to display information on. IfNone
, displays information on all existing maps.include_state (
bool
) – IfTrue
, include information on the state of the map’s components.include_meta (
bool
) – IfTrue
, include information about the map’s memory usage, disk usage, and runtime.
- Return type:
- Returns:
csv – A CSV-formatted table containing information on the given maps.
Delivery Methods¶
- htmap.register_delivery_method(name, descriptors_func, setup_func=None)[source]¶
Register a new delivery method with HTMap.
- Parameters:
name (
str
) – The name of the delivery method; this is what theDELIVERY_METHOD
should be set to to use this delivery method.descriptors_func (
Callable
[[str
,Path
],dict
]) – The function that provides the HTCondor submit descriptors for this delivery method.setup_func (
Optional
[Callable
[[str
,Path
],None
]]) – The function that does any setup necessary to running the map.
- Return type:
Transplant Installs¶
These functions help you manage your transplant installs.
- class htmap.Transplant(hash: str, path: Path, created: datetime, size: int, packages: Tuple[str, ...])[source]¶
An object that represents metadata information about a transplant install.
Create new instance of Transplant(hash, path, created, size, packages)
- classmethod load(path)[source]¶
- Parameters:
path (
Path
) – The path to the transplant install.- Return type:
- Returns:
transplant – The
Transplant
that represents the transplant install.
Settings¶
HTMap exposes configurable settings through htmap.settings
,
which is an instance of the class htmap.settings.Settings
.
This settings object manages a lookup chain of dictionaries.
The settings object created during startup contains two dictionaries.
The lowest level contains HTMap’s default settings, and the next level up is
constructed from a settings file at ~/.htmaprc
.
If that file does not exist, an empty dictionary is used instead.
The file should be formatted in TOML.
Alternate settings could be stored in other files or constructed at runtime. HTMap provides tools for saving, loading, merging, prepending, and appending settings to each other. Each map is search in order, so earlier settings can flexibly override later settings.
Warning
To entirely replace your settings, do not do
htmap.settings = <new settings object>
.
Instead, use the htmap.settings.Settings.replace()
method.
Replacing the settings by assignment breaks the internal linking between the
settings objects and its dependencies.
Hint
These may be helpful when constructing fresh settings:
HTMap’s base settings are available as
htmap.BASE_SETTINGS
.The settings loaded from
~/.htmaprc
are available ashtmap.USER_SETTINGS
.
- class htmap.settings.Settings(*settings)[source]¶
-
- to_dict()[source]¶
Return a single dictionary with all of the settings in this
Settings
, merged according to the lookup rules.- Return type:
- replace(other)[source]¶
Change the settings of this
Settings
to be the settings from anotherSettings
.- Return type:
- append(other)[source]¶
Add a map to the end of the search (i.e., it will be searched last, and be overridden by anything before it).
- prepend(other)[source]¶
Add a map to the beginning of the search (i.e., it will be searched first, and override anything after it).
- classmethod from_settings(*settings)[source]¶
Construct a new
Settings
from anotherSettings
.- Return type:
Logging¶
HTMap exposes a standard Python logging hierarchy under the logger named 'htmap'
.
Logging configuration can be done by any of the methods described in the documentation.
Here’s an example of how to set up basic console logging:
import logging
import sys
logger = logging.getLogger("htmap")
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler(stream=sys.stdout)
handler.setLevel(logging.DEBUG)
handler.setFormatter(
logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
)
logger.addHandler(handler)
After executing this code, you should be able to see HTMap log messages as you tell it to do things.
Warning
The HTMap logger is not available in the context of the executing map function. Trying to use it will probably raise exceptions.
Exceptions¶
- exception htmap.exceptions.TimeoutError[source]¶
An operation has timed out because it took too long.
- exception htmap.exceptions.NoMapYet[source]¶
The
htmap.MapBuilder
does not have an associatedhtmap.Map
yet because it is still inside thewith
block.
- exception htmap.exceptions.TagAlreadyExists[source]¶
The requested
tag
already exists (recover theMap
, then either use or delete it).
- exception htmap.exceptions.ReservedOptionKeyword[source]¶
The map option keyword you tried to use is reserved by HTMap for internal use.
- exception htmap.exceptions.MisalignedInputData[source]¶
There is some kind of mismatch between the lengths of the function arguments and the variadic map options.
- exception htmap.exceptions.UnknownPythonDeliveryMethod[source]¶
The specified Python delivery method has not been registered.
- exception htmap.exceptions.MapWasRemoved[source]¶
This map has been removed, and can no longer be interacted with.
- exception htmap.exceptions.InvalidOutputStatus[source]¶
The output status of the map component was not recognized.
- exception htmap.exceptions.MapComponentError[source]¶
A map component experienced an error during remote execution.
- exception htmap.exceptions.ExpectedError[source]¶
A map component that contained an OK result was unpacked as if it contained an error.
- exception htmap.exceptions.CannotTransplantPython[source]¶
The Python interpreter you are using cannot be transplanted.
- exception htmap.exceptions.CannotRerunComponents[source]¶
The given components cannot be rerun because they are currently active.