Sequences and iterators

Rolling averages

class satella.coding.sequences.RollingArithmeticAverage(n: int = 100)

A class to implement a rolling arithmetic average over n last entries

Parameters:n – amount of last entries to count
avg() → float

Compute current average

Returns:current average
Raises:ZeroDivisionError – the average buffer is empty
clear() → None

Clear the rolling average buffer

insert(x: float) → None

Add a value to the rolling average, discarding the previous entry if the buffer size is exceeded

Parameters:x – sample to insert

Standard routines


class satella.coding.sequences.IteratorListAdapter(iterator: Iterator[T_co])

A wrapper around an iterator that enables it to be processed as a list.

Ie. the generator will now support __contains__, __len__ and __getitem__. If a call to such a method is done, the generator will be unfolded in memory so this might take a ton of memory! You’ve been warned!

Deprecated since version 2.15.7: Use :class:`~satella.coding.sequences.ListWrapperIterator ` instead

Parameters:iterator – iterator to unfold



Convert function to an iterator. You can replace the following code:

>>> def iterator(x):
>>>     for y in x:
>>>         yield fun(y)


>>> @to_iterator
>>> def fun(y):
>>>     ...

and now call fun instead of iterator. fun will accept a single argument - the iterable, and assume that the function you decorate also takes a single argument - the item


satella.coding.sequences.map_list(fun: Callable, iterable: Union[Iterator[T], Iterable[T]]) → List[T]

A syntactic sugar for

>>> list(map(fun, iterable))
  • fun – function to apply
  • iterable – iterable to iterate over


satella.coding.sequences.unique(lst: Union[Iterator[T], Iterable[T]]) → Iterator[T]

Return each element from lst, but return every element only once.

Take care for elements of T to be __eq__-able and hashable!

This will keep internally a set of elements encountered, and skip them if same element appears twice

Parameters:lst – iterable to process
Returns:a generator yielding unique items from lst


satella.coding.sequences.iterate_callable(clbl: Callable[[int], V], start_from: int = 0, exc_classes=(<class 'IndexError'>, <class 'ValueError'>)) → Iterator[V]

Given a callable that accepts an integer and returns the n-th entry, iterate over it until it starts to throw some exception.

  • clbl – callable to call
  • start_from – number to start from
  • exc_classes – exceptions that being thrown show that the list was exhausted

an iterator


To return the first single element that returns true on given callable, use the following function:

satella.coding.sequences.choose(filter_fun: Callable[[T], bool], iterable: Union[Iterator[T], Iterable[T]], check_multiple: bool = False) → T

Return a single value that exists in given iterable.

Essentially the same as:

>>> next(iter(filter(filter_fun, iterable)))

but raises a different exception if nothing matches (and if there are multiple matches and check_multiple is True). If check_multiple is True this guarantees to exhaust the generator (if passed).

  • filter_fun – function that returns bool on the single value
  • iterable – iterable to examine
  • check_multiple – if True, this will check if there are multiple entries matching filter_fun, and will raise ValueError if so. If True, this will exhaust the iterator. If left at default, False, this may not exhaust the iterator.

single element in the iterable that matches given input


ValueError – on multiple elements matching (if check_multiple), or none at all


Does the same thing as choose, but exhausts the generator and checks if there are no multiple elements matching the callable. If there are, raises ValueError.

satella.coding.sequences.choose_one(filter_fun: Callable[[T], bool], iterable: Union[Iterator[T], Iterable[T]]) → T

Syntactic sugar for

>>> choose(filter_fun, iterable, check_multiple=True)

This exhausts the iterable.

  • filter_fun – function that returns bool on the single value
  • iterable – iterable to examine

single element in the iterable that matches given input


ValueError – on multiple elements matching, or none at all


class satella.coding.sequences.AlreadySeen

Class to filter out unique objects. Objects must be hashable, barring that they must be eq-able, however passing it an non-hashable object will result in O(n^2) complexity, as the class uses a list to keep track of the objects.


>>> als = AlreadySeen()
>>> for elem in sequence:
>>>     if als.is_unique(elem):
>>>         ... process the element ...
is_unique(key: K) → bool

Has the element been spotted first time?

Add it to the set.

Parameters:key – element to check
Returns:whether the element was seen for the first time


satella.coding.sequences.filter_out_nones(y: Sequence[T]) → List[T]

Return all elements, as a list, that are not None

Parameters:y – a sequence of items
Returns:a list of all subelements, in order, that are not None


satella.coding.sequences.index_of(predicate: Callable[[T], bool], seq: Sequence[T]) → int

Return an index of first met element that calling predicate on it returns True

  • predicate – predicate to apply
  • seq – sequence to examine

index of the element


ValueError – if no element found


satella.coding.sequences.index_of_max(seq: Sequence[T]) → int

Return the index of the maximum element

Parameters:seq – sequence to examine
Returns:index of the maximum element
Raises:ValueError – sequence was empty


satella.coding.sequences.f_range(*args) → Iterator[float]

A range() that supports float.

Note that this behaves correctly when given a negative step.

Call either:

>>> f_range(stop)   # will start from 0 and step 1
>>> f_range(start, stop)    # will start from start and continue until the result is gte stop
>>> # will start from start and continue by step until the result is gte stop
>>> f_range(start, stop, step)
Raises:TypeError – invalid number of arguments


satella.coding.sequences.filter_out_false(y: Sequence[T]) → List[T]

Return all elements, as a list, that are True

Parameters:y – a sequence of items
Returns:a list of all subelements, in order, that are not None


satella.coding.sequences.try_close(iterator: Iterator[T_co]) → None

Try to invoke close() on an iterator. Do nothing if provided iterator doesn’t have a .close() method.

Parameters:iterator – iterator to close


satella.coding.sequences.n_th(iterator: Union[Iterator[T], Iterable[T]], n: int = 0) → T

Obtain n-th element (counting from 0) of an iterable

  • iterator – iterable to process
  • n – element to return. Note that we’re counting from 0

IndexError – iterable was too short


satella.coding.sequences.append_sequence(seq: Iterator[tuple], *elems_to_append) → Iterator[tuple]

Return an iterator which append elem_to_append to every tuple in seq.


>>> a = [(1, ), (2, ), (3, )]
>>> assert list(append_sequence(a, 1, 2)) == [(1, 1, 2), (2, 1, 2), (3, 1, 2)]

If every element of seq is not a tuple, it will be cast to one.

  • seq – sequence to append
  • elems_to_append – element(s) to append

an iterator


For the rare moments, when you wish you could just do:

iterator: tp.Iterator[T] = iterator
n_elements: tp.List[T] = iterator[:n]

But it doesn’t let you do this, because iterator is not subscriptable. However, this function comes to the rescue:

satella.coding.sequences.take_n(iterator: Union[Iterator[T], Iterable[T]], n: int, skip: int = 0) → List[T]

Take (first) n elements of an iterator, or the entire iterator, whichever comes first

  • iterator – iterator to take from
  • n – amount of elements to take
  • skip – elements from the start to skip

list of p_len n (or shorter)


satella.coding.sequences.infinite_iterator(returns: Optional[T] = None, return_factory: Optional[Callable[[], T]] = None) → Iterator[T]

Return an infinite number of objects.

  • returns – object to return. Note that this will be this very object, it will not be copied.
  • return_factory – a callable that takes 0 args and returns an element to return.

an infinite iterator of provided values


A factory for filter functions that check if given object is an instance of something (or multiple classes, if passed a tuple of classes). Use like that

orders: tp.List[BaseOrder] = ...
read_orders = filter(is_instance(ReadOrder), orders)
satella.coding.sequences.is_instance(classes: Union[Tuple[type, ...], type]) → Callable[[object], bool]


satella.coding.sequences.is_last(lst: Union[Iterator[T], Iterable[T]]) → Iterator[Tuple[bool, T]]

Return every element of the list, alongside a flag telling is this the last element.

Use like:

>>> for is_last, element in is_last(my_list):
>>>     if is_last:
>>>         ...
Parameters:lst – list to iterate thru
Returns:a p_gen returning (bool, T)

Note that this returns a nice, O(1) iterator.


satella.coding.sequences.enumerate2(iterable: Union[Iterator[T], Iterable[T]], start: int = 0, step: int = 1) → Iterator[Tuple[int, T]]

Enumerate with a custom step

  • iterable – iterable to enumerate
  • start – value to start at
  • step – step to add during each iteration


satella.coding.sequences.smart_enumerate(iterator: Union[Iterator[T], Iterable[T]], start: int = 0, step: int = 1) → Iterator[Tuple]

An enumerate that talks pretty with lists of tuples. Consider

>>> a = [(1, 2), (3, 4), (5, 6)]
>>> for i, b in enumerate(a):
>>>     c, d = b
>>>     ...

This function allows you just to write: >>> for i, c, d in enumerate(a): >>> …

Note that elements in your iterable must be either a list of a tuple for that to work, or need to be able to be coerced to a tuple. Otherwise, TypeError will be thrown.

  • iterator – iterator to enumerate
  • start – value to start counting at
  • step – step to advance the enumeration with

TypeError – could not coerce the elements in your iterable to a tuple


satella.coding.sequences.smart_zip(*iterators) → Iterator[Tuple[T, ...]]

Zip in such a way that resulted tuples are automatically expanded.


>>> b = list(smart_zip([(1, 1), (1, 2)], [1, 2]))
>>> assert b == [(1, 1, 1), (1, 2, 2)]

Note that an element of the zipped iterator must be a tuple (ie. isinstance tuple) in order for it to be appended to resulting iterator element!

Parameters:iterators – list of iterators to zip together
Returns:an iterator zipping the arguments in a smart way


Sometimes you need to iterate through list and take also the next element.

satella.coding.sequences.add_next(lst: Union[Iterator[T], Iterable[T]], wrap_over: bool = False, skip_last: bool = False) → Iterator[Tuple[T, Optional[T]]]

Yields a 2-tuple of given iterable, presenting the next element as second element of the tuple.

The last element will be the last element alongside with a None, if wrap_over is False, or the first element if wrap_over was True


>>> list(add_next([1, 2, 3, 4, 5])) == [(1, 2), (2, 3), (3, 4), (4, 5), (5, None)]
>>> list(add_next([1, 2, 3, 4, 5], True)) == [(1, 2), (2, 3), (3, 4), (4, 5), (5, 1)]
  • lst – iterable to iterate over
  • wrap_over – whether to attach the first element to the pair of the last element instead of None
  • skip_last – if this is True, then last element, alongside with a None, won’t be output


Sometimes you need just a half of your Cartesian product, for example for operations that are commutative (eg. checking for collisions, if object A collides with B then B collides with A).

It helps you save time during computationally intensive operations.

This routine will return a iterator of tuple containing two elements from the same set (ie. it will do something like a cartesian power of two).


satella.coding.sequences.skip_first(iterator: Union[Iterator[T], Iterable[T]], n: int) → Iterator[T]

Skip first n elements from given iterator.

Returned iterator may be empty, if source iterator is shorter or equal to n.

Deprecated since version 2.14.22: Use itertools.islice instead


satella.coding.sequences.zip_shifted(*args) → Iterator[Tuple[T, ...]]

Construct an iterator, just like zip but first by cycling it’s elements by it’s shift factor. Elements will be shifted by a certain factor, this means that they will appear earlier.


>>> zip_shifted(([1, 2, 3, 4], 1), ([1, 2, 3, 4], 0)) == [(2, 1), (3, 2), (4, 3), (1, 4)]

This will work on arbitrary iterators and iterables.

Shift can be negative, in which case the last elements will appear sooner, eg.

>>> zip_shifted(([1, 2, 3, 4], -1), ([1, 2, 3, 4], 0)) == [(4, 1), (1, 2), (2, 3), (3, 4)]

Same memory considerations as shift() apply.

The resulting iterator will be as long as the shortest sequence.

Deprecated since version 2.14.22: Use zip(shift(…)) instead

Parameters:args – a tuple with the iterator/iterable and amount of shift. If a non-tuple is given, it is assumed that the shift is zero.

This is deprecated. Use zip(shift(...)) instead.


satella.coding.sequences.stop_after(iterator: Union[Iterator[T], Iterable[T]], n: int) → Iterator[T]

Stop this iterator after returning n elements, even if it’s longer than that.

The resulting iterator may be shorter than n, if the source element is so.

Deprecated since version 2.14.22: Use itertools.islice instead

  • iterator – iterator or iterable to examine
  • n – elements to return


satella.coding.sequences.group_quantity(length: int, seq: Union[Iterator[T], Iterable[T]]) → Iterator[List[T]]

Slice an iterable into lists containing at most len entries.


>>> assert list(group_quantity(3, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10])) == [[1, 2, 3], [4, 5, 6],
>>>                                                                     [7, 8, 9], [10]]

This correctly detects sequences, and uses an optimized variant via slicing if a sequence is passed.

You can safely pass ranges

  • length – p_len for the returning sequences
  • seq – sequence to split


satella.coding.sequences.iter_dict_of_list(dct: Dict[T, List[U]]) → Generator[Tuple[T, U], None, None]

Presents a simple way to iterate over a dictionary whose values are lists.

This will return the dictionary key and each of the value contained in the list attached to the key.


satella.coding.sequences.shift(iterable_: Union[Reversible[T], Iterator[T], Iterable[T]], shift_factor: int) → Iterator[T]

Return this sequence, but shifted by factor elements, so that elements will appear sooner by factor.


>>> assert list(shift([1,2, 3], 1)) == [2, 3, 1]

However note that this will result in iterators which have negative shift to be readed entirely into memory (converted internally to lists). This can be avoided by passing in a Reversible iterable.

  • iterable – iterable to shift
  • shift_factor – factor by which shift elements.

shifted sequence


satella.coding.sequences.other_sequence_no_longer_than(base_sequence: Union[Iterator[T], Iterable[T]], other_sequence: Union[Iterator[T], Iterable[T]]) → Iterator[T]

Return every item in other_sequence, but limit it’s p_len to that of base_sequence.

If other_sequence is shorter than base_sequence, the shorter one will be returned.

  • base_sequence – sequence whose p_len should be taken
  • other_sequence – sequence to output values from


satella.coding.sequences.count(sq: Union[Iterator[T], Iterable[T]], start: Optional[int] = None, step: int = 1, start_at: Optional[int] = None) → Iterator[int]

Return a sequence of integers, for each entry in the sequence with provided step.

Essentially the same (if step were ignored) as:

>>> (i for i, x in enumerate(sq, start=start_at))

Deprecated since version 2.14.22: Use start instead

  • sq – sequence to enumerate
  • start – alias for start_at. Prefer it in regards to start_at. Default is 0
  • step – number to add to internal counter after each element
  • start_at – deprecated alias for start

an iterator of subsequent integers


satella.coding.sequences.length(iterator: Union[Iterator[T], Iterable[T]]) → int

Return the length of an iterator, exhausting it by the way

Even and odd

satella.coding.sequences.even(sq: Union[Iterator[T], Iterable[T]]) → Iterator[T]

Return only elements with even indices in this iterable (first element will be returned, as indices are counted from 0)

satella.coding.sequences.odd(sq: Union[Iterator[T], Iterable[T]]) → Iterator[T]

Return only elements with odd indices in this iterable.


class satella.coding.sequences.Multirun(sequence: Iterable[T_co], dont_return_list: bool = False)

A class to launch the same operation on the entire sequence.


>>> class Counter:
>>>     def __init__(self, value=0):
>>>         self.count = value
>>>     def add(self, v):
>>>         self.count += 1
>>>     def __eq__(self, other):
>>>          return self.count == other.count
>>>     def __iadd__(self, other):
>>>         self.add(other)
>>> a = [Counter(), Counter()]

The following:

>>> for b in a:
>>>     b.add(2)

Can be replaced with

>>> Multirun(a).add(2)

And the following:

>>> for b in a:
>>>     b += 3

With this

>>> b = Mulirun(a)
>>> b += 3

Furthermore note that:

>>> Multirun(a).add(2) == [Counter(2), Counter(2)]
  • sequence – sequence to execute these operations for
  • dont_return_list – the operation won’t return a list if this is True


class satella.coding.sequences.ListWrapperIterator(iterator: Union[Iterator[T], Iterable[T]])

A wrapped for an iterator, enabling using it as a normal list.

The first time this is evaluated, list is populated with elements.

The second time, items are taken from the list.

It never computes more than it needs to.

Essentially a class that lets you reuse one-shot iterators.

This is additionally a generic class.

advance_to_item(i: int) → None

Makes the list be at least i in size

exhaust() → None

Load all elements of this iterator into memory.

next() → T

Get the next item

Raises:StopIteration – next element is not available due to iterator finishing
class satella.coding.sequences.ConstruableIterator(*args, **kwargs)

An iterator that you can attach arbitrary things at the end and consume them during iteration. Eg:

>>> a = ConstruableIterator([1, 2, 3])
>>> for b in a:
>>>   if b % 2 == 0:
>>>       a.add(6)

All arguments you provide to the constructor will be passed to underlying deque

add(t: T) → None

Schedule given value to be iterated over after current items

Parameters:t – value to iterate over
add_immediate(t: T) → None

Schedule given value to be iterated over during the next __next__ call

Parameters:t – value to iterate over
add_many(t: Iterable[T]) → None

Schedule given values to be iterated over after current items

Parameters:t – iterable of values
add_many_immediate(t: Iterable[T]) → None

Schedule given values to be iterated over during the next __next__ call

Parameters:t – values to iterate over
satella.coding.sequences.walk(obj: T, child_getter: Callable[[T], Optional[List[T]]] = <class 'list'>, deep_first: bool = True, leaves_only: bool = False) → Iterator[T]

Return every node of a nested structure.

  • obj – structure to traverse. This will not appear in generator
  • child_getter – a callable to return a list of children of T. Should return an empty list or None of there are no more children.
  • deep_first – if True, deep first will be returned, else it will be breadth first
  • leaves_only – if True, only leaf nodes (having no children) will be returned
satella.coding.chain(*args) → Iterator[T_co]

Construct an iterator out of provided elements.

If an element is an iterator, or an iterable it will be yielded-from. If it’s not, it will just be yielded.

A cast to iter() is used to determine iteratorness

satella.coding.exhaust(iterator: Iterator[T_co]) → None

Iterate till the end of the iterator, discarding values as they go

Parameters:iterator – iterator to exhaust
class satella.coding.hint_with_length(generator: Generator[T_co, T_contra, V_co], length: Optional[int], length_factory: Optional[Callable[[], int]] = None)

Accepting a generator, return it additionally providing a specified __length_hint__

You can provide generator-generating functions as well

  • generator – generator to decorate
  • length – length hint to provide
  • length_factory – a callable called with no arguments to get the length

You must provide either length or length_factory. Giving them both is wrong, and will result in ValueError

Sometimes, you need the entire body of the generator to run. It’d be a shame if someone decided to bail out on you in the middle of the for loop. That’s what this class is for:

class satella.coding.SelfClosingGenerator(generator: Union[Generator[T_co, T_contra, V_co], Callable[[Any], Generator[T_co, T_contra, V_co]]])

A wrapper to exhaust the generator in response to closing it.

This will allow generators to complete that don’t provide a .close() method.

This will additionally exhaust the generator upon deallocation of the generator.

You can feed it with either generators, or generator-functions, it will behave correctly each time.

You can also use it as a context manager, to decouple finalizing the generator from the GC collection

Using it on your generator objects will assure that they will run to completion.

Take care: this won’t work on PyPy due to it’s nondeterministic garbage collection!


Objects that allow you to easily (and rather quickly) remove elements from a list or a dict while iterating over them with minimum memory overhead.


class satella.coding.ListDeleter(list_to_process: MutableSequence[T], direction: int = 0)

Having problems deleting entries from your list while iterating on them? No problem. Just swap the following:

>>> entries_to_delete = []
>>> for entry in my_list:
>>>     if entry.should_delete():
>>>         entries_to_delete.append(entry)
>>> for entry in entries_to_delete:
>>>     my_list.remove(entry)

With the following:

>>> with ListDeleter(my_list) as ld:
>>>     for entry in ld:
>>>         if entry.should_delete():
>>>             ld.delete()

You can also use the alternative syntax of: >>> ld = ListDeleter(my_list) >>> while True: >>> try: >>> v = >>> except StopIteration: >>> break >>> if condition(v): >>> ld.delete() >>> ld.remove_items()

Note that a single ListDeleter running from a single context must be iterated on by only a single Thread as it keeps the state of iterator in itself, to prevent allocating new objects and slowing things down too much.

Note that calling reversed() on this will reset the pointer to the end of the list or the beginning of the list, respectively.

This allocates only a single object per a call to delete().

Calling the list deleter during iteration will yield the element.

You can pass any type of object here, as long as it supports pop(position) and __getitem__

next() → T
Returns:the next element
Raises:StopIteration – no more entries
prev() → T

Move to previous element, as per ordering.

Returns:the previous element
Raises:StopIteration – list is already at the first element!
remove_items() → None

After all of the items have been marked for deletion, delete them


class satella.coding.DictDeleter(dict_to_process:

Having problems deleting entries from your dict while iterating on them? No problem. Just swap the following:

>>> keys_to_delete = []
>>> for key, value in my_dict.items():
>>>     if value.should_delete():
>>>         keys_to_delete.append(key)
>>> for key in keys_to_delete:
>>>     del my_dict[key]

With the following:

>>> with DictDeleter(my_list) as ld:
>>>     for key, value in ld.items():
>>>         if value.should_delete():
>>>             ld.delete()

Note that a single DictDeleter running from a single context must be iterated on by only a single Thread as it keeps the state of iterator in itself, to prevent allocating new objects and slowing things down too much.

This allocates only a single object per a call to delete().