ruk·si

🐍 Python
Iterators

Updated at 2018-06-10 05:55

Generators are not always iterators, iterators are not always generators. Generator can produce seemingly infinite amount of values. Iterator allows you to go through a finite or infinite collection. Iterables can be used in a for-loop.

How Python iteration works:

  1. Call __iter__ if it exists, returning an iterator. The iterator has __next__ which is called until StopIteration.
  2. Calls __getitem__ with 0, 1, 2, ... if it exists.
  3. Raise TypeError if none of the previous work.

You can't create infinite lists but you can create infinite iterators.

import re
from typing import List

WORDS_RE: re = re.compile('\w+')

class Sentence:
    _text: str
    _words: List[str]

    def __init__(self, text: str) -> None:
        self._text = text
        self._words = WORDS_RE.findall(text)

    def __getitem__(self, index) -> str:
        return self._words[index]

    def __len__(self) -> int:
        return len(self._words)

    def __repr__(self) -> str:
        return f'Sentence({self._text})'

s = Sentence('"The time has come," the Walrus said.')
assert list(s) == ['The', 'time', 'has', 'come', 'the', 'Walrus', 'said']
assert s[1] == 'time'
assert s[-2] == 'Walrus'

# what for-loop does under the hood
it = iter(s)
while True:
    try:
        assert isinstance(next(it), str)
    except StopIteration:
        del it
        break

Or you can implement __iter__

import re
from typing import Iterable

WORDS_RE: re = re.compile('\w+')

class GenSentence:
    _text: str

    def __init__(self, text: str) -> None:
        self._text = text

    def __iter__(self) -> Iterable:
        # using lazy evaluation and generator expression
        return (match.group() for match in WORDS_RE.finditer(self._text))

    def __repr__(self) -> str:
        return f'Sentence({self._text})'

gs = GenSentence('Hello World!')
assert list(gs) == ['Hello', 'World']

If you want to stop iteration, raise StopIteration.

from typing import Generic, TypeVar

T = TypeVar('T')

class BoundRepeater(Generic[T]):

    def __init__(self, value: T, max_repeats: int) -> None:
        self.value: T = value
        self.max_repeats: int = max_repeats
        self.count: int = 0

    def __iter__(self) -> 'BoundRepeater':
        return self

    def __next__(self) -> T:
        if self.count >= self.max_repeats:
            raise StopIteration
        self.count += 1
        return self.value

for item in BoundRepeater[str]('Hello', 3):
    assert item == 'Hello'

Source

  • Python Tricks The Book, Dan Bader
  • Fluent Python, Luciano Ramalho