python - auto-repeat flag in a pack format string

Question

In php, unpack() has the "*" flag which means "repeat this format until the end of input". For example, this prints 97, 98, 99

$str = "abc";
$b = unpack("c*", $str);
print_r($b);

Is there something like this in python? Of course, I can do

str = "abc"
print struct.unpack("b" * len(str), str)

but I'm wondering if there is a better way.

Answer 1

In Python 3.4 and later, you can use the new function struct.iter_unpack.

struct.iter_unpack(fmt, buffer)

Iteratively unpack from the buffer buffer according to the format string fmt. This function returns an iterator which will read equally-sized chunks from the buffer until all its contents have been consumed. The buffer’s size in bytes must be a multiple of the size required by the format, as reflected by calcsize().

Each iteration yields a tuple as specified by the format string.

Let's say we want to unpack the array b'x01x02x03'*3 with the repeating format string '<2sc' (2 characters followed by a single character, repeat until done).

With iter_unpack, you can do the following:

>>> import struct
>>> some_bytes = b'x01x02x03'*3
>>> fmt = '<2sc'
>>> 
>>> tuple(struct.iter_unpack(fmt, some_bytes))
((b'x01x02', b'x03'), (b'x01x02', b'x03'), (b'x01x02', b'x03'))

If you want to un-nest this result, you can do so with itertools.chain.from_iterable.

>>> from itertools import chain
>>> tuple(chain.from_iterable(struct.iter_unpack(fmt, some_bytes)))
(b'x01x02', b'x03', b'x01x02', b'x03', b'x01x02', b'x03')

Of course, you could just employ a nested comprehension to do the same thing.

>>> tuple(x for subtuple in struct.iter_unpack(fmt, some_bytes) for x in subtuple)
(b'x01x02', b'x03', b'x01x02', b'x03', b'x01x02', b'x03')