Working with Bytes in Python

Biting through the bytes

How to create a temporary buffer in memory and working with data there.

Instantiate a buffer in memory.

from io import BytesIO
buf = BytesIO(); buf

<_io.BytesIO at 0x7e185d719350>

Now you can’t directly write data to this buffer.

buf.write('This is some data')

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[24], line 1
----> 1 buf.write('This is some data')

TypeError: a bytes-like object is required, not 'str'

You need a bytes-like object.

buf.write(b'This is some data')

17

17 bytes have been written to memory, which we can confirm here.

len(b'This is some data')

17

type(b'This is some data'), type('This is some data')

(bytes, str)

buf.getvalue()

b'This is some data'

Behind the scenes, the data is stored as binary. However, when printed out, if the byte is a printable character, Python directly renders it out.

We can view the bytes as hexadecimal.

buf.getvalue().hex()

'5468697320697320736f6d652064617461'

from textwrap import wrap
h = buf.getvalue().hex()
wrap(h, width=2)

['54',
 '68',
 '69',
 '73',
 '20',
 '69',
 '73',
 '20',
 '73',
 '6f',
 '6d',
 '65',
 '20',
 '64',
 '61',
 '74',
 '61']

?chr

def chr(
    i
):

Return a Unicode string of one character with ordinal i; 0 <= i <= 0x10ffff.

Type: builtin_function_or_method

''.join(chr(int(v, 16)) for v in wrap(h, width=2))

'This is some data'

Or as ASCII encoding.

list(buf.getvalue())

[84, 104, 105, 115, 32, 105, 115, 32, 115, 111, 109, 101, 32, 100, 97, 116, 97]

''.join(chr(v) for v in buf.getvalue())

'This is some data'

Hex escape format.

''.join(f'\\x{b:02x}' for b in buf.getvalue())

'\\x54\\x68\\x69\\x73\\x20\\x69\\x73\\x20\\x73\\x6f\\x6d\\x65\\x20\\x64\\x61\\x74\\x61'

Or binary, how it’s actually stored in memory.

bin(buf.getvalue()[0])

'0b1010100'

''.join(bin(v) for v in buf.getvalue())

'0b10101000b11010000b11010010b11100110b1000000b11010010b11100110b1000000b11100110b11011110b11011010b11001010b1000000b11001000b11000010b11101000b1100001'

type(b'text'), type('text')

(bytes, str)

Apart from .getvalue(), we can also treat buffer as a list with a pointer, and read the contents being pointed at onward.

buf.read()

b''

An empty string was returned. This is because the pointer is currently at the end of the list. We can seek back to the beginning of the list.

buf.seek(0)

0

buf.read()

b'This is some data'

buf.read()

b''

Let’s seek to the fourth byte.

buf.seek(4)
buf.read()

b' is some data'

We can see that the fifth byte is indeed blank space.

list(map(chr, buf.getvalue()))

['T',
 'h',
 'i',
 's',
 ' ',
 'i',
 's',
 ' ',
 's',
 'o',
 'm',
 'e',
 ' ',
 'd',
 'a',
 't',
 'a']

---

If you have any comments, questions, suggestions, feedback, criticisms, or corrections, please do post them down in the comment section below!