GH-145278: freeze encodings (partially) and linecache

[FFY00]

GH-145279 re-do

• Issue: Freeze modules used during startup #145278

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @FFY00 for commit 719d17d 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F148347%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

[FFY00]

The raspbian buildbot failure is weird. Grasping straws, the only possible connection I can see is this PR touching the IO initialization.

Per previous Windows failure, it seems that when executing the encodings module, before it is finalized, trying to import a non-frozen submodule can cause failures. I don't see anything missing, but if the buildbot failure is indeed connected, my best guess is that this situation happens under the buildbot environment.

Since I cannot identify an actual connection here, I am gonna go ahead and merge, keeping an eye on the bug reports.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM Raspbian 3.x (tier-3) has failed when building commit 0012686.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/424/builds/13416) and take a look at the build logs.
4. Check if the failure is related to this commit (0012686) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/424/builds/13416

Failed tests:

• test.test_io.test_fileio

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf7490000')


Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf7776000')


Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf78a2000')


Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf769e000')


Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf77f6000')


Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf7b30000')


Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf76b0000')


Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf79b0000')


Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf74ca000')


Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf7ae0000')


Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf76ea000')


Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf7a96000')

[vstinner]

https://buildbot.python.org/#/builders/424/builds/13416

Example of test failure:

FAIL: test_syscalls_read (test.test_io.test_fileio.CAutoFileTests.test_syscalls_read) (name='open in binary mode')
Check set of system calls during common I/O patterns
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_io/test_fileio.py", line 406, in check_readall
    self.assertIn(
    ~~~~~~~~~~~~~^
        fd_str,
        ^^^^^^^
        ev.args,
        ^^^^^^^^
        f"Looking for file descriptor in arguments|ev={ev}"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: '3' not found in ['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'] : Looking for file descriptor in arguments|ev=StraceEvent(syscall='mmap2', args=['NULL', '135168', 'PROT_READ|PROT_WRITE', 'MAP_PRIVATE|MAP_ANONYMOUS', '-1', '0'], returncode='0xf769e000')

The test gets an unexpected mmap2(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) syscall. I suppose that it should be ignored.

cc @cmaloney

[FFY00]

Yeah, looks like strace_helper.filter_memory is only filtering out mmap, but not mmap2.

[FFY00]

To be clear, this test is checking for IO syscalls, which is not the case for that mmap2 call. It simply isn't being filtered out by strace_helper.filter_memory, so it shows up here unexpectedly. GH-148648 fixes strace_helper.filter_memory, also taking mmap2 into account.