Preferences

> The standard library provides the LazyLoader class to solve some of these inefficiency problems. It permits imports at the module level to work mostly like inline imports do.

The use of these sorts of Python import internals is highly non-obvious. The Stack Overflow Q&A I found about it (https://stackoverflow.com/questions/42703908/) doesn't result in an especially nice-looking UX.

So here's a proof of concept in existing Python for getting all imports to be lazy automatically, with no special syntax for the caller:

  import sys
  import threading # needed for python 3.13, at least at the REPL, because reasons
  from importlib.util import LazyLoader # this has to be eagerly imported!
  class LazyPathFinder(sys.meta_path[-1]): # <class '_frozen_importlib_external.PathFinder'>
      @classmethod
      def find_spec(cls, fullname, path=None, target=None):
          base = super().find_spec(fullname, path, target)
          base.loader = LazyLoader(base.loader)
          return base
  sys.meta_path[-1] = LazyPathFinder
We've replaced the "meta path finder" (which implements the logic "when the module isn't in sys.modules, look on sys.path for source code and/or bytecode, including bytecode in __pycache__ subfolders, and create a 'spec' for it") with our own wrapper. The "loader" attached to the resulting spec is replaced with an importlib.util.LazyLoader instance, which wraps the base PathFinder's provided loader. When an import statement actually imports the module, the name will actually get bound to a <class 'importlib.util._LazyModule'> instance, rather than an ordinary module. Attempting to access any attribute of this instance will trigger the normal module loading procedure — which even replaces the global name.

Now we can do:

  import this # nothing shows up
  print(type(this)) # <class 'importlib.util._LazyModule'>
  rot13 = this.s # the module is loaded, printing the Zen
  print(type(this)) # <class 'module'>
That said, I don't know what the PEP means by "mostly" here.

Can you explain why does "threading" needs to be loaded? The rest seems decently straightforward, but what initialization from threading is required and why only in 3.13+?
I didn't investigate deeply (I just happened to stumble on the issue while figuring out the implementation), but it might be needed in older versions too. The loading process instantiates a `threading.RLock`, but from a source code file that doesn't import that (because it's in a file that has to bootstrap the import process). I'm not entirely sure how `threading` is supposed to get imported normally. The other complicating factor is that I was testing this at the REPL, and 3.13 has a new REPL implementation which might change how those initial libraries are brought in.

This item has no comments currently.

Keyboard Shortcuts

Story Lists

j
Next story
k
Previous story
Shift+j
Last story
Shift+k
First story
o Enter
Go to story URL
c
Go to comments
u
Go to author

Navigation

Shift+t
Go to top stories
Shift+n
Go to new stories
Shift+b
Go to best stories
Shift+a
Go to Ask HN
Shift+s
Go to Show HN

Miscellaneous

?
Show this modal