Solvedaiohttp "RuntimeError: Event loop is closed" when ProactorEventLoop is used

Long story short

When ProactorEventLoop is used, the lack of graceful shutdown in aiohttp results in RuntimeError. (For comparison, when SelectorEventLoop is used, it results in ResourceWarning.) Note that ProactorEventLoop is the default event loop on Windows in Python starting from 3.8.

Maybe the root cause of this issue is the same as of #1925. But the impact is higher (RuntimeError instead of ResourceWarning).

Steps to reproduce

import asyncio
import sys

import aiohttp

async def main():
	async with aiohttp.ClientSession() as session:
		async with session.get('https://example.com/'):
			pass

if __name__ == '__main__':
	if sys.version_info[:2] == (3, 7):
		asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())
	asyncio.run(main())

Actual behaviour

sys:1: ResourceWarning: unclosed <socket.socket fd=724, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=6, laddr=('2001:0db8:85a3:0000:0000:8a2e:0370:7334', 19283, 0, 0), raddr=('2606:2800:220:1:248:1893:25c8:1946', 443, 0, 0)>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
C:\Programs\Python37\lib\asyncio\proactor_events.py:94: ResourceWarning: unclosed transport <_ProactorSocketTransport fd=-1 read=<_OverlappedFuture cancelled>>
  source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x0000017FAC405048>
Traceback (most recent call last):
  File "C:\Programs\Python37\lib\asyncio\proactor_events.py", line 95, in __del__
    self.close()
  File "C:\Programs\Python37\lib\asyncio\proactor_events.py", line 86, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Programs\Python37\lib\asyncio\base_events.py", line 683, in call_soon
    self._check_closed()
  File "C:\Programs\Python37\lib\asyncio\base_events.py", line 475, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

Expected behaviour

No exceptions. Maybe warnings, but no exceptions.

Your environment

aiohttp (client) version: 3.6.2
Python version: 3.7, 3.8
OS: Windows 10

Additional information

I do not know any workaround.

A delay before closing the event loop suggested in the documentation does not help in a bit more complicated example (try to run it several times or add more URLs to the list):

import asyncio
import sys

import aiohttp

URLS = [
	# Insert several URLs here, e.g.:
	"https://www.python.org/ftp/python/",
	"https://github.com/python/",
	"https://www.cpan.org/src/5.0/",
	"https://aiohttp.readthedocs.io/",
	"https://example.com/",
]

async def main():
	async with aiohttp.ClientSession() as session:
		await asyncio.wait([fetch(session, url) for url in URLS])
	print("Done.")

async def fetch(session: aiohttp.ClientSession, url: str):
	async with session.get(url):
		print(f"Requested: {url}")

if __name__ == '__main__':
	if sys.version_info[:2] == (3, 7):
		asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())
	loop = asyncio.get_event_loop()
	try:
		loop.run_until_complete(main())
		loop.run_until_complete(asyncio.sleep(2.0))
	finally:
		loop.close()
31 Answers

✔️Accepted Answer

I found another solution for this problem if some still having issues with it. This involves directly manipulating the class method itself, it might be dirty in some peoples eyes. This applies to situations where you need to close your loops after a tasks like web apps and other similar situations.

from functools import wraps

from asyncio.proactor_events import _ProactorBasePipeTransport

def silence_event_loop_closed(func):
    @wraps(func)
    def wrapper(self, *args, **kwargs):
        try:
            return func(self, *args, **kwargs)
        except RuntimeError as e:
            if str(e) != 'Event loop is closed':
                raise
    return wrapper

_ProactorBasePipeTransport.__del__ = silence_event_loop_closed(_ProactorBasePipeTransport.__del__)

EDIT: You may want to do this only on Windows environment to avoid executing it in production (although ProactorEventLoop is not available on Linux, but just in case).

if platform.system() == 'Windows':
    # Silence the exception here.
    _ProactorBasePipeTransport.__del__ = silence_event_loop_closed(_ProactorBasePipeTransport.__del__)

Other Answers:

Running asyncio.get_event_loop().run_until_complete(...) instead of asyncio.run(...) doesn't show the error. Is there any difference between them ?

EDIT: Peraphs there is because run_until_complete does not close the loop while asyncio.run does and calls _ProactorBasePipeTransport.__del__ where it raises the exception, I've provided a way to handle it in a dirty manner below.

@paaksing I see, thanks.

You're wrong about this being fixed in python 3.9 though. Guess we'll wait for python 3.10?

More Issues: