Skip to content

LiveTradingNode fails to gracefully stop via Ctrl+C in Windows cmd (timeout & pending tasks) #2785

@daydayup789

Description

@daydayup789

Description:

Environment:

  • OS: Windows 10 (running via cmd)
  • Python: 3.11.x
  • Nautilus Trader: 1.219.0 and 1.220.0
  • Run method: script binance_futures_testnet_ema_cross.py

Problem:

After starting the strategy via binance_futures_testnet_ema_cross.py and attempting to stop it using Ctrl+C, the node fails to fully shutdown within the default 10-second timeout.

Observed log output:

[WARN] TradingNode: Timed out (10.0s) waiting for node to stop
ExecEngine.check_disconnected() == False
...
[ERROR] InvalidStateTrigger('RUNNING -> DISPOSE') state RUNNING
Task was destroyed but it is pending!
task: <Task cancelling name='inflight_check' coro=LiveExecutionEngine._inflight_check_loop() ...>

Details:

  • ExecEngine and DataEngine fail to disconnect within timeout;
  • Async tasks (such as _inflight_check_loop) are left pending or unawaited;
  • State transition RUNNING -> DISPOSE fails due to inconsistent shutdown state;
  • One or more async tasks are destroyed while still pending.

Steps to Reproduce:

  1. Run python binance_futures_testnet_ema_cross.py on Windows
  2. Let the strategy run normally
  3. Press Ctrl+C to stop it
  4. Observe the logs

Expected Behavior:

  • Node captures KeyboardInterrupt and fully shuts down all engines and clients
  • All background tasks are cleanly cancelled and awaited
  • No invalid state transitions occur
  • System logs show a clean shutdown without pending tasks

Suggestions:

  • Allow configurable shutdown timeout (default 10s may not be sufficient)
  • Ensure async tasks like _inflight_check_loop are properly cancelled and awaited
  • Add a lifecycle hook like on_exit() for user-defined cleanup
  • Include Windows-specific signal handling notes or compatibility fallbacks for asyncio
[WARN] TESTER-001.TradingNode: Timed out (10.0s) waiting for node to stop
Status
------
DataEngine.check_disconnected() == False
ExecEngine.check_disconnected() == False�[0m
�[1m2025-07-14T05:37:47.155703400Z�[0m [INFO] TESTER-001.DataClient-BINANCE: STOPPED�[0m
�[1m2025-07-14T05:37:47.155935500Z�[0m [INFO] TESTER-001.DataEngine: STOPPED�[0m
�[1m2025-07-14T05:37:47.156008400Z�[0m [INFO] TESTER-001.RiskEngine: STOPPED�[0m
�[1m2025-07-14T05:37:47.156042300Z�[0m [INFO] TESTER-001.ExecClient-BINANCE: Shutting down retry manager pool�[0m
�[1m2025-07-14T05:37:47.156050300Z�[0m [INFO] TESTER-001.ExecClient-BINANCE: STOPPED�[0m
�[1m2025-07-14T05:37:47.156128800Z�[0m [INFO] TESTER-001.ExecEngine: STOPPED�[0m
�[1m2025-07-14T05:37:47.156165200Z�[0m [INFO] TESTER-001.OrderEmulator: STOPPED�[0m
�[1m2025-07-14T05:37:47.156188800Z�[0m [INFO] TESTER-001.DataClient-BINANCE: DISPOSED�[0m
�[1m2025-07-14T05:37:47.156195600Z�[0m [INFO] TESTER-001.DataEngine: DISPOSED�[0m
�[1m2025-07-14T05:37:47.156206600Z�[0m [INFO] TESTER-001.RiskEngine: DISPOSED�[0m
�[1m2025-07-14T05:37:47.156218700Z�[0m [INFO] TESTER-001.ExecClient-BINANCE: DISPOSED�[0m
�[1m2025-07-14T05:37:47.156224500Z�[0m [INFO] TESTER-001.ExecEngine: DISPOSED�[0m
�[1m2025-07-14T05:37:47.156237500Z�[0m [INFO] TESTER-001.MessageBus: Closed message bus�[0m
�[1m2025-07-14T05:37:47.156268400Z�[0m �[1;31m[ERROR] TESTER-001.TESTER-001: InvalidStateTrigger('RUNNING -> DISPOSE') state RUNNING�[0m
�[1m2025-07-14T05:37:47.156273600Z�[0m �[1;31m[ERROR] TESTER-001.TESTER-001: InvalidStateTrigger('RUNNING -> DISPOSE_COMPLETED') state RUNNING�[0m
�[1m2025-07-14T05:37:47.156279600Z�[0m [INFO] TESTER-001.TradingNode: Shutting down executor�[0m
�[1m2025-07-14T05:37:47.156309500Z�[0m [INFO] TESTER-001.TradingNode: Closing event loop�[0m
Task was destroyed but it is pending!
task: <Task cancelling name='inflight_check' coro=<LiveExecutionEngine._inflight_check_loop() running at d:\Anaconda3\envs\nt\Lib\site-packages\nautilus_trader\live\execution_engine.py:535> wait_for=<Future cancelled>>
Traceback (most recent call last):
  File "E:\python_workspace\crypto2\nt_dev\test\live\binance_futures_testnet_ema_cross.py", line 139, in <module>
�[1m2025-07-14T05:37:48.157786300Z�[0m [INFO] TESTER-001.TradingNode: loop.is_running=False�[0m
    node.run()
  File "d:\Anaconda3\envs\nt\Lib\site-packages\nautilus_trader\live\node.py", line 277, in run
�[1m2025-07-14T05:37:48.157798400Z�[0m [INFO] TESTER-001.TradingNode: loop.is_closed=True�[0m
    self.kernel.loop.run_until_complete(self.run_async())
  File "d:\Anaconda3\envs\nt\Lib\asyncio\base_events.py", line 641, in run_until_complete
�[1m2025-07-14T05:37:48.157799400Z�[0m [INFO] TESTER-001.TradingNode: DISPOSED�[0m
    self.run_forever()
  File "d:\Anaconda3\envs\nt\Lib\asyncio\windows_events.py", line 321, in run_forever
    super().run_forever()
  File "d:\Anaconda3\envs\nt\Lib\asyncio\base_events.py", line 608, in run_forever
    self._run_once()
  File "d:\Anaconda3\envs\nt\Lib\asyncio\base_events.py", line 1898, in _run_once
    event_list = self._selector.select(timeout)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "d:\Anaconda3\envs\nt\Lib\asyncio\windows_events.py", line 444, in select
    self._poll(timeout)
  File "d:\Anaconda3\envs\nt\Lib\asyncio\windows_events.py", line 825, in _poll
    status = _overlapped.GetQueuedCompletionStatus(self._iocp, ms)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt
^C

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions