Skip to content

Instantly share code, notes, and snippets.

@muellerzr
Created August 27, 2025 22:28
Show Gist options
  • Save muellerzr/1d59d619827f7a29c8441a16f3ff74b2 to your computer and use it in GitHub Desktop.
Save muellerzr/1d59d619827f7a29c8441a16f3ff74b2 to your computer and use it in GitHub Desktop.
[balthasar.local:04124] btl:tcp: would block, so allowing background progress
[balthasar.local:04124] btl:tcp: now connected to 192.168.68.53, process [[49579,1],1]
[balthasar.local:04124] btl:tcp: connect() to 192.168.68.53:1024 completed (complete_connect), sending connect ACK
[melchior.local:01946] btl:tcp: now connected to 192.168.68.80, process [[49579,1],0]
[melchior.local:01946] btl: tcp: attempting to connect() to [[49579,1],0] address fdcd:ee7b:759e:1744:414:41e7:3d97:1761 on port 1024
[melchior.local:01946] btl:tcp: would block, so allowing background progress
[melchior.local:01946] btl:tcp: connect() to fdcd:ee7b:759e:1744:414:41e7:3d97:1761:1024 completed (complete_connect), sending connect ACK
[balthasar.local:04124] btl:tcp: now connected to fdcd:ee7b:759e:1744:f700:ca71:db72:f19a, process [[49579,1],1]
[melchior:01946] [0] func:0 libopen-pal.80.dylib 0x000000011a4c776c opal_backtrace_buffer + 56
[melchior:01946] [1] func:1 libmpi.40.dylib 0x000000011a69222c ompi_mpi_abort + 164
[melchior:01946] [2] func:2 libmpi.40.dylib 0x000000011a684358 ompi_mpi_errors_are_fatal_comm_handler + 80
[melchior:01946] [3] func:3 libmpi.40.dylib 0x000000011a684084 ompi_errhandler_invoke + 360
[melchior:01946] [4] func:4 libmpi.40.dylib 0x000000011a6c9894 MPI_Allgather + 416
[melchior:01946] [5] func:5 libmlx.dylib 0x000000010624840c _ZN3mlx4core9scheduler12StreamThread9thread_fnEv + 488
[melchior:01946] [6] func:6 libmlx.dylib 0x00000001062485e0 _ZNSt3__114__thread_proxyB8ne180100INS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEMN3mlx4core9scheduler12StreamThreadEFvvEPSA_EEEEEPvSF_ + 72
[melchior:01946] [7] func:7 libsystem_pthread.dylib 0x000000018ace7c0c _pthread_start + 136
[melchior:01946] [8] func:8 libsystem_pthread.dylib 0x000000018ace2b80 thread_start + 8
[melchior:00000] *** An error occurred in MPI_Allgather
[melchior:00000] *** reported by process [3249209345,1]
[melchior:00000] *** on communicator MPI_COMM_WORLD
[melchior:00000] *** MPI_ERR_TRUNCATE: message truncated
[melchior:00000] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
[melchior:00000] *** and MPI will try to terminate your MPI job as well)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment