Highlight v3.43/v3.44: CodeGenerator+ESC_TRUECOLOR may causes crash by out of memory

[smdn]

Smdn.LibHighlightSharp.Bindings.CodeGenerator.generateString seems to return an un-null-terminated string, may causing a crash.

Unhandled exception. System.ArgumentException: The string must be null-terminated.
  at System.String.ThrowMustBeNullTerminatedString()
  at System.String.Ctor(SByte* value)
  at System.StubHelpers.CSTRMarshaler.ConvertToManaged(IntPtr cstr)

This does not occur 100% of the time and seems to depend on the process circumstances.
Confirmed that this occurs when OutputType is ESC_TRUECOLOR, but it is not tested that this occurs in other OutputTypes.

[smdn]

dotnet test --diag test.log

output

テスト実行を開始しています。お待ちください...
次のファイルで Vstest 診断をログ記録しています: /home/smdn/Smdn.LibHighlightSharp/tests/Smdn.LibHighlightSharp/log.log
合計 1 個のテスト ファイルが指定されたパターンと一致しました。
アクティブなテストの実行が中止されました。理由: テストのホスト プロセスがクラッシュしました : Out of memory.


成功!   -失敗:     0、合格:   110、スキップ:     0、合計:   110、期間: 104 ms - /home/smdn/Smdn.LibHighlightSharp/tests/Smdn.LibHighlightSharp/bin/Debug/net6.0/Smdn.LibHighlightSharp.Tests.dll (net6.0)
テストの実行がエラー System.Exception: One or more errors occurred.
 ---> System.Exception: Unable to read beyond the end of the stream.
   at System.IO.BinaryReader.Read7BitEncodedInt()
   at System.IO.BinaryReader.ReadString()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.LengthPrefixCommunicationChannel.NotifyDataAvailable()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TcpClientExtensions.MessageLoopAsync(TcpClient client, ICommunicationChannel channel, Action`1 errorHandler, CancellationToken cancellationToken)
   --- End of inner exception stack trace --- により中止されました。

test.log

TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:46.119, 5222174064812, vstest.console.dll, InProgress is Generate_BreakingChangesOnOutputTypeValue_TrueColor
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:46.119, 5222174125863, vstest.console.dll, MulticastDelegateUtilities.SafeInvoke: TestRun.RunStatsChanged: Invoking callbacks was skipped because there are no subscribers.
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:46.119, 5222174315641, vstest.console.dll, TestRunRequest:SendTestRunStatsChange: Completed.
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:46.119, 5222174401690, vstest.console.dll, MulticastDelegateUtilities.SafeInvoke: LengthPrefixCommunicationChannel: MessageReceived: Invoking callback 1/1 for Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TestRequestSender+<>c__DisplayClass31_0.<StartTestRun>b__1, took 1 ms.
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:46.119, 5222174461731, vstest.console.dll, TcpClientExtensions.MessageLoopAsync: Polling on remoteEndPoint: 127.0.0.1:53200 localEndPoint: 127.0.0.1:41887
TpTrace Verbose: 0 : 36914, 4, 2022/12/02, 23:50:46.119, 5222174254784, vstest.console.dll, MulticastDelegateUtilities.SafeInvoke: InternalTestLoggerEvents.SendTestResult: Invoking callback 1/1 for Microsoft.VisualStudio.TestPlatform.CommandLine.Internal.ConsoleLogger.TestResultHandler, took 0 ms.
TpTrace Verbose: 0 : 36914, 4, 2022/12/02, 23:50:46.119, 5222174590374, vstest.console.dll, MulticastDelegateUtilities.SafeInvoke: InternalTestLoggerEvents.SendTestResult: Invoking callback 1/1 for Microsoft.VisualStudio.TestPlatform.CommandLine.Internal.ConsoleLogger.TestResultHandler, took 0 ms.
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:47.120, 5223175540798, vstest.console.dll, TcpClientExtensions.MessageLoopAsync: Polling on remoteEndPoint: 127.0.0.1:53200 localEndPoint: 127.0.0.1:41887
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:48.122, 5224176715951, vstest.console.dll, TcpClientExtensions.MessageLoopAsync: Polling on remoteEndPoint: 127.0.0.1:53200 localEndPoint: 127.0.0.1:41887
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:49.122, 5225177419652, vstest.console.dll, TcpClientExtensions.MessageLoopAsync: Polling on remoteEndPoint: 127.0.0.1:53200 localEndPoint: 127.0.0.1:41887
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:50.123, 5226178621191, vstest.console.dll, TcpClientExtensions.MessageLoopAsync: Polling on remoteEndPoint: 127.0.0.1:53200 localEndPoint: 127.0.0.1:41887
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:51.125, 5227179745497, vstest.console.dll, TcpClientExtensions.MessageLoopAsync: Polling on remoteEndPoint: 127.0.0.1:53200 localEndPoint: 127.0.0.1:41887
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:52.126, 5228180959868, vstest.console.dll, TcpClientExtensions.MessageLoopAsync: Polling on remoteEndPoint: 127.0.0.1:53200 localEndPoint: 127.0.0.1:41887
TpTrace Warning: 0 : 36914, 5, 2022/12/02, 23:50:52.328, 5228383332770, vstest.console.dll, TestHostManagerCallbacks.ErrorReceivedCallback Test host standard error line: Out of memory.
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:53.127, 5229182213987, vstest.console.dll, TcpClientExtensions.MessageLoopAsync: Polling on remoteEndPoint: 127.0.0.1:53200 localEndPoint: 127.0.0.1:41887
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:53.537, 5229592474779, vstest.console.dll, TcpClientExtensions.MessageLoopAsync: NotifyDataAvailable remoteEndPoint: 127.0.0.1:53200 localEndPoint: 127.0.0.1:41887
TpTrace Warning: 0 : 36914, 5, 2022/12/02, 23:50:53.538, 5229592729368, vstest.console.dll, TestHostManagerCallbacks.ErrorReceivedCallback Test host standard error line: 
TpTrace Error: 0 : 36914, 10, 2022/12/02, 23:50:53.538, 5229593183616, vstest.console.dll, Socket: Message loop: failed to receive message due to socket error System.IO.EndOfStreamException: Unable to read beyond the end of the stream.
   at System.IO.BinaryReader.Read7BitEncodedInt()
   at System.IO.BinaryReader.ReadString()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.LengthPrefixCommunicationChannel.NotifyDataAvailable()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TcpClientExtensions.MessageLoopAsync(TcpClient client, ICommunicationChannel channel, Action`1 errorHandler, CancellationToken cancellationToken), remoteEndPoint: 127.0.0.1:53200 localEndPoint: 127.0.0.1:41887
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:53.538, 5229593348346, vstest.console.dll, SocketServer.PrivateStop: Stopping server endPoint: 127.0.0.1:41887 error: System.IO.EndOfStreamException: Unable to read beyond the end of the stream.
   at System.IO.BinaryReader.Read7BitEncodedInt()
   at System.IO.BinaryReader.ReadString()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.LengthPrefixCommunicationChannel.NotifyDataAvailable()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TcpClientExtensions.MessageLoopAsync(TcpClient client, ICommunicationChannel channel, Action`1 errorHandler, CancellationToken cancellationToken)
TpTrace Verbose: 0 : 36914, 7, 2022/12/02, 23:50:53.541, 5229596159812, vstest.console.dll, TestHostProvider.ExitCallBack: Host exited starting callback.
TpTrace Error: 0 : 36914, 7, 2022/12/02, 23:50:53.541, 5229596329025, vstest.console.dll, TestHostManagerCallbacks.ExitCallBack: Testhost processId: 36928 exited with exitcode: 134 error: 'Out of memory.
'
TpTrace Verbose: 0 : 36914, 7, 2022/12/02, 23:50:53.541, 5229596365500, vstest.console.dll, DotnetTestHostManager.OnHostExited: invoking OnHostExited callback
TpTrace Verbose: 0 : 36914, 7, 2022/12/02, 23:50:53.541, 5229596409418, vstest.console.dll, CrossPlatEngine.TestHostManagerHostExited: calling on client process exit callback.
TpTrace Information: 0 : 36914, 7, 2022/12/02, 23:50:53.541, 5229596461918, vstest.console.dll, TestRequestSender.OnClientProcessExit: Test host process exited. Standard error: Out of memory.

TpTrace Information: 0 : 36914, 7, 2022/12/02, 23:50:53.541, 5229596494745, vstest.console.dll, SocketServer.Stop: Stop server endPoint: 127.0.0.1:41887
TpTrace Verbose: 0 : 36914, 7, 2022/12/02, 23:50:53.541, 5229596556967, vstest.console.dll, MulticastDelegateUtilities.SafeInvoke: HostProviderEvents.OnHostExited: Invoking callback 1/1 for Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.TestHostManagerHostExited, took 0 ms.
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:53.546, 5229601249809, vstest.console.dll, LengthPrefixCommunicationChannel.Dispose: Dispose reader and writer.
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:53.546, 5229601621780, vstest.console.dll, SocketServer.Stop: Raise disconnected event endPoint: 127.0.0.1:41887 error: System.IO.EndOfStreamException: Unable to read beyond the end of the stream.
   at System.IO.BinaryReader.Read7BitEncodedInt()
   at System.IO.BinaryReader.ReadString()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.LengthPrefixCommunicationChannel.NotifyDataAvailable()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TcpClientExtensions.MessageLoopAsync(TcpClient client, ICommunicationChannel channel, Action`1 errorHandler, CancellationToken cancellationToken)
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:53.547, 5229601851449, vstest.console.dll, TestRequestSender: OnTestRunAbort: Set operation complete.
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:53.547, 5229601966255, vstest.console.dll, TestRequestSender.SetOperationComplete: Setting operation complete.
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:53.547, 5229602041012, vstest.console.dll, SocketServer.Stop: Stop server endPoint: 127.0.0.1:41887
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:53.547, 5229602224458, vstest.console.dll, TestRequestSender.GetAbortErrorMessage: Exception: System.IO.EndOfStreamException: Unable to read beyond the end of the stream.
   at System.IO.BinaryReader.Read7BitEncodedInt()
   at System.IO.BinaryReader.ReadString()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.LengthPrefixCommunicationChannel.NotifyDataAvailable()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TcpClientExtensions.MessageLoopAsync(TcpClient client, ICommunicationChannel channel, Action`1 errorHandler, CancellationToken cancellationToken)
TpTrace Verbose: 0 : 36914, 10, 2022/12/02, 23:50:53.547, 5229602300241, vstest.console.dll, TestRequestSender.GetAbortErrorMessage: Client has disconnected. Wait for standard error.
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:53.547, 5229602405589, vstest.console.dll, TestRequestSender.GetAbortErrorMessage: Received test host error message.
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:53.548, 5229602823419, vstest.console.dll, AssemblyResolver.OnResolve: Microsoft.TestPlatform.CommunicationUtilities.resources: Resolving assembly.
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:53.548, 5229602947996, vstest.console.dll, AssemblyResolver.OnResolve: Microsoft.TestPlatform.CommunicationUtilities.resources: Searching in: '/usr/share/dotnet/sdk/6.0.403/Extensions'.
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:53.548, 5229603083715, vstest.console.dll, AssemblyResolver.OnResolve: Microsoft.TestPlatform.CommunicationUtilities.resources: Assembly path does not exist: '/usr/share/dotnet/sdk/6.0.403/Extensions/Microsoft.TestPlatform.CommunicationUtilities.resources.dll', returning.
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:53.548, 5229603184706, vstest.console.dll, AssemblyResolver.OnResolve: Microsoft.TestPlatform.CommunicationUtilities.resources: Assembly path does not exist: '/usr/share/dotnet/sdk/6.0.403/Extensions/Microsoft.TestPlatform.CommunicationUtilities.resources.exe', returning.
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:53.548, 5229603304548, vstest.console.dll, AssemblyResolver.OnResolve: Microsoft.TestPlatform.CommunicationUtilities.resources: Searching in: '/usr/share/dotnet/sdk/6.0.403'.
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:53.548, 5229603353230, vstest.console.dll, AssemblyResolver.OnResolve: Microsoft.TestPlatform.CommunicationUtilities.resources: Assembly path does not exist: '/usr/share/dotnet/sdk/6.0.403/Microsoft.TestPlatform.CommunicationUtilities.resources.dll', returning.
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:53.548, 5229603388472, vstest.console.dll, AssemblyResolver.OnResolve: Microsoft.TestPlatform.CommunicationUtilities.resources: Assembly path does not exist: '/usr/share/dotnet/sdk/6.0.403/Microsoft.TestPlatform.CommunicationUtilities.resources.exe', returning.
TpTrace Information: 0 : 36914, 10, 2022/12/02, 23:50:53.548, 5229603429529, vstest.console.dll, AssemblyResolver.OnResolve: Microsoft.TestPlatform.CommunicationUtilities.resources: Failed to load assembly.
TpTrace Error: 0 : 36914, 10, 2022/12/02, 23:50:53.549, 5229603723356, vstest.console.dll, TestRequestSender: Aborting test run because テストのホスト プロセスがクラッシュしました : Out of memory.

[smdn]

same issue with highlight v3.44

win-x64

A total of 1 test files matched the specified pattern.
  Failed Generate_BreakingChangesOnOutputTypeValue_TrueColor [8 s]
  Error Message:
     Expected: No Exception to be thrown
  But was:  <System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.String.CreateStringForSByteConstructor(Byte* pb, Int32 numBytes)
   at System.String.Ctor(SByte* value)
   at System.StubHelpers.CSTRMarshaler.ConvertToManaged(IntPtr cstr)
   at Smdn.LibHighlightSharp.Bindings.highlightPINVOKE.CodeGenerator_generateString(HandleRef jarg1, String jarg2)
   at Smdn.LibHighlightSharp.Bindings.CodeGenerator.generateString(String input) in /_/src/Smdn.LibHighlightSharp.Bindings/Smdn.LibHighlightSharp.Bindings/highlight-3.44/CodeGenerator.cs:line 103
   at Smdn.LibHighlightSharp.Highlight.Generate(String input) in /_/src/Smdn.LibHighlightSharp/Smdn.LibHighlightSharp/Highlight.CodeGenerator.Methods.cs:line 61

macos-x64

A total of 1 test files matched the specified pattern.
The active test run was aborted. Reason: Test host process crashed : Unhandled exception. System.ArgumentException: The string must be null-terminated.
   at System.String.ThrowMustBeNullTerminatedString()
   at System.String.Ctor(SByte* value)
   at System.StubHelpers.CSTRMarshaler.ConvertToManaged(IntPtr cstr)
   at Smdn.LibHighlightSharp.Bindings.highlightPINVOKE.CodeGenerator_generateString(HandleRef jarg1, String jarg2)
   at Smdn.LibHighlightSharp.Bindings.CodeGenerator.generateString(String input) in /_/src/Smdn.LibHighlightSharp.Bindings/Smdn.LibHighlightSharp.Bindings/highlight-3.44/CodeGenerator.cs:line 103
   at Smdn.LibHighlightSharp.Highlight.Generate(String input) in /_/src/Smdn.LibHighlightSharp/Smdn.LibHighlightSharp/Highlight.CodeGenerator.Methods.cs:line 61