c# – CSV Helper解析器抛出IOException.无法从传输连接中读取数据

2023年2月15日 263次阅读

我在我的c#项目中使用CSVHelper并将一个大的csv数据文件(大约2000条记录)读入内存.

https://github.com/JoshClose/CsvHelper

如果记录小于500,它可以正常工作,它总是在不同阶段抛出IOException取决于网络或数字是否上升.我目前部署到Azure云平台,所以从blob存储到服务器的读取,应该不是任何网络问题.

CsvHelper.CsvParserException: A parsing error occurred.

Row: '995' (1 based)
 ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.IO.StreamReader.ReadBuffer(Char[] userBuffer, Int32 userOffset, Int32 desiredChars, Boolean& readToUserBuffer)
   at System.IO.StreamReader.Read(Char[] buffer, Int32 index, Int32 count)
   at CsvHelper.CsvParser.GetChar(Int32& fieldStartPosition, Int32& rawFieldStartPosition, String& field, Boolean prevCharWasDelimiter, Int32& recordPosition, Int32& fieldLength, Boolean isPeek) in c:\Projects\CsvHelper\src\CsvHelper\CsvParser.cs:line 445
   at CsvHelper.CsvParser.ReadLine() in c:\Projects\CsvHelper\src\CsvHelper\CsvParser.cs:line 247
   at CsvHelper.CsvParser.Read() in c:\Projects\CsvHelper\src\CsvHelper\CsvParser.cs:line 108
   --- End of inner exception stack trace ---
   at CsvHelper.CsvParser.Read() in c:\Projects\CsvHelper\src\CsvHelper\CsvParser.cs:line 136
   at CsvHelper.CsvReader.Read() in c:\Projects\CsvHelper\src\CsvHelper\CsvReader.cs:line 173

它会抛出(csv.read())

 var wc = new WebClient();

            using (var sourceStream = wc.OpenRead(fileUrl))
            {

                using (var csv = new CsvReader(new StreamReader(sourceStream)))
                {


                    while (csv.Read())
                    {
                        try
                        {

//some reading operation
}
 catch (Exception ex)
                        {
                            _logger.Error(ex);
                        }
                    }
                    _logger.InfoFormat("Finished {0} reading data #{1}");

                }
            }

在哪里设置streamreader超时值？

最佳答案 使用云资源(无论是Azure还是任何其他云资源)时,您不应直接读取文件.最好你应该实现重试逻辑,以确保绕过任何瞬态错误(阅读有关瞬态错误
here,
here和
there或只是在互联网上搜索术语“瞬态错误”).

在你的情况下,我建议你通过CloudBlockBlob.DownloadToStream方法包裹你的电话.因此,您仍然可以使用Stream来解析文件,但您也将使用.NET Azure Blob API库的安全性,该库可以代表您处理所有瞬态错误.

您的代码看起来像这样：

// get the CloudblockBlob object
using(MemoryStream blobStream = new MemoryStream())
{
    blobObject.DownloadToStream(blobStream);
    using (var csv = new CsvReader(new StreamReader(blobStream)))
    {
        while (csv.Read())
        {
            try
            {
             //some reading operation
            }
            catch (Exception ex)
            {
                _logger.Error(ex);
            }
        }
        _logger.InfoFormat("Finished {0} reading data #{1}");
    }
}
点赞